racing commands in hammer prepare-system
Commands that are expected to run synchronously are not waited for, resulting in unexpected races.
I saw a job failure caused by this once, and @manu also failed to prepare-system
once when building an AMI.
Described in more detail by @manu. Quoting:
Recently introduced hammer.py
temp folder changes (new path and cleanup) caused me troubles with building AMI for Ubuntu 22.04
Reason: execute('rm -rf ~/.hammer-tmp')
is deleting things that are still needed to finish the previous command
==> amazon-ebs: [HAMMER] 2022-12-12 17:53:36,306 >>>>> Executing sudo make install in /.hammer-tmp/libyang/build
==> amazon-ebs: [HAMMER] 2022-12-12 17:53:36,307 [Errno 2] No such file or directory: '/.hammer-tmp/libyang/build'
Solution:
I solved it by removing the rm -rf
but it can probably be solved by adding interactive=True
to execute()
calls like execute('whatever_command', interactive=True)
which will force execute()
function to wait for the process to finish via p.wait()
(p.communicate()
could also be used) as it uses p = subprocess.Popen()
for the command execution
BTW I would expect more issues as subprocess.Popen()
is used for almost every execution of the "command" in hammer.py