How Can I Properly Quote Bash Commands In Python When Using `subprocess.check_output`?

I'm trying to run and read the output of time command in python. The problem is that time command works differently in /bin/sh and /bin/bash like so: sh $ time ls .. 0.00user 0.00s

Solution 1:

With /bin/sh, time ls runs the external program /usr/bin/time (your path may vary), which then runs ls and reports the timing information to its standard error.

With /bin/bash, time is a keyword which instructs the bash shell itself to report timing information for the execution of ls to standard error. As such, you need to redirect the standard error of the shell, not the command, to standard output so that check_output can capture it. It's a little ugly, but

s.check_output('exec 2>&1; time ls >/dev/null 2>&1', executable='/bin/bash', shell=True)

should work. The exec 2>&1 is the part that lets check_output capture anything. The redirections following ls are there to remove the output of ls from what is captured; you can delete them if you want the command output as well.

Solution 2:

The error is caused by the fact that you should not quote strings when proving a list as command line. In fact:

>>> s.check_output(['bash', '-c', "time ls"])

real0m0.002s
user0m0.000s
sys     0m0.000s
'some content produced by ls'

Doesn't raise any exception, while:

>>> s.check_output(['bash', '-c', "'time ls'"])
bash: time ls: comando non trovato
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.3/subprocess.py", line 586, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['bash', '-c', "'time ls'"]' returned non-zero exitstatus127

Raises the exception you see. Note the line before the traceback:

bash: time ls: comando non trovato

Quoting the 'time ls' made bash search for the program time ls instead of program time to be run with ls as argument.

Why quoting is not needed? Easy: the list of strings specifies how the command line should be splitted. Putting "time ls" in a single element of the list already provides the "quoting" you want.

---

As to how to obtain the output you want, since bash's time outputs on stderr, a solution is to redirect stderr to stdout (and remove ls output).

>>> s.check_output(['bash', '-c', "time ls > /dev/null"], stderr=s.STDOUT)
'\nreal\t0m0.001s\nuser\t0m0.000s\nsys\t0m0.000s\n'