Welcome to Pysh’s documentation!

Preparing command lines

pysh.shwords(format_string, *args, **kwargs)

Split format_string, then format using args and kwargs, producing a list.

Handy for producing the command line for invoking an external program, conveniently but without the complex gotchas of shell parsing. For example:

>>> shwords('rm -rf /tmp/{userdoc}', userdoc='1 .. 2')
['rm', '-rf', '/tmp/1 .. 2']

The format_string is split on spaces. Each word is then formatted through a minilanguage similar to str.format(). Each word of format_string produces exactly one item in the result (unless explicitly instructed otherwise with {!@}), regardless of the contents of the interpolated values.

The formatting minilanguage is exactly the same as for str.format(), except:

  • An additional conversion !@, as in {!@}. This must appear in format_string as a whole word. The argument must be an iterable, and each element of the iterable becomes an element in the result.

  • The conversions !r and !a are omitted, because they only make sense within a Python context.

  • No nested interpolation, as in {:{}}.

pysh.shwords_f(format_string)

Process (almost) like an f-string, but with splitting like shwords().

NB unlike an f-string, only the caller’s locals are available; not globals, and enclosing scopes are complicated.

Running commands

These provide thin wrappers around subprocess, mainly adding shwords().

The basic check_cmd() is a small convenience helper, just composing shwords() with subprocess.check_call().

Similarly, check_cmd_f() behaves like shwords_f() composed with subprocess.check_call(): it processes the command format-string much like an f-string.

The try_cmd() variant provides more appropriate semantics when the command’s failure is an expected condition, by returning a bool for success/failure rather than raising an exception on failure. Similarly try_cmd_f().

The slurp_cmd() function and the rest of its family run a command for its output, with subprocess.check_output(), and strip trailing newlines. This matches the behavior of $(...) in Bash and fits nicely with conventional semantics for Unix CLI tools.

The behavior of slurp_cmd_f(), try_slurp_cmd(), and try_slurp_cmd_f() relate to slurp_cmd() in the same way as check_cmd_f(), try_cmd(), and try_cmd_f() do to check_cmd().

In short, these functions fill the following table:

check_cmd      try_cmd
check_cmd_f    try_cmd_f

slurp_cmd      try_slurp_cmd
slurp_cmd_f    try_slurp_cmd_f
pysh.DEVNULL

Has the same meaning for stdout/stderr arguments of these functions, and of cmd.run, as for subprocess.Popen.

pysh.STDOUT

Has the same meaning for stderr arguments of these functions, and of cmd.run, as for subprocess.Popen.

pysh.check_cmd(fmt, *args, _stdin=None, _stdout=None, _stderr=None, _cwd=None, _timeout=None, **kwargs) → None

Just like subprocess.check_call(), but with shwords().

The named keyword arguments are passed through, with stdin=_stdin etc. All other arguments are passed to shwords().

pysh.check_cmd_f(fmt, **kwargs) → None

Just like check_cmd(), but with shwords_f() instead of shwords().

This means fmt is processed much like an f-string, with access to the caller’s locals. See shwords_f() for details.

Also, unlike check_cmd() all keyword arguments are passed straight through to subprocess.check_call().

pysh.try_cmd(fmt, *args, **kwargs) → bool

Just like check_cmd(), but returns success/failure rather than raise.

pysh.try_cmd_f(fmt, **kwargs) → bool

Just like check_cmd_f(), but returns success/failure rather than raise.

pysh.slurp_cmd(fmt, *args, _stdin=None, _stderr=None, _cwd=None, _timeout=None, **kwargs) → str

Run the command and capture output, stripping any trailing newlines.

Stripping trailing newlines is the same behavior as $(...) has in Bash. It fits nicely with conventional semantics for Unix CLI tools.

See also pysh.slurp().

pysh.slurp_cmd_f(fmt, **kwargs) → str

Just like slurp_cmd(), but with shwords_f() instead of shwords().

Also, like check_cmd_f() in contrast to check_cmd(), all keyword arguments are passed straight through to subprocess.check_output().

pysh.try_slurp_cmd(fmt, *args, **kwargs) → Optional[str]

Just like slurp_cmd(), but on failure returns None rather than raise.

pysh.try_slurp_cmd_f(fmt, **kwargs) → Optional[str]

Just like slurp_cmd_f(), but on failure returns None rather than raise.

Running pipelines

pysh.slurp(filter)

Run the pipeline and capture output, stripping any trailing newlines.

Stripping trailing newlines is the same behavior as $(...) has in Bash. It fits nicely with conventional semantics for Unix CLI tools.

See also pysh.slurp_cmd.

pysh.to_stdout(filter)

Run the pipeline, with output directed to our stdout.

Commands in pipelines

pysh.cmd.run(fmt, *args, _check=True, _stderr=None)

Run the given external command, as a pipeline filter.

Like any filter, the return value of cmd.run() only has an effect when a pipeline containing it is run by a function like slurp() or to_stdout(). When run:

  • The given fmt and *args are interpreted by shwords() to produce a command line.

  • The command line is executed (using subprocess.Popen), with stdin and stdout connected to the filter’s input and output in the pipeline. Input is optional.

  • The external command’s stderr can be controlled with _stderr, using values None, pysh.DEVNULL, or pysh.STDOUT.

  • If _check is true (the default), an exception is raised on failure just like check_cmd(). Otherwise, the external command’s return code is ignored.

NOTE: In the current implementation, the input may be read completely into a buffer before any of it is passed to the external command. This is just fine for lots of scripts, but will be fixed before Pysh 1.0. If you have a use case where this matters, please file an issue, to help prioritize fixing it.

pysh.cmd.splitlines()

Split the input stream into an iterator of lines.

The meaning of “line” follows the standard Unix convention:

  • Each newline byte (b'\n') terminates a line.

  • If there are any bytes after the last newline, they form one last line.

This is also the same behavior as Python’s bytes.splitlines(), except that only b'\n' counts as a newline: b'\r' is treated the same as any other byte.

pysh.cmd.cat(*filenames)

Read each of the given files in turn, and output their contents.

This is very similar to the Unix command cat, but with fewer features:

  • The filenames are never interpreted as options; they’re simply filenames.

  • The filename -, or an empty list of filenames, are not special; instead of reading from stdin, they cause reading from a file named -, and an empty output, respectively.

pysh.cmd.echo(*words, ln=True)

Write the given words to the output.

The words are separated by blanks b' '. If ln is true (the default), a newline b'\n' follows at the end.

This corresponds to the Unix command echo, with ln=False corresponding to echo -n.

For example:

>>> pysh.to_stdout( cmd.echo(b'hello', b'world') )
hello world
pysh.cmd.devnull()

Ignore any input, and provide nothing as output.

This corresponds to a shell redirection from or to /dev/null. In particular:

  • a shell command like … </dev/null corresponds to a Pysh pipeline cmd.devnull() | …;

  • a shell command like … >/dev/null corresponds to a Pysh pipeline … | cmd.devnull().

Indices and tables