Shlex
Tuesday, September 29, 2026
Something I’ve always respected about Python is their infamous batteries included philosophy:
The Python source distribution has long maintained the philosophy of “batteries included” – having a rich and versatile standard library which is immediately available, without making the user download separate packages. This gives the Python language a head start in many projects.
We have a similar approach in Factor, which leads us to our extensive vocabulary index. Sometimes we see new things that we could add. One recent example is Python’s shlex module for splitting a command into arguments in a shell-like manner, keeping quoted strings together.
I’m happy to say that we now have a shlex vocabulary in the main repository.
We can use it to read a command with a quoted argument:
USING: io prettyprint shlex ;
IN: scratchpad "echo -n 'hello world'" parse-shlex .
{ "echo" "-n" "hello world" }
This is also useful for options containing spaces or an empty value:
IN: scratchpad "--title='My Project' --label ''" parse-shlex .
{ "--title=My Project" "--label" "" }
For command-like lines in a configuration file, we might want to allow
comments. The two flags to shlex-split enable comments and POSIX mode
(although perhaps instead of word arguments, maybe these should be
dynamic variables…):
IN: scratchpad "copy 'annual report.txt' archive # keep a backup" t t shlex-split .
{ "copy" "annual report.txt" "archive" }
IN: scratchpad "echo '#hello' # a comment" t t shlex-split .
{ "echo" "#hello" }
Going in the other direction, we can quote a filename as one argument for a POSIX shell:
IN: scratchpad "annual report.txt" shlex-quote print
'annual report.txt'
Or turn a whole sequence of arguments into a command string, ready to copy into a terminal:
IN: scratchpad { "cp" "annual report.txt" "backup copy.txt" } shlex-join print
cp 'annual report.txt' 'backup copy.txt'
Joining and splitting preserves the original arguments, including empty strings:
IN: scratchpad { "echo" "" "hello world" } shlex-join .
"echo '' 'hello world'"
IN: scratchpad { "echo" "" "hello world" } shlex-join parse-shlex .
{ "echo" "" "hello world" }
These words work with strings; they do not execute commands or expand variables and wildcards. The quoting is intended for POSIX-style shells.
This is now available in the development version of Factor!