Because FLOSS is handy, isn’t it?

Parsing command line arguments

Posted by isilanes on December 20, 2006

In UNIX-like environments, such as GNU/Linux, command line is often used to operate on a bunch of files, such as:

rm -f *.dat

In the command above, “*.dat” is expanded by the shell (the command interpreter), to all matching files in the present directory (e.g.: “file1.dat file2.dat dont_delete_me.dat this_file_is_rubbish.dat“). However, this expansion is performed as a first step, and then the expanded command line is executed, e.g.:

rm -f file1.dat file2.dat dont_delete_me.dat this_file_is_rubbish.dat

This behaviour can potentially fail if a lot of files match the *.dat, because there is an upper limit to how wide a command line can be (brutally high, but finite). This can happen, for example, if you try to delete the contents of a directory with 100,000 files, and use rm -f * (yes, this can happen). For example, a ls in a directory with 100,000 files works fine, but an rm * does not:

Bart[~/kk]: rm *
/bin/rm: Argument list too long.

To avoid this problem, we can make use of xargs and echo (since echo does not seem to suffer from this argument number limitation), in the following way:

echo * | xargs rm -f

Now, xargs takes care of the argument list, and processes it so that rm does not see an endless argument list.

xargs can also be given other uses. For example, consider the following example: We want to convert some OGG files to MP2 (I won’t be caught dead near an MP3, due to its patent issues), so that a Windows luser can hear them. We can use oggdec to convert OGG to WAV, then toolame to convert WAV to MP2. Now, oggdec accepts multiple arguments as input files to convert, so the following works:

oggdec *.ogg

The above generates a .wav for each .ogg in the argument list. However, toolame does not work like that; it expects a single argument, or, if it finds two, the second one is interpreted as the desired output file, so the following fails (too many arguments):

toolame *.wav

This is where xargs can come in handy, with its -n option (see the xargs man page). This option tells xargs how many arguments to process at the same time. If we want each single argument to be processed separately, the following will do the trick:

echo *.wav | xargs -n 1 toolame

In the above example, toolame is called once per WAV file, and sees a single WAV file as argument each time, so it generates a .mp2 file per .wav file, as we wanted.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: