Terminal Velocity – 4 Great Ways To Work With Automatic Expansion In The Shell

Written by
Venditan
Published on
24/7/2015
Share this post
Other posts by:
Venditan
Company
10
Min read
22/12/2023
10 retail and eCommerce trends for 2024
Trends that we believe are going to shape retail and eCommerce in 2024 and beyond.
6
Min read
22/12/2023
Venditan Commerce: The 2023 highlights
The significant updates and additions that have shaped Venditan Commerce in 2023.

With so much getting done in the shell, lots of little shortcuts can all add up to make things quicker in the future. After seeing people doing it the long way round, here’s the first in our series on efficiency hacks.

Note that in this series I’m only going to cover the 3 most common shells people use today, Bash, Ksh and Zsh. If you use something else you may find some of this advice doesn’t work or is unreliable. In that case I suggest consulting the manual for your specific shell.

Globbing – An oldie but goodie that bears repeating

Globbing is the name given to the pattern matching for files. This should be something most regular users have used, but it’s always handy to have a cheat sheet:

Globbing in practice

# Globbing basics
# Create the sample files
touch a ab bc cd de
# * Matches anything and everything
echo *
a ab bc cd de
echo a*
a ab
# ? Matches a single character
echo ?
a
echo ??
ab bc cd de
# Square brackets(and contents) match a single character
echo [abd]*
a ab bc de
echo [a-d]?
ab bc de
echo [ad-z]?
ab de
# ~ on it’s own matches your home directory
echo ~
/home/llord
# ~ followed by a username matches that user’s home directory
echo ~tom
/home/tom
echo ~root
/root

Extglob – Expanding on the available options

Almost all shells come with an expansion to the typical glob system that allows even more options, but before it can be used it needs to be turned, although the command to do so depends on your exact shell. If you’re unsure you can run

echo $SHELL
/bin/bash

to see what shell you’re running at the moment.

Extglob in practice

# extglob
# Create sample files
touch cheesecake chocolate-cake apple-pie pear-pie rice-pudding
echo *@(cake|pudding)
-bash: syntax error near unexpected token `(‘
shopt -s extglob
echo *@(cake|pudding)
cheesecake chocolate-cake rice-pudding
# You can even embed other globbing patterns, like ? or *
echo *-@(???|c*)
apple-pie chocolate-cake pear-pie
# Be careful using the not pattern, * is greedy and
# the not operator !(…) could match a single character.
echo *!(pie)
apple-pie cheesecake chocolate-cake pear-pie rice-pudding
echo !(*pie)
cheesecake chocolate-cake rice-pudding

More Globbing Options

There are more features than just extglob that can be turned on through shell options that affect the nature of globbing, although not all are available on all possible shells:

Other options in practice:

# Other glob options
# Set up
mkdir 5th 6th 6th/7th
touch .first .second third fourth 5th/a 5th/b 6th/c 6th/7th/d
# By default * ignores hidden files
echo *
5th 7th fourth third
shopt -s dotglob
# with dotglob they should match
echo *
5th 6th .first fourth .second third
># By default unmatched params are interpreted as literals
echo cat*
cat*
shopt -s nullglob
# with nullglob they’re removed from the param list
echo cat*

# Turn nullglob back off and turn failglob on
shopt -u nullglob
shopt -s failglob
# with failglob on you get an error
echo cat*
-bash: no match: cat*
# without globstar there are no recursive calls
echo **
5th 6th .first fourth .second third
shopt -s globstar
# With globstar it is a recursive directory search
echo **
5th 5th/a 5th/b 6th 6th/c 6th/7th 6th/7th/d .first .second third fourth

Brace Expansion

While globbing will only expand to match filenames that actually exist, brace expansion will always expand to every possible combination of the provided input.

Globbing is useful for working with already existing files. Whereas brace expansion is a general purpose tool that may be used to generate file or folder names that don’t exist, as well as a wide range of options not related to the filesystem.

Brace expansion works in two ways:

1. As a comma separated list:

{a,b,c}

# Brace expansion – lists
# Brace expansion is really useful for producing cartesian products,
echo {a,b,c}{1,2,3}
a1 a2 a3 b1 b2 b3 c1 c2 c3
# Creating a whole dir structure without repeating names,
echo {application/{controllers,library,modules,plugins,view},pub}/
application/controllers/ application/library/ application/modules/
application/plugins/ application/view pub/
# Or even just rename a file with a minor change
$ echo V{ei,ie}wHelper
VeiwHelper ViewHelper

2. As a range:

{a..c}

# Brace expansion – ranges
# Range expansion is good for lists and loops
for i in {1..25};do echo -n “$i “; done
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
# For example, loop over every letter in the alphabet,
# and count file starting with that letter:
for i in {a..z}; do echo -n “$i “; ls -1 $i* | wc -l; done
a 0
b 1
c 7
d 0
e 12

Final thoughts

We hope you find these shortcuts useful, next part on making your life easier in directories coming soon.

Our recent posts

Keep up to date with the latest news and insight from the team at Venditan

7
Min read
1/3/2024
Multilocal SEO for bricks-and-clicks retailers
The key techniques that form a successful multilocal SEO strategy.
Andrew Flynn
Head of Digital Marketing
3
Min read
29/2/2024
Meet the team - Steph
In our second staff profile of 2024 we're chatting with Steph Fenton.
Andrew Flynn
Head of Digital Marketing
4
Min read
29/2/2024
Get to know Force 4 Chandlery
In conversation with Lawrence Parr, Managing Director of Force 4 Chandlery.
Andrew Flynn
Head of Digital Marketing
6
Min read
27/2/2024
Gmail & Yahoo's new email sender requirements
Google and Yahoo are introducing mandatory email authentication requirements.
Callum Brocklehurst
Senior Web Developer
9
Min read
31/1/2024
Our guide to first-time cross-border eCommerce
Simplifying the mechanics of a successful international eCommerce strategy.
Andrew Flynn
Head of Digital Marketing
10
Min read
30/1/2024
eCommerce KPIs - 59 metrics that matter
A complete shopping list of KPIs that you can use to monitor your eCommerce performance.
Steph Fenton
Senior Account Manager