Shell variables

When you execute a shell command/script from TextMate, it exposes a lot of information as shell variables (file path, project folder, current word, selection, caret position a.s.o.) which you can work with in your script.

You may often want to do miscellaneous transformations on the variables, and bash supports quite a lot of neat syntax to do so, so what follows is a short tutorial with examples. I have picked what I consider the most useful stuff, for the full list there's man bash.

Quoting variables

You should be aware that if a variable contain spaces (or other characters set in the input field separator variable) the variable will be expanded to several words.

For example:

str="foo bar"
for word in $str; do echo $word; done

Will output:

foo
bar

To avoid this we need to quote the variable like this:

for word in "$str"; do echo $word; done

Generally you should always quote variables, unless you want it to be interpreted as multiple words.

Providing a default value

There are times when a variable may not be set and we want to use a default value instead. For example if we want to print the current file name (TM_FILENAME) and still want our command to output something when the file is untitled. The naive way to do this is:

if [[ "${TM_FILENAME}" ]]
    then echo "${TM_FILENAME}"
    else echo "untitled"
fi

A simpler approach is to use the ${«VARIABLE»:-«default value»} notation. With this, the above can be reduced to:

echo "${TM_FILENAME:-untitled}"

The default value doesn't have to be text, e.g. we can use another variable. So if we want to lookup the selected text using ri (Ruby doc lookup) but fallback on the current word, we can execute:

ri -T "${TM_SELECTED_TEXT:-$TM_CURRENT_WORD}"

A variant is ${«VARIABLE»:=«default value»}, which also expands to «default value» when «VARIABLE» is not set, but in addition assigns the default value to the variable. This is useful if we want to use the variable again later and don't want to add the default value each time.

Replacements

The general way to replace one string with another in a variable is by using: ${«VARIABLE»/«pattern»/«replacement»}.

The pattern is the same as used for filename expansion, that is (w/o enabling extended glob patterns), you can use:

  • * to match anything (e.g. "*.txt"),
  • ? to match a single (arbitrary) character (e.g. "f??.txt"), and
  • […] to express a set of characters to match. The set of characters can contain:
    • POSIX classes (e.g. "[[:alpha:]]"),
    • ranges (e.g. "[a-c]"),
    • single characters (e.g. "[abc]"), and
    • you can negate the set by using ^ or ! as first character (e.g. "[^[:digit:]]").

If you use a variable in place of the pattern, this variable will be interpreted as a pattern, so it shouldn't contain *, ? or [. If it does, you need to escape them, which can be done using replacements on the variable. It's clumsy, but we can do it with a bash function like this:

glob_esc () {
    res="${1/\\[/\\[}"
    res="${res/\\?/\\?}"
    echo -n "${res/\\*/\\*}"
}

And then instead of $var we use $(glob_esc "$var"). For the rest of this post, I'll assume that this is not a problem.

So with all that, if we want to replace Users with home in ${TM_FILEPATH} we can do:

${TM_FILEPATH/Users/home}

This only does a single replacement, if we want to replace all occurrences we need to use ${«VARIABLE»//«pattern»/«replacement»}. For example to replace all spaces with dashes in the path, we can use:

${TM_FILEPATH// /-}

To indicate that the pattern should only be replaced if it's either at the beginning or end of the variable, we need to prefix it with # or % respectively. So if we want to remove the home directory prefix, we can do (here the # is mostly just a safety precaution):

${TM_FILEPATH/#$HOME/}

When we just want to chop off the prefix or suffix of our variable, there's a special notation for that, namely ${«VARIABLE»#«pattern»} and ${«VARIABLE»%«pattern»}. Unlike the general replacement, these will use the shortest match. So for example if we want to get rid of the file extension, we may try something like this (using the general replacement notation):

${TM_FILEPATH/%.*/}

But if the path contains more than one dot, it will chop off everything from the first dot (since * is greedy). Instead we need to use the non-greedy:

${TM_FILEPATH%.*}

There are actually also greedy versions of this cut prefix/suffix specialization of the general replace. These have two # or %'s. E.g. to cut off everything up until the last slash (i.e. to get the filename from the full path) these two lines both achieve that task:

${TM_FILEPATH/#*\//}  # general replace
${TM_FILEPATH##*/}    # cut prefix (greedy)

Subsets

To skip the first n characters from a variable, one writes: ${«VARIABLE»:«n»}. If we want m characters starting at position n, the syntax is: ${«VARIABLE»:«n»:«m»}.

Here n and m can both be math expressions. So for example TextMate exports the contents of the current line as TM_CURRENT_LINE and the carets column position as TM_COLUMN_NUMBER (one based). If we want to get everything to the right of the caret (i.e. cut everything to the left of it) we'd do:

${TM_CURRENT_LINE:$TM_COLUMN_NUMBER-1}

If instead we want everything to the left of the caret, we can do:

${TM_CURRENT_LINE:0:$TM_COLUMN_NUMBER-1}

Length of variable

The length of a variable can be obtained using ${#«VARIABLE»}. We may need this if we lookup the current word in a list of potential completions, and want to insert only the missing part. Here's an example:

match="$(grep "^$TM_CURRENT_WORD" <<'EOF'|head -n1
what
when
which
EOF)"
echo -n "${match:${#TM_CURRENT_WORD}}"

E.g. if the current word is "whi" then the result from the above will be "ch".

Here-strings

The above should cover the majority of situations where you need to transform a variables value, but there are cases where it would be nice to run the variable through tr, a perl regular expression or similar.

Normally shell commands read their data from stdin which may first prompt us to do things like:

printenv TM_SCOPE|tr ' ' '\n' # convert spaces to newlines

But it is possible to provide a single string as stdin for a command using a here-string, which looks like this:

tr <<<$TM_SCOPE ' ' '\n'

We can wrap that expression in $(…) to use it where we need the result. So if for example we want to create an alternate title for an image by extracting the base name from the image file path, replace -, _, and . with a space, and title case it, we may want to run TM_DROPPED_FILE through a small perl script, with data taken from stdin, e.g.:

echo '<img … alt="'"$(perl -pe <<<$TM_DROPPED_FILE \
   's%.*/(.*?)\.[^.]*%ucfirst $1%e; y/-_./ /')"'">'

Granted, we could have avoided stdin here, but I was running out of examples :)

9 Comments

  1. 26 Sep 2005 | # zzen wrote…

    Excellent tips, thank you! Geez – and I thought I knew a lot about shells…

  2. 26 Sep 2005 | # domenico wrote…

    allan pwnz us all :)

  3. 01 Oct 2006 | # marios wrote…

    What I am trying to figure out is, what would be the best way to shop off both the suffix and the file-path , to retain just the filename without extension. I had a look at the man page, but couldn't find any special notation for such a task.

    regards, marios

  4. 01 Oct 2006 | # Allan Odgaard wrote…

    marios: If you want to cut the extension of a filename (contained in a variable) you can do something like "${VARIABLE%.*}". You would combine that with basename or have to do two variable replacements to also cut the directory part of the path.

  5. 07 Oct 2006 | # marios wrote…

    Terrific, just came across this, thank you so much Allan. I just ported some of these excellent examples over to a new Bash Tutorial Bundle that I created, so I can memorize the Syntax a little better and tool-tip test, what I'm doing.

    regards, marios

  6. 08 Nov 2006 | # kyle wrote…

    I can't get any of this to work…

    i do the ${TM_FILEPATH/Users/Home} and nothing shows up..

    can someone send me a same script that actually works

  7. 09 Nov 2006 | # Allan Odgaard wrote…

    kyle: where do you do this (bash, TextMate, …)? How do you output the value? And does ‘echo ${TM_FILEPATH}’ alone output something?

  8. 28 Nov 2007 | # Dean wrote…

    Nice work Allan, just what I was looking for, had no idea where to start with the search/replace stuff. Nicely written, easy/clear to read :-)

  9. 02 Jul 2008 | # Yves Van Walle wrote…

    Excelent! just what I looking for!

Comments closed, you can use the mailing list for discussion.