TextMate News

Anything vaguely related to TextMate and macOS.

Working With History in Bash

Yesterday we talked about favorite bash features (on the ##textmate IRC channel). I figured it was worth posting mine to this blog, they mostly revolve around history, hence the title.

Setup

My shell history collects a lot of complex command invocations which take time to figure out. To ensure that I have access to them at a later time, I have the following 3 lines in my bash init:

export HISTCONTROL=erasedups
export HISTSIZE=10000
shopt -s histappend

The first one will remove duplicates from the history (when a new item is added). For example if you switch between running make and ./a.out in a shell, you may later find that the last 100 or so history items is a mix of these two commands. Not very useful.

The second one increase the history size. With duplicates erased, the history already holds a lot more actual information, but I still like to increase the default size of only 1,000 items.

The third line ensures that when you exit a shell, the history from that session is appended to ~/.bash_history. Without this, you might very well lose the history of entire sessions (rather weird that this is not enabled by default).

History Searching

Now that I have my history preserved nicely in ~/.bash_history there are a few ways to search it.

Using Grep

The most crude is grep. You can do:

history|grep iptables

For me (on this particular Linux server) I get:

4599  iptables -N http-block
4600  iptables -A http-block -s 58.60.43.196 -j DROP
4601  iptables -A INPUT -p tcp --dport 80 -j http-block
4602  iptables -L http-block
4603  iptables-save -c
4604  history|grep iptables

I do this often enough to have an alias for history (which is just h).

From the output I can either copy/paste the stuff I want, or repeat a given history event. You’ll notice that each history event has a number, you can repeat e.g. event number 4603 simply by running:

!4603

I will write a bit more about referencing history events in History Expansion.

Prefix Searching

Similar to how you can press arrow up for the previous history event, there is a function you can invoke for the previous history event with the same prefix as what is to the left of the insertion point.

This function is called history-search-backward and by default does not have a key equivalent. So to actually reach this function, I have the following in ~/.inputrc (or /etc/inputrc when I control the full system):

"\ep": history-search-backward

This places the function on ⎋P (escape P). So if I want to repeat the iptables-save -c history event we found in previous section, all I do is type ipt and hit ⎋P. If it finds a later event with the same prefix, hit ⎋P again to go further back.

This functionality is offered by the readline library, so if you setup this key, you have access to prefix searching in all commands which use this library.

It is possible to press ⌃R to do an incremental (interactive) search of the history.

Personally I am not a big fan of this feature, so I will leave it at that :)

Update: The reason I dislike ⌃R is both because the interactive stuff just seems to get in the way (when ⎋P is what I need 99% of the time) and because it fails in cases where I “switch shell”, for example I may do: ssh mm press return, then instantly type: f⎋P and again hit return (to execute free -m on the server called mm). I enter this before the connection to the server has been fully established, and here ⌃R would have been taken by the local shell, but it is the shell history at the server I want to search.

History Expansion

History Expansion was what we did above when we ran !4603. It is a DSL for referencing history events and optionally run transformations on these.

Anyone interested in this should run man bash and search for History Expansion, but just to give you a feel for what it is, I will reference a subset of the manual and provide a few examples.

Event Designators

First, an event designator starts with ! and then the event we want to reference. This can be:

«n»      Reference event number «n».
-«n»     Go «n» events back.
!        Last line (this is the default).
#        Current line.
«text»   Last event starting with «text».
?«text»  Last event containing «text».

So if we want to re-run our iptables-save -c we can do: !ipt.

What’s more useful though is to use history references as part of larger commands.

For example take this example:

% which ruby
/usr/bin/ruby
% ls -l $(!!)
lrwxr-xr-x  1 root  wheel  76 30 Oct  2007 /usr/bin/ruby -> ../../System/Library/Frameworks/Ruby.framework/Versions/Current/usr/bin/ruby

Or something like:

% make some_target
(no errors)
% path/to/target some arguments
(no errors)
% !-2 && !-1

Word Designators

In the previous section we referenced entire history events. It is possible to reference just a subset of a history event by appending a : to the event designator and then the word of interest, the two most useful are:

«n»      Reference the «n»’th word.
$        Reference the last word.

So for example we can do:

% mkdir -p /path/to/our/www-files
(no errors)
% chown www:www !$
(no errors)

Here we reference last word of last line. We can also reference stuff on the same line, e.g.:

% cp /path/to/important/file !#:1_backup

To reference the last word of last line one can also press ⎋_ which will immediately insert that word.

Modifiers

To make history substitution even more useful (and harder to remember), one can also add a modifier to the event designator.

The most useful modifiers are in my experience :h and :t, these are head and tail respectively or better know as dirname and basename.

An example could be:

% ls -l /path/to/some/file
(listing of file)
% cd !$:h
(change directory to that of file)

Brace Expansion

Somewhat related to the backup example where we reference the first argument as !#:1 and append _backup to this, another approach is bracket expansion.

Anywhere on a command line, one can write {a,b,c} which will expand to the 3 words a, b, and c. If we include a prefix/suffix, that will be part of each of the expanded words. We can also leave the word in the braces empty, and have it expand to just the prefix/suffix, so for example we can do:

% cp /path/to/important/file{,_backup}

This is functionally equivalent to:

% cp /path/to/important/file !#:1_backup

But lack of hardcoded word number is IMO an improvement.

categories General

46 Comments

28 June 2008

by Anonymous

This is interesting, but… any updates on 2.0? A year ago I thought for sure it would be out by now…

Yo, Frakking lay off with the 2.0 shit. It’s not like he’s spent a year looking at YouTube.

yea, and where’s textmate for the iphone anyway? :P

28 June 2008

by Steve L

Wow … and all this time I’ve been thinking that I was pretty handy with the shell.

Thanks for the schooling.

Thanks Allan, this is great!

Simply add bind Space:magic-space to your .bash_profile to be able to auto-expandy those crazy ! commands inline before executing them.

Personally, instead of using the escape-p business, I just replaced the standard up/down character history search with the history search with context. If you want it to work without the context just move your cursor to the left.

bind '"\e[A"':history-search-backward bind '"\e[B"':history-search-forward

bind '"\e[A"':history-search-backward

bind '"\e[B"':history-search-forward

no, I’m not a Markdown pro :’(

28 June 2008

by Naveed

You should consider putting the following 2 lines in your .inputrc “\e[A”: history-search-backward “\e[B”: history-search-forward It will allow you to search backward just by pressing the up key.

I have written an article which teaches how to work efficiently on the command line.

It’s called “[The Definite Guide to Bash History])http://www.catonmat.net/blog/the-definitive-guide-to-bash-command-line-history/)”

Sincerely, Peter

Fixing my previous comment:

Article: The Definite Guide to Bash History

The reason I dislike ⌃R is both because the interactive stuff just seems to get in the way

Here’s how to use it properly.

^rgit will show you the last line in your shell history with git.

From here you can:

  1. Press enter to re-execute your command immediately.
  2. Press ESC or CTRL-[ to edit the command line found, for what you need.
  3. And much better, type ^r to search more for previous instance, from the current position.

In any case it is quicker than ESC-P and more powerful.

Rather than using ^r to search the history, I usually type the start of the command, and then use the up/down arrows to complete it. Details here: http://www.pixelbeat.org/settings/.inputrc

29 June 2008

by lee doolan

Just a little note on history expansion. If you do this

<event-designator>M-^

the event will be moved into the command line buffer but it will not be executed. You can use editing commands to modify the line before typing to execute. For instance:

!iptableM-^

will move the last iptables command into the buffer and

!4601

would move this command (from the example in the main article) into the buffer:

iptables -A INPUT -p tcp --dport 80 -j http-block

This is a very handy facility. Try it, you’ll like it.

29 June 2008

by lee doolan

Just one further note:

M-

is the meta key. On my keyboard it is bound to ALT, but you can change this binding. Consult the man page for details.

[…] Working with History in Bash - Some tips from Allan Odgaard. […]

30 June 2008

by David Maas

Didn’t know the brace trick, very very very useful

A few other bash tricks:

Add “cdspell” to the shopt -s line. This allows bash to correct for minor misspellings when doing cd

export HISTIGNORE="&:ls:[bf]g:exit:cd ..:make:svn up:svn commit"
# a bunch of command-lines separated by colons

There’s also a pile of great bash completions at Caliban.org. I use them so often that I find myself wondering why tab-completion “doesn’t work” on someone else’s computer.

[…] TextMate Blog » Working With History in Bash - Great tips for Bash. Lots of great comments too. This was written by Kris Markel. Posted on Monday, June 30, 2008, at 9:13 pm. Filed under del.icio.us. Tagged bash, commandline, history, howto, Linux, reference, shell, terminal, tips, Unix. Bookmark the permalink. Follow comments here with the RSS feed. Post a comment or leave a trackback. […]

Thank you. A lot.

01 July 2008

by Håkan Eriksson

Hey, I’ve recently been looking for a way to this kind of stuff in bash.

One question; at a place where I used to work, someone had set up our environment so that Ctrl-ArrowUp (or Ctrl-P) did what is described as Esc-P above, i.e. jump to the previous line in history that starts like the one you’re currently typing.

I’d like to have Ctrl-P (for instance) instead of Esc-P for “history-search-backward” - how do I do that?

erasedups for HISTCONTROL isn’t available for bash 2.05 (like that of OSX Tiger) insead you can use

export HISTCONTROL=ignoreboth

more info on: http://aplawrence.com/Linux/bash_history.html

[…] http://blog.macromates.com/2008/working-with-history-in-bash/ […]

If you’re man (or woman) enough to have learned vi, the same search commands (/, n, N), line navigation (k, l) and line editing (cw, dw, C, r, etc.) are all available to you on the command line if you just issue this command: “set -o vi”.

There’s an interactive equivalent to the “Word Designators” section that I really like:

M-. (that’s meta-dot) yanks the last word of the previous command. Give it an argument (like M-2 M-.) to yank the /n/th word, counting from zero.

Very useful for running several commands on the same file without having to think about editing.

Another one: fixing mistakes. To correct something in a command (let’s say the last command):

!!:s/old/new/

There is a shortcut for this (only works on !!):

^old^new

But it doesn’t let you replace every occurence on the line, like so:

!!:gs/old/new/

One more and I’m done. Let’s say you wanted to run the sequence of iptables commands from the first example, but with a different name for the chain. Doing an s/// substitution on each command would get annoying. The sequence is from numbers 4599 to 4603 in the history. So you run:

fc 4599 4603

and bash launches your text editor ($EDITOR, maybe vi if you don’t have that set) with all the commands in it. Do a global search-and-replace to change “http-block” to whatever else (or make any other edits you want). Then save and quit, and bash will run each edited command, printing them before they are run.

With no arguments, fc just edits the last command.

[…] TextMate Blog » Working With History in Bash Great collection of tips for managing your history in Bash. This has already helped me out. (tags: bash terminal tips development) […]

Man the load of useful stuff here is amazing.

Not much to add here but you can add the following to your .bash_profile you can tab through other completions rather than trying to remember what letter to write next. Useful if you don’t use the bash very often and have a hard time remembering commands.

bind '"\t":menu-complete'

[…] TextMate Blog: Working with History in Bash […]

10 July 2008

by Thomas

Where can we find product updates? I’m also interested in how close we are to the 2.0 release. Generally, I think it’s a good idea to let customers know something about the software they’re purchasing, and an expected arrival date/month/quarter would be nice. More accurately, knowing that a new release will make it out in 2008, for example, would certainly dissuade me from buying Komodo IDE. Or using MacVim as my primary editor, since I use GVim on Windows at the office.

wow this is a great compilation of useful commands, thanks a lot

31 July 2008

by nairda

@Thomas: TextMate is dead, get used to it.

TextMate isn’t dead, it’s just stagnating because Allan’s spending his time working on 2.0 instead of incremental updates to 1.x. And the bundles are being updated all the time by the community.

[…] Working With History in Bash [TextMate Blog] - Allan Odgaard wrote a hugely useful tutorial to help us master bash history. I found this very useful and I’m sure you will, too. […]

Thanks a lot

[…] Working With History in Bash - Some very good tips for the history command. […]

[…] Recent public urls tagged “commandline” → TextMate Blog “ Working With History in Bash […]

[…] the bash prompt I would like to show you is the history. This comes in part from the TextMate blog. What you can do is increase or decrease the history size, remove duplicates and make sure it saves […]

[…] the bash prompt I would like to show you is the history. This comes in part from the TextMate blog. What you can do is increase or decrease the history size, remove duplicates and make sure it saves […]

[…] If you’re new here, you may want to subscribe to my RSS feed. Thanks for visiting!Working WIth History in Bash […]

[…] have them in any of it’s shells. I mean seriously, how does anyone live without a searchable history in a command […]

[…] TextMate Blog » Working With History in Bash […]

[…] Allan Odgaard’s excellent Working With History in Bash: export HISTCONTROL=erasedups export HISTSIZE=10000 shopt -s histappendPop out1 2 3 export […]

[…] by marinz on May 5, 2009 Ho trovato questo articolo, in pratica basta inserire nel […]