Subversion Relocation

The URL of the main Subversion repository has been changed from the forgettable untypeable http://macromates.com/svn/Bundles/ to a much nicer http://svn.textmate.org/. We made this change a while ago, but we didn’t put in a redirect before because we weren’t sure the new server could handle the load (there were some problems with the first one). We are putting in the redirect now.

Subversion doesn’t handle this change gracefully (it will just report an error and fail), so anyone using svn to keep up with bundle development needs to manually update the checkout to the new location. Here are the steps required to relocate your checkout.

In the terminal:

# assuming this is where your old chekout is
cd /Library/Application\ Support/TextMate

svn switch --relocate \
    http://macromates.com/svn/Bundles/trunk/ \
    http://svn.textmate.org/trunk/

If you have “switched” subsets of the checkout, for example to test the “work in progress” (WIP) bundles, then this relocate command won’t fix the switched bundles. Run svn status in the root of your checkout, which will indicate any switched bundles with an “S” marker. For each switched bundle, find the from URL by changing directories into the bundle, and running:

svn info

This will show the URL on the second line. To make a to URL from that, just replace “macromates.com/svn/Bundles” with “svn.textmate.org”. We combine the from and to URLs when running the relocate command, for example:

cd Bundles/Subversion.tmbundle

svn switch --relocate \
    'http://macromates.com/svn/Bundles/branches/Subversion%20(Interactive%20Input).tmbundle/' \
    'http://svn.textmate.org/branches/Subversion%20(Interactive%20Input).tmbundle/'

(This was detailed in this mailing list post; instructions repeated here for convenience.)

Git for Bundle Development

For those developers who prefer distributed revision control, we’re now also mirroring the bundle repository on GitHub, at http://github.com/textmate/. This should make it much easier for you to customize your bundles and contribute back: just fork the bundle from GitHub, work on it locally, and then tell us when you have changes for us to pull back into the main repository. GitHub bundles will be kept in sync with the svn bundles, and changes to the main svn repository will be reflected at GitHub. As a bundle maintainer, you’ll be able to commit either to a fork of our GitHub bundle, or, as before, directly to svn. Anyone who isn’t a bundle maintainer will now also be able to work under version control.

For those existing authors without GitHub accounts, we used «nick»@svn.textmate.org as the commit email address. To link these changes to your account on GitHub so you’ll be properly credited (or blamed), just add that email to your account.

If you already forked a bundle obtained from svn, it would help us if you “rebase” that fork onto our GitHub mirror, as we carefully ensured proper history and author info for these mirrors. Also, our mirrors will serve as the starting point when looking for forks in the community. It is a bit of a nuisance to rebase all of these bundle forks, we realize, but it can be finished all at once, and will save pain later on. The sooner we can manage it, the better. To learn how to rebase your checkout, see Allan’s recent mailing list post.

This was initially intended as a two-way mirror (so that changes to the bundles in git would be reflected in svn as well), but the way git-svn works, we must rebase our “master” using the svn trunk, breaking the updates in one direction. So for now it is only a passive mirror. Practically, this means that when we pull changes, we must go through svn, so these changes will be given a new address (hash) and authorship info when they arrive in our master branch, requiring the fork we pulled from to do a new merge. We are looking into how to avoid this (and for this reason we are hesitant to pull changes right now: manually updating the dropped author info is tedious).

In short, TextMate development is going strong: TextMate 2 isn’t done yet, but progress is steady, it is starting to take shape, and the end is in sight. The rewrite has been a slow and careful process, but the ideas behind it are exciting. I hope to publicly describe some new abstractions in the coming weeks and months. Moreover, the community continues to churn out new bundles and features for TextMate 1.5, and I’ve been building up a backlog of posts describing them. While I am not writing to announce a release date for TextMate 2, I do hope that this post will be the first in a series showing a bit more transparency.

The requests for TextMate 1 have mostly been incremental additions such as split views, chunked undo, and editing over SFTP. But TextMate 2 is about more than new surface features. Every part has been completely rewritten to take advantage of the lessons learned from the years of version 1. Not only are the low-level data structures chosen for increased flexibility, but the abstractions on which TextMate is built—snippets, scope-based language grammars, context-dependent settings—have been rethought and are more powerful than ever. In the coming months, I’ll try to describe some of these new abstractions, but for now, know that I am excited about the new ideas involved.

So where does development stand for 2.0? It feels to me like most of the modules are getting close, say 90%. But as they say, on the horizon, mountains look small. While I use 2.0 for my own work, day-to-day, and the basic infrastructure is pretty solid, much of the front-end still needs work, and for now it’s all lacking the spit and polish of a finished app. Hopefully an alpha version will be ready before too long, but I can’t make any promises about dates.

And why haven’t I been better about keeping the world informed? It is a combination of many things really, but the main issue is that I am not good at writing for a large audience. I am more into informal conversations, for instance over mailing lists or on IRC. So while I started a lot of posts, I end up unhappy with them halfway through, and they don’t get finished or published. I am taking measures: I have enlisted a technical writer to help bring this blog back to life, and I’ll try to communicate more of TextMate’s status and direction through him.

Bigger than either of those problems though, as I mentioned, is that TextMate 2 is no minor facelift. It’s a major undertaking with a long timeline and its final form isn’t fully settled. I don’t want to hype vaporware, and I don’t want to get anyone’s hopes up before I know I can meet their expectations.

Furthermore, I haven’t wanted to throw ideas onto the internet without having a chance to implement them myself. I’m humbled that TextMate has served as inspiration for many other products, and I hope that it continues to be a model for other developers in the future, but I want to see my ideas done my way first, before I feed them to the competition.

I am trying to slowly turn this boat. With this post, I hopefully am showing that a hand is at the wheel. I know I’ve been quiet too long about my plans. I can’t make up for that, but going forward, I aim to do better.

Presently most bundles at GitHub use -tmbundle but likely because a dot was previously not allowed in a project name.

A few things still need to be translated (there is an issue tracker) but it should already be very useful for people who prefer reading Japanese over English.

And no, this does not mean that a new CJK-compliant TextMate has been released. The best approach for using CJK with TextMate is still via the plug-in.

Setup

My shell history collects a lot of complex command invocations which take time to figure out. To ensure that I have access to them at a later time, I have the following 3 lines in my bash init:

export HISTCONTROL=erasedups
export HISTSIZE=10000
shopt -s histappend

The first one will remove duplicates from the history (when a new item is added). For example if you switch between running make and ./a.out in a shell, you may later find that the last 100 or so history items is a mix of these two commands. Not very useful.

The second one increase the history size. With duplicates erased, the history already holds a lot more actual information, but I still like to increase the default size of only 1,000 items.

The third line ensures that when you exit a shell, the history from that session is appended to ~/.bash_history. Without this, you might very well lose the history of entire sessions (rather weird that this is not enabled by default).

History Searching

Now that I have my history preserved nicely in ~/.bash_history there are a few ways to search it.

Using Grep

The most crude is grep. You can do:

history|grep iptables

For me (on this particular Linux server) I get:

4599  iptables -N http-block
4600  iptables -A http-block -s 58.60.43.196 -j DROP
4601  iptables -A INPUT -p tcp --dport 80 -j http-block
4602  iptables -L http-block
4603  iptables-save -c
4604  history|grep iptables

I do this often enough to have an alias for history (which is just h).

From the output I can either copy/paste the stuff I want, or repeat a given history event. You’ll notice that each history event has a number, you can repeat e.g. event number 4603 simply by running:

!4603

I will write a bit more about referencing history events in History Expansion.

Prefix Searching

Similar to how you can press arrow up for the previous history event, there is a function you can invoke for the previous history event with the same prefix as what is to the left of the insertion point.

This function is called history-search-backward and by default does not have a key equivalent. So to actually reach this function, I have the following in ~/.inputrc (or /etc/inputrc when I control the full system):

"\ep": history-search-backward

This places the function on ⎋P (escape P). So if I want to repeat the iptables-save -c history event we found in previous section, all I do is type ipt and hit ⎋P. If it finds a later event with the same prefix, hit ⎋P again to go further back.

This functionality is offered by the readline library, so if you setup this key, you have access to prefix searching in all commands which use this library.

Incremental Search

It is possible to press ⌃R to do an incremental (interactive) search of the history.

Personally I am not a big fan of this feature, so I will leave it at that :)

Update: The reason I dislike ⌃R is both because the interactive stuff just seems to get in the way (when ⎋P is what I need 99% of the time) and because it fails in cases where I “switch shell”, for example I may do: ssh mm press return, then instantly type: f⎋P and again hit return (to execute free -m on the server called mm). I enter this before the connection to the server has been fully established, and here ⌃R would have been taken by the local shell, but it is the shell history at the server I want to search.

History Expansion

History Expansion was what we did above when we ran !4603. It is a DSL for referencing history events and optionally run transformations on these.

Anyone interested in this should run man bash and search for History Expansion, but just to give you a feel for what it is, I will reference a subset of the manual and provide a few examples.

Event Designators

First, an event designator starts with ! and then the event we want to reference. This can be:

«n»      Reference event number «n».
-«n»     Go «n» events back.
!        Last line (this is the default).
#        Current line.
«text»   Last event starting with «text».
?«text»  Last event containing «text».

So if we want to re-run our iptables-save -c we can do: !ipt.

What’s more useful though is to use history references as part of larger commands.

For example take this example:

% which ruby
/usr/bin/ruby
% ls -l $(!!)
lrwxr-xr-x  1 root  wheel  76 30 Oct  2007 /usr/bin/ruby -> ../../System/Library/Frameworks/Ruby.framework/Versions/Current/usr/bin/ruby

Or something like:

% make some_target
(no errors)
% path/to/target some arguments
(no errors)
% !-2 && !-1

Word Designators

In the previous section we referenced entire history events. It is possible to reference just a subset of a history event by appending a : to the event designator and then the word of interest, the two most useful are:

«n»      Reference the «n»’th word.
$        Reference the last word.

So for example we can do:

% mkdir -p /path/to/our/www-files
(no errors)
% chown www:www !$
(no errors)

Here we reference last word of last line. We can also reference stuff on the same line, e.g.:

% cp /path/to/important/file !#:1_backup

To reference the last word of last line one can also press ⎋_ which will immediately insert that word.

Modifiers

To make history substitution even more useful (and harder to remember), one can also add a modifier to the event designator.

The most useful modifiers are in my experience :h and :t, these are head and tail respectively or better know as dirname and basename.

An example could be:

% ls -l /path/to/some/file
(listing of file)
% cd !$:h
(change directory to that of file)

Brace Expansion

Somewhat related to the backup example where we reference the first argument as !#:1 and append _backup to this, another approach is bracket expansion.

Anywhere on a command line, one can write {a,b,c} which will expand to the 3 words a, b, and c. If we include a prefix/suffix, that will be part of each of the expanded words. We can also leave the word in the braces empty, and have it expand to just the prefix/suffix, so for example we can do:

% cp /path/to/important/file{,_backup}

This is functionally equivalent to:

% cp /path/to/important/file !#:1_backup

But lack of hardcoded word number is IMO an improvement.

Speaking of the Ruby bundle, Ciarán Walsh has a post about the design philosophy behind the mnemonics of the Ruby bundle and a few highlights.

And while there seems to be a preference for writing TextMate commands in Ruby, anything that you can run from a terminal (shell) can be used. A step-by-step tutorial about how to write TextMate commands in PHP is available at Ciarán’s blog. I pushed him to write this one, as I believe there are a lot of people who work with PHP in TextMate that never got around to writing custom commands in the false belief that they would have to learn elisp or some similarly obscure language :)

I’ve never seen TextMate work well on a network drive. With ExpanDrive, it’s a dream. And the drives stay mounted perfectly when I move my laptop between home and work.

John Gruber just posted a review about ExpanDrive in which he writes:

My first impression after reading ExpanDrive’s promotional description last week was that it sounded too good to be true. One week later, I’m pretty sure it actually is that good.

If it wasn’t clear from the above, ExpanDrive lets you mount remote sftp drives and it actually works! I.e. no long delays or dropped connection in the middle of a save. Their introductory price is $29.

2008-03-11: Update: There is a $5 discount for the first 100 who use UWQ0OUQZWIB1ZMTP as coupon code.

Ian Baird recently released Changes which is an application to show differences between two folders and merge these intelligently.

I know this is not an uncommon task among many TextMate users, as I have received quite a lot of requests for building such functionality — Changes of course has TextMate integration, so this application will hopefully satisfy a lot of you.

Version 1.0 of Changes is available for $39.95 (free trial) but if you use TEXTMATE0308 as coupon code you can get it for $29.95 (offer lasts throughout March 2008).

Fortunately Tim Harper recently picked up on my initial efforts and has done a great job at not only making this bundle functional but also downright impressive :)

The much improved Git bundle can be found at Gitorious and discussion about it can be directed to its Google Group.

For those too lazy to click the link above, here are the short install instructions:

mkdir -p /Library/Application\ Support/TextMate/Bundles
cd !$
git clone git://gitorious.org/git-tmbundle/mainline.git Git.tmbundle
osascript -e 'tell app "TextMate" to reload bundles'

After having installed it you can press ⌃⌘T in TextMate and enter git to find the “Administration → Update Git Bundle” action. Use this to update the bundle (it will automatically reload after having performed the update).