09 June 2005

by Tony

Allan, how can we help? I’m more than happy to pitch in, as TextMate is my most-used program at this point. I’ve been ferreting away at compiling syntax files for the script files for our (proprietary) CMS at work - code completion would be fantastic! (as the Doctor would say).

10 June 2005

by Jamie Hoover

If TextMate’s code sensing would support linked XML schemas, I could kiss my crappy Java IDEs goodbye. Please keep up the amazing progress that you’ve been making!

11 June 2005

by Mathias Meyer

Gotta back Jamie on this one! Using Eclipse is not that productive on my iBook, so having that kind of stuff in TextMate would be awesome!
The Xcode stuff mentioned is great stuff, too.

Cheers, Mathias

16 June 2005

by Bud Gibson

What I am really interested in is the sftp support. I just want a light weight editor that will reach out and allow me to edit on servers. Is there a time frame for sftp support?

16 June 2005

by TobiasLuetke

Bud Gibson: I really don’t understand this request. Whats wrong with using Cyberduck or Transmit which lets you edit files over sftp with editors like textmate. I find this functionality is much better left to specialized apps.

You are not developing on a remote machine are you? In that case you need to fix your setup and pull your tool chain to localhost for much more convenience.

Most requests for remote editing are just symptoms of bad setups.

22 June 2005

by Noah

As a newbie Ruby developer, I’m very interested in the prospect of code-sense for Ruby. There’s already fantastic code-completion in irb, but that’s because it’s a full Ruby interpreter, so it knows what methods a given object can respond to. What might be the best way to try to get Ruby code-sense into TextMate?

29 June 2005

by Cody

I too would love code sense feature! I can’t wait until this is included, awesome news.

03 July 2005

by joachimm

Since I do not know very much about parsing, I was wondering what is wrong with the C++ parser on this page:
http://www.antlr.org/grammar/list

Also I was wondering how customizable the popup will be. I can think of a few enhancements I would like to see (Think Quicksilver like).

04 July 2005

by Allan Odgaard

joachimm: there’s nothing wrong with the C++ parser at the page you link to (AFAIK), but it’s not really suited for code sense either (in current form). I need to cache information per file, the parser works on the result of the preprocessor (which expands includes etc.), although I might be able to do caching like gcc’s pre-compiled headers. It expects well-formed code (I need to parse the code while the user is actually typing it), it gives one big abstract syntax tree (AFAIK) where I need the type of the variable to the left of the caret etc.

As for customization of the popup, there’ll be none, other than having the plugin populate it, unless I’m convinced of the advantage. I’m not sure the enhancements you have in mind are really suitable for this though – the popup appears automatically and is unobtrusive, when you say Quicksilver I’m thinking of something you actively call upon and perform the “search”.

Noah: yes, IRB uses run-time introspection for the completions, which isn’t possible to do at “write time”. Given all source files though, it would often be able to find at least a superset of the methods implemented for a given object.

07 July 2005

by Dan Ridley

The main “Quicksilver-like” enhancement that springs to mind is QuickSilver’s ability to give higher weight to often-used entries, moving them higher in the list.

I don’t think re-sorting the list makes sense here, but maybe positioning the list such that the most recently or frequently used completion is the choice nearest the pointer, and gets special treatment for keyboard selection somehow or another. (Down arrow selects completions from the top of the list and right arrow selects the most frequent completion first, or something).

Of course, joachimm may have been thinking of something completely different.

10 July 2005

by joachimm

Dan Ridley. Yes that is one of the enhancements I would like to see, I like the idea with right/left arrow. I was just running Eclipse writing “System.out.println” and to get to println I had to wade through variations of print() (Without the ln suffix). Perhaps there is some key that would complete up to print just like hitting tab in the terminal, I did not find that key and in the end it would have been faster for me to just type the whole word and be done with it.

This could have been avoided using the scheme you describe, perhaps it would also have been better if all the “print” functions had been folded together, then I would not have been forced to scroll through two screens.

Using my method the popup from the screenshot above would have looked like.

dictionaryWithContentsOf*
dictionaryWithDictionary
dictionaryWithObject*

Perhaps some optional heuristic could break out

dictionaryWithObjects*

Ofcourse when all the methods fit in the popup the win is small or a loss.

Anyway Allan the minimum is that you give us terminal like completion for the words in the popup ;) . I would much rather do a tabcompletion and get “dictionaryWith” then hit the first letter in the rest of the word I wanted, than hitting down arrow perhaps up to five times (or 25 in the java println case). Perhaps this is already implemented in the PHP bundle. Just don’t implement it like completion in Safari, I hate having to outtype the computer ;) .

20 October 2005

by Jan-Henrik

Allan: “(I need to parse the code while the user is actually typing it), it gives one big abstract syntax tree (AFAIK) where I need the type of the variable to the left of the caret etc.”

Don’t parse the code on the fly. Here’s what you should do,

1) Parse all files in a project (auto-completion is only appropriate for a project). Let this parsing (lets call it indexing) be activated by the user. I.e. he must push a button to index the project.

2) Build a symbol-table and you may not even need an AST. Lower the ambitions in the first version. Besides an AST is mostly needed if you must analyze (type-check etc) and generate code. While for auto-completion you are mostly interested in function names and signatures. This you can lift right out in a semantic rule and put in the symbol-table.

3) Once you have a symbol-table in-memory it’s easy to perform auto-completion. The user write some text and press ESC. Take that text and look up the symbol-table and show a pop-up list with all symbols that match that prefix. Color the items in the list after type.

4) That’s it! The hard part is to get the grammars for different language and you are on the wrong path with ANTLR, you moron! Use bison/flex, most of the grammars recognized by TM is written in bison anyway. Simply look in the CVS for Ruby, PHP etc and you can find the bison grammars right there!

Ps, another thing, you should really open-source TM, you cant do all this work yourself and I see that your backlog is growing proportional.

20 October 2005

by Allan Odgaard

Not parsing code on-the-fly gives a pretty bad code completion, something you can already do yourself by overloading the completion function, granted you won’t get the menu, but you will get the candidates you propose.

A public bundle for PHP and a non-public for Ruby/Rails which makes use of this scheme already exist.

As for an AST, you misread my comment. I discarded the parser linked-to saying it produced an AST.

Though for only grabbing function signatures, that completion would be pretty worthless to me, take e.g. this code:

class foo {
    foo* var;
    void bar ();
    void fud ();
}

void foo::bar () {
    var->f«caret»
}

Here the completion should be fud, but for that it needs to know that it’s inside a method of the foo class, that this class has a member variable named var, the type of that member variable, and what member data starts with f in an instance of that type.

And this is actually a rather simple example. It sounds like you propose showing all member data that exist in the entire project.

When I’m favoring ANTLR over flex/bison, then it has to do with the code produced, which is encapsulated in a namespace, object oriented, re-entrant/thread safe, works with arbitrary input (memory buffer, i/o stream), etc. which is unfortunately not the case for code produced by flex/bison — I’m also highly skeptical about the gains by using bison to get “free” grammars, both because ANTLR has likewise a huge library of existing grammars (even for latest Java, Obj-C etc.) and because there’s a long way from a bison grammar to what I need.

As for open sourcing TM, I think it’s far to early to do that (and I am currently dependent on the income it gives me). But despite it being closed source, it shouldn’t hinder a third party from e.g. writing the stuff you outline above and send me (e.g. as a library with a function to call when a file is saved, and another to get completions for a line) — the question is, will people start contributing with this, just because I open source TM? I’m skeptical (after all, there are already several other F/OSS text editors for the Mac platform), but I’m willing to do the experiment, when I feel TextMate is in a state where it is better suited for having multiple individuals working on the source.

21 October 2005

by Anonymous

Allan: “Though for only grabbing function signatures, that completion would be pretty worthless to me, take e.g. this code: *”

When parsing a OO language you can scope the methods of the enclosing class and can save this information in the symbol-table. This information can then be looked up when ‘var’ is found (parsed) to be an object of class X.

Antler has a steep learning curve, the grammar is awkward and antler is targeted primarily at writing parsers in Java, most of the good stuff like tree parsers are Java AFAIK. Antler is certainly a well written parser generator for Java, not sure how good it is at generating C++ parsers.

Personally I prefer LALR parsers such as bison before LL(k) like antler, IMO the grammar is easier to write and read. As I mentioned before most programming languages out there is also written with LALR parsers. Ruby for instance.

You can create reentrant parsers with bison. The latest release of flex should be thread-safe, otherwise you can use scanners such as re2c together with bison to get a perfect reentrant parser.

Allan: “ the question is, will people start contributing with this, just because I open source TM?”

Well, others have had success with that model, but it’s a leap of faith.

07 October 2006

by Tom

Have you looked at Boost.Spirit for parsing? If you like STL and generic programming, this might hit the spot.