Sunday, August 20, 2006

Lisp is Low-Level

Lisp is built out of an amazing idea: like everything else in the machine, code is also data. This is true in any language, but Lisp's syntax makes the code especially easy to manipulate as data.

This feature allows new language constructs to be implemented as program transformations, without touching the core language. Given this uncommon ability, how could any language ever beat Lisp in terms of level of abstraction? Any missing functionality, it would seem, can only be temporarily missing, awaiting a sufficiently dedicated macro writer. Surely, Lisp must be the most high-level language ever designed!

Well, I think not. Lisp is low-level.

Macros may allow you to write programs more abstractly, but writing macros themselves is more painful than needed. A macro is manipulating a raw syntax tree, devoid of meaning, especially since it may contain new macro constructs it has never heard of. How can it be expected to do the right thing?

The only hope, for the poor macro, is to leave the new constructs alone. Thus, some constructs shadow the influence of other constructs, effectively preventing a class of independently developed language extensions from being used together. That's a shame, and that's a direct consequence of Lisp's macro facilities being too low-level tools.

What would a high-level macro system look like? Personally, my bet is on more advanced reflection mechanisms. In an upcoming post, I'll explain how reflection and macros are interrelated, and provide a much needed example to illustrate the kind of things which Lisp cannot do.