Sunday, August 20, 2006

Lisp is Low-Level

Lisp is built out of an amazing idea: like everything else in the machine, code is also data. This is true in any language, but Lisp's syntax makes the code especially easy to manipulate as data.

This feature allows new language constructs to be implemented as program transformations, without touching the core language. Given this uncommon ability, how could any language ever beat Lisp in terms of level of abstraction? Any missing functionality, it would seem, can only be temporarily missing, awaiting a sufficiently dedicated macro writer. Surely, Lisp must be the most high-level language ever designed!

Well, I think not. Lisp is low-level.

Macros may allow you to write programs more abstractly, but writing macros themselves is more painful than needed. A macro is manipulating a raw syntax tree, devoid of meaning, especially since it may contain new macro constructs it has never heard of. How can it be expected to do the right thing?

The only hope, for the poor macro, is to leave the new constructs alone. Thus, some constructs shadow the influence of other constructs, effectively preventing a class of independently developed language extensions from being used together. That's a shame, and that's a direct consequence of Lisp's macro facilities being too low-level tools.

What would a high-level macro system look like? Personally, my bet is on more advanced reflection mechanisms. In an upcoming post, I'll explain how reflection and macros are interrelated, and provide a much needed example to illustrate the kind of things which Lisp cannot do.

7 comments:

山宗原人 said...

where is your "reflection" post???

gelisam said...

it rots on my hard-disk, awaiting completion...

gelisam said...

...and it has now grown into the research subject of my Master's thesis. Patience! Another year or two!

gelisam said...

I take that back: Brigitte thinks I'm being way too ambitious. Oh well. Maybe I'm still going to work secretly on it, once my thesis is complete. Patience! Another couple more years!

Anonymous said...

I call bullshit.

gelisam said...

Actually, I don't even remember where I was going with this reflection thing. But I stand by my claim that it would be beneficial to have a more semantically-relevant representation for code.

Nice argumentation, by the way.

gelisam said...

Oh, I remember the reflection thing now! Let me write it here before I forget again.

Instead of manipulating the AST as a concrete piece of data, we should manipulate the AST as an abstract type. For example, we could have a class Block with a method to obtain the list of statement it surrounds, and there would be classes like While and For that would subclass Block.

This way, we could extend the language by implementing more subclasses. More details to follow in a couple more years.