5.1.2 Optimization

The main barrier to efficient Lisp programs is not that there is no efficient way to code the program in Lisp, but that it is difficult to arrive at that efficient coding. Common Lisp is a highly complex language, and usually has many semantically equivalent ``reasonable'' ways to code a given problem. It is desirable to make all of these equivalent solutions have comparable efficiency so that programmers don't have to waste time discovering the most efficient solution.

Source level optimization increases the number of efficient ways to solve a problem. This effect is much larger than the increase in the efficiency of the ``best'' solution. Source level optimization transforms the original program into a more efficient (but equivalent) program. Although the optimizer isn't doing anything the programmer couldn't have done, this high-level optimization is important because:

• The programmer can code simply and directly, rather than obfuscating code to please the compiler.
• When presented with a choice of similar coding alternatives, the programmer can chose whichever happens to be most convenient, instead of worrying about which is most efficient.

Source level optimization eliminates the need for macros to optimize their expansion, and also increases the effectiveness of inline expansion. See sections 5.4 and 5.8.

Efficient support for a safer programming style is the biggest advantage of source level optimization. Existing tuned programs typically won't benefit much from source optimization, since their source has already been optimized by hand. However, even tuned programs tend to run faster under Python because:

• Low level optimization and register allocation provides modest speedups in any program.
• Block compilation and inline expansion can reduce function call overhead, but may require some program restructuring. See sections 5.8, 5.6 and 5.7.
• Efficiency notes will point out important type declarations that are often missed even in highly tuned programs. See section 5.13.
• Existing programs can be compiled safely without prohibitive speed penalty, although they would be faster and safer with added declarations. See section 5.3.6.
• The context declaration mechanism allows both space and runtime of large systems to be reduced without sacrificing robustness by semi-automatically varying compilation policy without addition any optimize declarations to the source. See section 5.7.5.
• Byte compilation can be used to dramatically reduce the size of code that is not speed-critical. See section 5.9