The Knuth Effect

I've used the term so often that I believe it is time to coin it with a blog-post! The Knuth Effect, I reckon, would apply to many computer entusiasts, or generally people aware of the possibilities of computer science, and generally aware of the marginally inefficient world around them.

The Knuth Effect requires a little history, and clearly has to do with Donald Ervin Knuth, the current God of computer science, and his two most known masterpieces, namely the typesetting system TeX, and his volumen-set(bible of computer science) The Art Of Computer Programming.

Upon receiving his Ph. D.(1963) from the California Institute of Technology he was regarded as a specialist in writing compilers, and so he wanted to write a book about it. After coming up with the first draft, he found out that the printed book would be about 2000 pages long, which was clearly far too much. Furthermore Knuth soon realized that the scope of the book wasn't wide enough - he had more to say. So eventually he decided to split the book up into 7 volumes.

The first volume came out in 1969 and was typeset using a hot metal typesetting machine from way back in the 19th century. Knuth always loved the classical style that this typesetting produced, but when the time came around for the second edition, the technology was no longer available. So he decided to spend 8 subsequent years working on a computer-based typesetting system that followed these classical traditions. Thus came TeX into being, a typesetting system, still very widely used by anyone involved in science, especially maths, physics and computer science. Also, today, 47 years later, we have four subvolumes in volume 4, and volume 5 is planned for 2015.

Thus, the Knuth Effect should be clear - it is the process that starts with a simple goal, such as write a book about compilers, but along the way, diverges to a number of adjacent goals, absolutely required if one is to reach the originally simple goal, and be satisfied with the result.

Most people simply drop such goals, and engage too much in walking off the side roads, but the Knuth Effect subsumes an extreme will in completing what it was originally you set out to do, and the will to do it flawlessly.

-------------

I've realized that I, myself, am in the middle of such a Knuth Effect, for the past year or so I've been working a philosophical work, but along the way I've realized several things - firstly that I cannot phrase it in a book, the philosophy itself contradicts the constant structure, and a predefined for a reader to follow. However, a web technology akin to Wikipedia, with a flavor of TeX fundamentals, would surely do the job. However, to write it I will need a far more efficient database management system that what one can get for free today, or at a low price anyway. To write that, I'll need a programming language that doesn't irritate my mind with redundancy and general clumsiness, and even to write that, I cannot use the conventional context-free-grammar definition syntaxes that exist today.

(The last two sentences are poorly explained since it isn't a programming language that I'm after, but rather a programming language framework, which will help me advance the web project that I've got going on, at one of the higher levels of abstraction.)

Eventually, I'm afraid, I'll think I have to write my own operating system, if I am to make it all as efficient as I really wish for it to be.

Well, at least I'm not abstracting from the philosophical ideas! Rather, I'm getting deeper and deeper into details, one can only hope that at some point I'll reach the fundamental building blocks, from which I can really start to make some progress towards my final goal.