Why is simplicity so important for the OSS movement?

Small is beautiful

Without simplicity the value of open source is much less. My main conviction is that open source movement first of all was and is about simplicity, about attempt to avoid overload connected with what Harvard Business School professor Clayton M. Christensen called the technology mudslide -- a perpetual cycle of extending existing architecture in pursuit of the latest techno fashion that Microsoft represents so well. But this is no longer the case, and complexity of the leading OSS products is now pretty close to the complexity of commercial counterparts. For example let's take SMP -- Linux is about desktop and with current CPU speeds is SMP really necessary or beneficial in the desktop environment and justifies corresponding complications in kernel?

Will open source MS Word clone solve the problem connected with MS Word dominance?  Or will it  just repeat the same cycle of growth until it will be unable to run in, say, 16M? BTW early Linux kernels run in less than 2M and it was possible to use Linux on a PC with just 2M of memory (this is a palmtop by today's standards ;-) although 4M was preferable.  This vicious cycle in which "Coping with the relentless onslaught of technology change was akin to trying to climb a mudslide raging down a hill. You have to scramble with everything you've got to stay on top of it. And if you ever once stop to catch your breath, you get buried." is pretty disturbing.

Initially OSS was a movement at least partially connected with seeking "the other way" -- a simpler solution to the closed software complexity and connected with this luck of flexibility and reliability. And  as such it was a constrictive way to avoid information overload. The philosophy that "small is beautiful"  and that smaller, simpler tools with source that are adaptable by owner can be better and more productive that complex and untouchable commercial monsters is now lost. The current mood (the mood of "world domination") is "I want absolute parity, no matter what even if you (developer) need to die for it".

The greatest idea of OSS software seems to be the idea that tool should be simple enough to be understandable and  modifiable by a power user. Moreover such modifiable simple tool often is able to be more flexible and powerful that tools that are much more complex for this part of  user population. Unless a power user is able to learn how to modify the tool to meet his/her needs the difference between open and closed tools exists mainly in price. That' why I think that Perl-based tools are superiors to similar tools written in lower level languages like C-- they are more easily modifiable. And this is a great idea -- a real alternative to Microsoft vision of huge all dancing all singing products that meet any foreseeable requirement (that most users will never have). Actually I think the fact that major scripting languages are open sourced has a very deep meaning indeed -- this help to achieve the core idea of the movement -- simple and powerful tools. BTW from this point of view fetchmail (a small product that can be easily written by one programmer that paradoxically was used in CatB as an example of an OSS project similar to the Linux kernel -- a really complex project) use wrong implementation language. It definitely need to be rewritten in a different language -- a scripting language would be much better implementation language for such a tool.

What I am afraid is that in the current race to match commercial competitors this advantage of open source may be completely lost. In my understanding OSS benefits are not scalable -- like sound barrier there is a complexity barrier after which they became closer to proprietary commercial tools than one may think. Many OSS products now are produced by salaried programmers and naturally are free from the limitations of volunteer development and underling requirement of architectural integrity and simplicity.  They are becoming as hairy and complex as commercial products they compete with. In order to meet (often conflicting) demands of the commercial marketplace they are moving farther and farther along the road called "one Microsoft way".  And along this road they lose both simplicity and conceptual integrity  the principal ingredients of open source if we assume that the users need to be able to read the source.  In a way, Open-Source development may be doomed by its own success. As Davor Cubranic in his paper Open-Source Software Development recently noted:

Ted Lewis in his ``Binary Critic'' column in IEEE Computer [10] voiced a rare criticism aimed at the open-source development process itself (in particular the ``bazaar'' approach) and its ability to cope with its own success. Specifically, Lewis thinks that as open-source projects' popularity grows and they begin multiplying, the pool of talented programmers who devote their free time to such projects will become increasingly scarce. Furthermore, Lewis claims, as those projects mature, they will also grow in features and complexity, eventually overwhelming the resources of their handful of core developers and the capability of their typically ad-hoc organizational structure to cope with those stresses. This sentiment is also echoed by Microsoft: ``The biggest roadblock for open source software projects is dealing with exponential growth and management costs as a project is scaled up in terms of innovation and size'' [19].

In his osOpinion paper  Simplify! Monty Manley wrote:

I was an English major in college, and this probably influences how I approach programming: I view source code as I do certain kinds of technical literature -- not just as *instructions*, but as a thing in and of itself. Well-written source-code can be beautiful if you know how to read it; it can be a marvel of clarity or of devilish subtlety. Sometimes the code scans wonderfully, reading almost like a digital-age poem.

Many years ago I read a very battered tenth-generation photocopy of "Lions Commentary on Unix" (also called the Lion Book), and I remember thinking that I had passed some kind of event horizon. Looking at the original Unix sources (with John Lions' wonderful commentary), I came to realize that good computer programs are not those that simply *work*; good computer programs are also elegant, efficient, and (when they need to be) devious and clever. Ken Thompson, Brian Kernighan, and Dennis Ritchie had discovered something profound: small things could be both beautiful and amazingly powerful.

Begin with the C language itself: it consists of perhaps 50 keywords, and has a fairly simple syntax. It is not a complicated language to learn, although it can be used in amazingly complicated ways. But rather than try to invent a language for beginners (like BASIC), or a teaching language (as Niklaus Worth did with Pascal), they invented a language for *programmers*. That is, C was written to get things done on digital computers. It was portable. In a time when most pundits thought operating systems had to be coded in machine-language, Thompson, Kernighan, and Ritchie built an operating system almost entirely out of platform-neutral C code. About 95% of the original Unix kernel was -- and is -- straight C. The original Unix kernel was composed of about 10,000 lines of code. (John Lyons wryly noted that later versions of Unix fixed this problem.)

... ... ...

But why is this a good thing? Why is it desirable?

Not everyone would agree that it *is* desirable. A certain company in Redmond, for example, subscribes to the belief that more and bigger is better. It is estimated that Windows 2000 contains between thirty and fifty million lines of code. By contrast, a basic Linux installation (kernel, shell, and essential libraries and tools) takes about five million lines of code, more or less. Why is one approach "better" than the other?

The answer depends on how you feel about software stability and robustness. For better or worse, most software users have decided to value features over stability -- we would rather have something colorful and featureful that crashes occasionally rather than something more quotidian that crashes not at all. Every line of code added to a piece of software increases the chances that something will go wrong: not just programming bugs like a misplaced minus sign or a rounding error, but more obscure problems. Everyone who has used Windows is familiar with "DLL hell" -- incompatible libraries that cause software not to work correctly. Each library by itself works fine, but they cannot reliably work together. Multiply this by several hundred libraries and you begin to see the scope of the problem. (And lest we forget, Linux has its own library dependency problems; we all remember the libc5/glibc conversion nightmare of a couple of years ago.)

It may be that software, like the biosphere itself, has a natural tendency to complexity. Certainly history lends credibility to this point of view -- almost no program ever gets *smaller* as time goes by. Features are added, capabilities are enhanced, and the code grows and grows. Programmers have neither the time nor the tools to exhaustively test these programs completely, and so they always ship with unfound bugs...

...The hidden evil here is the "ease of use" fallacy. Programmers assume that end-users need lots of hand-holding and visual cues in order to be productive with their software, and oftentimes this is true. But the *reason* for this is that software is too complex; so it is a circular problem. For every feature added to a piece of software, another wizard, help file, or speed-key combination must be added to access that feature. This feature must interact with all *other* features, sometimes in unexpected and contrary ways. These so-called "easy to use" programs require five-pound tomes to explain their workings. Users can sometimes spend years learning all the features of a given program, and become terrified when faced with the prospect of having to learn something else. Users will put up with a great deal -- crashes, anomalous behavior, bad implementations -- rather than switch due to the learning curve involved. And yet, if asked, they will usually wish for a more stable system.