I have called this principle, by which each slight variation, if useful, is preserved, by the term of Natural Selection.

I've just had a flash of insight that makes me feel like Darwin visiting Galapagos!

What is survival for open source? Getting customers to pay you for support. What kind of customers are willing to pay for support? Customers who can't trivially fix their own problems.

Now we know why nothing is ever documented. Now we know why everything must be thrown out and rewritten anytime it becomes likely that a google search might reveal the desired information. No matter how powerful google becomes, it can't find information that doesn't yet exist anywhere except in the minds of a few fanatical software developers (who tell themselves they are improving things, but who are really driven by the unseen evolutionary pressure to obfuscate).

Evidence? It abounds! Try a simple example in The Case of the Non Missing Disk, or the more complex and still in flux saga in xdbusd, or the truly insane Linux Is Easy!.

And let's not even get into the linux kernel driver model being changed everytime anyone has a whim (and of course all these changes are throughly tested on the vast array of devices that use all these drivers - not). Just look at the glowing success of the recent complete rewrite of the video drivers. The 3 video cards the developers use work OK, not so much success for the other linux users with any of the 47,621,992 different model cards churned out over the years.

So, do I actually believe there is some mystic force acting on developers to encourage obfuscation? Not at all. The actual origins of "the force" can be explained much as Darwin explained "the force" of evolution:

Let's say you are a free software development house. You love software, you want people to use your software, so you not only make it highly functional, but you also include excellent documentation to encourage people to use it, and you test and review it extensively to make sure it is as bug free as possible and does the job promised in the documentation.

If you are that software house, you ain't making any money :-). People download your free software, use it for free, it works, and they don't need any support from you. You can afford to do this if you are independently wealthy and you run the software company as a hobby, otherwise you are soon out of the software business.

On the other hand, let's say you are a different kind of software company. Your managers learn that hireing the sorts of folks who could make this kind of classic comment get them promoted, because their projects inevitably wind up signing up more customers for support contracts. (Don't even begin to think this is a unique comment. You can find similar ones across the open source universe, I merely include one specific example to prove I'm not making this stuff up :-).

Eventually you grow big enough to start sponsoring and hosting all sorts of wonderful new open source projects, all of which are naturally being developed primarily by the kinds of developers who made you a success.

See? No mysticism required, yet obfuscation and bugs is what gets you success as a business and infuses the corporate culture with the attitude that keeps it that way.