It was 18 years this month since when the coder world first got to read the Cathedral & The Bazaar.This is the seminal essay by Eric S. Raymond that sets out 19 lessons in creating quality open source software. It was first presented at the Linux Kongress in May of 1997. To celebrate this we have officially launched our brand new ‘Open Source Hero’ quiz. Designed using the tech-knowledge of the Verify team and community, this is the first in a series of tech-themed quizzes we hope to release over the coming months. Take ‘The Open Source Hero Quiz‘ & don’t forget to share and challenge your friends!
Achieving economic efficiency has always depended on the availability of relevant knowledge: for any commercial operation the requisite information is fragmented and distributed randomly throughout the market.
Friedrich A. Hayek, an economist who was important in the post-war neo-liberal movement, contributed to the solution of this problem in his 1945 essay “The Use of Knowledge in Society“.
He was making the case against central economic planning that was in vogue after 1945; the essay looked past centralism and asked how can people with expert knowledge be motivated to contribute to an enterprise and help make it commercially viable.
The problem is precisely how to extend the span of our utilization of resources beyond the span of the control of any one mind; and therefore, how to dispense with the need of conscious control, and how to provide inducements which will make the individuals do the desirable things without anyone having to tell them what to.
Friedrich A. Hayek, The Use of Knowledge in Society, American Economic Review.
Eric S. Raymond, an American coder activist and Libertarian, believed he came across an answer to Hayek’s question when he wrote “The Cathedral and the Bazaar“, an essay he published in in 1997 (he published an extended version of the essay as a book with the same name in 1999). The anniversary of the essay’s first public presentation is coming up on 27th of this month.
In an interview with Robert Rossin EconTalk in 2009, he acknowledged the influence that Von Hayek had on his essay.
The Cathedral and the Bazaar used Linus Torvalds’ Linux project as a case study. This was an early example of the open source model of software development. In the essay Raymomd compares the open source approach (the bazaar) with the proprietorial development model (the cathedral) where a company hires a team of engineers to develop a platform, but all the code is the property of their employer.
The bazaar is crowded, ostensibly chaotic but for some reason it always attracts huge numbers of people; the cathedral is static and was built by a small team of craftsmen.
When he was developing Linux, Torvalds used the internet to engage coders from all over the world to work on the project. And crucially, no-one was paid for their labour and no-one owned the code, arguably the twin essences of the open source model of development.
One of the most powerful motives for the software engineers who get involved in projects like this is the ego boost they got when they spot or resolve a bug that everyone else has missed.
A related motivator is the opportunity to build a reputation within in the industry; this is an established way for inexperienced or junior coders to prove themselves and break into their first engineering job or get a promotion.
Raymond’s “Bazaar” model, and counter to expectations, it has an intrinsic and efficient self-organising feature.
Open source projects can effectively call on an infinite amount of resources like talent, experience, man-hours and eyes scanning the code for bugs, compared to the finite number of salaried engineers and company resources invested by a firm in a private project.
This is the kernel of the open source model’s advantage: Torvalds’ global co-developer base meant he had more resources than a team working for a private company. Raymond argues this allowed the Linux people to execute two critical tasks a lot quicker: 1.Finding bugs (Torvalds thinks this is the tougher of the two) and 2.fixing them. However, there are limits to the efficacy of debugging in the open source model: the Heartbleed bug stayed in open source code for a number of years before it was found by debuggers.
These problems could hold up a proprietorial software project for months but using Torvalds’ open source approach they were solved in hours or even less.
As a result, he could do something hitherto unheard of: daily releases of new high quality versions of the code. Not only were the bugs caught and fixed quickly the additional advantage to open source was that they weren’t replicated in successive code iterations.
Another efficiency in the open source model is the amount of re-usable code will be bigger than the competing model.
The Bazaar model is anarchic but it does have what Raymond calls an architect or an architectural team. Among other things they decide on the code that’s finally committed. In the Linux project it’s Torvalds and his lieutenants who make that decision.
The open source approach to software development has created a lot of successful projects. This is demonstrable in platforms like the Apache web server that’s used to support many high availability web services and also Openstack, which supports so much of the world’s cloud infrastructure.
But Raymond’s essay in general and more particularly the 19 lessons he learned when writing the essay isn’t without controversy in the codersphere to this very day.
Despite this debate the model for software development outlined in the essay has thrived in the open market that Hayek envisaged back in 1945.