On my way down to Illinois today, I listened to a podcast giving an overview of American history before 1870. In it, Dr. Reilly talks about the industrial revolution. I was enthralled as I contemplated the changes that the revolution brought about and the similarities to the changes taking place in web development.
In the 1800’s manufacturing moved from a guild system toward the use of unskilled labor and interchangeable parts. Under the guild system, an apprentice would live and work for a master tradesman who provided food, shelter, and training. Goods were produced from start to finish by a single tradesman. Volume of production was limited and a great deal of skill was required of the tradesmen, who needed to have a holistic understanding of the products they produced.
The guild system remained primarily because it was most economical to have a local tradesman produce goods. It was too costly to import goods from across a continent. Once canals and railroads provided cheap freight services, it became economical to produce a high volume of goods and ship them to distant customers. Thus, the need for mass production arose. Instead of hiring ten times more tradesman to produce ten times more products, it made more sense to hire unskilled workers to perform specialized tasks, thus “pipelining” the production process. Tradesman began to take more managerial roles, training the unskilled laborers in their specific jobs.
At the same time, the idea of using interchangeable parts arose. With interchangeable parts, it was much easier to repair and maintain products. Before the revolution, a product was custom made and its design was known intimately by the tradesman who made it. If it needed to be fixed, the best person to fix it was the one who made it. With interchangeable parts, a product’s parts could be easily replaced if broken and could be repaired or maintained by anyone familiar with the industry. Furthermore, a factory producing widgets can make the widget almost completely out of interchangeable parts produced by other companies. No specialized knowledge of how each part is made is required. The factory can simply make new widgets out of preexisting components.
The industrial revolution was a drastic change for the tradesman who were accustomed to the old guild system. The tradesman who prided themselves on quality (and high wages for specialized work) strongly opposed the revolution. They considered the mass-produced goods to be inferior to their custom, hand-made products. They formed unions to try and preserve their wages and job security. But ultimately, mass production won out and in many industries completely replaced the guild system.
So how does this relate to web development? In many ways there has been an “industrialization” of how web applications are developed. In many cases today, web applications are (or have been) written by one or a few programmers who are intimately familiar with their custom implementation. If a programmer leaves, the web development company is in a bit of bind because knowledge of all of the intimate details of the system is lost and a great deal of time is required for new programmers to wrap their heads around the system in order to maintain or improve it. Over the past seven or so years, there has been a trend toward specialization of work and an increase in the use of libraries, code generators, and frameworks, similar to the idea of interchangeable parts.
Prior to our current days of web 2.0, the only division of labor that normally existed was between graphic design and programming. As web applications have become more complex, many companies have moved from hiring jack-of-all-trades web developers to hiring specialized Flash developers, UI developers, database developers, and server-side develoeprs. Companies seem to care more about whether a programmer is intimately familiar with J2EE or DOM scripting rather than a holistic understanding of web applications and computer science. To run with the industrial revolution analogy, it’s like an employer who is more interested in how well a worker can operate a welder, rather than being able to build an entire car by hand.
Libraries, frameworks, code generators, web services, and mashups can be considered the “interchangeable parts” of the web. As long as an unskilled programmer knows a specific framework, say CakePHP, he can jump into any CakePHP project and easily maintain it. Not as much specialized knowledge about the application is needed compared to a custom-written, classic PHP site.
While there are similarities between the effects that the industrial revolution had on industries like blacksmithing, shoe making, and automobile making, and the effects of industrialization on web development, there are some differences. While the industrial revolution nearly completely replaced tradesmen with unskilled workers (and even robotics) in industries where simple products are produced, it has not replaced tradesman in every industry. For example, in most cases skilled contractors are still needed to build houses and large buildings.
As I see it, there are currently three types of web applications being produced today. The first type is the static, informational web site. There’s not much for a programmer to do here. Most of the work is for the designer. At one time, someone who knew HTML was needed, but apps like Dreamweaver have replaced that need. The second type is the CMS-style site that might have news, events, subscriptions, or blogs. These sites used to require dedicated developers, but now packages like Drupal, PHPNuke, and wikis have replaced this need. Some companies still develop these type of sites from scratch, but as customers become more educated and competitors outsell the custom shops with prepackaged CMS systems, the custom-designed CMS site is a dying breed. The third type of web app either has complex business rules or it stands on the bleeding edge of technology. These type of apps will need developers, but not necessarily the “jack of all trades” developer.
On the one hand, it is comforting to know that, at least for the time being, there is a need for skilled developers. The web development guild system hasn’t yet been eliminated by unskilled labor. However, it is a bit disturbing to see companies move toward hiring specialized developers who are “intimately familiar” with specific languages or platforms, rather than developers who have a holistic understanding of web applications. In addition, many companies are hiring programmers with little skills to churn out sloppily-coded applications. What is a “craftsman developer” to do?