As organizations get larger they tend to become stupider. We suspect this is because coordination constraints and overhead come to dominate responsiveness. Unfortunately, much computer system design tends to reinforce and freeze in place these constraints. Nearly everything typically called “information technology” (IT) exhibits this problem. The worst problems tend to be limited to large corporations and other bureaucracies, but to a degree the IT mindset tends to percolate out into the design of web services and other broadly disseminated user interfaces. We've written up some (rather abstract) thoughts on this in more detail here.
On the other hand, many ways of coordinating people don’t seem to have this tendency to stupidity. Some of the most interesting examples right now are networks of companies and open source development efforts. So far, this type of coordination hasn’t been tied to an explicit design philosophy (unlike the “IT” software mentioned above, for which organizations have spent hundreds of millions of dollars on “methodology”). Perhaps the closest we have to an explicit statement of ideology is David Weinberger's book Small Pieces Loosely Joined.
In her book Regional Advantage, Annalee Saxenian analyzed the different styles of high tech companies in the Boston area and Silicon Valley. She claimed that much of the persistent advantage gained by Silicon Valley could be attributed to its tendency to use networks of small companies, rather than large, vertically integrated companies, as was more prevalent in the Boston area. Similar claims have been made for light industry in the Birmingham area in England, and textile companies in Northern Italy. Such a network seems to be more robust in difficult economic times, and more capable of quickly adapting to changing technology, market demands, and similar environmental pressures — in other words, more intelligent.
These companies interact through an information-rich environment in which they have repeated contact. The actual explicit contracts between companies only very partially define their relationships. Companies have long-term concerns about maintaining their reputation, building expertise to track market and technology changes, and sustaining good working relationships with other companies that have more or less complementary businesses. As a result, most “knowledge workers” in the company have a significant interest in maintaining a rich set of external relationships with peers in other companies. This is a dramatic contrast with more vertically integrated companies, in which external relationships are more the province of sales and purchasing.
Projects such as Linux, Apache, Mozilla and other large open source software development efforts depend heavily on loosely coupled collections of hundreds or thousands of volunteer, part time contributors. Because the contributors are very geographically dispersed, most of their interaction goes through mailing lists, source repositories, and bug databases. Nonetheless, they seem to be able to make progress on large software projects at a rate comparable to large, well funded, monolithic software development teams, such as those internal to Microsoft, Sun Microsystems, and IBM.
The results of successful open source development efforts seem very comparable, even superior in functionality and quality to the much more expensive efforts of large corporations. Thus it seems reasonable to conclude that the open source projects are at least as “smart” as corporate development projects, and achieve their intelligence much more cost effectively. In contrast with the networks of companies discussed above, much of their coordination is carried out through publicly accessible electronic forums, making investigation of their processes relatively easy.
These examples seem to establish that self-organizing communities are possible, and that such communities may achieve a better or at least equal quality of coordination with significantly less coordination overhead. Furthermore the open source examples show that computer technology of the right sort can be an essential part of coordinating such communities in some circumstances. It seems clear from existing examples that the right kind of computer technology is not “coordination software” such as scheduling, project management, workflow, or other “automated bureaucracy.” Instead, the right direction seems to be automated services that help people organize and carry out social discourse in a flexible and open-ended way.
So far, the most successful mechanisms for such social discourse seem to be mailing lists and bug databases of open source projects, and community sites like Slashdot, where the members contribute items and pointers, the “staff” filters the items for relevance and fit, and then the members comment on them. The work done by open source groups is fairly clear. Community discussion sites also do significant “work” but it is less obvious. The most easily visible work is simply sharing current information of interest to the community. A large, alert community usually generates a comprehensive, timely flow of news and reviews, and often the community comments contain vital information not otherwise available. In addition, the community site can provide analysis (often quite detailed and expert) of current issues, collection of important questions relevant to a given topic for offline response, as well as motivation and guidance (but not typically organization) of short term efforts.