Initially, online content and social activities were organized using pre-existing models for other forms of content and social activity. Gradually we’re learning ways to organize things online that fit the medium better: open source development groups, search engines, blogs, etc. These new forms of online organization are influenced by three characteristics of online media that are very different from physical media:
Limitations could be imposed to make online media more like physical media, such as copy prevention, password access, etc., but this is always extra work and only partially successful. The more interesting option is to explore the possibilities and problems raised by online media.
Our experience so far with the fluid, non-rival, and open characteristics of online activities shows that they dramatically heighten at least two phenomena:
• Content is being produced and released much more incrementally, and the contributions are made by smaller groups (often individuals). This difference spans a wide range of content types: software, news, academic discourse, encyclopedias, etc.
• Much new content is a modification or commentary on existing content — in software, a diff or patch; in text often a comment on quoted or linked material, or appended in a discussion area.
Large groups have been able to create and maintain large, complex products through such incremental accretion, which Linux, Apache, and Mozilla exemplify. This success is now being extended to major online knowledge repositories such as Wikipedia. These groups have relatively open memberships but follow a highly structured process of validating and incorporating changes. At the same time, loosely coupled groups are providing information services such as product support, investigative journalism, product reviews, and so forth. These services have shown the ability to provide rapid and accurate information and to efficiently discover and correct errors. These groups are often entirely open, but they typically have mechanisms for internal review and rating that help to guard against malicious or mistaken misinformation. Even so, as they grow they experience many forms of “discourse pollution” due to their openness.
We do not yet seem to be anywhere near the inherent limits of the potential of online media. The shift to more incremental contributions seems to have a fundamental role in this potential. To get a better sense of how this may play out, let us try to briefly imagine how online, incremental processes could transform some familiar institutions:
News. Google News is already demonstrating the possibility of creating a “virtual newspaper” by aggregating content from many sources. Blogs and discussion sites like SlashDot are demonstrating the ability of individuals “on the ground” to provide key information, and other volunteers to provide background research and insightful commentary. Ultimately individual contributors plus aggregation might provide considerably more comprehensive and deeper news coverage than our existing newspapers, magazines and broadcast networks.
Textbooks. Incremental online processes have already been used to write documentation and a few computer “how to” books. These processes have been relatively tightly controlled, in ways similar to open source projects. Textbooks in nearly any field could be incrementally produced and updated using similar processes. The burden on authors would be greatly reduced and the timeliness and completeness of the material would be increased. Furthermore, an online textbook would naturally be integrated with other online material including primary sources.
Urban design. Managing growth and change in a city affects most citizens in various ways, but citizen involvement is typically limited to hearings organized around major decisions, or membership in interest groups that may or may not influence decisions. Even most interested citizens have relatively limited information about what decisions are being made and what arguments have been made for and against different options. An incremental online planning process could allow much deeper engagement of citizens on issues where they have expertise or insight to contribute, and where their interests are affected.
Entertainment. Some computer games allow users to create mods — essentially new “worlds” with the same “engine” — and some mods have been created by online incremental processes. In a few cases, parodies or edited version of feature films have been created by fans. A few “novels” have been created using round-robin collaboration. However, as yet we don’t know whether an open authoring process can rise to the same aesthetic level as an individual or a small tightly knit group. On the other hand, open authoring does allow many more entertainment consumers to explore being (at least casual) producers as well, and may give rise to entirely new art forms, such as collaboratively maintained online worlds.
Health care regulation. Right now the regulation of health care is overwhelmingly bureaucratic. Some parts of the bureaucracy, such as the Centers for Disease Control, seem to work quite well; others, such as the FDA, are less successful but still do an adequate job. But federal and state regulation of the health care delivery system seems to be very poor, leading to an opaque market that serves consumers poorly at high cost. Almost every aspect of health care could benefit from more open, incremental online participation, such as reporting to the CDC and FDA, tracking the reputations of providers, sharing information about diseases and drugs, etc.
Anyone who is familiar with current debates about online activities will immediately see a number of problems with these rosy scenarios.
• Current content aggregation mechanisms often require a great deal of committed human judgment and in such cases do not scale well. In effect, we fall back on bureaucracy, just a smaller and more fluid one.
• Online processes are typically open to garbage and quickly become polluted. They wouldn’t produce reliable information without some sort of strong filtering mechanism.
• Often groups that “shout loudest” online dominate the discussion, drowning out and eventually driving away more reasonable voices and eventually destroying the possibility of consensus (except possibly on their terms).
• In many online social activities there is an inherent tension between effectiveness of the process and respect for the privacy of participants.
Let us take these in reverse order, last first. The problem of tension between effectiveness and privacy will ultimately require architectural solutions, but we do not yet have enough experience to see the form that these will take. In the mean time, existing authentication mechanisms are fully adequate for case by case approaches that will eventually give us the deeper understanding needed to design an architectural response.
The problems of pollution and interests group domination are very real and urgent, have killed many online communities, vitiated many promising efforts, and severely limit the credibility of online social processes. Adequate ad hoc solutions have been developed by discussion communities like SlashDot, which have a completely open membership and which also have had to surmount major problems with pollution and interest group domination as they have grown. However, these solutions have severe intrinsic limitations, and only address some aspects of these problems.
We believe these problems are ripe for fundamental architectural solutions and that such solutions would have a big multiplier effect, substantially increasing the value of online social activity.
The limitations of content aggregation have not yet been aggressively addressed, and may be susceptible to dramatic improvement. In some areas, such as source code, we have basic tools in place for managing incremental changes but even these tools, which are essential to all large and medium sized development projects, are still recognized as quite weak. Looking more broadly we can see that major new forms of incremental content production and delivery are having significant social impact, with the rapid growth of blogging and RSS. However, the problem of aggregating such incremental content into easily accessible form has barely been addressed, much less solved.
We believe that an effort to provide adequate infrastructure for incrementally contributing, managing and viewing content in an open-ended range of formats would be very fruitful, and would both benefit from and contribute to solutions to the problems of pollution and interest group domination. Such an effort involves architectural questions at many levels, from content storage and indexing up to user interfaces.
The problems discussed above lend themselves to architectural solutions. We believe that some thought should also be invested in understanding the organization of online social processes themselves. As Yochai Benkler has pointed out, we have historically used two models for analyzing production activities: markets and bureaucracies. However, there are important production activities which do not fit either of these models, for example the production of scientific knowledge and the production of works of art. Benkler argues effectively that much online content, including such obviously important cases as open source products, is produced by processes which cannot be explained well by either markets nor bureaucracies, and which more closely resemble the processes that underlie the self-organizing practices of scientific and artistic communities. Benkler calls this mode of production commons-based peer-production, or peer production for short.
We believe that deep understanding of the potential and limitations of peer production requires a new analytical framework, distinct from the neo-classical synthesis used to analyze market production, and the sociological and legal frameworks for analyzing bureaucracy. Peer production has generated immense social value through science and art, and offers immense potential to generate further social value through online content production. Such a powerful self-organizing process seems very likely to depend upon principles as fundamental as the self-organization of markets, and these principles must be clearly understood and clearly differentiated from market and bureaucratic mechanisms if we are to support peer production effectively. Clearly, in creating such a framework we should take as much advantage as possible of existing tools such as economic theory and game theory, but we must also be careful not to import inapplicable assumptions or biases when adapting these tools to our subject matter. We believe that adapting existing analytical tools may permit relatively rapid explanation of much of the power of peer production. It is in this direction that we intend to aggressively pursue efforts.
We also need to recognize that many existing interests will be threatened by the potential of online peer production, and will tend to resist changes of the sort described. Such problems arise whenever significant social changes are underway. Interests opposed to the changes can usually delay and sometimes coopt them, but in the end the underlying trends usually overwhelm them. In many areas, such as news, textbooks and entertainment, existing interests have little control of future distribution channels (as opposed to current distribution channels) and thus have relatively little power to prevent the sorts of development sketched above. In more highly regulated areas, such as urban design and health care, existing interests may have a much easier time stalling the advance of open online processes. We see our role as enhancing the power of the underlying trends rather than addressing problems in specific areas posed by existing interests. The proponents of online peer production in any given domain will, of course, need to deal with such resistance, and in every case will have easier jobs if the underlying strengths of online peer production are greater.