The Internet as an Agent of Creative Collision

Summary

The average person in the modem world and much of the Third World receives most of his or her information about society from a small number of communications channels: primarily television, with newspapers and magazines a distant second. These media are funded almost entirely by advertising revenue; their true customers are not their viewers and readers, but large advertisers such as General Motors and Procter & Gamble. The enormous value of information distribution -- television facilities and physical production and delivery of print media are among the most expensive global assets -- has concentrated tremendous socioeconomic power in the hands of a small number of media owners and their often-uneasy allies in business, government and religion.

In order to sustain and enrich the power that results from domination of popular communications channels, today's media rely on an escalating level of sensory stimulation containing illusory promises of health, wealth and happiness; entertainment and propaganda disguised as news; and threats of pain and suffering if viewers and readers do not accept the ideas, products and services they present. (See The Internet and the Anti-Net.) My strong criticism of popular news media is not only as an outside observer, I was a reporter and writer in radio, television, magazines and newspapers for many years, covering news and events from the Reagan shooting to corruption in Silicon Valley's transportation politics.

Although many arguments may be made against the concentration of socioeconomic power in the hands of a few, especially when their primary goal is to sell, this paper focuses on the fact that this results in a lack of diverse viewpoints. Advertising-based media's habit of reporting "both" sides of every issue, no matter how many legitimate "sides" there might be, should not be confused with rich communications that offer a true diversity of points of view. The media's habitual polarization does more to inflame than to inform, with unfortunate side effects such as an increasing public disrespect for experts. This dismal turn of events is unsurprising when the popular media will trot out "experts" who will take just about any position to get on camera.

Under these conditions, technologies that quickly expand the availability of communications channels, such as the personal computer and the Internet, are likely to be eagerly adopted, at first by the literati, who have the power to transform technologies into drivers of profound change. The closest historical precedent for the current situation is the invention of printing and paper technologies in early modem Europe, which broke the Church's then-corrupted domination of communication channels to the average person, despite a relatively low literacy rate. (The literacy rate did not significantly increase for at century or so when the Industrial Revolution created the demand for an educational system to produce factory workers.)

One of the immediate effects of inexpensive access to alternative points of view in early modern Europe was a widespread debate of Martin Luther's criticism of church practices. The church had many rules and policies that furthered her self-professed role as the only path to truth and salvation, which had become, much as in today's advertising, for sale at low cost. Ironically, the church was the primary financier of the printers who eagerly reproduced and distributed Luther's critiques.

The rise of the publishing industry brought together as co-workers people with dramatically different world-views: technologically minded printers, academics from the church and publishers with a mercantile approach. Some of the creativity of the period appears to be a direct result of these groups' efforts to see the world through one another's eye in order to take advantage of the new technologies, The breakdown of the social strata of feudal times, which was due to factors in addition to publishing, led to increased sharing of viewpoints, which appears to be a strong force for invention and creativity. Less directly, the press and paper brought forth new ideas by easing access to multiple points of view in books themselves. No longer was it necessary for one to become a mendicant scholar to see the world through the eyes of one's predecessors; the local bookseller put "information at your finger-tips," to quote the more recent technologist-merchandiser, Microsoft Chairman Bill Gates.

Although today's Internet is, as it often appears, a disorganized mess, the best hope for bringing order to the chaos is not, as many technologists imagine, through more intelligent software, but, as many librarians and scholars believe, by distributing the problem among many people. This is the source of the title of this paper. The term "wetware," referring to the human brain and mind, was coined by science fiction writer Rudy Rucker as a parallel to computer "hardware" and "software."

One of the first projects likely to be tackled by Internet users on a massively parallel basis is the evaluation of newly available documents, an approach dubbed "social filtering" and "collaborative filtering" by its proponents. A collaborative filtering system would allow software agents to take into account the opinions of large numbers of people -- friends, co-workers, professional opinion leaders and members of other communities -- to determine the relevancy of new documents to the agents' owners. By observing correlations among peoples' opinions, such agents are expected to offer predictive data without having to solve the much more difficult problems associated with understanding causality or why a particular new document, such as a news report, might be interesting to the agent's owner.

In combination with today's search and relevancy-ranking software, social filtering holds the promise of helping people learn about relevant ideas from fields other than their own, reproducing the serendipity of discoveries that come from consciously putting oneself in the right place at the right time.

Introduction

This paper represents my first attempt to bring together a set of ideas about the state of mass communications in our time and the role of technologies, particularly technologies that allow us to exchange ideas, as agents of change for humanity. I hope and plan to expand it into a book over the next few months. This examination was originally inspired by Elizabeth Eisenstein's fascinating book, "The Printing Press as an Agent of Change," (See also the Canto edition, "The Printing Revolution in Early Modem Europe.") in which she examines the role of Gutenberg's invention, coincidentally with the invention of an inexpensive paper supply, in the changes that swept through early modem Europe. (See James Bradford De Long's review.)

Today's corresponding technologies are the personal computer and the Internet. Like the printing press, personal computers allow rapid creation of many copies of documents. Like paper, the Internet supports speedy distribution of these documents. Each of these technologies is orders of magnitude less expensive than its predecessors, just as the printing press and paper dramatically lowered the cost of reproducing and distributing information.

For example, a typical desktop-published newsletter of 12 pages costs approximate US$1 to print and mail. Therefore, to reach 1,000 people with approximately 5,000 words costs about $1,000. Publishing these 12 pages over the Internet using the same computer that produced them, with the nearly inconsequential additional cost of a $200 modem and a phone line that costs $15 or $20 a month takes at most 30 seconds, so that the 1,000 people can be reached in 500 minutes, or about half a day. Assuming that our 12-page newsletter is monthly, this means that we can reach about 30,000 people (assuming that our modem is only busy half the time) at something like 1/50th the cost of printing and mailing. Of course, this is a rough estimate. Cut the number by a factor of 10 if you want to be extremely conservative and it is still an astounding reduction in the cost of distributing information. The resulting temporary problem is that so much information is published that little can be found, but I will come back to that opportunity.

In early modern Europe and on the Internet today, the new technologies of copying and distribution required documents to give up a great deal in terms of sensory appeal. The Gutenberg Bible was quite ugly compared with the illustrated manuscripts that preceded it. Today, the highest-quality World Wide Web server produces audiovisual imagery that hardly compares to Hollywood's most mediocre production values. Clearly, "better" does not apply exclusively, or perhaps even primarily, to the audiovisual character of communications media. The director of the MIT Media Lab, Nicholas Negroponte, puts this more glibly in his pointed criticism of high-definition television as the next step in the evolution of media: "When you ask average people on the street what's wrong with TV, they don't say, 'It doesn't have enough pixels,' they say, 'The problem is that it is stupid.'"

Like Church at its most corrupt, television has pushed salesmanship to a high art. It is not surprising that television programmers are incredibly adept at keeping viewers tuned in and buying its customers' products. It is a powerful, lucrative profession in which very, very few people rise to the top, thus they are, in their own way, some of the brightest, cleverest people in the world. Television may be stupid, but the people running it certainly are not.

If you accept the idea that television is stupid and the idea that the Church at the time of the Reformation was promoting ideas that were stupid at best, the next question might be, "Why could they get away with this?" Why was the church able to maintain its dominance of Europe in the Dark Ages, using stupid ideas to bring in cash that allowed its leaders to live like royalty, to build great cathedrals and to finance expensive holy wars? Why has "stupid" advertising-based media, especially television, become arguably the most powerful institution in the modern world, depended upon by business and politicians? These organizations offer demonstrably biased information and often, outright lies, yet they remain powerful. Why? The answer seems to be revealed not by looking at societies under the domination of concentrated socioeconomic power, but by examining the events that led to the redistribution of power, in particular, the role of technology in the Reformation and subsequent development of the West.

The Wittenberg Door was an FTP site

The Reformation dramatically eroded the Church's power, sparked by Martin Luther's nailing of his 95 theses to the Wittenberg church door. One might imagine that Luther's ideas were so original that they contained power in themselves to transform society. This seems unlikely however; the church practices and philosophies to which Luther objected were well-known and widely disliked; others objected and were generally excommunicated and often executed for their heresies.

The unprecedented event accompanying Luther's heresies is that they spread throughout Europe within a few weeks. Luther seems to have intended for the Wittenberg Door to play the role of an FTP server circa 1990 - a means of distributing ideas to a small academic community, in which the church tolerated discussion of heretical ideas. But just as the New York Times' John Markoff and other early net-surfing reporters sometimes surprised Internet publishers with mass media articles that contained information from the net, the early printers appear to have surprised Luther.  One could hardly fault Luther for failing to predict the consequences. Judging by his defense before Pope Leo X six months after he posted the theses to the Wittenberg server, er, door, it seems that Luther had little idea that the new inventions of printing and paper would spread his ideas throughout Europe in a few weeks. Iormation had never traveled so far and wide so quickly.

"It is a mystery to me how my theses, more so than my other writings, indeed, those of other professors were spread to so many places. They were meant exclusively for our academic circle here... They were written in such a language that the common people could hardly understand them. They... use academic categories." (Luther, in an interview with Pope Leo X).

Unfortunately for the status quo, the newly available means of inexpensively reproducing and distributing information proved more effective at distributing power and money than consolidating it. The printers who rapidly spread Luther's theses throughout Europe had been capitalized by the church itself, for example, the first dated printed product from Gutenberg was an indulgence. By Luther's time, indulgences had become big business, with printers sometimes paid by the granting of rights to print further indulgences, making them close to a form of currency.

Today's critics and reformers

Negroponte and other media critics, particularly those whose ideas are disseminated primarily via the Internet, are modern-day Martin Luthers, pointing damning fingers not at the Church, but at advertising-based media, questioning their authority and revealing their largely corrupt motives (despite the presence of many well-intentioned, ethical journalists, just as the Church in early modem Europe had many such priests). Luther condemned the church leadership for selling salvation in the form of indulgences, for claiming that ordinary people could gain access to truth only through its priests, and all the while funding the curiae to live and act as royalty. This description aptly describes today's advertising-based media.

The policies against which Luther forcefully argued, such as the exclusive use of Latin during mass, can be viewed as mechanisms to suppress heretical ideas. Today, advertising-based media control the tiny number of television channels, newspapers and magazines available to would-be mass communicators. "The media" have revealed their interests most clearly in the television networks' strong opposition to proposed ratings systems such as the V-chip, which would provide the first-ever mechanism for distributing opinions about the quality of programming. Once again, those who control distribution of popular information argue that theirs is the only path to Truth. Incredibly, they argue the First Amendment to the U.S. Constitution, one of the strongest expressions of freedom of speech, in their defense. Clearly, the major media would prefer if no one were free to use technology to associate criticism directly with their programming. This is, of course, the modern equivalent of the church's silencing of heretical viewpoints, especially those that suggested there were direct paths to Truth available to the ordinary person.

This observation begs the question of why availability of new points of view is threatening to those who control limited communications channels. Perhaps the answer is somewhat self-evident, but if you assume that this control is diminishing dramatically, much more difficult and fascinating questions arise:

• How does the availability of new viewpoints spur creativity and invention, as seems to have transpired during the Reformation and subsequent Industrial Revolution?
• Will the personal computer and Internet spur a rise in creativity and invention with a primarily positive impact on the world?
• What are the tools and products that will enable and encourage the positive effects of widespread access to new points of view?

These are some of the ideas that this paper and the subsequent book will explore. To a certain extent, this writing is a product of the phenomenon it attempts to examine. My own access to the Internet has helped me understand the points of view of a variety of communities whom I would have not otherwise routinely encountered, including writers, programmers, religious scholars, librarians and publishers, to name a few. There is one more sweeping question that has arisen in my most recent readings. It is said best by the title of a recent book of essays from the MIT Press, "Does Technology Drive History: The Dilemma of Technological Determinism?" (1995, Merritt Roe Smith and Leo Marx, editors).

Technological Determinism

Is technology intrinsically able to change the world? Does the creation of an invention inevitably lead to change of a certain kind'? Or by worshipping technology itself, do we choose to create a world in which technology is able to make changes? Is it that we want the Internet to make our lives better, even though it doesn't seem to do so yet? Have we just not yet figured out what it's good for and in the meantime our intuition is screaming that the something is just around the corner?

Looking at these questions, I find myself in agreement with "soft determinism," the idea that technology is one factor among many others as a driver of history. However, in the face of limited resources, such as mass communications channels in early modem Europe, technologies that dramatically increase the supply of that resource become powerful forces indeed. This is the situation that I believe exists today - one-to-many and many-to-many communications channels are in the hands of a few, but new technology is changing that before our eyes.

I am equally convinced that blind belief in technology as a force of good is dangerous. Widespread support for "progress" allowed and encouraged General Motors and other to conspire to destroy mass transit systems in the United States. (For another viewpoint on this conspiracy theory, see "Who Killed Mr. Red Car?") Belief in the purity of the cause of science and knowledge was also the justification offered by Nazi scientists who committed atrocities.

Moral and ethical arguments aside, pure technology, uninformed by other points of view, should be avoided if for no other reason than the fact that it is bad business. Everything we know about Gutenberg, the inventor of world-changing technology, comes from his bankruptcy records. Meanwhile, his contemporary, Aldus Manutius, built alliances, bringing together the mechanistic, technological printers with the philosophical, scholarly and spiritualistic academics of the church. If Aldus were alive today, he'd probably be running one of the big media companies and would be regarded as a brilliant "packager," in media industry lingo.

The fascinating aspect of access to new points of view is that it seems to be an essential ingredient for humans to be inventive and creative. Authors such as Norbert Wiener ("Invention: The Care and Feeding of Ideas," MIT Press, 1993) argue that serendipitous discoveries are only recognized by those who appreciate the relevance of more than one field of knowledge. Wiener, writing in the 1950s, expressed alarm that the technologies of publishing, microfilm and automatic text indexing were already isolating researchers from interesting ideas in fields other than their own. Before diving further into the nature of invention and serendipity, I offer a short detour examining the nature of serendipity, which should not be confused with randomness. If all that were needed for creative inspiration was random association of differing ideas, the Internet would have long since been a medium of inspiration; randomness abounds! But serendipity is not the same as randomness or wild chance.

Serendipity and randomness

Great "accidental" discoveries were accomplished by people who put themselves into a situation in which interesting accidents were likely to happen. Benjamin Franklin put himself into a thunderstorm to increase the chances of a lightning strike -- a random event, but quite likely to occur under his chosen circumstances. Imagine if Franklin had chosen a random time and place to conduct his experiment -- chances are he would have failed many times. (It is little wonder that one of Microsoft Chairman Bill Gates' favorite criticisms is, "That is so random!")

There is an old saying that enough monkeys with enough typewriters will eventually write the Bible and Shakespeare. This is a misleading expression of the nature of randomness and probability. It may be true, but if it were, you would have a tough time finding the Bible in the midst of the monkeys' nonsense. Of course, today's Internet sometimes looks like it was created by an infinite number of monkeys working for an infinite amount of time. There is a possibility that there is some Shakespeare in cyberspace, but we can't find it in the midst of the noise... yet. We are still creating the tools for cyberspace that will allow the equivalent of the sculptor or bonsai artist who explains his technique by saying that be cuts away everything that doesn't look like the end result. We have many tools to grow the net, but few to sculpt it.

Serendipity doesn't merely depend on the wisdom of those who put themselves in the right place at the right time. The historical record of simultaneous inventions suggests that invention is also the results of information that has become generally known. The fact that inventions happen simultaneously in isolated places also suggests that other constraints may have been simultaneously removed, such as the technology that underlies their creation. For example, computer chips are designed by software that depends on ever more powerful computer chips in order to design newer, more powerful chips. In this example, the circle of dependency is quite tight and thus obvious. More often, one can assume, the dependencies are less obvious, especially when time has passed, so that simultaneous invention seems to be more coincidence than causal.

Wiener makes the most relevant argument for access to diverse points of view as a mechanism of invention. He suggests that an inventor with knowledge of multiple fields is more likely to realize the usefulness of the thing that he or she invented, often along the way to doing something else. This observation immediately brings to mind the Silicon Valley put-down: "another product in search of a market." Perhaps the problem is that technologists have so little access to others' points of view that they simply fail to recognize the usefulness of their inventions.

Events such as Apple's Steve Jobs' visit to Xerox PARC, where Apple obtained many of the ideas that became standard on today's personal computers, become legend. Although Jobs likes to portray himself as a technologist, his real strength lies in packaging products. Apple has done a brilliant job of merging the design strengths of the academic/research community with technology, Unfortunately, Apple has consistently failed to build the right products; the company endlessly seems to build too many of the machines that people don't want and too few of the ones that sell well. When you consider the company's roots this way, it is not surprising that its great successes were in selling to schools and the artistic professions. Microsoft, in contrast, has not had strong ties to the research, academic and artistic communities, but has done a spectacular job of balancing the points of view of technology and merchandising. Its technology often lags, but its packaging more than makes up for technological impurities. Of course, to be fair to Apple, the time and expense of shifting product lines in hardware is much greater than in software.

Cultural collision as a catalyst for creativity

There is much evidence to support the notion that when cultures collide, even within a stratified society where paths might have literally crossed, the result may be intense periods of creative invention. Wiener echoes Plato's idea that in the ideal state, philosophers must become kings and kings must become philosophers, saying that invention is only encouraged when artisans and philosophers communicate. Wiener's idea may be supported by the dramatically differing histories of Mexico and the United States since the arrival of Europeans. The two countries are virtually identical in terms of natural resources. Mexico, dominated by a single, highly stratified culture, has not prospered, while the United States, the "melting pot" of cultures, rose to world dominance.

Japan, even more than Germany since World War II, has prospered as a nation that has been highly inventive, particularly in manufacturing. Could this be due in large part to the fact that it threw open the doors to the outside world after so many centuries of isolation?

Clearly, there is no assurance that cultural collision turns into creative inventiveness; only opportunity. Technology is also a tool of genocide and repression; the native Americans certainly experienced a cultural collision, but their world diminished rather than expanding, as Europeans used the technologies of warfare to dominate.

Personal experiences also seem to teach us that exposure to a variety of points of view is a stimulant for learning, if not creative invention. (Perhaps this is redundant; the process of learning should probably be viewed as creative invention of self.) The tradition of spending an academic year abroad, of learning a second language, being broadly read - the values of a liberal arts education are not intended to communicate a particular set of facts, but to experience the world from a variety of points of view. Although few have well articulated why this kind of activity is valuable, it clearly is so.

The opportunities of diversity

If personal computers and the Internet indeed are increasing and will continue to increase the availability of relevant, diverse points of view, at least to the computer literati for the present, and if this will in fact be a positive agent of change in a world whose popular communications is dominated by a handful of self-serving media organization, then there must opportunities for developing related products and services. This is to say that there are and will be problems with digital publishing on the Internet, problems that will call for fresh new goods and services. Or, in business terms, there are market segments emerging and fortunes to be made.

Getting lost

Mendicant scholars in early modern Europe had to spend a great deal of time getting from place to place, figuring out idiosyncratic punctuation and organizational schemes, and memorizing a great deal, since paper wasn't readily available. This is not too different from today's Internet surfer. Those who organize and categorize information at Yahoo, Excite, etc. might be dubbed today's "" mendicant sysops ("sysop" is a contraction of "system operator"). They don't add value directly, they make connections and bring some order to the chaos.

Today, the volume of information being generated on the Internet seems to greatly exceed the ability of any one organization that attempts to categorize all Web pages and other resources. Although there will surely continue to be broad categorizations of Internet resources, a large number of narrower indexes, sometimes called 'virtual libraries' is likely to emerge. These are driven by the desire to create and/or serve a discrete community on the Internet. For example, Xilinx, a manufacturer of semi-custom computer chips, maintains a virtual library of Internet resources related to the type of chips it makes. As these community-oriented resources grow, tools for building and maintaining them will be developed, as well as tools and services that integrate information from multiple sources.

"Nobody goes there anymore because it's too crowded"

The heading above is a quote from baseball coach Yogi Berra, describing a restaurant that was formerly a hangout for the elite. He could have been describing a Usenet newsgroup, where popular discussions often die of their own growth. So many messages accumulate that no one can keep up with the volume and so the participants drift away. Mailing list-based discussions, which send every message to every subscriber, tend to be more self-policing, although they also can become overwhelming. However, since there is no definitive source to find out about mailing lists, they tend to be populated by people who are quite interested in the list topic and they don't grow as quickly as Usenet groups.

One of the significant short-term opportunities for developing collaborative filtering tools would be to leverage the power of Usenet, mailing lists and similar online discussions. Software that would assist people in selecting the most relevant messages (as defined by a combination of their interests and the accrued opinions of the communities to which they choose to belong, for example) would be welcomed by many overloaded Usenet surfers and list subscribers.

Libraries and academia

The potential of the 'virtual library' has not been ignored by the existing library community. Realizing that they possess the indexing, categorization and retrieval skills necessary for the Internet's growth, many progressive librarians are developing software tools, as well as participating in discussions and debate about the future of these technologies on the net. First-hand observation of these activities reveals an interesting aspect of the relationship between librarians and technologists. Librarians generally ask for inclusion of much more "meta-information" (author, subject, title, etc.) than commercial software vendors and services are willing to provide. This is a present-day example of the kind of compromise in quality that is necessary to take advantage of a new medium of distribution. The Internet is too primitive to take the leap to supporting the sophisticated indexing, search and retrieval services that libraries use.

Other forces are pushing librarians onto the net. As public information access centers, libraries are serving their traditional role by providing Internet access. More importantly in the long run, they are working to put card catalogs and other resources on the Internet, to increase accessibility.

Re-learning rhetorical skills

As people engage in much more written debate than was possible in the age of television, they have been forced to re-learn rhetorical skills that were lost to many. It is not unusual to see engineers and programmers pointing out a straw man argument or ad hominem attack. The recipients of the experienced Internet debaters' critiques are usually new participants, who seem invariably to bring the black-and-white rhetoric of television. The usual result of poor argument skills is a "flame war," a discussion that degenerates at best into name-calling.

The opportunities represented by new forums for debate are probably going to be educational in nature, teaching people how to bring others to their point of view through discourse.

Rating systems

Faced with the prospect of Congress' passing of the Communications Decency Act (struck down by a federal court), a group of technologists rapidly developed a standard for distributing ratings of pages on the World Wide Web. The primary intent of this effort was to demonstrate an industry solution for parents who want to restrict their children's access to sexual, violent or otherwise potentially harmful Web pages. However, the rating standard (Platform for Internet Content, or PICS) is open to many other uses. It is potentially the information system that software agents would use to find interesting information based on input from communities that you, as a user, consider relevant, or those for which your agents observe correlations.

Although the use of PICS for deliberate censorship by parents was its intent, it may also be used by nations to stop unwanted Web pages from being carried by their domestic Internet Service Providers. For example, France reportedly recently passed a law requiring such service providers to block access to Web pages that will be identified by a government committee. PICS could be the mechanism by which this censorship is carried out. Those implementing PICS are of two minds on this issue. On the one hand, virtually all of those involved are motivated by a desire to prevent the U.S. and other governments from censoring the Internet. On the other, some believe that a nation such as France, like smaller, non-governmental communities, has the right to self-censorship.

Regardless of the political implications, PICS is providing the first set of large-scale opinions about the content of the Web. Numerous applications could draw from this data to help people sift through it, looking for their Shakespeare in the noise.