Depth-First: Tag collectiveintelligence

Building Chempedia: Learning About Contributors

Rich Apodaca — Wed, 02 Jul 2008 11:50:00 -0400

Chempedia is a free online chemical encyclopedia similar in concept to the Merck Index, but radically different in implementation. One key difference: the Merck Index is compiled by a small number of paid professionals while Chempedia is compiled by thousands of unpaid volunteers. Although this distinction raises a host of intriguing questions, one of the most basic revolves around what can be said about these volunteers in the aggregate. This article, the first in a series, explores this issue with some statistics compiled from Chempedia.

Learning About Contributors

Chempedia works in part by aggregating content from Wikipedia dealing with single molecular entities, or "Compound Monographs." This content is created by the now famous process of individuals taking upon themselves the responsibility of fixing what's broken in Wikipedia. (Some take it upon themselves to break what's working, but that's another topic.)

Chempedia associates each of its Compound Monographs with the last Wikipedia user to edit it. The current interface to these relationships is available on the Chempedia contributors page.

The interface to this page is currently limited. The analyses reported here were made for the most part by querying the Chempedia database directly.

Each contributor is linked to a contributor summary page containing links to that user's Wikipedia homepage and talk page, as well as a complete listing of all active contributions. For example, you can view the contributor page for one of Chempedia's most active contributors, Arcadian.

The data model is also limited. Because Chempedia only records the last Contributor to edit a Monograph, when another Contributor edits a Monograph, the link between the previous Contributor is lost. As a result, many Contributors have no associated Monographs.

How Many Monographs?

Chempedia currently hosts 6,308 Compound Monographs.

How Many Contributors?

Chempedia currently lists 2,516 Contributors. Of these, 1,046, or 42% are associated with one or more Monographs, meaning that they were the last to edit. The remainder are associated with no Monographs for which they were the last to edit.

Here is a list of the top 20 Contributors and the number of Monographs they were the last to edit:

anonymous	1022
DOI bot	904
Edgar181	378
Fvasconcellos	170
Meodipt	151
Arcadian	144
Chem-awb	133
Chowbok	122
Rifleman 82	114
SmackBot	105
Thijs!bot	99
ChemNerd	85
Puppy8800	80
DumZiBoT	78
Axiosaurus	63
Chempedia	63
Carlo Banez	55
Benjah-bmm27	52
OKBot	51
Cacycle	50

These Contributors represent 1.9% of all active Contributors and collectively are responsible for being the last to edit 62% of all Monographs. Although not performed here, a histogram plotting number of contributions would be expected to follow a power law.

'Anonymous' is an aggregation of all users who edited a Monograph without a Wikipedia account. 16% of all Monographs were last edited by an anonymous user. Leaving out the aggregated 'anonymous' users indicates that roughly half of all Monographs were last edited by the top 19 Contributors.

What is a Contributor?

Although it's difficult to say a lot about individual Contributors, most appear to have some training in science, although that training may not have involved chemistry or biology. Still others (for example, SJP) appear to have been drawn to contribute to a Monograph based on their nonscientific experience with the title compound or in an effort to fight vandalism or otherwise improve the nonscientific content of the Monograph. The ability of services like Wikipedia (and by extension Chempedia) to provide a platform for those without formal training in a particular area to make useful contributions is without question one of its most useful (and controversial) features.

Some Contributors are not even human, but rather robots designed to improve the quality of Wikipedia articles in general. For example, SmackBot performs an array of tedious quality control jobs such as fixing bad checksum ISBNs (CAS Numbers, anyone?) and capitalization errors.

Conclusions

Wikipedia's collaboration model has made the creation of a free and continuously-updated chemical encyclopedia feasible. Applying chemistry-specific user interfaces and data models exposes this hidden treasure. Although it's tempting to think of this process as mainly being the work of a handful of trained scientists, the numbers suggest a much broader base of contributors. Future articles will explore this idea.

What Makes Wikipedia Tick?

Rich Apodaca — Fri, 05 Oct 2007 10:28:00 -0400

Whatever your views on Wikipedia, it's clear that the volunteer online encyclopedia has left it's mark on society. But the most important things about Wikipedia have less to do with its contents and more to do with the people contributing and using the service. To understand how and why people collaborate on the Web, you have to understand Wikipedia.

An interview with three leading Wikipedia figures sheds some light on Wikipedia as a collaborative activity.

There is a myth about online collaboration that Open Source practitioners are very familiar with. It goes something like this: "I'll start building something and release it to the community. I'll get feedback from a lot of users, some of whom will fix bugs, write documentation, and build extensions. All of that feedback will create a better product."

Now, this does happen, of course. The reason I consider it a myth is that it happens so rarely that you might as well not count on it. Virtually all Open Source software is designed, written, documented, debugged, and promoted by a single developer with the help of a tiny fraction (say 2-10%) of the committed user base. Pick any good example of Open Source software that works and behind it you'll find a committed user base large enough to make 2-10% a number greater or equal to one. It's not clear this is necessarily a bad thing.

The interview with the Wikipedia leaders confirmed this view. When asked about the idea that lots of contributors makes a good article, Elisabeth Bauer, of the English Wikipedia, had this to say:

The best articles are typically written by a single or a few authors with expertise in the topic. In this respect, Wikipedia is not different from classical encyclopedias.

Her view was shared by Kizo Naoko, of the Japanese Wikipedia who added that short articles tend to remain short and of poor quality.

There doesn't seem to be anything complicated here. Wikipedia places a very low barrier to contribution. It has created a system where active contributors with specialized knowledge feel a sense of ownership over their contributions. Checks and balances insure that these contributors can monitor changes to their work, and correct errors. Finally, the subject matter is so broadly appealing (All of Human Knowledge) that 2-10% of the user base is a massive number.

It may not be complicated, but it's far from easy.

Collective Intelligence and the Dumbness of Crowds

Rich Apodaca — Thu, 18 Jan 2007 15:07:00 -0500

It's the sharp edges, gaps, and differences in individual knowledge that make the wisdom of crowds work, yet the trendy (and misinterpreted) vision of Web 2.0 is just the opposite--get us all collborating [sic] and communicating and conversing all together as one big happy collborating [sic], communicating, conversing thing until our individual differences become superficial.

-Kathy Sierra, The "Dumbness of Crowds"

"Web 2.0" has gotten a lot of people thinking about exciting new forms of collaboration made possible through the Internet. Services like Digg, YouTube, Flickr, and Wikipedia, and especially the way they harness the selfish impulses of individuals for the common good, are seen by many as just the start of even better things to come.

Given the astonishing advances in hardware made possible by Moore's law, and the relentless progress of Open Source software, starting one of these services on your own can be done for practically nothing. Of course, starting a Web 2.0 service and actually seeing it become a raging success are two very different things. What's the deciding factor?

It was while thinking about this question that I ran across a thought-provoking article on Kathy Sierra's Creating Passionate Users. Although mainly focused on software, Kathy's blog is essential reading for anyone trying to create remarkable products. The article that caught my eye was titled "The 'Dumbness of Crowds'". In it, Kathy makes some interesting distinctions between "Collective Intelligence" and "Dumbness of Crowds."

The distinction arises from the number of people involved, the nature of what they're building together, and the process by which differing views are reconciled. For example:

Collective Intelligence a bunch of people writing Amazon book reviews
Dumbness of Crowds a bunch of people using a Wiki to write a book
Collective Intelligence - Flickr's photo collection and tags
Dumbness of Crowds - a bunch of people trying to make a photo together

I could add my own observations:

Collective Intelligence an online RSS aggregator that combines blog feeds from multiple sources within the community
Dumbness of Crowds a publicly-writable Wiki that serves as your community's public face
Collective Intelligence ten users of an Open Source software library stress-testing it in their own work, fixing bugs, and requesting features
Dumbness of Crowds ten programmers trying to design an API
Collective Intelligence ten developers who each start their own project to build their own distinct application based on a piece of Open Source software
Dumbness of Crowds ten developers who set out to design a single application that does the work of ten

The distinction really revolves around the degree to which individual contributions are blindly averaged verses being allowed to retain their individuality. Committees rarely create great works. Even worse - sometimes consensus is fatal.