Depth-First: User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information

User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information

Rich Apodaca — Thu, 17 Apr 2008 17:50:00 +0000

Printed encyclopedias of chemical information like the Merck Index suffer from the problem of becoming obsolete on publication. When new compounds are discovered, or when the information about a compound changes, those changes can take many months or years to appear in print form due to the high cost of publication. It doesn't have to be that way. This article introduces a new feature to the free online chemical encyclopedia Chempedia that lets working scientists update is contents via Wikipedia.

About Chempedia.net

A recent article introduced Chempdia, the free online chemical encyclopedia. This service is built on two of the largest free and open repositories of chemical information in existence: Wikipedia and PubChem. PubChem supplies low-level chemical information such as connection tables, and Wikipedia supplies free-text descriptions of the properties and uses of certain molecules.

Which Molecules?

Currently, Chempedia.net only includes compound monographs for about 1,000 of its over 300,000 molecules. These monographs were located by a manual process in which the titles for all Wikipedia articles were downloaded in alphabetized form; this process clustered titles that represented IUPAC nomenclature due to its use of leading numbers and symbols. IUPAC nomenclature titles were extracted, and then a script was written to extract the chemical information from these titles and combine it with that from PubChem.

This method, although useful for getting a service running, is clearly flawed. The biggest problem is in how to discover new compound monographs.

Why Not Put Users in Control?

Chempedia users themselves are in the best position to know when an existing Wikipedia compound monograph should appear in Chempedia but doesn't, when an existing monograph needs to be updated, or when a new monograph is written and needs to be linked.

How can the process be automated?

As a partial answer to this question, users now have the ability to notify Chempedia of any changes to a Wikipedia compound monograph, and to have those changes immediately reflected in the next viewing of a Chempedia compound monograph.

An Example

As an example, let's take anandamide, a compound I've had some experience with during my time as a medicinal chemist. Although the Chempedia entry for ananandamide exists, there is (or as of today - was) no link to the Wikipedia compound monograph. Let's create one.

At the top of Chempedia's main menu, you'll see a link titled 'Update'. Choosing this link leads to a form that will ask for two pieces of information: (1) the title of the Wikipedia article to which you want Chempedia to link - in this case 'anandamide'; and (2) reCaptcha text to keep robots from making mischief.

Submitting this information is all that's needed to create a new or updated link from Chempedia to Wikipedia. Chempedia handles the rest.

Conclusions

Wikipedia is a vast source of free, high-quality, semi-structured chemical information just waiting to have good chemically-aware interfaces applied to it. Chempedia.net is an attempt to do just that, but it's a bit more as well. Although it may appear that Chempedia is the major beneficiary in this relationship, Wikipedia also benefits. When chemists have a tool that allows them to query and visualize Wikipedia using their native language (the chemical structure) they're in a better position to both use and contribute to Wikipedia itself - something I've started to do.

This positive feedback effect is the real value of exposing Web services. The question is: who in cheminformatics is willing and able to take the risk to discover this simple principle and its benefits?

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Rich Apodaca

Fri, 25 Apr 2008 09:23:15 +0000

Kris, thanks for reminding me about the ChemDB SMILES Depict service. Unfortunately, several attempts yesterday to use it resulted in no response. Today it worked.

It would be nice to have some online documentation of the parameters. For example, when I clicked on the 'Options' link, I got Ogham options despite using the 'ChemAxon Marvin method'. And this list of options looked incomplete - for example not including 'width' and 'height'.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Rich Apodaca

Fri, 25 Apr 2008 08:49:19 +0000

Noel, Rajarshi's REST cheminformatics services are quite interesting. The SMILES depictor (it would be helpful if this link displayed the service's documentation - the home page if you will) has some potential.

For example, what parameters does the service accept? How can I customize the returned image in terms of size, aspect ratio, border, etc?

These are all questions that need to be answered longer term, but Rajarshi's groups is clearly onto something.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by baoilleach

Fri, 25 Apr 2008 02:25:22 +0000

Are you aware of Rajarshi's CDK service at http://www.chembiogrid.org/projects/proj_rest.html which returns a PNG given a SMILES structure.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Kris

Thu, 24 Apr 2008 16:46:26 +0000

I am interested in this type of API as well. ChemDB offers a few options for generating images here: http://cdb.ics.uci.edu/CHEM/Web/cgibin/Smi2DepictWeb.py

The api should have the options ChemDB offers. I am particularly interested in rendering of SMARTS strings which Marvin does not do very well.

I think the best idea for performance is to cache the images for awhile after generating them. I don't think this is a problem that is too hard to scale. I would like to see a comparison of the speeds of various 2d chem image generating tools.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Rich Apodaca

Thu, 24 Apr 2008 11:56:55 +0000

Noel, I look forward to seeing that post. One of the challenges with building a Web-based system that serves 2D molecular images is deciding exactly how it should work.

Clearly, the rendering technology is already there with ChemWriter. But the rest of it is also important:

What will the service do and what will it not do?
What should the Web API look like?
What's the best way to build the system so that it scales?
How does the service support itself?

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Rich Apodaca

Thu, 24 Apr 2008 11:49:39 +0000

James, thanks again for the comments and your system info. You're helping me put some things into perspective.

User experience and speed of product development iteration are trade-offs that Chempedia needs to balance. Because the system is under heavy development, speed of iteration is paramount right now.

The image server idea is one I've been playing around with for some time now. But it would be a product unto itself. Getting something running wouldn't be hard. Creating something great would take more time - and pull resources away from making Chempedia great.

As Chempedia stabilizes, the user experience will take a front seat. But to be honest, the rendering performance is quite good, except for the 5% or fewer using Firefox/OS X. The flicker on Safari/OS X is something I've noticed and is a little distracting, but may be fixable - it's just a matter of looking into it.

I'm not saying that users with niche systems aren't important - they definitely are. I'm saying that getting Chempedia into a stable form is more important right now.

And once Chempedia is stable, it will be much clearer what the ideal 2D structure image server needs to do - if that becomes necessary.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by James

Thu, 24 Apr 2008 10:35:08 +0000

Hi Rich

First off, this idea is fascinating to me: "It's also a problem sufficiently widespread that a solution in the form of a general purpose Web service (a la reCAPTCHA) might be worth creating...". It would be fun to implement and really quite easy. ChemTube?

You have indeed nailed my preferred browser configuration. I checked the performance on Safari 3.x, where the speed is somewhat better, but I still experience some flickering on scrolling. Is your test system beefy? Or, perhaps more appropriately, what is the average performance on your potential customers' machines?

One point: "This results in fast Webapp development iterations because the server doesn't need to worry about generating and delivering an expensive raster image."

As a developer, I appreciate the agility of your setup. But that doesn't make the flickering any less annoying. And who are you to offload rendering work to me? I just want the thing to work efficiently!

But, if as you suggest the market with my setup is very small, I suppose I'm willing to concede the point.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by baoilleach

Thu, 24 Apr 2008 09:19:21 +0000

Regarding webserving 2D molecular images, if you build the web API I will use it. I'm preparing a blog post on 2D SDG and depiction, so if you can get a Restful web service running within a week that accepts SMILES strings and returns a jpeg/png, we can see how ChemWriter stacks up against the rest.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Rich Apodaca

Wed, 23 Apr 2008 22:14:18 +0000

How about using chemwriter as a backend rendering engine for serving the static images?

James, to answer your question, which I forgot to do...

The system Chempedia uses to display 2D molecular images (ChemWriter PainterApplets) distributes the workload over the client and the server. This results in fast Webapp development iterations because the server doesn't need to worry about generating and delivering an expensive raster image.

There are actually a couple of strategies for serving up 2D molecular images, which might make a good post at some point. It's also a problem sufficiently widespread that a solution in the form of a general purpose Web service (a la reCAPTCHA) might be worth creating...

If you confirm my suspicion and tell me you're using Firefox on OS X, then that really leaves just that one browser/OS combination as a problem area. According to Google Analytics, Firefox/OS X represents about 5% of the readers on Depth-First, which is actually slightly higher than the share for Safari/OS X.

So this looks like something that needs to be dealt with eventually.

Unfortunately, Firefox 3 beta 5 on OS X doesn't fix the performance problem.

In short, I plan to stick with the current system as the functionality on Chempedia gets refined and look for ways to improve the performance on Firefox/OS X.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Rich Apodaca

Wed, 23 Apr 2008 20:04:47 +0000

James, thanks for the feedback.

I've only found one configuration on which applets in general are slow - Firefox 2.x for OS X. For that particular combination, Chempedia's way of doing things is not optimal. For all the other major browser/OS combos (Firefox Windows, Firefox Linux, Safari OS X, IE 6/7), performance has been plenty fast in my hands.

Could you tell a little about the system you're using (browser, OS, JVM version, system memory, processor)?

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by James

Wed, 23 Apr 2008 19:25:02 +0000

I think you'd be better off serving static images rather than individual applet-clones of chemwriter on your main page. It's incredibly slow and forces me to stop scrolling before the redraw calls get a chance to fire.

How about using chemwriter as a backend rendering engine for serving the static images?

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by Kris

Thu, 17 Apr 2008 21:23:48 +0000

I have talked to Martin who posted on your blog earlier and he pointed me in the direction of these 2 lists of compounds: http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Chemistry/Curation http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Chemicals/Inorganics

There is also the list of organic compounds: http://en.wikipedia.org/wiki/Listof_organiccompounds

Hope this helps. You could also just download the latest version of wikipedia and do a one time scan for all chemical identifiers regardless of title.

"User-Created Compound Monographs on Chempedia.net: Open Sourcing the Collation and Indexing of Chemical Information" by ChemSpiderMan

Thu, 17 Apr 2008 20:48:42 +0000

Rich...an update on our own efforts in this area

http://www.wichempedia.org/ is an isolated Wikipedia Chemistry Set of structures. Click on any of the structures in the table and we will take you to the start of the monograph with a link back to the article in Wikipedia. For example, see Bezaldehyde - http://www.wichempedia.org/Chemical-Structure.235.html

Includes safety/tox data, NMR spectra and IR spectra, Compound monograph segment and other related data. The data is now over 100 data sources.

I notice your intention to put advertising on ChemPedia. Good idea. Hope that people don't slap you too hard for that but you're doing the work and deserve the support! Good for you I say!

We are going to do similarly on ChemSpider shortly to support our own efforts and I am sure that certain segments of the community will deem it inappropriate but c'est la vie.