Good news for cheminformatics: Chemical Abstracts Service (CAS) has agreed to help Wikipedia users curate its collection of CAS numbers. As a result of the diligence of some hard-working volunteers, chemistry's most universal system for referring to chemicals can now be used far more effectively by the worlds biggest open repository of knowledge.

Wouldn't it be great to be able to pull these CAS numbers from Wikipedia programmatically?

Perspective

Estimates place the number of Wikipedia pages dealing with individual inorganic and organic substances in the thousands. (I'll use the term "compound monographs" to describe them.) One factor acting to keep this number low is poor visibility of these entries. Unlike most chemical databases, Wikipedia can't, by itself, be easily searched by structure. As chemically-aware tools for indexing Wikipedia begin to emerge, look for six things to happen:

1. The number of Wikipedia compound monographs will increase significantly.
2. The quality of monographs for intermediate- to well-known compounds will increase substantially.
3. Demand for user-friendly interfaces to Wikipedia's chemical content will increase.
4. Wikipedia users will become interested in storing and finding ever more diverse kinds of information about each compound.
5. Bench chemists will start to include Wikipedia as one of their preferred literature search techniques, leading to...
6. More creative tools for using the chemical content of Wikipedia.

As noted previously, it wasn't too long ago that indexing of the chemical literature was done solely by volunteers. Wikipedia offers an intriguing way to channel the innate drive for chemists to combine their own work and experience with that of others to build useful information tools for the community.

But for now we are left with the question of how to index the chemical content of Wikipedia. Although a few systems have been proposed, the only practical method is through the use of CAS numbers. Which brings us to the subject of today's tutorial.

A Quick CAS Number API for Wikipedia

The Ruby program below will accept the title of any Wikipedia compound monograph title and return the CAS number for the compound being discussed, or an error message if none was found:

require 'rubygems'
require 'hpricot'
require 'open-uri'
require 'cgi'

class Wikikemi
  @cas = nil

  attr_reader :cas

  def initialize title
    uri = URI.escape("http://en.wikipedia.org/wiki/#{title}")
    puts "loading... #{uri}"
    doc = Hpricot(open(uri))
    table = (doc/"table")[0]

    table.inner_html.match(/([0-9]{2,7}?\-[0-9]{2}\-[0-9])/) if table

    @cas = $1
  end
end

# Returns the CAS number present in the Wikipedia monograph with
# the indicated title, or an error message if none is found. Try, for example,
# "benzene.".
while true
  puts "Enter the title of the Wikipedia page, for example: 'benzene'"
  monograph_title = gets.chomp
  w = Wikikemi.new monograph_title
  puts w.cas ? "[#{w.cas}]" : "CAS number not found"
end

This program makes use of the excellent Ruby HTML parser, Hpricot.

Saving the above code to a file called wikikemi.rb, we can run it with:

$ ruby wikikemi.rb

For example, we can look up the CAS numbers for Ferrocene, Lipitor, or 1,2,3,4,4a,5,6,7,8,8a-Decahydronaphthalene:

$ ruby wikikemi.rb
Enter the title of the Wikipedia page, for example: 'benzene'
ferrocene
loading... http://en.wikipedia.org/wiki/ferrocene
[102-54-5]
Enter the title of the Wikipedia page, for example: 'benzene'
lipitor
loading... http://en.wikipedia.org/wiki/lipitor
[134523-00-5]
Enter the title of the Wikipedia page, for example: 'benzene'
1,2,3,4,4a,5,6,7,8,8a-Decahydronaphthalene
loading... http://en.wikipedia.org/wiki/1,2,3,4,4a,5,6,7,8,8a-Decahydronaphthalene
[91-17-8]

All this method requires is that the Wikipedia page lists the correct CAS number in its Drugbox or Chembox template. Fortunately, CAS has agreed to help make this happen.

Conclusions

A little Ruby code is all it takes to build a working CAS number lookup system using Wikipedia. Although this may be useful as a standalone tool, it becomes much more powerful when made part of a larger cheminformatics system. But that's a story for another time.

See also Antony Williams' announcement on CAS and Wikipedia.

A recent article described how PubChem could be used to quickly search for CAS numbers. Although useful, the approach is limited in that only an array of PubChem CIDs was returned. What would be really useful would be a simple way to create a report with entries hyperlinked into the PubChem site itself to aid in visual inspection. In this tutorial, we'll see how an HTML template and a few extra lines of code can do just that.

The Template

Ruby supports a number of HTML templating mechanisms. In this example, we'll use an ERB template resurrected from the Molbank graphical table of contents tutorial:

<html>
  <head>
    <title>
      <%= "PubChem Search for #{term}" %>
    </title>
  </head>
  <body>
    <h1>
      <%= "Search: #{term}" %>
    </h1>
    <table>
      <tr>
      <% col = 0 %>
      <% cids.each do |cid| %>
        <td>
          <% image = "http://pubchem.ncbi.nlm.nih.gov/image/imgsrv.fcgi?cid=#{cid}" %>
          <% summary = "http://pubchem.ncbi.nlm.nih.gov/summary/summary.cgi?cid=#{cid}" %>
          <a href="<%= summary %>">
            <img src="<%= image %>" border="2"></img>
          </a>
          <center>
            <span style="font-size: 8px">
              <a href="<%= summary %>"><%= "CID-#{cid}" %></a>
            </span>
          </center>
        </td>
        <% col += 1 %>
        <% if col > 5 %>
          <% col = 0 %>
          </tr>
          <tr>
        <% end %>
      <%end %>
      </tr>
    </table>
  </body>
</html>

The above template uses a search term and an array of CIDs to build a table of results. Each cell in the table contains a color 2D image and the CID, both hyperlinked into PubChem itself.

Saving this library to a file called template.rhtml is all we need to do.

The Library

The library is a modification of the one shown in the previous article in this series:

require 'rubygems'
require 'mechanize'
require 'erb'

module PubChemTerms
  def report term
    cids = get_cids term
    erb = ERB.new(IO.read("template.rhtml"))

    File.open "output.html", 'w+' do |file|
      file << erb.result(binding)
    end
  end

  def get_cids term
    agent = WWW::Mechanize.new
    page = agent.get "http://www.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pccompound&retmax=100&term=#{term}"

    (page.parser/"id").collect {|id| id.innerHTML}
  end
end

The method report accepts a search term and uses our template to render a report.

Testing

By saving the above library in a file called pubchem.rb, we can search by keyword via interactive ruby (irb):

$ irb
irb(main):001:0> require 'pubchem'
=> true
irb(main):002:0> include PubChemTerms
=> Object
irb(main):003:0> report 'esomeprazole'
=> #

This produces a file called output.html that can be viewed with any browser:

As in the original version of the library, we can also query by CAS number:

$ irb
irb(main):001:0> require 'pubchem'
=> true
irb(main):002:0> include PubChemTerms
=> Object
irb(main):003:0> report '119141-88-7'
=> #

Conclusions

The simple approach outlined here could be extended in many ways. For example, we could easily retrieve molfiles based on keyword or CAS number search. We could pipe queries together or work with query lists. We could blend in ChemSpider data. We could even build a simple Web application (with Rails) that returned customized reports. Mixing in Ruby CDK or Ruby Open Babel offers still more possibilities.

Increasingly, the most important question in cheminformatics is not "What can we build?", but rather "What should we build?" Success in this new world requires a much deeper understanding of how cheminformatics software is being used by real chemists and where it's not.

Although the PubChem system has been discussed in numerous recent D-F articles and elsewhere, there's much more to the story that hasn't been told. One of the more intriguing things PubChem can do is look up CAS Numbers for free. In this tutorial, we'll see how a simple Ruby script can be used to automate the conversion of CAS numbers into PubChem Compound IDs (CIDs).

The Library

Our library needs to accept a CAS number and return an array of PubChem CIDs in response. To do this, we'll make use of the Entrez eUtils system. Although Entrez is incredibly complex, the only two things that matter now are that the NIH requires automated scripts to access most of its databases through Entrez, and that Entrez can be used to perform PubChem keyword queries.

The library is simplicity itself:

require 'rubygems'
require 'mechanize'

module PubChemTerms
  def get_cids term
    agent = WWW::Mechanize.new
    page = agent.get "http://www.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pccompound&retmax=100&term=#{term}"

    (page.parser/"id").collect {|id| id.innerHTML}
  end
end

The excellent Ruby library Mechanize is used for submitting queries and processing the results. (This is the same library that was used to extract full bibliographical information from nothing more than a DOI). The only remarkable thing about the library above is how unremarkable it is.

A Test

$ irb
irb(main):001:0> require 'entrez'
=> true
irb(main):002:0> include PubChemTerms
=> Object
irb(main):003:0> get_cids '64318-79-2'
=> ["5282237", "6434870"]

Our library has returned a Ruby array containing two compound identifiers. We can use PubChem to view their records here and here. Visual inspection reveals these two compounds to be isomers of each other, with the first member of the array containing the direct hit.

Let's try another CAS number selected from another random Merck index entry:

irb(main):004:0> get_cids '66981-73-5'
=> ["68870", "169125"]

Again we've obtained two CIDs, with the first one being the neutral form and the second one being the sodium salt of the antidepressant tianeptine.

Applications

Now, instead of converting one or two CAS numbers, imagine we've got a few thousand. Our library could be easily adapted to this purpose. The only caveat is that we'd need to observe the Entrez use policy and not overload the server with too many requests. We could build in a delay with Ruby's sleep method.

Notice that the library can be used to search for any keyword - not just CAS numbers. For example:

$ irb
irb(main):001:0> require 'entrez'
=> true
irb(main):002:0> include PubChemTerms
=> Object
irb(main):003:0> get_cids 'anandamide'
=> ["5281969", "5283455", "5283388", "4671", "5353407", "5283452", "5283456", "5283451", "5283450", "5283449", "5283448", "5283447", "5283445", "5283444"]

Like our previous queries, we've obtained multiple CIDs associated with the term 'anandamide', with the first one being the direct hit.

Conclusions

Our little library isn't perfect, but it performs a very difficult task cheaply and conveniently in the majority of cases. By mashing up this functionality with other Ruby cheminformatics libraries (for example Ruby Open Babel and Ruby CDK), a variety of tough and highly practical cheminformatics problems can be solved elegantly. Look to further installments of the Hacking PubChem series to find out how.

CAS Registry Numbers simplify the thorny problem of referring to chemical substances. These short numerical sequences are arguably the most widely-used form of molecular identifier, appearing on reagent bottles, in publications, in patents and patent applications, and MSDS sheets.

During my time as a synthetic organic chemist, I would sometimes run into the problem of finding the structure of a molecule represented by a CAS number. A common case was when an ambiguous, incomprehensible, or blurred IUPAC name was printed on a reagent bottle along with a CAS number. By looking up the CAS number, I could confirm the bottle's contents.

Your first impulse when looking up a CAS number might be to fire up SciFinder. For years this was the only option. Those days are quickly starting to seem as quaint as when people actually wrote on pieces of paper and dropped them in mailboxes (dropping DVDs in a mailbox is a different matter).

A little-publicized feature of PubChem makes it an ideal way to quickly find the structure associated with a CAS Number. To use it, you need nothing more than a computer, a browser, and an internet connection.

Browse over to the PubChem welcome page. At the top you'll find a search box. Enter your CAS number and press "Go." For this example, I'm using the CAS number for 2,5-Pyrazinedicarboxylic acid dihydrate:

If all goes well, you should see a results screen containing the structure of your compound and a link to its summary page:

Does this seem a little too good to be true? Try it for yourself. Pick up a copy of the Aldrich catalog, Merck index, or anything else that lists lots of CAS numbers. Choose several structures at random and see how PubChem performs.

There are limitations to this method. PubChem generally doesn't index large molecules such as polymers and peptides, so they won't be found by this method. Similarly, if a CAS number doesn't point to a distinct molecular entity (e.g. "mineral oil"), PubChem won't find it either. But these are hardly limitations in the vast majority of cases.

With the recent addition of Sigma-Aldrich as a PubChem compound supplier, it won't be long before smaller companies begin following suit. What we're seeing with PubChem is a classic example of a network effect. The end result should come as a surprise to nobody.

The InChI team has announced a proposal for a standardized InChI hashing mechanism. This would create a free, fixed-length, alphanumeric molecular identifier.

This is an excellent proposal. One of the biggest problems in working with InChIs (and other line notations such as SMILES) is that even medium-sized molecules produce very long identifiers. Another problem is the use of characters that must be escaped in URLs. The hashing proposal addresses both of these issues, getting very close to creating the optimal molecular identifier.

For example, imagine the convenience of being able to refer to a molecule by a universally-recognized, machine-generated string like the one shown below:

AAAAAAAAAAA-BBBBBBB-XYZ

This is something that actually stands a chance of getting printed on reagent bottles, in catalogs, in patent applications, or anywhere else chemists are using chemical information. Aside from its length, it's not too different from that other molecular identifier system, but without the perpetual use tax.

There are at least three downsides to this approach:

1. For most purposes, hashing is a one-way process. It would become virtually impossible to computationally convert this hashed identifier back into its InChI or molecular representation . On the other hand, this could create a market for cryptography experts in cheminformatics. A hashed-InChI lookup service would start to look very useful.

2. Because of the one-way nature of hashing, the authenticity of a hashed InChI couldn't be directly verified. Checksums will help, but the fundamental problem remains. InChI itself can be decoded, and therefore authenticated.

3. It's possible, although extremely unlikely, that two different molecules will end up having the same hashed InChI. Reducing the collision probability means increasing the length of the identifier.

As in any design decision, the question is whether the benefits outweigh the disadvantages.

Anyone is free to develop their own InChI hash system. Several, including me, already have. But by introducing a standard mechanism, the InChI team has the potential to create both a free and easy-to-use molecular identifier.