Hacking ChemSpider: Query by SMILES and InChI with Ruby

Rich Apodaca — Mon, 17 Sep 2007 08:19:00 +0000

Slowly but surely, cheminformatics Web APIs are starting to appear. What's the big deal, you may ask? By exposing Web APIs, service providers enable third parties to develop new applications that "mash up" functionality from two or more sites, or which take the original service in directions its founders never considered.

By way of Antony Williams' blog, I came across the announcement for the ChemSpider Web API. What can this API do for Web developers? To find out, let's write a small Ruby library.

The Library

Our library will accept a SMILES string or InChI identifier and returns a URL pointing to the corresponding ChemSpider compound summary page. Like previous Web API demos, this one uses the powerful Ruby library Mechanize, leading to very concise code:

require 'rubygems'
require 'mechanize'

module ChemSpider
  def url_for_inchi inchi
    agent = WWW::Mechanize.new
    page= agent.get "http://www.chemspider.com/inchi.asmx/InChIToCSID?inchi=#{inchi}"
    csid = (Hpricot(page.body)/"string").innerHTML

    csid == "" ? nil : "http://www.chemspider.com/RecordView.aspx?id=#{csid}"
  end

  def url_for_smiles smiles
    agent = WWW::Mechanize.new
    page= agent.get "http://www.chemspider.com/inchi.asmx/SMILESToInChI?smiles=#{smiles}"
    inchi = (Hpricot(page.body)/"string").innerHTML

    raise "Invalid SMILES: #{smiles}" if inchi == ""

    url_for_inchi inchi
  end
end

The url_for_inchi method directly uses the ChemSpider API to query by InChI. The url_for_smiles method first uses the ChemSpider API to convert a SMILES string to an InChI identifier, and then calls the url_for_inchi method.

Two points are worth noting. First, although for convenience the InChI identifier isn't escaped before being appended to the API URL, strictly speaking it should be. Second, both methods invoke the underlying Mechanize library Hpricot to parse the raw XML returned by ChemSpider.

Testing

Saving the above code to a file called chemspider.rb, we can get the URL to ChemSpider's benzene page from its InChI identifier via interactive Ruby (irb):

$ irb
irb(main):001:0> require 'chemspider'
=> true
irb(main):002:0> include ChemSpider
=> Object
irb(main):003:0> url_for_inchi "InChI=1/C6H6/c1-2-4-6-5-3-1/h1-6H"
=> "http://www.chemspider.com/RecordView.aspx?id=236"

We can work with SMILES strings just as easily as with InChIs:

$ irb
irb(main):001:0> require 'chemspider'
=> true
irb(main):002:0> include ChemSpider
=> Object
irb(main):003:0> url_for_smiles 'c1ccccc1'
=> "http://www.chemspider.com/RecordView.aspx?id=236"

Both the InChI and the SMILES string yield a URL pointing to the same Chemspider page for benzene.

Conclusions

Like most chemical databases, ChemSpider uses a compound summary page as a way of organizing the available resources for a given molecule. With a method in hand for accessing these pages based on arbitrary SMILES or InChIs, we can begin to think of manipulating ChemSpider independently of its current user interface. But that's a story for another time.

Easily Convert Publisher URLs and DOIs to Bibliographical Citations: Synthesis, Synlett, Ruby, and Mechanize

Rich Apodaca — Wed, 27 Jun 2007 08:45:00 +0000

Just ten years ago, the thought of accessing all of the world's scientific literature online struck many as optimistic at best. Today, an increasing number of scientists use the Web as their only means of reading the literature.

This shift has brought with it a significant, but rarely discussed problem: converting a publisher URL or DOI to a bibliographical citation (title, authors, journal, page, volume, etc.). This is a problem because bookmarking and linking URLs are the way we reference Web documents, but the bibliographical citation is still how we reference paper documents. We may well see the day when the need for bibliographical citations disappears, but until that happens there's a need for user-friendly tools that manage the conversion.

This article discusses remarkably simple and flexible solution to this problem using Ruby and the outstanding Mechanize library. As test subjects, I'll use two of my favorite journals: Synthesis and Synlett.

What is Mechanize?

From the Mechanize documentation:

The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history.

Think of Mechanize as a programmable Web browser controlled by Ruby. This powerful idea offers possibilities that go far beyond the relatively simple example I'll describe here.

A Simple Library

Our library consists of the following code:

require 'rubygems'
require 'mechanize'

module Thieme
  def get_ris url
    agent =  WWW::Mechanize.new
    page = agent.get url
    ris_link = page.links.text /[Bb]iblio/
    ris_url = "http://" + page.uri.host + ris_link.href

    agent.get_file ris_url
  end
end

After saving this code in a file called thieme.rb, we can test it on this Synthesis article with interactive ruby (irb):

$ irb
irb(main):001:0> require 'thieme'
=> true
irb(main):002:0> include Thieme
=> Object
irb(main):003:0> ris=get_ris 'http://www.thieme-connect.com/ejournals/abstract/synthesis/doi/10.1055/s-2007-966071'
=> "\nTY  - JOUR\nID  - 101055S2007966071\nAU  - Gil,Mar\355a Victoria\nAU  - Ar\351valo,Mar\355a Jos\351\nAU  - L\363pez,\323scar\nT1  - Click Chemistry - What?s in a Name? Triazole Synthesis and Beyond\nJO  - Synthesis\nPY  - 2007///\nIS  - 11\nSP  - 1589\nEP  - 1620\nER  - \n\n"
irb(main):004:0> ris.match /T1  - (.*)/
=> #
irb(main):005:0> title = $1
=> "Click Chemistry - What?s in a Name? Triazole Synthesis and Beyond"

Let's say that instead of a deep link to an article in the Thieme site we have a DOI. Can we still get the bibliographical citation?

irb(main):006:0> ris=get_ris 'http://dx.doi.org/10.1055/s-2007-966071'
=> "\nTY  - JOUR\nID  - 101055S2007966071\nAU  - Gil,Mar\355a Victoria\nAU  - Ar\351valo,Mar\355a Jos\351\nAU  - L\363pez,\323scar\nT1  - Click Chemistry - What?s in a Name? Triazole Synthesis and Beyond\nJO  - Synthesis\nPY  - 2007///\nIS  - 11\nSP  - 1589\nEP  - 1620\nER  - \n\n"
irb(main):007:0> ris.match /T1  - (.*)/
=> #
irb(main):008:0> title = $1
=> "Click Chemistry - What?s in a Name? Triazole Synthesis and Beyond"

It worked! Mechanize had no problem following the redirect from dx.doi.org. Similar results would be obtained with a Synlett article or DOI.

For this approach to be truly useful, our software would need to gracefully handle character encoding to avoid garbled strings such as "What?s".

How it Works

Our library relies on two important things being provided by the publisher: (1) a downloadable version of the RIS file for every article; and (2) a consistent way to access it across journals. By simply telling Mechanize to follow a link labeled as "Download bibliographical data", we can easily retrieve the full citation. Fortunately, nearly every scientific publisher follows this practice.

Conclusions

Just a few lines of Ruby code have solved a significant scientific information management problem, at least for one journal. A complete solution to the problem would require code for every scientific journal, a task well underway at CiteULike. While nothing here can pretend to be an end-user application, it's not difficult to imagine how to build one (or a few) using these basic concepts. But that's a story for another time.

Depth-First: Tag mechanize

Hacking ChemSpider: Query by SMILES and InChI with Ruby

The Library

Testing

Conclusions

Easily Convert Publisher URLs and DOIs to Bibliographical Citations: Synthesis, Synlett, Ruby, and Mechanize

What is Mechanize?

A Simple Library

How it Works

Conclusions