Hacking DOI: Interconvert Bibliographic References and DOIs with CrossRef and OpenURL

Rich Apodaca — Tue, 06 May 2008 15:50:00 -0400

Science is in the middle of a transition from print to the internet as the primary medium of communication. This transition, although a boon for many scientists, creates a host of problems for those dealing with scientific information. For example, how would you interconvert a DOI and its corresponding bibliographic reference?

A previous Depth-First article discussed a screen-scraping method as one solution. Unfortunately, this system relies on an intimate understanding of how individual publishers' Websites work, requires a different implementation for each publisher, and can break at any time without warning.

This article discusses a far more robust solution to the problem of interconverting bibliographic references and DOIs.

Background: OpenURL and CrossRef

CrossRef is the official DOI link registration agency for scholarly and professional publications. One of the less well-known services offered by CrossRef is a free, Web-based bidirectional DOI/bibliographic reference converter based on OpenURL.

A Simple Ruby Library

The following Ruby library is all we need to begin using CrossRef and OpenURL:

require 'rubygems'
require 'hpricot'
require 'open-uri'

module DOI
  # Convert a doi into a bibliographic reference.
  def biblio_for doi
    doc = Hpricot(open("http://www.crossref.org/openurl/?id=doi:#{doi}&noredirect=true&pid=ourl_sample:sample&format=unixref"))

    journal = (doc/"abbrev_title").inner_html
    year = (doc/"journal_issue/publication_date/year").inner_html
    volume = (doc/"journal_issue/journal_volume/volume").inner_html
    number = (doc/"journal_issue/issue").inner_html
    first_page = (doc/"pages/first_page").inner_html
    last_page = (doc/"pages/last_page").inner_html

    "#{journal} #{year}, #{volume}(#{number}) #{first_page}-#{last_page}"
  end

  # Convert a bibliographic reference into a DOI.
  def doi_for journal, year, volume, issue, page
    doc = Hpricot(open("http://www.crossref.org/openurl/?title=#{journal.gsub(/ /, '%20')}&volume=#{volume}&issue=#{issue}&spage=#{page}&date=#{year}&pid=ourl_sample:sample&redirect=false&format=unixref"))

   (doc/"doi").inner_html
  end
end

This code makes use of the excellent Ruby HTML parser library Hpricot.

Testing the Library

Saving the Ruby code to a file named doi.rb, we can test it using the interactive Ruby shell:

$ irb
irb(main):001:0> require 'doi'
=> true
irb(main):002:0> include DOI
=> Object
irb(main):003:0> biblio_for "10.1021/cr00032a009"
=> "Chem. Rev. 1994, 94(8) 2483-2547"
irb(main):004:0> doi_for "Chem. Rev.", 1994, 94, 8, 2483
=> "10.1021/cr00032a009"

Notice how the journal abbreviation Chem. Rev. was used; we'd get the same result if we used Chemical Reviews.

Of course, the implementation described here could be refined a lot. With a DOI, it's trivial to construct a URL to the example paper. But we could take it further than that. With some carefully crafted regular expressions, our doi_for method could accept a freeform bibliographical citation rather than separately identified fragments. From there we might start to think about creating living HTML and/or Wikis from old PDFs and Word documents.

With a little creative thought, other possibilities are well within reach.

Caveat

Before extensively experimenting with CrossRef's OpenURL system, you might want to sign up for a free account. CrossRef is understandably interested in tracking usage and this is their way to do it.

Conclusions

DOIs and traditional bibliographical citations now coexist in a variety of settings, from literature citation managers to journals themselves. Using CrossRef, OpenURL and a little bit of code, it's now possible to make a great deal more sense of it all.

Harvesting bibliographical citations must be one of the least sexy topics in cheminformatics. But as Google demonstrated (building on the approach taken by Science Citation Index), cataloging citation behavior leads to a unique and highly productive way to view many tough problems. Future articles will discuss how this might apply to cheminformatics.

Image Credit: ecstaticist

Hacking ChemSpider: Query by SMILES and InChI with Ruby

Rich Apodaca — Mon, 17 Sep 2007 08:19:00 -0400

Slowly but surely, cheminformatics Web APIs are starting to appear. What's the big deal, you may ask? By exposing Web APIs, service providers enable third parties to develop new applications that "mash up" functionality from two or more sites, or which take the original service in directions its founders never considered.

By way of Antony Williams' blog, I came across the announcement for the ChemSpider Web API. What can this API do for Web developers? To find out, let's write a small Ruby library.

The Library

Our library will accept a SMILES string or InChI identifier and returns a URL pointing to the corresponding ChemSpider compound summary page. Like previous Web API demos, this one uses the powerful Ruby library Mechanize, leading to very concise code:

require 'rubygems'
require 'mechanize'

module ChemSpider
  def url_for_inchi inchi
    agent = WWW::Mechanize.new
    page= agent.get "http://www.chemspider.com/inchi.asmx/InChIToCSID?inchi=#{inchi}"
    csid = (Hpricot(page.body)/"string").innerHTML

    csid == "" ? nil : "http://www.chemspider.com/RecordView.aspx?id=#{csid}"
  end

  def url_for_smiles smiles
    agent = WWW::Mechanize.new
    page= agent.get "http://www.chemspider.com/inchi.asmx/SMILESToInChI?smiles=#{smiles}"
    inchi = (Hpricot(page.body)/"string").innerHTML

    raise "Invalid SMILES: #{smiles}" if inchi == ""

    url_for_inchi inchi
  end
end

The url_for_inchi method directly uses the ChemSpider API to query by InChI. The url_for_smiles method first uses the ChemSpider API to convert a SMILES string to an InChI identifier, and then calls the url_for_inchi method.

Two points are worth noting. First, although for convenience the InChI identifier isn't escaped before being appended to the API URL, strictly speaking it should be. Second, both methods invoke the underlying Mechanize library Hpricot to parse the raw XML returned by ChemSpider.

Testing

Saving the above code to a file called chemspider.rb, we can get the URL to ChemSpider's benzene page from its InChI identifier via interactive Ruby (irb):

$ irb
irb(main):001:0> require 'chemspider'
=> true
irb(main):002:0> include ChemSpider
=> Object
irb(main):003:0> url_for_inchi "InChI=1/C6H6/c1-2-4-6-5-3-1/h1-6H"
=> "http://www.chemspider.com/RecordView.aspx?id=236"

We can work with SMILES strings just as easily as with InChIs:

$ irb
irb(main):001:0> require 'chemspider'
=> true
irb(main):002:0> include ChemSpider
=> Object
irb(main):003:0> url_for_smiles 'c1ccccc1'
=> "http://www.chemspider.com/RecordView.aspx?id=236"

Both the InChI and the SMILES string yield a URL pointing to the same Chemspider page for benzene.

Conclusions

Like most chemical databases, ChemSpider uses a compound summary page as a way of organizing the available resources for a given molecule. With a method in hand for accessing these pages based on arbitrary SMILES or InChIs, we can begin to think of manipulating ChemSpider independently of its current user interface. But that's a story for another time.

Depth-First: Tag hpricot

Hacking DOI: Interconvert Bibliographic References and DOIs with CrossRef and OpenURL

Background: OpenURL and CrossRef

A Simple Ruby Library

Testing the Library

Caveat

Conclusions

Hacking ChemSpider: Query by SMILES and InChI with Ruby

The Library

Testing

Conclusions