Hacking PubChem: Visually Inspect Results for CAS Number and Keyword Searches 1

Posted by Rich Apodaca Tue, 25 Sep 2007 14:55:00 GMT

A recent article described how PubChem could be used to quickly search for CAS numbers. Although useful, the approach is limited in that only an array of PubChem CIDs was returned. What would be really useful would be a simple way to create a report with entries hyperlinked into the PubChem site itself to aid in visual inspection. In this tutorial, we'll see how an HTML template and a few extra lines of code can do just that.

The Template

Ruby supports a number of HTML templating mechanisms. In this example, we'll use an ERB template resurrected from the Molbank graphical table of contents tutorial:

<html>
  <head>
    <title>
      <%= "PubChem Search for #{term}" %>
    </title>
  </head>
  <body>
    <h1>
      <%= "Search: #{term}" %>
    </h1>
    <table>
      <tr>
      <% col = 0 %>
      <% cids.each do |cid| %>
        <td>
          <% image = "http://pubchem.ncbi.nlm.nih.gov/image/imgsrv.fcgi?cid=#{cid}" %>
          <% summary = "http://pubchem.ncbi.nlm.nih.gov/summary/summary.cgi?cid=#{cid}" %>
          <a href="<%= summary %>">
            <img src="<%= image %>" border="2"></img>
          </a>
          <center>
            <span style="font-size: 8px">
              <a href="<%= summary %>"><%= "CID-#{cid}" %></a>
            </span>
          </center>
        </td>
        <% col += 1 %>
        <% if col > 5 %>
          <% col = 0 %>
          </tr>
          <tr>
        <% end %>
      <%end %>
      </tr>
    </table>
  </body>
</html>

The above template uses a search term and an array of CIDs to build a table of results. Each cell in the table contains a color 2D image and the CID, both hyperlinked into PubChem itself.

Saving this library to a file called template.rhtml is all we need to do.

The Library

The library is a modification of the one shown in the previous article in this series:

require 'rubygems'
require 'mechanize'
require 'erb'

module PubChemTerms
  def report term
    cids = get_cids term
    erb = ERB.new(IO.read("template.rhtml"))

    File.open "output.html", 'w+' do |file|
      file << erb.result(binding)
    end
  end

  def get_cids term
    agent = WWW::Mechanize.new
    page = agent.get "http://www.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pccompound&retmax=100&term=#{term}"

    (page.parser/"id").collect {|id| id.innerHTML}
  end
end

The method report accepts a search term and uses our template to render a report.

Testing

By saving the above library in a file called pubchem.rb, we can search by keyword via interactive ruby (irb):

$ irb
irb(main):001:0> require 'pubchem'
=> true
irb(main):002:0> include PubChemTerms
=> Object
irb(main):003:0> report 'esomeprazole'
=> #

This produces a file called output.html that can be viewed with any browser:

As in the original version of the library, we can also query by CAS number:

$ irb
irb(main):001:0> require 'pubchem'
=> true
irb(main):002:0> include PubChemTerms
=> Object
irb(main):003:0> report '119141-88-7'
=> #

Conclusions

The simple approach outlined here could be extended in many ways. For example, we could easily retrieve molfiles based on keyword or CAS number search. We could pipe queries together or work with query lists. We could blend in ChemSpider data. We could even build a simple Web application (with Rails) that returned customized reports. Mixing in Ruby CDK or Ruby Open Babel offers still more possibilities.

Increasingly, the most important question in cheminformatics is not "What can we build?", but rather "What should we build?" Success in this new world requires a much deeper understanding of how cheminformatics software is being used by real chemists and where it's not.

My InChI Runneth Over 2

Posted by Rich Apodaca Thu, 17 May 2007 12:59:00 GMT

The only solution to this problem I've found is to set the CSS overflow property to "scroll":

InChI=1/C50H70O14/c1-25(24-51)14-28-17-37(52)50(8)41(54-28)19-33-34(61-50)18-32-29(55-33)10-9-12-46(4)42(58-32)23-49(7)40(62-46)21-39-47(5,64-49)13-11-30-44(60-39)26(2)15-31-36(56-30)22-48(6)38(57-31)20-35-45(63-48)27(3)16-43(53)59-35/h9-10,16,24,26,28-42,44-45,52H,1,11-15,17-23H2,2-8H3/b10-9-

Cheminformatics for the Web: Convert SD Files to HTML with Ruby CDK

Posted by Rich Apodaca Mon, 13 Nov 2006 20:23:00 GMT

The Structure Data File (SDF) format is the de facto standard for cheminformatics data exchange. One of the problems that arises when working with SD Files, especially large ones like those distributed by PubChem, is "seeing" the structures they contain. Although commercial software packages are available for doing so, they are generally closed, unreasonably expensive, or overly complex. This article describes a simple solution to the SDF visualization problem that uses Open Source tools controlled from the elegant and agile Ruby programming language.

Cut to the Chase

This page shows the output produced by the software. You'll see a neatly arranged grid of colorful 2-D chemical structures in a Web page that was generated directly from a PubChem SDFGZ file. Each structure has a number below it, the PubChem Compound ID (CID). Both the structure and CID are hyperlinked to the Compound Summary page on PubChem. A partial screenshot is provided below.

Prerequisites

For this tutorial, you'll need Ruby CDK (RCDK). A recent article described the small amount of system configuration required for RCDK on Linux. Another article showed how to install RCDK on Windows.

Download the Software

The software described in this article can be downloaded here. Inflate this file and make it your working directory. You should see a 14 MB SDFGZ file, a RHTML template, and three Ruby files.

Ripping PubChem SD Files

The software is designed to work with PubChem SDFGZ files. The SDFGZ format simply results from the application of the gzip compression algorithm to an ordinary SD file.

Ripping the example SDFGZ file is just a matter of running test.rb:

$ ruby test.rb

You'll see some output indicating that various CIDs are being processed. On completion, the software has created a directory called rip containing a HTML file and an images directory.

The Little Engine That Could: CDK's StructureDiagramGenerator

If you've ever worked with PubChem's SD Files, you'll no doubt have noticed that the molfile section encodes all hydrogen atoms, which is not general practice. Rendering these hydrogens results in a very cluttered image.

To solve this problem, the software creates its graphics from the PUBCHEM_OPENEYE_CAN_SMILES field encoded by the SDFGZ file. This SMILES string is converted into a molecular representation and coordinates are assigned by CDK's StructureDiagramGenerator.

When an image can't be rendered in this way, it is left out. This was done for CIDs 18, 115, 147, 148, 222, and 223, for example. There are three common themes in these missing structures: metals, phosphorous, and molecules with a single heavy atom. The problem may, in fact, lie in the underlying Structure-CDK software, rather than with CDK. Stay tuned for more on this.

PubChem for Debugging

In developing this SD File Ripper program, I realized that it could be used as a powerful debugging tool. Notice how the missing structures (and their SMILES strings) can easily be examined via PubChem by clicking the empty cell. The alternative would have been for the program to spit out a list of SMILES that didn't process properly and to then try to construct a mental image of what this string represents. With PubChem, we do away with this tedium altogether.

I doubt the creators of PubChem envisioned this application of their work. Surely it's but one of many still to be discovered.

Another Cool Thing About Ruby: eRuby Templates

Our SDF Ripper program creates HTML output, something for which Ruby is well-suited through its eRuby ERB library. Among other uses, ERB enables Ruby code to be embedded within HTML. This inside-out scripting capability resembles that of other templating languages such as PHP, ASP, and JSP (ERB is used extensively by the Ruby on Rails web application framework). The file template.rhtml contains the rippers's ERB template. The separation of program logic from presentation makes it very simple to customize the appearance of the output.

Room to Grow

Our SDF Ripper only works with SDFGZ files from PubChem. The program is short enough that it should be simple to adapt it for your specific needs. It would not be much work at all, for example, to create an HTML table containing all fields encoded by the SDFGZ file. Similarly, adding support for non-compressed SD files is straightforward. If JavaScript is your medium, the possibilities become even more interesting. How about a pop-up menu showing an enlarged structure and data summary, a la Netflix when the user mouses over an image?

Paging is a technique that divides large Web pages into smaller pages linked to one another. For example, Google's search results are divided into groups of ten by default. Adding paging support to the software described here would also not be difficult, and would enable the convenient browsing of much larger datasets.

Other Software That Does This

I am aware of no product, commercial or otherwise, that performs the SDF to HTML conversion in the way shown here. SciTegic does offer an HTML table component as part of its Pipeline Pilot framework, but as far as I know, no standalone version is available.

Rajarshi Guha, among his many other interesting projects, has written a Java SDF to PDF convertor that uses CDK.

Conclusions

This article has demonstrated how the combination of RCDK and Ruby makes short work of converting the contents of an SD file into a Web-ready format. As usual, we've only scratched the surface of what's easily within reach. Watch for future articles to build on the concepts outlined here.