Depth-First: Tag rest

Just a Flesh Wound

Rich Apodaca — Wed, 30 Apr 2008 18:24:00 -0400

SEMANTIC KNIGHT: None shall pass without formally defining the ontological meta-semantic thingies of their domain something-or-others!

HACKER: What?

SEMANTIC KNIGHT: None shall pass without using all sorts of semantic meta-meta-meta-stuff that we will invent Real Soon Now!

HACKER: I have no quarrel with you, good Sir Knight, but I must get my work done on the Web. Stand aside!

SEMANTIC KNIGHT: None shall find anything on the Internet without semantic metadata!

HACKER: So be it!

HACKER and SEMANTIC KNIGHT: Aaah!, hiyaah!, etc.

[HACKER chops the SEMANTIC KNIGHT's first argument off by building efficent statistical/heuristic search engines]

HACKER: Now stand aside, worthy adversary.

SEMANTIC KNIGHT: 'Tis but a scratch.

HACKER: A scratch? Your argument has been cut off!

SEMANTIC KNIGHT: No, it isn't.

HACKER: Well, what's that, then?

SEMANTIC KNIGHT: I've had worse. None shall have an effective syndication network without RDF Site Summaries!

[clang]

Hiyaah!

[clang]

Aaaaaaaah!

[HACKER chops the SEMANTIC KNIGHT's second argument off by building the blogs/RSS/Aggregators/Bloglines/etc. network ]

HACKER: Victory is mine!

SEMANTIC KNIGHT: Have at you!

[kick]

HACKER: Eh. You are indeed brave, Sir Knight, but the fight is mine.

SEMANTIC KNIGHT: Oh, had enough, eh?

HACKER: Look, you stupid &^%$# You've got no arguments left.

SEMANTIC KNIGHT: Yes, I have.

HACKER: Look!

SEMANTIC KNIGHT: Just a flesh wound.

[kick]

HACKER: Look, stop that.

SEMANTIC KNIGHT: You won't be able to get machine-machine services without an ontology to formally describe all the relationships!

[kick]

HACKER: Right!

[whop]

[HACKER chops the SEMANTIC KNIGHT's third argument off by building SOAPy and RESTful services with only implicit semantic descriptions]

SEMANTIC KNIGHT: Right. I'll do you for that!

HACKER: You'll what?

SEMANTIC KNIGHT: Come here!

HACKER: What are you going to do, bleed on me?

SEMANTIC KNIGHT: I'm invincible!

HACKER: You're a looney.

SEMANTIC KNIGHT: The SEMANTIC Knight always triumphs! Have at you! Come on, then. I have an battalion of KR theorists on my side!

[whop]

[HACKER chops the SEMANTIC KNIGHT's last argument off with an army of actual code writers]

SEMANTIC KNIGHT: Oh? All right, we'll call it a draw.

HACKER: Come on, folks, let's go.

SEMANTIC KNIGHT: Oh. Oh, I see. Running away, eh? You yellow ^&^%$s! Come back here and take what's coming to you. I'll bite your legs off!

-Michael Champion, xml-dev list

Streamlining Cheminformatics on the Web: Let InChI Do the Heavy Lifting and Get Some REST

Rich Apodaca — Mon, 01 Oct 2007 10:53:00 -0400

A recent Depth-First article discussed the advantages of minimal Web APIs in Cheminformatics. Recently, Antony Williams unveiled some simplified ChemSpider URL schemes, mainly from the perspective of enabling Google indexing. However, it's possible to take this scheme much, much further. Here I present a proposal for radically simplifying (and unifying) the development of cheminformatics Web APIs and the software that interacts with them.

The New ChemSpider URLs

ChemSpider now has several new kinds of URLs. For the purposes of this article, the most interesting of these are of the format:

These URLs may seem unremarkable, but there's much more than meets the eye. They let anonymous developers query ChemSpider about specific substances - without needing to know much at all about how ChemSpider itself works. Goodbye API. Goodbye API support. Goodbye API documentation. Goodbye angle brackets. Hello to getting stuff done. It's all very RESTful. Well, at least it could be that way with some minor modification.

Some Recommendations

ChemSpider hasn't quite reached that place where the API just disappears. The problem is that the ChemSpider URLs listed above point to query results pages, not compound summary pages. Were these URLs to redirect to a summary page, we could construct the following URLs to extract ChemSpider resources (I've replaced the '=' sign with a '/' for simplicity):

.../InChIKey/DEIYFTQMQPDXOT-RERXVCSDCZ Get all resources for the molecule identified by the given InChIKey - i.e., "Compound summary page"
.../InChIKey/DEIYFTQMQPDXOT-RERXVCSDCZ/molfile.mol Get the molfile for the molecule identified by the given InChIKey
.../InChIKey/DEIYFTQMQPDXOT-RERXVCSDCZ/small_image.png Get the small image for the molecule indentified by the given InChIKey.
.../InChIKey/DEIYFTQMQPDXOT-RERXVCSDCZ/large_image.png Get the large image for the molecule identified by the given InChIKey.
.../InChIKey/DEIYFTQMQPDXOT-RERXVCSDCZ/citations.xml Get the list of citations for the molecule identified by the given InchIKey, in XML format.

Jane, a developer building Web applications on top of this new ChemSpider API, would immediately notice that things just work. Let's say her online database stores IC₅₀s at the dopamine D₂ receptor. On the summary page for each molecule, she wants to link out to the ChemSpider compound summary page, if available. She would simply construct the InChIKey on her server, build the needed ChemSpider URL and GET it. An HTTP 404 would indicate no molecule with that Key exists on ChemSpider and so no link would be shown. An HTTP 200 would indicate ChemSpider has the molecule, and so the link would appear.

Conclusions

It would be interesting enough if ChemSpider adopted a system like that described here. But the real power of this approach would emerge if multiple Web services were to adopt it. By following a simple set of conventions, these services would enable third party developers to elegantly mashup all manner of cheminformatics resources into applications unimaginable today.

Technically, there's nothing that prevents this system from being implemented on every free chemistry database in existence today. However, doing so would transfer a significant degree of control from service operators to third-party developers. Not all providers will be comfortable with that idea.

Cheminformatics Web service providers need to carefully consider whether they're trying to develop a platform or an integrated service. As history has shown, the strategies, and upside potential, for each approach can differ dramatically.

The Best API May Be No API At All: PubChem and PDB

Rich Apodaca — Mon, 13 Aug 2007 07:55:00 -0400

Both PubChem and the Protein Data Bank (PDB) maintain vast collections of molecular data. Individual users are free to view and search these collections via standard Web browsers. But what are the options if you're developing software to interact with these databases?

Various application programming interfaces (APIs) are available for accessing PubChem and PDB records. For example, PubChem recently introduced its Power User Gateway (PUG), an XML-based query language. But writing APIs is extremely difficult; reconciling the need for simplicity with the need for rich functionality is a tough balancing act. Where do you draw the line?

Recently, Bosco described a remarkably short method to retrieve PDB records using nothing more than standard Python. Given the similarities between Python and Ruby, it seemed reasonable that his method could be adapted to Ruby.

The following Ruby library accepts a PDB identifier and returns the corresponding PDB record:

require 'net/http'

module PDB
  # Returns a PDB record for the given id
  def self.get_record id
    Net::HTTP.get_response('www.rcsb.org', "/pdb/files/#{id}.pdb").body
  end
end

Notice how the business end of this library is nothing more than a single line of Ruby code. The library can be tested by saving it in a file called pdb.rb and invoking interactive Ruby (irb):

$ irb
irb(main):001:0> require 'pdb'
=> true
irb(main):002:0> puts PDB::get_record('1hpn')
HEADER    GLYCOSAMINOGLYCAN                       17-JAN-95   1HPN
TITLE     N.M.R. AND MOLECULAR-MODELLING STUDIES OF THE SOLUTION
TITLE    2 CONFORMATION OF HEPARIN

[truncated]

Several months ago, a D-F article described a related, but somewhat lengthier approach to retrieving PubChem molfiles. Using the same approach we used for PDB, we can create the world's shortest PubChem library:

require 'net/http'

module PubChem
  # Returns a molfile for the given PubChem CID
  def self.get_molfile cid
    Net::HTTP.get_response('pubchem.ncbi.nlm.nih.gov', "/summary/summary.cgi?cid=#{cid}&disopt=DisplaySDF").body
  end
end

This library can be tested by saving it in a file called pubchem.rb followed by running irb:

$ irb
irb(main):001:0> require 'pubchem'
=> true
irb(main):002:0> puts PubChem::get_molfile('969472') #eszopiclone (Lunesta)
969472
  -OEChem-08130700422D

 44 47  0     1  0  0  0  0  0999 V2000
    9.2619   -2.2732    0.0000 Cl  0  0  0  0  0  0  0  0  0  0  0  0

[truncated]

Both of these Ruby libraries leverage one the most versatile and robust protocols ever developed: plain old http. The last few years have witnessed a renaissance in using bare http as platform for building simplified yet powerful Web APIs with less software. Referred to as REST, the approach has gained traction partly in response to the wasteful complexities introduced by various XML-based approaches. Although slow to catch on in cheminformatics, REST has enormous potential in unifying a diverse array of isolated database systems.

One limitation of the approach described here is that the PubChem (or PDB) folks may get upset if you use it a lot. For example, if you examine the PubChem robots.txt file, you'll notice that access to the summary.cgi resource, which our library makes use of, is prohibited to robots:

...

User-agent: *

...
Disallow: /summary/summary.cgi
...

What makes a "robot" and does your software qualify for exclusion? The answer is not enirely clear-cut, especially in the era of browser-side scripting.

Regardless, it looks like PubChem's policy was put in place in 2004, long before PubChem had experience with usage patterns for its service. It may be that this restriction could be relaxed without adversely affecting PubChem's ability to operate efficiently. It may even be possible to offer a low-level http retrieval method alongside PubChem's PUG interface on a machine dedicated to automated queries (i.e., Entrez eUtils).

As developers, our mission is to deliver functionality, not to write software. We should extract every possible ounce of value from established protocols and APIs before writing a single line of additional code. REST, and the creative use of good old http, are powerful tools to do so.

Image Credit Dru!

RESTful Cheminformatics

Rich Apodaca — Wed, 30 May 2007 10:42:00 -0400

Names are powerful. For example, if you work in a field that has a well-recognized title, it's easy to talk to non-experts about what you do. Doctors, lawyers, NASCAR drivers, actors, and airline pilots all fall into this category. If, on the other hand, you're a medicinal chemist - well, you've got your work cut out for you.

Names are especially important in software design. Many of the toughest design problems occur because we fail to see an underlying concept and name it. Unfortunately, our natural impulse in these situations is to get busy creating complexity (or just give up). But if we allow ourselves to take a few steps back, more often than not we find a central concept waiting to be named.

Representational State Transfer (REST) has received a lot of attention lately, and for good reason. REST is fundamentally about discovering, naming, and exposing the elements of a problem domain. Object-Oriented designers have been doing this for decades. With REST it's possible to do the same thing with Internet resources, and achieve many of the same benefits.

To draw an analogy between REST and Object-Oriented design, resources are to the Internet what objects are to software. Resources have names. They have capabilities and well-defined behavior. They can be nested, extended, created, queried, and destroyed. They act as nouns upon which a limited number of verbs can act. Most importantly, resources have consistent and discoverable behavior.

REST's "big idea" is that the HTTP protocol already provides all the verbs necessary to operate on resources: GET, PUT, POST, and DELETE. If you're willing to accept the notion that (almost) everything in the world can be modeled as resources being acted on by those four verbs, your life as a Web developer suddenly gets much simpler.

The technical aspects of REST have been covered in many excellent blogs, articles, and talks. In particular, the release of Ruby on Rails 1.2 has lead to a flood of excellent material. Two works I've found especially helpful are:

David Heinemeier Hanson's 2006 Rails Conf Presentation: The slide deck for this presentation is essential.
RESTFul Rails Development: Written by Ralf Wirdemann and Thomas Baustert, this free booklet describes REST and Rails in much more detail than the Agile Book.

What does REST have to do with Cheminformatics? Quite a lot. As more and more free chemistry services, including free databases, become available online, lack of interoperability will start to become very painful - not just to developers, but to chemists themselves. There are many valid approaches to solving this problem. REST may well be the most workable.