Depth-First

Introducing MX: Lightweight and Free Cheminformatics Tools for Java

Rich Apodaca — Fri, 21 Nov 2008 18:28:00 +0000

If you want to build cheminformatics software of any kind, you'll need a basic toolkit. Ideally, this toolkit contains all of the low-level functionality used over and over in your projects. Tools for building an in-memory molecular representation, exact- and substructure comparison, and reading/writing molfiles all fall into this category. Also ideally, this toolkit should be free. Not free in the sense of free to use if you work at a university, free to try, or even free to use provided that you make your changes public when you redistribute the toolkit. But free in the sense of "do whatever you want with it and all you have to do is include a copyright notice."

This article introduces MX, a suite of lightweight and free cheminformatics tools for Java designed to fill these needs.

Download

A Google Code page has been set up for MX. Both a source distribution and compiled jarfile representing MX in its current state can be downloaded.

A subsequent article will show how to get started with MX.

Origins: A Chemical Structure Editor for Web Applications

In 2007 my company, Metamolecular, set out to build a lightweight and easy-to-use chemical structure editor for Web applications. Realizing the increasing importance the Web would play as a chemical communication medium in the next decade, a truly Web-based, platform-independent alternative to ChemDraw and ISIS/Draw seemed to be a good direction to pursue. The resulting product became known as ChemWriter.

Minimizing deployment footprint was a key consideration with ChemWriter; the last thing a chemist using a Web site wants to spend his or her time doing is waiting around for a large applet to download. With many chemical structure editor applets available today, download times on the order of one minute or longer are not uncommon. This is simply unacceptable.

To create ChemWriter, an ultra-lightweight cheminformatics toolkit was need. How lightweight? We were targeting 100 KB for the complete editor. A good chemical structure editor is a fairly complex piece of UI software involving multiple drawing tools with state-dependent behavior, not to mention some fairly sophisticated vector graphics rendering and molfile input/output. The only way we could reach our 100 KB target for ChemWriter is if the basic cheminformatics toolkit were 20 KB or smaller.

At the time, there was no cheminformatics toolkit, free or otherwise, that could fill the need.

So it was created from scratch.

High Performance in ChemPhoto

Eventually, the same cheminformatics toolkit used in ChemWriter was adapted for ChemPhoto, the chemical structure imaging application.

ChemPhoto was designed to dynamically display 100,000 or more 2D chemical structures in a grid-like GUI using minimal memory. Rather than pre-loading all 100,000 molecule objects into memory, which would not be feasible on most systems, ChemPhoto uses a lazy approach in which an in-memory index of the target SD file is built. Every time a new structure needs to be displayed to the user during a scrolling event, it's created from scratch: the molfile text is loaded from disk, a molecule object is created, the molecule is rendered, and then the entire construct is thrown away.

The performance of ChemPhoto was so good, even though everything was being created on demand and immediately thrown away, that it appeared the cheminformatics toolkit being used had potential in high-performance situations as well.

Substructure Search and Mapping

Recently, Rajarshi Guha reported his port of the VF library to Java for use with the Chemistry Development Kit (CDK). This began a thought process starting with "how can it be improved" and leading to the conclusion that the creation of flexible, Java-centric substructure search utilities would offer the most bang for the buck. A subsequent article described a simple strategy that could be used to get there.

To implement this idea, a cheminformatics toolkit was needed. The one used successfully in ChemWriter and ChemPhoto was an ideal candidate.

The result, a complete substructure search and mapping utility built from scratch, is available in MX under the package com.metamolecular.mx.map.

Free to Use Anytime, Anyplace - No Strings Attached

Licenses can be a problem with nearly all open source cheminformatics toolkits. If your work is mostly done in an academic environment for free, you're likely to experience no problem at all. However, if you run a company that sells licenses to software containing code you'd rather not reveal to the world, the reciprocity provisions in licenses such as those in the GPL, Mozilla (MPL), and IBM (CPL) families lead to major problems.

The problem isn't so much the open source license itself - it's the fact that the original copyright owners either won't give their permission to dual-license their contributions, or in many cases, can't even be tracked down to ask.

This is an unacceptable position for a software distributor wanting to use open source as a cost-effective means to boost their developer productivity.

To address these issues, MX is being distributed under the extremely permissive MIT License. In a nutshell it says you are free to modify and incorporate MX into any software you distribute without any obligation to release a line of your own source code. It also says if MX doesn't do the job, you're on your own. And that's about all it says. Your only obligation is to include the original copyright notice on all copies or substantial portions of the software.

To my knowledge, only one major cheminformatics toolkit is licensed under an academic-style open source license - RDKit, which is licensed under the New BSD License.

Conclusions

A basic cheminformatics toolkit is a vital component of most chemistry-related software. For maximal cost-effectiveness as a software distributor, a free toolkit licensed under a permissive open source license is ideal. MX is a free and lightweight cheminformatics toolkit written in Java that has been used successfully in two commercial products.

Future articles will describe the many ways MX can be used and extended.

ChemPhoto Beta-2

Rich Apodaca — Thu, 20 Nov 2008 23:59:00 +0000

The second beta version of ChemPhoto, the chemical structure imaging application developed by Metamolecular, is now available for testing.

Thanks to detailed reports from several ChemPhoto Beta-1 testers, a number of issues were discovered and have been addressed by this newest release.

If you're interested in seeing how easy it can be to create high-quality 2D chemical structure images from your compound collection, please drop me a line.

SciFinder Web, Greasemonkey, and REST: Embracing Divergence in Chemical Information Systems

Rich Apodaca — Wed, 19 Nov 2008 17:41:00 +0000

Recently, Alain Borel posted a message to the CHMINF-L list describing his successful attempt to get links to external datasources to show up in SciFinder Web:

I'm dreaming of a world where chemical data is hyperlinked as thoroughly as text data is today (or even more)... here's my small contribution to this goal.

...

Basically, Greasemonkey allows you to rewrite the HTML content of a web page before it is rendered in the browser window. Thanks to this, I've been able to write a script that adds links to external databases through registry numbers. Currently, ChemSpider, Chempedia and Chemexper [links added] are supported - I also have a private version that links to our Intranet chemical stocks application. The pleasant side for those who worry about intellectual property is that neither side of the link needs to know what's on the other side, and even the plugin doesn't know what's inside the database records.

Greasemonkey is the client-side script plugin engine that enables users to change the appearance and content of any site. Although originally designed as a Firefox extension, as least some Greasemonkey scripts can be run in Safari and Chrome. A recent paper describes some of the potential for this form of scripting in the life sciences.

Alain's script, which can be freely downloaded, uses CAS numbers to link SciFinder records to the external databases. Although I don't have access to SciFinder Web, Alain's description makes it sound like each entry for a specific substance in SciFinder Web is given an additional set of links out to external datasources.

What's REST Got To Do With It?

One of Alain's external datasources is Chempedia. A unique feature of Chempedia is the way it exposes a an electronic paper trail for CAS numbers. Rather than just reporting a CAS registry number, it fully discloses which organization is asserting that a particular CAS number belongs with a structure.

For example, see this entry on [525-66-6].

Chempedia was designed from the start to apply the principles of REST. The big idea behind REST is that every resource on the Web, such as a CAS number, can be manipulated by exactly four methods: GET; PUT; POST; and DELETE.

The highly-desirable side-effect of designing Web sites around the concept of resources being acted on by exactly four methods is that sites applying this philosophy become orders of magnitude easier to mash up.

Alain's Greasemonkey script is an example of a client-side mashup. But how did he do it?

CAS Numbers are First-Class Citizens

Each CAS number on Chempedia is a resource that can be accessed by a URL taking the form:

http://chempedia.com/registry_numbers/REGISTRY_NUMBER

where REGISTRY_NUMBER is the CAS number of interest. For example, acetaminophen has the registry number [103-90-2] and it can be accessed with this URL:

http://chempedia.com/registry_numbers/103-90-2

If you request a CAS number that doesn't exist, you should receive a 404 error, although a bug in Chempedia is currently preventing that from happening.

To link SciFinder Web to Chempedia, Alain's user script simply looks for which CAS number the SciFinder page is talking about and constructs the RESTful URL. It doesn't get much simpler than that.

Conclusions

Chemistry is a large, established field. Not surprisingly, specialization is an essential part of being a chemist. It's therefore to be expected that chemical databases will diverge into a variety of specialized forms. One size will almost certainly not fit all.

We can deny this simple fact and build ever more complex and unusable chemical information systems. Or we can accept it and custom-build our services for the job at hand.

RESTful server architectures and mashups offer a powerful way to accomplish this goal.

Image Credit: betenoir

Substructure Search From Scratch in Java Part 1: The Atom Mapping Problem

Rich Apodaca — Mon, 17 Nov 2008 19:17:00 +0000

One of the most important capabilities in cheminformatics is mapping the atoms of a query structure onto the atoms of a target structure. Although useful in itself, the main value of atom mapping comes from the software that gets built on top of it: exact structure comparators, substructure search systems, and query atom/bond search systems such as SMARTS. The fundamental nature of atom mapping means that correctness, efficiency and adaptability are essential features of a good mapping implementation. Recently, a D-F article made the case that atom mapping software written in Java needs to be Java-centric to achieve these goals. This article, the first in a series that describes a complete substructure search system written in Java, takes the first step by offering some simple interface definitions and code for the atom mapping problem.

The Problem

Given a query molecule (query) and a target molecule (target), our atom mapping software needs to find ways to match the atoms of query onto target such that the mapping describes a substructure embedded in target. The software might stop at one mapping, continue on to find all of them, or stop at some point in the middle. It all depends on the specific cheminformatics problem we're trying to solve.

The Recursive Function

Our implementation will gradually build up an atom mapping by traversing the atoms of query in depth-first order and trying to map each found atom onto an atom in target. At each step in the process, we will have a partial atom map that maps some of the atoms in query onto target. That map, and any other information needed to complete the analysis will be kept in an instance of a class implementing the State interface.

A State will be manipulated by a recursive method, mapFirst that returns when the first atom map is found:

// create a list to hold atom maps
List<Map<Atom, Atom>> maps = new ArrayList<Map<Atom, Atom>>();

// create initial state
State state = ...; 

boolean mapFirst(State state)
{
  if (state.isDead())
  {
    return false;
  }

  if (state.isGoal())
  {
    maps.add(state.getMap());

    return true;
  }

  boolean found = false;

  while (!found && state.hasNextCandidate())
  {
    Match candidate = state.nextCandidate();

    if (state.isMatchFeasible(candidate))
    {
      State nextState = state.nextState(candidate);
      found = mapFirst(nextState);

      nextState.backTrack();
    }
  }

  return found;
}

Comparison of the mapFirst method to the pseudocode VF algorithm Match procedure given in the previous article shows some similarities. In fact, something similar to the mapFirst method forms the basis of many atom mappers in use today.

Although it may be clear from the code, it's worth re-iterating that each time mapFirst is recursively called, an attempt is made to branch off a new State that maps an additional pair of atoms from query to target. If that branch leads to a possible solution, it's followed. Otherwise the next possible mapping is explored.

The `State` Interface

The recursive mapFirst method determines all of the methods the State interface needs to define:

public interface State
{
  /**
   * Returns the current mapping of query atoms onto target atoms.
   * This map is shared among all states obtained through nextState.
   */
  public Map<Atom, Atom> getMap();

  /**
   * Returns true if another candidate match can be found or
   * false otherwise.
   */
  public boolean hasNextCandidate();

  /**
   * Returns the next candidate match.
   */
  public Match nextCandidate();

  /**
   * Returns true if the given match will work with the current
   * map, or false otherwise.
   */
  public boolean isMatchFeasible(Match match);

  /**
   * Returns true if all atoms in the query molecule have been
   * mapped.
   */
  public boolean isGoal();

  /**
   * Returns true if no match will come from this State.
   */
  public boolean isDead();

  /**
   * Returns a state in which the atoms in match have been
   * added to the current mapping.
   */
  public State nextState(Match match);

  /**
   * Returns this State's atom map to its original condition.
   */
  public void backTrack();
}

Finally, State uses an instance of the Match class:

public class Match
{
  private Atom query;
  private Atom target;

  public Match(Atom query, Atom target)
  {
    this.query = query;
    this.target = target;
  }

  public Atom getQueryAtom()
  {
    return query;
  }

  public Atom getTargetAtom()
  {
    return target;
  }
}

Conclusions

With just a few lines of Java, we've managed to reduce the fundamental cheminformatics problem of atom mapping to the far simpler problem of implementing the State interface.

How many ways are there to implement the State interface? Probably as many as there are subgraph isomorphism algorithms. Notice that the way we've set up the problem lets us use the same recursive method to test all State implementations, an essential prerequisite for benchmarking and optimization.

Future articles in this series will describe one way to implement the State interface.

Image Credit: Dollar Bin

One of These Things is Not Like The Others

Rich Apodaca — Thu, 13 Nov 2008 01:49:00 +0000

You can't get very far in cheminformatics without the ability to compare one molecule to another to find either an exact structure or substructure match. For example, if you want to build chemical databases, a good substructure matcher comes in very handy. As luck would have it, the substructure match problem (a variant of the subgraph isomorphism problem) is both computationally expensive and difficult implement. This article discusses one approach to the problem.

Background

Recently, Rajarshi Guha described some benchmarking studies suggesting that it was possible to greatly improve the speed of the Chemistry Development Kit (CDK) substructure matching code. His code employed the widely-used Ullmann algorithm.

There's just one problem: the Ullmann algorithm detects edge-induced isomporphisms. This means, for example, that if your query molecule is propane and your test molecule is cyclopropane, you won't find a match with an Ullmann-backed tool. I'm still not sure if it's possible to modify an Ullmann implementation to make its matches node-induced. Based on the implementations I've seen, the answer appears to be "no."

For substructure matching, we need an atom-induced isomorphism algorithm.

What's Wrong with Existing Implementations?

To begin with, it must be pointed out that working isomorphism code is valuable and hard-won.

Having said that, many Java implementations are written in a way that makes optimization difficult at best. Some start out as C code that then gets ported, mostly verbatim. Other are written with an understandable emphasis on speed over readability. For developers used to working with classes, objects, shallow loops, and short methods with expressive names, the impedance mismatch can be jarring to say the least.

Here's an example, taken from the CDK, that while functional, would take a great deal of time to understand well enough to change:

public static List makeAtomsMapOfBondsMap(List l, IAtomContainer g1, IAtomContainer g2) {
  if(l==null)
    return(l);
  List result = new ArrayList();
  for (int i = 0; i < l.size(); i++) {
    IBond bond1 = g1.getBond(((RMap) l.get(i)).getId1());
    IBond bond2 = g2.getBond(((RMap) l.get(i)).getId2());
    IAtom[] atom1 = BondManipulator.getAtomArray(bond1);
    IAtom[] atom2 = BondManipulator.getAtomArray(bond2);
    for (int j = 0; j < 2; j++) {
      List bondsConnectedToAtom1j = g1.getConnectedBondsList(atom1[j]);
      for (int k = 0; k < bondsConnectedToAtom1j.size(); k++) {
        if (bondsConnectedToAtom1j.get(k) != bond1) {
          IBond testBond = (IBond)bondsConnectedToAtom1j.get(k);
            for (int m = 0; m < l.size(); m++) {
              IBond testBond2;
              if (((RMap) l.get(m)).getId1() == g1.getBondNumber(testBond)) {
                testBond2 = g2.getBond(((RMap) l.get(m)).getId2());
                for (int n = 0; n < 2; n++) {
                  List bondsToTest = g2.getConnectedBondsList(atom2[n]);
                  if (bondsToTest.contains(testBond2)) {
                    RMap map;
                    if (j == n) {
                      map = new RMap(g1.getAtomNumber(atom1[0]), g2.getAtomNumber(atom2[0]));
                    } else {
                      map = new RMap(g1.getAtomNumber(atom1[1]), g2.getAtomNumber(atom2[0]));
                    }
                    if (!result.contains(map)) {
                      result.add(map);
                    }
                    RMap map2;
                    if (j == n) {
                      map2 = new RMap(g1.getAtomNumber(atom1[1]), g2.getAtomNumber(atom2[1]));
                    } else {
                      map2 = new RMap(g1.getAtomNumber(atom1[0]), g2.getAtomNumber(atom2[1]));
                    }
                    if (!result.contains(map2)) {
                      result.add(map2);
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  return (result);
}

VFLib

Rajarshi's implementation of substructure search was based on a Java port of the VFLib C++ library. VFLib was developed by an Italian group to compare the performance of the VF algorithm with that of Ullmann.

VFLib defines a single interface (State) that a variety of subgraph isomorphism matchers can implement in order to work interchangeably.

What makes this so interesting is that when you can boil a software problem down to implementing an interface, it can become orders of magnitude simpler. But more on that later.

Another interesting aspect of VFLib is that the code can be easily converted from an edge-induced implementation to a node-induced implementation. In other words, if we had a Java port of the VFLib2 code, we could begin to build families of Java-based substructure matchers that could be easily compared and optimized.

The View from 10,000 Feet

One of the difficult aspects of implementing subgraph isomorphism algorithms is dividing the process up into understandable chunks. One way forward might be to look for commonalities among all of the approaches currently used. What might those be? Here are some possibilities:

Recursion. At the heart of any implementation typically lives a method that repeatedly calls itself (without creating a stack overflow).
Gradual accumulation of state. What's that recursive method doing? Building up a map of the atoms from a query structure to a target structure, one pair of atoms at a time. Sometimes it fails and needs to go back to the last successful match. Sometimes it succeeds and needs to report that information to avoid accessing an out-of-bounds index. At every stage, the accumulated state must be sufficient to finish the mapping attempt.
Mapping comes for free. The implementation typically uses an internal map to keep track of what it's done, so getting one mapping (or more) of the query structure onto the target tends to be as easy as simply detecting that a match exists.
Optimization heuristics. Where to begin, what order to compare structural features, and what features should be compared anyway? The possibilities for taking advantage of simple optimization rules are significant. It should, therefore, be easy to run many implementations side-by-side in performance tests.

A paper describing the VF algorithm, and the way VFLib implements it is freely available.

In it, a high-level overview of the VF algorithm is presented:

PROCEDURE Match(s)
  INPUT: an intermediate state s; the initial state s0 has M(s0)=∅
  OUTPUT: the mappings between the two graphs
  IF M(s) covers all the nodes of G2 THEN
    OUTPUT M(s)
  ELSE
    Compute the set P(s) of the pairs candidate for inclusion in M(s)
    FOREACH (n, m)∈ P(s)
      IF F(s, n, m) THEN
        Compute the state s' obtained by adding (n, m) to M(s)
        CALL Match(s')
      END IF
    END FOREACH
     Restore data structures
  END IF
END PROCEDURE

The Match(s) procedure plays the role of recursive function, while s and s' play the dual roles of state accumulators and feature comparators.

VFLib, together with the paper describing it, does a good job of breaking the process up into manageable chunks from which unit tests, interface definitions, and ultimately working code can created in a variety of languages.

Conclusions

Substructure matching is one of the most difficult and most useful cheminformatics tasks. Although many Java cheminformatics toolkits support substructure search, their implementations can be difficult to understand, modify, and optimize. VFLib has some interesting features that could help to change that.

Image Credit: Santa Rosa OLD SKOOL

Casual Saturdays: Business Plan

Rich Apodaca — Sat, 08 Nov 2008 23:13:00 +0000

Building ChemWriter: What to Do When Requesting Applet Keyboard Focus Leads to Disappearing Popup Windows

Rich Apodaca — Thu, 06 Nov 2008 19:35:00 +0000

Recently a customer reported a problem in which mousing over an instance of the ChemWriter editor applet caused browser popup windows to disappear behind the parent window. Although many view browser popup windows as bad UI design, there are situations in which no alternative exists. This article describes the window focus problem in detail and outlines one solution.

Background

One of the ways ChemWriter makes chemists more efficient at drawing chemical structures is through the use of keyboard shortcuts. Rather than having to mouse back and forth between a tool palette and drawing canvas to put in atom labels, simply hover the mouse over the atom to label, and press a key on the keyboard. In addition to atom labels, there are keyboard shortcuts for chains (keys 1-9 while hovered over atom), for benzene rings ("a" key), and to edit bond order (1-3 while hovered over bond). A short movie explains the feature in more detail.

Although quite helpful to users, this feature requires some behind-the-scenes magic. Keyboard focus is one of those topics at the boundary between applet and browser for which very little documentation exists and, not surprisingly, one sees the most variation in platform and browser behavior.

The approach ChemWriter takes is to wait for a signal that the mouse cursor has entered the canvas area. When this happens, keyboard focus is requested through Component.requestFocus().

Scope of the Problem

It turns out that on Windows, Component.requestFocus() also causes the hosting window to be pulled to the top of the windows stack, explaining the behavior described above. On Linux and OS X, this doesn't happen, which is the behavior you'd expect.

All Windows browsers, except the much maligned Internet Explorer 6, show this behavior. This includes Internet Explorer 7, Firefox 3, and Google Chrome. Internet Explorer 8 beta 2 also shows the behavior, but only once per popup window.

Minimizing, then maximizing the popup window eliminated the problem some of the time. But a new popup window would still show the behavior.

The Solution

The root of the problem is that on Windows, keyboard focus is granted to an object regardless of whether the object's hosting browser window is focused. What's needed, therefore, is a way for the applet to implement a window focus check.

Apparently, nothing in the Applet API itself can solve this problem. The Applet, JApplet, and AppletContext classes only deal with much higher-level considerations.

However, it's possible to take advantage of support for LiveConnect technology on Windows, which is actually quite good. Using LiveConnect in combination with JavaScript's Document.hasFocus() method offers the makings of a solution. For example, the following code can be used as a starting point within a Java applet to determine if the containing browser window is focused:

netscape.javascript.JSObject js = netscape.javascript.JSObject.getWindow(this);
Object result = js.eval("document.hasFocus();");

if ("true".equals(result.toString()))
{
  requestFocusMethod();
}

Of course, requestFocusMethod() needs to be defined, we need to check for null, and we need to handle exceptions that could arise from a missing netscape.javascript package. You'll also need to ensure that non-windows browsers such as Safari/OS X never even try to execute the LiveConnect code due to a very buggy implementation.

Issues

I've seen mixed signals about the status of LiveConnect in the next major release of the Java plugin. Regardless of the specific way it's implemented, it seems safe to say that Java-Javascript communication is far too valuable to abandon. The only question is what form support for this feature will take going forward.

Conclusions

ChemWriter's ability to accept keyboard input is a helpful user interface feature, but one that resulted in window focus issues on Windows. Using LiveConnect in combination with some simple JavaScript in the focus-management code offered an effective solution.

Billions and Billions

Rich Apodaca — Mon, 03 Nov 2008 18:00:00 +0000

One of the things that makes organic chemistry such a fascinating and useful subject (not to mention profitable) is the way mind-boggling levels of diversity arise from the application of very simple rules.

In 1924, the American chemist Eugene Markush was awarded a new kind of patent (US 1,506,316). Rather than claiming manufacturing processes listing specific input materials, Markush claimed processes listing families of compounds as inputs - and pretty large families at that. For example:

The process for the manufacture of dyes which comprises coupling with a halogen-substituted pyrazolone, a diazotized unsulfphonated material selected from the group consisting of aniline, homologues of aniline and halogen substituted products of aniline.

The phrase "selected from the group" has since become the signature of a special kind of patent claim called a Markush claim. The approach significantly influenced the way chemical intellectual property was created and protected.

A Markush structure is a special kind of chemical structure in which variables are used in place of specific substituents. It turns out that even the most basic Markush structures can specify very large numbers of specific chemical structures. For example, consider a Markush structure specifying all halogenated naphthalenes:

where X = H, F, Cl, Br, I

To a first approximation, we can calculate the number of compounds specified by this structure with the formula:

5 x 5 x 5 x 5 x 5 x 5 x 5 x 5 = 5⁸ = 390,625

To get the real number of halogenated naphthalenes, which would be less than 390,625, we'd need to account for symmetry of the enumerated structures.

Applying a Markush perspective to cheminformatics leads to some interesting product ideas. For example, a few years ago Coalesix released a computational tool called Mobius (screenshots) designed specifically to work with very large families of compounds encoded as Markush structures. And for years, patent databases such as MARPAT have been capable of searching the Markush claims of patents.

Given the centrality of Markush structures to the theory, practice, and law of modern industrial chemistry, it may be worthwhile to consider ways of incorporating the Markush perspective in to existing, or new, cheminformatics products and services.

Image Credit: Space Ritual

A Simple Vector Graphics API for Chemical Structure Output Part 1: In Search of a Simplifying Approach for ChemPhoto

Rich Apodaca — Fri, 31 Oct 2008 18:25:00 +0000

One of the main design goals of ChemPhoto, the chemical structure imaging application, was to support all Web-relevant image output formats, both vector-based and pixel-based. Like most things in software development, there are far more approaches that add complexity to this problem than there are approaches that remove it. And for some reason, the complexity-reducing methods tend to be the last to be considered. This article, the first in a series, will discuss how ChemPhoto simplifies the problem of supporting multiple chemical structure image output formats from a common representation.

The Problem in a Nutshell

ChemPhoto uses an internal representation of molecular structure based closely on the industry-standard MDL molfile format. Given this representation, ChemPhoto needs to be able to write a variety of vector- and raster-based image formats. Raster formats are fortunately limited to PNG and JPG, which are supported directly by the standard Java library.

Vector formats, on the other hand are more diverse and less accessible. Currently, ChemPhoto supports Scalable Vector Graphics (SVG) and Encapsulated PostScript (EPS). Complete support for Adobe Flash (SWF) output is expected soon. Proof of concept for Microsoft's Vector Markup Language (VML) has already been demonstrated. Support for Adobe Acrobat format, through the iText library is anticipated. Last but not least is Java2D itself for use in Swing components such as renderers and editors.

Clearly, supporting all of these formats requires rendering code that is minimally coupled to the underlying display system. But how to do this in practice?

The Batik Approach: Extend Graphics2D

Batik is a widely-used library for creating and processing SVG in Java. At its core is the SVGGraphics2D class which extends Graphics2D, overriding many of its methods in the process. The idea seems simple enough - create your drawing code using the Java2D API like you normally would. When you want to generate SVG, just pass an instance of SVGGraphics2D and then read out the SVG document using stream method.

The problem with this approach is that every new image output format to be supported needs to extend Graphics2D and essentially re-implement most of its methods. Graphics2D is a large and complex class with many associated helper classes. Just knowing when you've completely covered the API is a major challenge, aside from the even bigger challenge of implementing the overridden methods.

Fine, you might say, given that so many SVG interconverters exist, why not just use SVG (created by Batik) as the universal interconversion format and get a third-party-library to convert SVG into other vector formats?

This approach is appealing in principle, but fails in practice. Many SVG implementations are partial at best - and many lack the documentation that would warn that a problem might exist with the exact form of SVG you're using. For example, in an early iteration of ChemPhoto, Batik was used to create SVG from some representative chemical structures. Unfortunately, the way Batik represented path data was not fully interpreted by any of the SVG->SWF converters that were examined. The result was bumpy instead of smooth curves for atom labels, and other unacceptable abnormalities.

Finally, after spending some time reading J. David Eisenberg's excellent book about SVG, it became clear that for drawing 2D chemical structures and even reactions and reaction schemes, only a fraction of the SVG specification was relevant.

In this case, Batik, and its approach of extending Graphics2D was simply overkill that made the problem more complex than it needed to be.

A Better Approach: Create a Custom Vector Graphics Interface

Batik has the right idea: isolate drawing code from the specific format being generated. The problem is that the Graphics2D class wasn't really designed for this purpose. For one thing, it's a concrete class that inherits from another concrete class. And as mentioned before, Graphics2D a very complex class with many dependencies.

How can we create a simple vector graphics API tailored to chemical structure image creation, which is easily re-implemented, and which works with the existing Java2D API?

Part 2 of this series will describe one approach.

Conclusions

Creating the ChemPhoto rendering engine has been an evolutionary process. It started with the idea of directly using the Graphics2D class in rendering code, but has since moved on to the definition of a vector graphics abstraction layer to simplify the addition of new image formats.

I'd like to thank those beta testers who have already offered valuable feedback on ChemPhoto. If you'd like an unlimited 30-day trial for yourself, please drop me a line.

Image Credit: estherase

Fast Substructure Search Using Open Source Tools Part 6: Modelling a One-To-Many Relationship Between Fingerprints and Compounds in Ruby

Rich Apodaca — Wed, 29 Oct 2008 17:15:00 +0000

We can think of a fingerprint as a bucket into which every molecule in the universe can be reproducibly placed. Each molecule will belong to a single bucket, but each bucket may contain any number of molecules. In other words, there exists a one-to-many relationship between a fingerprint and its associated molecules. The previous article in this series discussed how to model this relationship using SQL. This article will take the idea one step further by describing one way to model this relationship in Ruby.

All Articles in this Series:

Part 1: Fingerprints and Databases
Part 2: Fingerprint Screen With SQL
Part 3: A CRUD API for Fingerprints in Ruby
Part 4: Creating Fingerprints from Chemical Structures
Part 5: Relating Molecules to Fingerprints with SQL
Part 6: Modelling a One-To-Many Relationship Between Fingerprints and Compounds in Ruby

SQL Recap

So far, we've set up a fingerprints database:

mysql> describe fingerprints;
+--------+---------------------+------+-----+---------+----------------+
| Field  | Type                | Null | Key | Default | Extra          |
+--------+---------------------+------+-----+---------+----------------+
| id     | int(11)             | NO   | PRI | NULL    | auto_increment | 
| byte0  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte1  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte2  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte3  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte4  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte5  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte6  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte7  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte8  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte9  | bigint(64) unsigned | YES  |     | 0       |                | 
| byte10 | bigint(64) unsigned | YES  |     | 0       |                | 
| byte11 | bigint(64) unsigned | YES  |     | 0       |                | 
| byte12 | bigint(64) unsigned | YES  |     | 0       |                | 
| byte13 | bigint(64) unsigned | YES  |     | 0       |                | 
| byte14 | bigint(64) unsigned | YES  |     | 0       |                | 
| byte15 | bigint(64) unsigned | YES  |     | 0       |                | 
+--------+---------------------+------+-----+---------+----------------+
17 rows in set (0.00 sec)

This database contains a single (empty) fingerprint:

mysql> select * from fingerprints;
+----+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+--------+--------+--------+--------+--------+--------+
| id | byte0 | byte1 | byte2 | byte3 | byte4 | byte5 | byte6 | byte7 | byte8 | byte9 | byte10 | byte11 | byte12 | byte13 | byte14 | byte15 |
+----+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+--------+--------+--------+--------+--------+--------+
|  1 |     0 |     0 |     0 |     0 |     0 |     0 |     0 |     0 |     0 |     0 |      0 |      0 |      0 |      0 |      0 |      0 | 
+----+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+--------+--------+--------+--------+--------+--------+
1 row in set (0.00 sec)

We've also set up a compounds database containing a foreign key (fingerprint_id) into the fingerprints table:

mysql> describe compounds;
+----------------+---------+------+-----+---------+----------------+
| Field          | Type    | Null | Key | Default | Extra          |
+----------------+---------+------+-----+---------+----------------+
| id             | int(11) | NO   | PRI | NULL    | auto_increment | 
| fingerprint_id | int(11) | YES  |     | NULL    |                | 
| smiles         | text    | YES  |     | NULL    |                | 
+----------------+---------+------+-----+---------+----------------+
3 rows in set (0.00 sec)

In this hypothetical example, the compounds database is populated by two molecules, benzene and bromobenzene, both of which share the same fingerprint:

mysql> select * from compounds;
+----+----------------+------------+
| id | fingerprint_id | smiles     |
+----+----------------+------------+
|  1 |              1 | c1ccccc1   | 
|  2 |              1 | c1ccccc1Br | 
+----+----------------+------------+
2 rows in set (0.00 sec)

Adding the Ruby Layer

In Part 3, we created a CRUD API for fingerprints in Ruby. We now need to modify the class we created there, Fingerprint, to make it aware of the Compounds it will be associated with.

For brevity, you can view the updated Fingerprint class here. The main change has been to add a single line of code that tells Fingerprint that it's now associated with a class called Compound:

  has_many :compounds

All that remains is to bring the Compound class into being:

require 'rubygems'
require 'active_record'
require 'fingerprint'

ActiveRecord::Base.establish_connection(
  :adapter    => 'mysql',
  :host       => 'localhost',
  :username   =>  'root',
  :password   =>  '',
  :database   =>  'compounds'
)

class Compound < ActiveRecord::Base
  belongs_to :fingerprint
end

The belongs_to line is the counterpart to Fingerprint's has_many line. Together, both Fingerprint and Compound create a system in which each Fingerprint can reference multiple Compounds and each Compound references one Fingerprint. Let's test this with interactive Ruby:

$ irb
irb(main):001:0> require 'fingerprint'
=> true
irb(main):002:0> f=Fingerprint.find 1
=> #<Fingerprint id: 1, byte0: 0, byte1: 0, byte2: 0, byte3: 0, byte4: 0, byte5: 0, byte6: 0, byte7: 0, byte8: 0, byte9: 0, byte10: 0, byte11: 0, byte12: 0, byte13: 0, byte14: 0, byte15: 0>
irb(main):003:0> f.compounds
=> [#<Compound id: 1, fingerprint_id: 1, smiles: "c1ccccc1">, #<Compound id: 2, fingerprint_id: 1, smiles: "c1ccccc1Br">]

Looks good. Our code has made the correct association between a Fingerprint and its Compounds. What about the other way around?

$ irb
irb(main):001:0> require 'compound'
=> true
irb(main):002:0> c=Compound.find 1
=> #<Compound id: 1, fingerprint_id: 1, smiles: "c1ccccc1">
irb(main):003:0> c.fingerprint
=> #<Fingerprint id: 1, byte0: 0, byte1: 0, byte2: 0, byte3: 0, byte4: 0, byte5: 0, byte6: 0, byte7: 0, byte8: 0, byte9: 0, byte10: 0, byte11: 0, byte12: 0, byte13: 0, byte14: 0, byte15: 0>

As expected, the first Compound became associated with the correct Fingerprint.

Conclusions

Our system can now store and query molecular fingerprints in a relational database. It also associates multiple compounds with each fingerprint.

We have a complete fingerprint screening system, but not a substructure search system.

What's missing? For one thing, we'd need a way to perform atom-by-atom searches (ABAS) of all candidate structures after the fingerprint screening process is complete. Recall that just because a query fingerprint matches a candidate fingerprint doesn't necessarily mean that a substructure match has been found.

We'd also need a way to conveniently get real compounds with real fingerprints into our database. Only then would we be able to test the chemical validity of substructure queries.

The remaining articles in this series will discuss approaches to each of these requirements.

Image Credit: leeechy

Depth-First

Introducing MX: Lightweight and Free Cheminformatics Tools for Java

Download

Origins: A Chemical Structure Editor for Web Applications

High Performance in ChemPhoto

Substructure Search and Mapping

Free to Use Anytime, Anyplace - No Strings Attached

Conclusions

ChemPhoto Beta-2

SciFinder Web, Greasemonkey, and REST: Embracing Divergence in Chemical Information Systems

What's REST Got To Do With It?

CAS Numbers are First-Class Citizens

Conclusions

Substructure Search From Scratch in Java Part 1: The Atom Mapping Problem

The Problem

The Recursive Function

The State Interface

Conclusions

One of These Things is Not Like The Others

Background

What's Wrong with Existing Implementations?

VFLib

The View from 10,000 Feet

Conclusions

Casual Saturdays: Business Plan

Building ChemWriter: What to Do When Requesting Applet Keyboard Focus Leads to Disappearing Popup Windows

Background

Scope of the Problem

The Solution

Issues

Conclusions

Billions and Billions

A Simple Vector Graphics API for Chemical Structure Output Part 1: In Search of a Simplifying Approach for ChemPhoto

The Problem in a Nutshell

The Batik Approach: Extend Graphics2D

A Better Approach: Create a Custom Vector Graphics Interface

Conclusions

Fast Substructure Search Using Open Source Tools Part 6: Modelling a One-To-Many Relationship Between Fingerprints and Compounds in Ruby

SQL Recap

Adding the Ruby Layer

Conclusions

The `State` Interface