Depth-First: Tag opensource

PerlMol: A Case Study in Open Source Cheminformatics Software

Rich Apodaca — Thu, 15 Nov 2007 09:49:00 -0500

How does open source software happen? Although many factors come into play, the majority of answers seem to revolve around a simple theme: developers building solutions to fill their own needs. Yet only a fraction of these solutions end up becoming open source software. And only a fraction of those end up being used by a wider audience. What's the key ingredient? There's still a lot to learn from studying individual cases.

Readable discussions about the origins of specific open source projects are pretty rare, but those dealing with the origins of open source cheminformatics software are more uncommon still. So it was with great interest that I came across Ivan Tubert-Brohman's account of how PerlMol was created.

PerlMol is an open source "collection of Perl modules for cheminformatics and computational chemistry." Many software packages fit into this category, and some of them are open source, so why write another? For Tubert-Brohman, the deciding factor was being able to work in his preferred environment, Perl:

I was surprised that CPAN [The Comprehensive Perl Archive Network] was sorely lacking in terms of modules for chemistry. The only available modules were Chemistry::Element, which allows you to convert between atomic number, element symbol, and element name and store other elemental information; and Chemistry::MolecularMass, which calculates the mass from the molecular formula. There were no modules that actually dealt with the structure of molecules. While some of the options in other languages are not bad, I was looking for something with the simplicity and conciseness of Perl that could allow me to write "chemical one-liners" to solve small problems very quickly, without having to compile anything. Hence, PerlMol was born.

The elimination of the need to compile, and relaxed syntaxes that promote succinct code are two of the biggest reasons to try a cheminformatics scripting environment.

There's a lot of great software still to be written in cheminformatics, and some of it will be open source. Although open sourcing that side project you've been working on may not be the best option for your career or your company, studying case studies like that of PerlMol gives plenty of food for thought.

The Business Case for Open Source and the Small Company

Rich Apodaca — Fri, 09 Nov 2007 09:37:00 -0500

Few would argue against small companies using open source software - indeed many owe their very existence to it. But what real, tangible good can come from a small company releasing open source software?

Signal to Noise, the company blog of 37signals, offers a worthwhile perspective on this issue. To summarize the business case:

Certain kinds of software, like infrastructure software, take vast amounts of time and resources to get right - something that few small companies can afford. Open sourcing can accelerate the process.
Open sourcing provides a public arena in which your own company's developers can learn from other great developers.
That public arena provides unique access to a pool of smart, motivated developers - and offers a way to evaluate their work before even deciding to interview them.
Open source generates press attention and goodwill from potential customers.

And about the elephant in the room:

A big fear that a lot of people have is that they’ll somehow be giving away their secret sauce. Unless your actual product is what you’re open sourcing, it really doesn't matter (and there are even plenty of examples of that working well). It’s unlikely that the piece of code that’s only seen internal development is such a silver bullet that you’re going to outshine your competition by its use alone.

The distinction between infrastructure software and a company's secret sauce is particularly important.

By just about any standard, 37signals is a leader in the deliberate use of open source software to achieve business objectives. We can all learn from them.

Building Rubidium: Creating a RubyForge Project Space

Rich Apodaca — Fri, 26 Oct 2007 10:21:00 -0400

Recent articles have discussed Rubidium, the cheminformatics toolkit for Ruby. In this article, the first in a series, I'll go beyond the Ruby code to discuss the technical aspects of taking an Open Source idea from concept to release.

Finding a Home

Before setting up your Open Source project, you'll need to decide on how to host it. Project hosting can be as simple or elaborate as you wish, but the basic services include: a website; a mailing list; a discussion forum; a source code repository (typically CVS or Subversion); a bug tracking system; and a file release system.

The multitude of choices can be broken down into two basic options: host the project yourself or use a free hosting service. Fortunately, Ruby-based projects enjoy two excellent free hosting options: SourceForge and RubyForge. Although SourceForge could certainly be used for a Ruby project, RubyForge is a more popular option. One of the reasons is that any RubyGem your project releases automatically becomes installable through the RubyGems package management system with a simple one-line incantation:

$ sudo gem install <yourprojectname>

Another reason to use RubyForge is discoverability. RubyForge only hosts projects related in some way to Ruby. So, your project will stand out a lot more in its category than with a much larger site like SourceForge.

Given RubyForge's advantages, and my own interest in minimizing the work needed to maintain an Open Source project, Rubidium will be hosted on RubyForge.

Requesting a Project Space

Having decided on RubyForge as Rubidium's host, all that's left is to ask for free services. You'll need to register for a user account if you haven't done so already. Then, simply apply for project space. After about three business days, you should be notified whether your project was accepted.

Several days ago, I completed this process for Rubidium. Its new home on RubyForge will be:

http://rubyforge.org/projects/rbtk

The Rubidium home page can be found at:

http://rbtk.rubyforge.org

There's nothing useful there yet, a situation that will hopefully be fixed in a few weeks.

Next Steps

With powerful free services now available for the Rubidium project, we'll want to start taking advantage of them. The next articles in this series will discuss some ways of doing so.

Cheminformatics in Korean: An Example of Scientific Self-Organization

Rich Apodaca — Fri, 31 Aug 2007 09:24:00 -0400

Hanjo Kim of the Bioinformatics & Molecular Design Research Center (BMDRC) in Seoul, South Korea writes in to tell about his Korean-language cheminformatics blog Agile2Robust. Some of his articles, such as this one on PubChem have been translated from Depth-First; others are based on Depth-First articles.

This completes a circle. It was Kim's post to the Ruby mailing list in late 2005 and the responses to it that helped me in the early days of developing cheminformatics software in Ruby (and writing articles about it here), although Kim probably didn't know it at the time.

During this process, none of the usual scientific protocols have been observed: no journal articles were written or published; no grants to do any of the work were prepared or funded; no presentations were made; and no central authority was involved (at least on my end). In fact, if any of these conventions had been observed, the work would have never seen the light of day. Free tools, free services, and open standards enabled one scientist to connect with another in a way that's impossible to achieve through conventional means.

Golden Rules for Open Source

Rich Apodaca — Mon, 30 Apr 2007 09:27:00 -0400

Greg Beaver has written an interesting set of ten golden rules for running open source projects. Many of these rules apply to running (or working on) anything. My favorite rule is "3. Despite the evidence, it's probably your fault." Greg develops the popular Open Source project PEAR, a packaging tool for PHP that looks similar in concept to RubyGems.

Cheminformatics, like all vertical markets, has is own peculiarities when it comes to Open Source. Most importantly, cheminformatics software has a limited, but captive audience. When done well, it is both highly profitable and expensive to develop. For insightful perspectives on Open Source software as it applies to cheminformatics and vertical markets, see Matt Stahl, Warren DeLano, Craig James, and Eric S. Raymond.

Roll Your Own Chemical Database With Free Components

Rich Apodaca — Fri, 13 Apr 2007 10:27:00 -0400

Are you thinking of building a free chemical database but would rather not rent and maintain a bunch of proprietary software components? Norbert Haider has thought a lot about this problem and offers some helpful resources to get you started:

Creating a web-based, searchable molecular structure database using free software Step-by step case study
How to create a web-based molecular structure database with free software A presentation
checkmol/matchmol Open source command-line utility for 2D (sub)structure matching
mol2ps Command-line utility for converting molfiles into Postscript files

Haider's system can be deployed on commodity hardware running open source operating systems. In other words, the cost of setting up a system like the one he describes is practically zero.

Creating and open sourcing your own custom components is one way to go. Building on top of existing open source tools like CDK, Open Babel, Octet and JOELib is another.

Haider's work raises an interesting question. Has anyone assembled a complete, ready to install general purpose chemical database package built from open source components? It for no other reason, such an exercise would give an excellent idea of what the dogfood tastes like.

Source Code, Science, and Advertising

Rich Apodaca — Mon, 26 Mar 2007 11:05:00 -0400

The minimum level of disclosure that Nature Methods requires depends on how central the software is to the paper. If a software program is the focus of the report, we expect the programming code to be made available. Without this code, the software - and thus the paper - would become a black box of little use to the scientific community. In many papers, however, the software is only an ancillary part of the method, and the focus is on the methodological approach or an insight gained from it.

-Nature Methods Editorial, March 2007

It's refreshing to see Nature Methods publicly acknowledge the importance of software availability in the sciences. On the other hand, the editorial still leaves wriggle room for an author determined to keep his or her source code private.

For example, consider the term "programming code." Does this mean "source code", or does it also include "object code"? The rest of the editorial makes it clear that both are acceptable to Nature Methods, although source code is preferred. Releasing binaries only "... may be acceptable if the operations performed by the software are sufficiently clear." Can any amount of description ever substitute for source code itself?

Over twenty years ago, John Figueras made the case that authors wishing to keep their source code private fundamentally misunderstood the difference between science and advertising. This may be a distinction lost on certain authors, but journals play along at their own risk.

Web 2.0 and Chemistry

Rich Apodaca — Mon, 12 Mar 2007 10:23:00 -0400

Chemistry on the World Wide Web is picking up speed like a runaway train. A remarkable number of groups are devoting considerable time, effort, and money to a wide variety of chemical web applications.

-Stu Borman, Chemical & Engineering News, September 16, 1996

Whatever happened to chemistry and the Web? Stu Borman's article is a wonderful read, if for no other reason than to illustrate what makes technology predictions so tricky. Borman cites these developments, among others, as evidence of the rise of chemistry on the Web circa 1996:

An AltaVista search for the word "chemistry" returned 400,000 documents. (Remember AltaVista? Google now lists over 115 million documents containing the word "chemistry").
Popular Web browsers such as Navigator 3.0 and Explorer 3.0 support Greek characters, an important characteristic of chemical information. (Support for chemistry in Web browsers has barely improved in the meantime.)
A Java browser for Chemical Markup Language (CML) was soon to be released. (CML is still in use and supported by many software packages, although it has not been widely adopted. For example, the builders of PubChem opted for a custom XML format over CML.)
Virtual Reality Markup Language (VRML) could be used display molecules in three-dimensions. (This article gives a brief overview of the rise and fall of VRML.)
Chemscape Chime, a Navigator plugin that interprets chemical information, could be freely downloaded. (~~MDL has apparently since discontinued free distribution of Chime.~~ The free as in beer plugin is rarely seen in use on the public Web.)
Chemically oriented Java applets such as "WebSketch" are proliferating. (Java has had a difficult time making it as a browser technology. Security has had little to do with it, though.)

Stu Borman's article serves as a clear reminder that chemistry on the Web is almost as awkward today as it was at the dawn of the Internet age. The problem isn't lack of content; it's the lack of robust, widely-adopted, open standards that take the peculiarities of chemical information into account, and the free software to support them. Coming to terms with past failures in this area is one way to increase the chances of future success.

Making the Case: Flux-2

Rich Apodaca — Mon, 26 Feb 2007 09:41:00 -0500

... The Flux software makes extensive use of the Chemistry Development Toolkit [sic] (CDK) as a cheminformatics library. Every descriptor that implements the Java interface for CDK descriptors can be utilized for the chemical similarity calculations; this includes the 69 descriptors available in CDK at the time of writing. ...

...

... CDK is employed as a cheminformatics toolkit (http://cdk.sf.net). The input and output of chemical structures in SDF format and the basic functionality for all structure manipulation are provided by CDK. Compounds are tested for uniqueness by means of canonical SMILES generated by CDK. Our ligand-based fitness function and molecule filter are based on descriptors that implement the Java interface for descriptors defined by CDK.

-Uli Fechner and Gisbert Schneider, J. Chem. Inf. Model. ASAP Articles

The number of peer-reviewed publications using the Open Source Java library Chemistry Development Kit (CDK) just keeps growing. The latest addition comes by way of a paper by Fechner and Schneider on ligand-based de novo design. In many areas, Open Source software is used not because it's free, but because it's perceived as superior to alternatives. How long will it be before this is true in cheminformatics?

Google for Molecules with InChIMatic

Rich Apodaca — Mon, 19 Feb 2007 10:18:00 -0500

InChIMatic is a simple Web application that uses Google to perform exact structure searches on the Web. After drawing your structure in the editor window, click the "InChI!" button to get a link. This link takes you to a Google query that displays matches for your molecule. You'll need both Java and JavaScript enabled in your browser to use InChIMatic.

The Technical Details

The technology at the heart of InChIMatic is the IUPAC International Chemical Identifier (InChI). An InChI is an alphanumeric string that uniquely identifies a molecular structure. By converting molecular structures to text, InChI makes it easy to use standard Internet tools to do exact structure searches.

The earliest reference in the peer-reviewed literature to using Google for searching InChIs is contained in a 2005 paper. More recently, a service called QueryChem has taken this idea one step further by using the Google API to perform substructure searches based on InChI.

InChIMatic works differently. Unlike a raw Google search, InChIMatic builds a Google query link for you. Unlike QueryChem, InChIMatic doesn't use the Google API and so has none of its restrictions. This does result in a limitation: InChIMatic can only currently be used to for exact structure queries.

The InChIMatic Web application has been discussed in greater technical detail in a previous article. The rapid Web application development framework Ruby on Rails made building InChIMatic a snap. InChIMatic is served by the Ruby application container Mongrel, which is hosted on a Linux server running Apache. Rino provided the Ruby interface to the IUPAC/NIST InChI toolkit. The 2-D structure editor is Java Molecular Editor (JME) by Peter Ertl, which is used with his kind permission.

Aside from JME, all components of InChIMatic, from the operating system it runs on to the InChI system itself, are Open Source software.

Using InChI to Raise the Visibility of Your Content

InChIMatic returns many Google results for common molecules. But less common, known molecules return no hits at all. Three factors are responsible: (1) Google doesn't index all InChIs on the Internet; (2) few content providers currently use InChI; and (3) there is no standard and convenient mechanism to embed InChIs into Web pages for indexing by Google.

For these reasons, I consider InChI to be bleeding edge technology. Some will find it useful, most will not. Unfortunately, this state of affairs will persist until problems (1) and (3) are solved.

Nevertheless, if you're technically adventurous, InChIMatic offers a relatively painless way to begin incorporating InChIs into your content and verifying that they get indexed. There's no software to download, install, or upgrade. Forget about operating system incompatibilities (hopefully!). Just point your Java-enabled browser to inchimatic.com.

Although there's no standard method to encode InChIs in Web pages, some interesting ideas have been put forward. Egon Willighagen has proposed a system based on RDFa. Future iterations of InChIMatic may include support for generating scripts and/or markup for including InChIs into blogs and other online content.

Conclusions

InChI is a complex new technology in need of easy-to-use tools. InChIMatic is one such tool that makes it possible to perform exact structure queries using Google.

One of the exciting things about Web applications is how quickly they can evolve. If in trying out InChIMatic you find something you'd like changed or added, please feel free to write me.