Virtual Laboratory for e-Science
HOWTO: Submit your software for the contrib/ track.

by Dennis H. van Dok (dennisvd@nikhef.nl)

Updated Nov. 16, 2006

Introduction

Besides the officially endorsed VL-e software in the Proof-of-Concept distribution, there is a special ‘contributions’ section for software that is

• provided by a contributor outside VLeIT or the P4 team,
• provided ‘as is’ without guarantees,
• certified only to pass some basic installation tests and
• not supported by the P4 team, but exclusively by the original contributor.

This page outlines the basic requirements that the software has to meet before it is eligible to be installed in the contrib/ section. Simply speaking, it boils down to testing that it is installable through some automatic procedure. We don't make any demands as to the actual usability of the product. However, meeting these requirements is still no guarantee that the software is allowed. It still needs to pass the close scrutiny of the P4 team to check if the software does not misbehave itself.

When you think your software complies with these requirements, you should verify this by running a script in a PoC environment. If the script says you're OK, email your web link to the P4 team explaining that you want your software in the contributions.

Summary of the basic requirements

The basic requirements are summarily as follows: the software should be simple to obtain as a bundle (a tar or zip archive) and simple to build and install in the right location. It should require no special tricks, no manual intervention and no root privileges or starting of services; it should not even need network connectivity, the original download aside. All this should be automatic from a single script (detailed below) that does little more than (./configure && make && make install).

If you think this bears a striking resemblance to the GNU way of distributing software, you're right. The GNU build system is a model for the way we like to do things. There is no such thing as a perfect build system, but GNU is making a good effort and packaging software like, say, rpm and dpkg are well-tuned to it. Also recommended are the GNU coding standards.

So if your software is already set up to make use of autoconf, automake and maybe libtool, good for you! Chances are your software will install straight away. If not, you may be able to get by with a minimalistic approach, providing only a wrapper script that follows the rules detailed below. You give up some portability, but we don't hammer on portability too much yet.

The detailed requirements below are each accompanied by a rationale (the “Why?”) and the procedure we follow to test it (the “What?”). The measures you should take to comply (the “How?”) are what this HOWTO is really all about, but the “what” and “why” are provided for perspective (and, for perspective, in a grey background). An overview is give in Table 1. To help distinguish whether a phrase is meant as a hard requirement or merely as a guideline, I've adopted the keywords (must, should, may and derived forms) as described in RFC 2119, consistently emphasised like this.

Release and Distribute

Name your package

The package should be given a name that is all lowercase, less than 20 characters in length, and containing only letters, digits, hyphens (‘-’) and plus signs (‘+’).

The release number should follow the schema of x.y.z, that is major number, minor number and patchlevel. The distribution shall be fully named <name>-<version>.

Note: Avoid ambiguity about the name of your package by following this simple rule: the first occurrence of a digit (‘0’–‘9’) after a hyphen (‘-’) marks the start of the version number.

Note: If you prefer using dated versions, use the format 0.0.yyyymmdd because that is what GNU suggests and this way, comparing dates numerically actually makes sense.

Tag your release

It is recommended to maintain the original sources of your software with version control software, such as subversion.

The versions of the files that are going to be released should be tagged in this version control system. The name of the tag should clearly identify the release version.

Make a source distribution

• It is required that a zip or tar archive is created of the complete sources. This should be named after the package and release number, for instance foo-1.0.2.tar.gz.
• The archive should unpack into a single directory named after the package and release number, for instance archive foo-1.0.2.tar.gz contains a single directory foo-1.0.2.
• The archive should not contain any files that are irrelevant for building, such as CVS directories and files generated as part of the build.
• The archive should be put up for download from a central repository, such as gforge.vl-e.nl.

Provide contact information

Along with the distributed archive, you must provide contact information, so that users of the software know how to contact you for questions and bug reports. This information should be on the web page from where the archive is linked, and it should also be present in a README or MAINTAINER file inside.

The contact information should include at least an email address, and optionally a web page URL, a postal address and/or a telephone number. You should provide a (web or email) address for reporting bugs.

Providing documentation is optional, but it is recommended that documentation follows the GNU guidelines (see GNU coding standards, Chapter 6: Documenting Programs).

Configure and Build

Your released sources are required to build.

Provide a configure script

The source distribution must contain a shell script named configure in the top-level directory. This script should accept the command-line arguments as given in testing the build; the placeholders between < > are paths of the various locations where the files will ultimately be installed. These paths shall not exist during the build step, or even at install time; they may only be referred to at run-time, but it is recommended that the software remains independent of the exact installed location.

The script must not use the network in any way. The script is required to function when run as an ordinary user. The script should be portable to other UNIX-like environment, and therefore it should only use common shell constructs and programs.

The script may generate files in the source tree, such as Makefiles, config.h and the like.

Provide a Makefile

The source tree must contain (after the configure script has been run successfully), a Make input file called Makefile. It's default target must build the software. The make step must not write files outside the current directory tree, other than temporary files. The build must not use the network.

Mind the dependencies

The build step should not rely on any software other than what is provided in the source distribution or what is provided in the VL-e PoC.

./configure  --prefix=<prefix> \
	     --bindir=<bindir> \
	     --sbindir=<sbindir> \
	     --sysconfdir=<sysconfdir> \
	     --datadir=<datadir> \
	     --includedir=<includedir> \
	     --libdir=<libdir> \
	     --localstatedir=<localstatedir> \
	     --mandir=<mandir> \
	     --infodir=<infodir>

Self Check

Provide your own unit tests

The Makefile should have a target called check, which is called on to initiate a test suite right after the build phase. What this target actually accomplishes is up to you, but it is recommended that at least some sanity checks are performed.

Installing the built software

When the test suite is run successfully, the last step is to install the built files in various locations, ready for re-archiving as a binary distribution.

Provide an install target

The Makefile must have a target called install. This target should copy the files from the working tree to the target locations and give them the correct modes.

The installation must not use the network. It is required to function with normal user privileges.

Install file in the right locations

The install must use the Make variables as indicated in Table 2 to derive the target locations. It must also use the DESTDIR variable as the root of the filesystem to install to, because the script does not have the privileges of writing to the real file system (that is for real file system administrators). So instead of writing files to $(prefix)/, you should write to $(DESTDIR)$(prefix)/.

The install must not try to write files to locations other than those indicated by the variables.

The install should follow the Filesystem Hierarchy Standard when installing files: executable binaries go into bindir, libraries into libdir, etc.

The install may create further directory structures below those given in the table, but it is not recommended that directories are created directly under <prefix>, <bindir> or <sbindir>.

Installed software should still work

After the install step the software is re-archived into a binary package and subsequently installed in the locations given in the configure step. Installed so, the software should be functioning normally. This is not really trivial, as where the software is built, installed and finally ends up are three different locations.

The location /opt must be considered read-only; the program's variable data should be written to the location indicated by <localstatedir> during the configure step; likewise, configuration data should be stored in the <sysconfdir>.

make DESTDIR=<buildroot> install

Variables for locations

Verifying your compliance

Before you submit your software to the P4 team, you should test your software against the above rules. The best way is to download and run the PoC VMWare image, and to run (in the VM) this validation script.

If the script says you're OK, it made a tar archive named <package-name>-<version>-packaged.tar.gz that you should unpack (as root) in the VM. If your software installed in this manner is functioning correctly, you are welcome to send it to us. Send an email to us at: vle-pfour-team@lists.vl-e.nl with a subject like “PoC contrib submission” and saying in the body something along the lines of:

I've read and followed the guidelines of your contrib HOWTO, and successfully tested my software with the validation script. You can obtain my software from this-and-this website.

We will then repeat the tests of the software and package it (if all is well) in the PoC middleware release.