Port Testing Guide

Index

• Introduction
• How
• First step
• Testing
• Commenting
• More testing

Introduction

The ports tree is an huge piece of work that permits OpenBSD users to use third-party programs without wasting time patching, configuring and installing each one individually. This work is done by a group of volunteers who spend their time porting and testing applications across the range of OpenBSD platforms. Many people think that they cannot help this process because they do not have enough knowledge; this is false because they can help porters work better and faster.

How

Testing submitted updates or new ports which are posted on the ports mailing list. By doing this you reduce the latency of commits and also increase the number of ports to be committed (many ports are not committed because of lack of testing!).

First step

The ports tree is developed against OpenBSD-current; there is no guarantee that new ports will work correctly on the other branches. This also means you should upgrade your system and ports tree to -current (instructions on how to do this can be found at the anoncvs page and in the FAQ). It is also recommended that you subscribe to the ports and ports-changes mailing-lists. This way you will be notified about new or updated ports and about changes in the ports tree.

Testing

There are two types of submissions on the mailing-lists; new ports and updates. New ports are generally posted as tarball attachments or urls. A good idea is to extract them into the /usr/ports/mystuff/ directory and test from there. Updates are generally a diff against the -current ports tree, so it is best to copy the port to mystuff/ and apply the diff to prevent tree breakage.

Step-by-step building is needed to verify that every target, see ports(7), is achieved correctly :

• fetch
  • Needed to verify that distfile(s) are correctly downloaded. Try to test all of the MASTER_SITES specified to make sure they are all valid sources.
• checksum
  • Verify that downloaded distfiles match the checksums recorded in the distinfo file.
• extract
  • Each of the distfiles should install any dependent programs (such as bzip2) beforehand.
• patch
  • Patches should apply cleanly without any warnings.
  • There shouldn't be any .orig files left behind in the patches/ directory.
  • Another common mistake is to include RCS tags in a patch; this will break when the port is checked into the repository and the RCS tag expanded.
• configure
  • Check that configure scripts correctly detect features on your platform.
  • GNU libtool is notorious for undesired 'features' on OpenBSD, so check that the port is using a recent version which creates correct libraries on your architecture.
  • The configure script should not detect stray applications already installed on your system without explicit dependencies being set in the port.
• build
  • Check for build errors and suspicious warnings.
  • Warnings about tmpnam(3) issues should be resolved by using mkstemp(3).
  • Try to set the SEPARATE_BUILD variable to 'concurrent' and test if the build still works.
  • Make sure dependencies on GNU make are really necessary.
• fake
  • This target installs the application into a fake working directory, to ensure that all files can be easily packaged up without affecting the base system.
  • The port should never install files outside of the fake directory (such as into /usr/local).
  • GNU libtool occasionally has trouble relinking libraries during the fake process on some architectures.
• package
  • Package creation can break if pkg/PLIST* and/or pkg/PFRAG* are wrong.
• install
  • Packages should install all of the files from their packaging lists successfully and with the correct permissions. Be especially careful of files with the setuid bits set.
  • Make sure that the package INSTALL script works correctly, and does not overwrite any files in /etc.
• deinstall
  • This should remove all files installed by the package, except those in /etc.
  • Make sure that the DEINSTALL script works as expected.

Remaining pkg/ files like DESCR and MESSAGE should be checked for grammar and typos. Paragraphs should be formatted using fmt(1) and wrapped at 70 characters.

Commenting

At the end of the test comes the really important thing : comments. Even if the port is working fine comments must be done. If we have ten posts where people say that the port runs fine under different architectures then the commit is done faster. If it does not work then some information must be given. There are tools that can help in this task, like portslogger which is like an "intelligent tee" that redirects output into a log file.

Example :

    # make install | /usr/ports/infrastructure/build/portslogger .

This will redirect the output into a log file located in the current directory.

More testing

Check the port Makefile for typos, incorrect links, useless or missing variables, correct licensing and categories. For those who are more skilled other targets can be tested like lib-depends-check, examining patches, as well as providing diffs to correct bugs, add flavors, or other enhancements.

These diffs should be done with the -uNr --exclude=CVS options. cvs diff -uN can also be used to generate patches against the CVS repository.