CTM is a system which was designed by Poul-Henning Kamp for making changes to a source tree available on a regular basis by email. This is a good way to stay up to date with the current source tree or CVS repository if you have a slow Internet connection, for instance via modem.
OpenBSD CTM distributes all modules of the OpenBSD CVS repository plus checked out versions of several of these modules. CTM uses very low bandwidth for distributing the changes to the tree, and compresses them using gzip -9. In contrast to CVSup, CTM is written in plain C language and therefore available for every platform supported by OpenBSD. CTM implements a so called push system, while all other methods to get access to the OpenBSD CVS repository are pull systems.
For any problems, suggestions, reports and questions regarding CTM, contact the CTM maintainer Hans Günter Weigand. (Please do not ask other OpenBSD people for CTM support, as they are busy doing other things. Thank you.)
The basic idea of CTM is that you subscribe to a special mailing list for a particular CVS module or source tree. Each day you receive email messages containing all the changes to that tree during that period. One such set of differences is called a delta. Actually, the CTM server makes new deltas every twelve hours.
You begin with a base set, against which later deltas are patched. To begin using CTM you download the latest base set and all deltas generated after that. Once you have processed those you can process later deltas you receive from the mailing list. The base sets are split into pieces of 1400kB each for easier downloading over bad lines.
Base sets are generated once every 50 relative deltas, so you never have to grab more than 50 deltas to catch up.
There are currently two kinds of base set files. For quite a while we were making tar.gz files manually, but we are back to real CTM base deltas. The latter contain md5(1) checksums and are generated automatically, thus are a bit safer. You can distinguish these two kinds of base sets by their filename. An example for a tar.gz file is OpenBSD-cvs.1450.tar.gz, whereas a CTM base delta would be named OpenBSD-cvs.1500A.gz. See below for detailed instructions.
You obtain the base set and all relative deltas up to the recent one via ftp from the sites listed below. Sizes are:
There are several mailing lists related to CTM. There is a list for updates to the CVS repository, a list for the source tree, a list for the X11 part of both, and one for the ports tree, plus a list for log messages from the CTM delta generation runs.
The following commands will subscribe you to the appropriate lists:
For the CVS repository (CVSROOT, ports, src and www modules, but not X11 and XF4):
echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.orgFor the X11 module of the CVS repository:
echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.orgFor the XF4 module of the CVS repository:
echo subscribe OpenBSD-cvs-xf4 | mail majordomo@OpenBSD.orgFor a checked out copy of the src module:
echo subscribe OpenBSD-src | mail majordomo@OpenBSD.orgFor a checked out copy of the X11 module:
echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.orgFor a checked out copy of the ports module:
echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.orgFor the CTM log list:
echo subscribe ctm-log | mail majordomo@OpenBSD.org
cat split/OpenBSD-cvs.1500A.gz.* > OpenBSD-cvs.1500A.gz cd target ctm -v -v -v .../OpenBSD-cvs.1500A.gz
or
cat split/OpenBSD-cvs.1450.tar.gz.* > OpenBSD-cvs.1450.tar.gz cd target tar -xzvf .../OpenBSD-cvs.1450.tar.gz
where split is the directory in which you have placed the files you downloaded and target is the place you have chosen for unpacking the source set.
It is not necessary to keep these base set files around, if you're sure you've got a complete tree. But you can keep the base set file(s) for backup purposes. However, be sure to note the number of the base set before continuing.
cd target ctm -v -v -v deltas/OpenBSD-cvs.*
where target is the directory where your source set tree is, and deltas is where you have stored the deltas.
ctm_rmail -p /tmp -d deltas -b target folder
where folder is the mail folder containing the delta mail. This will decode the CTM delta and save it in the deltas directory. The delta will be a file of the form OpenBSD-cvs.XXXX.gz, where XXXX is the number of the delta. Then the delta will be applied to the directory tree located at target, the place where you unpacked the base set.
Repeat this last step every time you receive a delta from the mailing list. If you like, you can automate this step using procmail, maildrop, or any similar program from the ports collection.
Here is the list of FTP servers carrying CTM deltas:
ftp://openbsd.rug.ac.be:/pub/OpenBSD-ctm/ .
This server is automatically updated by the CTM delta generation process. Base sets are available as big single files in the base directory. The split files are in base-split.
This server is updated by CTM deltas received from the mailing lists. It additionally has the split base deltas in the base subdirectory, mirrored from ftp://openbsd.rug.ac.be:/pub/OpenBSD-ctm/ .
is identical to
and is a mirror of ftp://ctm.se.OpenBSD.org .
NOTE: I'm very interested in finding other sites around the world that would like to mirror these deltas.
This depends on your diskspace, among other factors. Since CTM doesn't deal very well with files which have been modified locally, the better choice is surely the CVS repository. You can check out your working copy of the source tree from your CVS repository then, can track a stable branch, and keep your local modifications in your source tree. CVS is smart enough to merge in your local changes. You can also use the full range of CVS commands except commands like cvs commit and cvs tag.
The drawback, however, is the amount of diskspace it takes. A checked-out source tree takes about 370MB or so, but if you have your own CVS repository you need more than 1Gb for it, plus another 370MB for the checked out tree. This doesn't count the 100MB or more (depending on the architecture) you will need for generated files during a build, i.e. compiling the sources.
If you just get the source tree, you will need to deal with local modifications. One way to do this is to use the union filesystem, (see mount_union(8)) although there have been reports that the union filesystem can be unstable if both the upper and lower layers reside on the same physical filesystem. If you place your upper and lower layers on separate filesystems you should have no problems. A sample of this is as follows:
Suppose your CTM-updated tree is in /usr/src-ctm, and your real source tree, the one you make your modifications to and do your builds from, is in /usr/src. /usr/src should be initially an empty directory. The following command will set up the union mount:
mount -t union -o -b /usr/src-ctm /usr/src
Modifications made to files in /usr/src will become a file contained within /usr/src, hiding the one in src-ctm. If changes are made via CTM to the underlying src-ctm tree, those changes will not be seen if there is a file in the upper layer hiding it.
You should periodically unmount the union and search for files which are local to the union filesystem.
umount /usr/src find /usr/src -type f
The commands ls -W and rm -W will be useful too, as objects named "whiteouts" in /usr/src will also hide files in src-ctm.
If you get the CVS repository, you can use the cvs checkout command to check out a source tree from it. Each time you update with CTM you can use the cvs update command to update your source repository.
To get an initial src repository:
cd /usr cvs -qd YOUR_CVS_REPOSITORY checkout -P src
and after each CTM update:
cd /usr/src cvs -q update -Pd
Important notes and announcements about CTM will be posted to the announce@OpenBSD.org mailing list.
OpenBSD/CTM logo designed for the OpenBSD Project by Phillip F Knaack.
www@openbsd.org