[BACK]Return to ctm.html CVS log [TXT][DIR] Up to [local] / www
File: [local] / www / Attic / ctm.html (download) (as text)

Revision 1.5, Thu Sep 4 21:57:18 1997 UTC (26 years, 8 months ago) by flipk
Branch: MAIN
Changes since 1.4: +296 -210 lines 

make pretty, add graphic at top -- note for later, make graphic smaller.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
   <TITLE>CTM</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#23238E">

<center>
<img alt="CTM for OpenBSD" src=bsdctm.gif width=401 height=126>
</center>

<H3>What is ctm?</H3>

<P>
Ctm is a system which was designed by Poul-Henning Kamp for making 
changes to a source tree available on a daily basis by email.
This is a good way to stay up to date with the current source- or
cvs-tree if you have a bad internet connection - for instance via modem.

<P>
Ctm uses very low bandwith for distributing the changes to the
tree, and compresses them using gzip -9.

<H3>How does it work?</H3>

<P>
The basic idea of ctm is that you subscribe to a special mailing list
for a particular source tree. Each day you receive an email message
containing all the changes to that tree during that period.
One set of differences is called a &quot;delta&quot;.

<P>
You begin with a base set against which later deltas are patched.
To begin using ctm you must grab the latest base set and all deltas
after that; once you have processed those you can process later
deltas you receive on the mailing list.

<P>
Bases are generated once approximate each 100 relative deltas, so you never
have to grab more than 100 deltas to catch up.

<H3>How much bandwidth does ctm use?</H3>

<P>
As mentioned above, the base set contains a full source tree in TAR format,
and is thus quite large.  You must obtain the base set and all relative
deltas up to present via ftp from the site listed below.  Sizes are:

<ul>
<li> approximately 55Mb for base/OpenBSD-src.XXXX.tar.gz
<li> uncompressed, this is 250M or so
<li> approximately 80Mb for base/OpenBSD-cvs.XXXX.tar.gz
<li> uncompressed, this is 370M or so
<li> relative deltas vary from 10k to 100k, and occasionally a couple of meg
<li> ctm always splits deltas into 100k messages which are reassembled
     automatically
</ul>

<P>
Larger ctm updates are rare, though, usually occuring only after a major
import of something like binutils, perl, etc.

<H3>How do I use ctm?</H3>

<UL>
<LI>Subscribe to the ctm mailing list for the source set you want.
<p>
   There are several mailing lists related to ctm.  There is a list for
   updates to the source tree, a list for updates to the cvs tree, a list
   for announcements where I'll post important announcements to ctm users,
   and a list for seeing log messages from the ctm delta generator runs.
   <p>
   The following commands will subscribe you to the appropriate lists:
<br><br><tt>
For the source tree: <br>
    <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b><br>
For the cvs tree:<br>
    <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b><br>
For the announce list:<br>
    <b>echo subscribe ctm-announce | mail majordomo@OpenBSD.org</b><br>
For the ctm log list:<br>
    <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b><br>
</tt><br>

<LI>Wait until you get your first delta.

<LI>Unpack and assemble the ctm delta using ctm_rmail.
<br>
<br>
<tt>
    <b>ctm_rmail -p. -d. -b. folder</b>
</tt>
<br><br>
where <b>folder</b> is the mail folder containing the delta mail.
<br>
This will decode the ctm delta and place it in the working directory.
The delta will be a file of the form OpenBSD-XXX.YYYY.gz, where XXX is
either "src" or "cvs", and YYYY is the number of the delta.
<p>

<li>Grab the base set from the ftp site below, in the subdirectory 
<b>base-splitted/src</b> (for OpenBSD-src) or <b>base-splitted/cvs</b>
(for OpenBSD-cvs).  Grab all files in that directory, and once you have
them all you can create the base source tree with the following command:

<br><br>
<tt>
    <b>cd target <br>
    cat /splitted/OpenBSD-* | tar zxf - <br>
</b></tt><br>

where <b>splitted</b> is the directory in which you placed the files you
grabbed.

<p>
It is no longer necessary to keep these files around, if you're sure
you've got a complete tree.  However, be sure to note the number of the
base, before continuing.

<p>
You now need to grab the deltas which have been generated since the base
was generated.  Go to the ftp site below in the subdirectory <b>src</b> or
<b>cvs</b> and grab every file whose number is <it>larger</it> than the 
number of the base you used.

<p>
The address of the base and deltas is:

<ul>
<li><a href="ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD/">
     ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD/
</a>
</ul>

<p>
<ul>
<li><b>NOTE:</b> I'm very interested in finding other sites around the world
   that would be interested in mirroring these deltas.
</ul>
<p>

<LI> Run ctm.
<br>
  <br>
  <tt>
     <b>cd target<br>
     ctm -v -v /deltas/OpenBSD-src.*</b><br></tt>
or<br><tt>
     <b>cd target<br>
     ctm -v -v /deltas/OpenBSD-cvs.*</b><br>
  </tt><br>
where <b>target</b> is the directory where you want your source tree to be,
and <b>deltas</b> is where you have stored the deltas.
<p>
It will take some time to run all of the deltas, but the daily invocations
of <b>ctm_rmail</b> and <b>ctm</b> will not take nearly as long.
</ul>

<h3>Should I choose the source or cvs tree?</h3>

<P>
This depends on your diskspace, among other factors.  Since ctm doesn't 
yet deal with files which have been modified outside of the ctm process, 
the "better" choice is probably the cvs-tree.  Then you can check out
your working source tree from your cvs-tree, and keep your local mods
in your source tree, leaving only ctm to touch your cvs tree.

<p>
The drawback, however, is the amount of diskspace it takes.  A checked-out
source tree takes about 250MB or so, but if you have your own cvs tree you
need 370MB or so for it, <b>plus</b> another 250MB for the checked out tree.

<p>
This doesn't count the 100MB or more you will need to do a build (depending
on the architecture).

<p>
If you just get the source tree, you will need to deal with local
modifications.  One way to do this is to use the <b>union</b> filesystem,
although there have been reports that the <b>union</b> filesystem can be
unstable if both the upper and lower layers reside on the same physical
filesystem.  If you place your upper and lower layers on separate filesytems
you should have no problems.  A sample of this is as follows:

<p>
Suppose your ctm-updated tree is in <b>/usr/src-ctm</b>, and your real
source tree, the one you make your modifications to and do your builds from,
is in <b>/usr/src</b>.  <b>/usr/src</b> should be initially an empty 
directory.  The following command will set up the union mount:
<br><br>
<tt>
    <b>mount -t union -o -b /usr/src-ctm /usr/src</b>
</tt>
<br><br>
Modifications made to files in /usr/src will become a file contained within
/usr/src, hiding the one in src-ctm.  If changes are made via ctm to the
underlying src-ctm tree, those changes will <b>not</b> be seen if there is
a file in the upper layer hiding it.

<p>
You should periodically unmount the
union and search for files which are local to the union filesystem.
<br><br>
<tt>
    <b>umount /usr/src <br>
    find /usr/src -type f</b> <br>
</tt>
<br><br>
The commands <b>ls -W</b> and <b>rm -W</b> will be useful too, as objects
named "whiteouts" in /usr/src will also hide files in src-ctm.

<p>
If you get the cvs-tree, you can use the "cvs checkout" command to check
out a source tree from it, and each time you update with ctm you can use
the "cvs update" command to update your source tree.

<br><br>
To get an initial src tree:<br>
<tt>
    <b>cd /usr <br>
    cvs -qd YOUR_CVS_TREE checkout src</b> <br><br>
</tt>
and after each ctm update:<br>
<tt>
    <b>cd /usr/src <br>
    cvs -q update -PAd</b> <br>
</tt>

<H3>How stable is ctm for OpenBSD?</H3>

Ctm has been in use for over a year for OpenBSD, and for FreeBSD for a
number of years.  Ctm is quite reliable and stable.

<H3>More information</H3>

If you have problems with ctm, a good place to start is the FreeBSD 
handbook:

<ul>
<li> <a href="http://www.FreeBSD.org/handbook/ctm.html">
         http://www.FreeBSD.org/handbook/ctm.html
     </a>
</ul>

There are also man pages for all of the ctm utilities included in the
source package.  If you have OpenBSD, you should already have the man
pages, source, and binaries.  If not you can obtain them from the same
ftp site as the deltas below.

<p>
You can also contact the OpenBSD mailing lists or myself if you have 
difficulties with ctm, although my free time tends to be rather limited.

<p>
Summary:

<p>

<ul>
<li> Mailing lists:

   <ul>
   <li> <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b>
   <li> <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b>
   <li> <b>echo subscribe ctm-announce | mail majordomo@OpenBSD.org</b>
   <li> <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b>
   </ul>

<li> FTP (for base and relative deltas; and OpenBSD has ctm in the source
     tree, but if you don't have source or binaries, the source is here too):

   <ul>
   <li> <a href="ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD">
         ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD
	   </a>
   <li> <a href="ftp://ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD">
         ftp://ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD
	   </a>
   </ul>
</ul>

<p>
I will make any important notes and announcements about ctm in:
<ul>
<li> ctm-announce@OpenBSD.org
<li> misc@OpenBSD.org
<li> announce@OpenBSD.org
</ul>

<p>
<b>NOTE:</b> If you use ctm for OpenBSD, please send me email so I
know how many people are using it!

<H3>Acknowledgements</H3>

<li> Poul-Henning Kamp (phk@FreeBSD.org) for making ctm possible and helping
     me getting the delta generation running.

<li> Theo de Raadt (deraadt@theos.com) for making OpenBSD possible and
     giving me the resources i needed for creating the ctm
     deltas on cvs.OpenBSD.org

<li> Wolfram Schneider (wosch@FreeBSD.org) for setting up the ftp space
     for the deltas on ctm.OpenBSD.org

<li> ... and all the others who contributed indirectly ..
</ul>

<P>
Good luck!

<P>
t

<HR>

<ADDRESS><A HREF="mailto:graichen@OpenBSD.org">Thomas Graichen</A></ADDRESS>
<br>
OpenBSD/CTM logo designed for the OpenBSD Project by 
   <a href="mailto:flipk@openbsd.org">Phillip F Knaack</a>.

<P>
<!-- Created: Mon Oct 28 22:20:54 MET 1996 -->
<!-- hhmts start -->
  $Id: ctm.html,v 1.5 1997/09/04 21:57:18 flipk Exp $
<!-- hhmts end -->

</BODY>
</HTML>