Annotation of www/ctm.html, Revision 1.13
1.1 graichen 1: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
2: <HTML>
3: <HEAD>
4: <TITLE>CTM</TITLE>
5: </HEAD>
1.5 flipk 6: <BODY BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#23238E">
1.1 graichen 7:
1.5 flipk 8: <center>
1.13 ! naddy 9: <img alt="CTM for OpenBSD" src="images/bsdctm.gif" width="401" height="126">
1.5 flipk 10: </center>
1.1 graichen 11:
1.5 flipk 12: <H3>What is ctm?</H3>
1.1 graichen 13:
14: <P>
1.5 flipk 15: Ctm is a system which was designed by Poul-Henning Kamp for making
16: changes to a source tree available on a daily basis by email.
17: This is a good way to stay up to date with the current source- or
1.13 ! naddy 18: CVS tree if you have a bad Internet connection, for instance via modem.
1.1 graichen 19:
1.5 flipk 20: <P>
1.13 ! naddy 21: Ctm uses very low bandwidth for distributing the changes to the tree,
! 22: and compresses them using gzip -9. In contrast to <a
! 23: href="cvsup.html">CVSup</a>, ctm is written in plain C and therefore
! 24: available for every platform supported by OpenBSD.
1.1 graichen 25:
1.5 flipk 26: <H3>How does it work?</H3>
1.1 graichen 27:
1.5 flipk 28: <P>
29: The basic idea of ctm is that you subscribe to a special mailing list
30: for a particular source tree. Each day you receive an email message
31: containing all the changes to that tree during that period.
1.13 ! naddy 32: One set of differences is called a "delta".
1.1 graichen 33:
1.5 flipk 34: <P>
1.13 ! naddy 35: You begin with a base set, against which later deltas are patched. To
! 36: begin using ctm you download the latest base set and all deltas
! 37: generated after that. Once you have processed those you can process
! 38: later deltas you receive from the mailing list. The base sets are split
! 39: into pieces of 1400kB each for easier downloading over bad lines, and
! 40: you can even put them onto floppies this way.
1.1 graichen 41:
1.5 flipk 42: <P>
1.13 ! naddy 43: Base sets are generated once every 50 relative deltas, so you never
! 44: have to grab more than 50 deltas to catch up.
! 45:
! 46: <P>
! 47: There are currently two kinds of base delta. For quite a while we
! 48: were making <tt><b>tar.gz</b></tt> files manually, but we are
! 49: switching back to real ctm base deltas. The latter contain md5
! 50: checksums and are generated automatically, thus are a bit safer. You
! 51: can distinguish these two kinds of base delta by their filename. An
! 52: example for a <tt><b>tar.gz</b></tt> file is
! 53: <tt><b>OpenBSD-cvs.1450.tar.gz</b></tt>, whereas a ctm base delta
! 54: would be named <tt><b>OpenBSD-cvs.1500A.gz</b></tt>. See below for
! 55: detailed instructions.
1.1 graichen 56:
1.5 flipk 57: <H3>How much bandwidth does ctm use?</H3>
1.1 graichen 58:
1.5 flipk 59: <P>
1.13 ! naddy 60: You obtain the base set and all relative deltas up to the recent one
! 61: via ftp from the sites listed below. Sizes are:
1.5 flipk 62:
63: <ul>
1.13 ! naddy 64: <li>Approximately 115MB for OpenBSD-cvs.
! 65: <li>Approximately 37MB for OpenBSD-cvs-x11.
! 66: <li>Approximately 65MB for OpenBSD-src.
! 67: <li>Approximately 35MB for OpenBSD-src-x11.
! 68: <li>Approximately 2MB for OpenBSD-ports.
! 69: <li>Keep in mind that these are compressed sizes. The uncompressed
! 70: files are about five times as large.
! 71: <li>Relative deltas vary from 10kB to 100kB, and occasionally a couple of
! 72: megabytes.
! 73: <li>ctm always splits deltas into 100kB messages which are reassembled
! 74: automatically.
! 75: <li>If a delta is larger than 5MB, it is not sent via mail. Instead, you
! 76: will receive a notification to get the delta by ftp.
1.5 flipk 77: </ul>
1.1 graichen 78:
1.5 flipk 79: <P>
1.6 todd 80: Larger ctm updates are rare, though, usually occurring only after a major
1.13 ! naddy 81: import of something like binutils, perl, gcc, etc.
1.1 graichen 82:
1.5 flipk 83: <H3>How do I use ctm?</H3>
1.1 graichen 84:
85: <UL>
1.5 flipk 86: <LI>Subscribe to the ctm mailing list for the source set you want.
87: <p>
88: There are several mailing lists related to ctm. There is a list for
1.13 ! naddy 89: updates to the CVS tree, a list for the source tree, a list
! 90: for the X11 part of both, and one for the ports tree, plus
! 91: a list for log messages from the ctm delta generation runs.
1.5 flipk 92: <p>
93: The following commands will subscribe you to the appropriate lists:
1.13 ! naddy 94: <p>
! 95: For the CVS tree (without the X11 part):
! 96: <pre>
! 97: echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org
! 98: </pre>
! 99: For the X11 part of the CVS tree:
! 100: <pre>
! 101: echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org
! 102: </pre>
! 103: For the source tree (without the X11 part):
! 104: <pre>
! 105: echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org
! 106: </pre>
! 107: For the X11 part of the source tree:
! 108: <pre>
! 109: echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org
! 110: </pre>
! 111: For the ports tree:
! 112: <pre>
! 113: echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org
! 114: </pre>
! 115: For the ctm log list:
! 116: <pre>
! 117: echo subscribe ctm-log | mail majordomo@OpenBSD.org
! 118: </pre>
! 119:
! 120: <p>
! 121: <li>Install the ctm program from a package or from the ports
! 122: collection. You'll find the port at
! 123: <tt><b>/usr/ports/devel/ctm</b></tt>.
! 124:
! 125: </ul>
! 126: <b>Note:</b> The examples below just cover OpenBSD-cvs for clarity. The
! 127: other source sets are handled in the same fashion.
! 128:
! 129: <p><ul>
! 130:
! 131: <li>Download the base set from an FTP site. You'll find it in the
! 132: subdirectory <tt><b>base/OpenBSD-cvs</b></tt>. Get all files in
! 133: that directory. Once you have them all you can create the base set
! 134: file and unpack it with the following commands, depending on the kind
! 135: of base delta you got:
! 136:
! 137: <pre>
! 138: cat split/OpenBSD-cvs.1500A.gz.* > OpenBSD-cvs.1500A.gz
! 139: cd target
! 140: ctm -v -v -v .../OpenBSD-cvs.1500A.gz
! 141: </pre>
! 142: <p>
! 143: or
! 144:
! 145: <pre>
! 146: cat split/OpenBSD-cvs.1500.tar.gz.* > OpenBSD-cvs.1500.tar.gz
! 147: cd target
! 148: tar -xzvf .../OpenBSD-cvs.1500.tar.gz
! 149: </pre>
! 150:
! 151: <p>
! 152: where <tt><b>split</b></tt> is the directory in which you have
! 153: placed the files you downloaded and <tt><b>target</b></tt> is the
! 154: place you have chosen for the source set.
1.5 flipk 155:
1.13 ! naddy 156: <p>
! 157: It is not necessary to keep these files around, if you're sure you've
! 158: got a complete tree. But you can keep the base set file(s) for backup
! 159: purposes. However, be sure to note the number of the base set before
! 160: continuing.
1.5 flipk 161:
162: <p>
1.13 ! naddy 163: <li>You now need to get the deltas which have been generated since the
! 164: base set. Go to the FTP site in the subdirectory
! 165: <tt><b>OpenBSD-cvs</b></tt> and download every file whose number is
! 166: <em>larger</em> than the number of the base set you used. Then apply
! 167: these deltas:
1.5 flipk 168:
1.13 ! naddy 169: <pre>
! 170: cd target
! 171: ctm -v -v -v deltas/OpenBSD-cvs.*
! 172: </pre>
1.5 flipk 173:
1.13 ! naddy 174: <p>
! 175: where <tt><b>target</b></tt> is the directory where your source set
! 176: tree is, and <tt><b>deltas</b></tt> is where you have stored the
! 177: deltas.
1.5 flipk 178:
1.13 ! naddy 179: <p>
! 180: <LI>Wait until you get your first delta from the mailing list.
! 181: <p>
! 182: <LI>Unpack, assemble, and apply the ctm delta using ctm_rmail.
! 183: <p>
! 184:
! 185: <pre>
! 186: ctm_rmail -p /tmp -d deltas -b target folder
! 187: </pre>
1.5 flipk 188:
189: <p>
1.13 ! naddy 190: where <tt><b>folder</b></tt> is the mail folder containing the delta
! 191: mail. This will decode the ctm delta and save it in the
! 192: <tt><b>deltas</b></tt> directory. The delta will be a file of the
! 193: form <tt><b>OpenBSD-cvs.<var>XXXX</var>.gz</b></tt>, where <var>XXXX</var>
! 194: is the number of the delta. Then the delta will be applied to the
! 195: directory tree located at <tt><b>target</b></tt>, the place where you
! 196: unpacked the base delta.
1.5 flipk 197:
198: <p>
1.13 ! naddy 199: Repeat this last step every time you receive a delta from the
! 200: mailing list. If you like you can automate this step using procmail
! 201: or any similar program.
! 202: </ul>
1.5 flipk 203:
204: <p>
1.13 ! naddy 205: The addresses for the deltas are:
1.10 deraadt 206:
207: <ul>
208: <li><a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/">
209: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/
210: </a>
211: </ul>
1.5 flipk 212:
213: <ul>
1.10 deraadt 214: <li><a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/">
215: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/
216: </a>
217: </ul>
218:
219: <ul>
220: <li><a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/">
221: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/
1.5 flipk 222: </a>
223: </ul>
224:
225: <p>
1.13 ! naddy 226: <b>NOTE:</b> I'm very interested in finding other sites around the world
! 227: that would like to mirror these deltas.
1.5 flipk 228: <p>
1.1 graichen 229:
1.13 ! naddy 230: <h3>Should I choose the source or CVS tree?</h3>
1.1 graichen 231:
1.5 flipk 232: <P>
1.13 ! naddy 233: This depends on your diskspace, among other factors. Since ctm
! 234: doesn't deal with files which have been modified locally, the "better"
! 235: choice is probably the CVS tree. You then can check out your working
! 236: source tree from your CVS tree, and keep your local modifications in
! 237: your source tree, because cvs is smart enough to merge in your local
! 238: changes.
1.5 flipk 239:
240: <p>
241: The drawback, however, is the amount of diskspace it takes. A checked-out
1.13 ! naddy 242: source tree takes about 350MB or so, but if you have your own cvs tree you
! 243: need about 1Gb for it, <b>plus</b> another 350MB for the checked out tree.
1.5 flipk 244:
245: <p>
1.13 ! naddy 246: This doesn't count the 100MB or more (depending on the architecture)
! 247: you will need to do a build, i.e. compile the sources.
1.5 flipk 248:
249: <p>
250: If you just get the source tree, you will need to deal with local
251: modifications. One way to do this is to use the <b>union</b> filesystem,
252: although there have been reports that the <b>union</b> filesystem can be
253: unstable if both the upper and lower layers reside on the same physical
1.6 todd 254: filesystem. If you place your upper and lower layers on separate filesystems
1.5 flipk 255: you should have no problems. A sample of this is as follows:
256:
257: <p>
1.13 ! naddy 258: Suppose your ctm-updated tree is in <tt><b>/usr/src-ctm</b></tt>, and
! 259: your real source tree, the one you make your modifications to and do
! 260: your builds from, is in <tt><b>/usr/src</b></tt>.
! 261: <tt><b>/usr/src</b></tt> should be initially an empty directory. The
! 262: following command will set up the union mount:
! 263:
! 264: <pre>
! 265: mount -t union -o -b /usr/src-ctm /usr/src
! 266: </pre>
! 267:
! 268: <p>
1.5 flipk 269: Modifications made to files in /usr/src will become a file contained within
270: /usr/src, hiding the one in src-ctm. If changes are made via ctm to the
271: underlying src-ctm tree, those changes will <b>not</b> be seen if there is
272: a file in the upper layer hiding it.
273:
274: <p>
275: You should periodically unmount the
276: union and search for files which are local to the union filesystem.
1.13 ! naddy 277:
! 278: <pre>
! 279: umount /usr/src
! 280: find /usr/src -type f
! 281: </pre>
! 282:
! 283: <p>
! 284: The commands <tt><b>ls -W</b></tt> and <tt><b>rm -W</b></tt> will be
! 285: useful too, as objects named "whiteouts" in /usr/src will also hide
! 286: files in src-ctm.
! 287:
! 288: <p>
! 289: If you get the CVS tree, you can use the <tt><b>cvs checkout</b></tt>
! 290: command to check out a source tree from it, and each time you update
! 291: with ctm you can use the <tt><b>cvs update</b></tt> command to
! 292: update your source tree.
! 293:
! 294: <p>
! 295: To get an initial src tree:
! 296: <pre>
! 297: cd /usr
! 298: cvs -qd YOUR_CVS_TREE checkout -A src
! 299: </pre>
! 300: <p>
! 301: and after each ctm update:
! 302: <pre>
! 303: cd /usr/src
! 304: cvs -q update -PAd
! 305: </pre>
1.5 flipk 306:
307: <H3>How stable is ctm for OpenBSD?</H3>
308:
1.13 ! naddy 309: Ctm has been in use for over four years for OpenBSD, and for FreeBSD
! 310: for an even longer period of time. Ctm is very reliable and stable.
1.5 flipk 311:
312: <H3>More information</H3>
313:
1.13 ! naddy 314: If you want to learn more about ctm, a good place to start with is the
! 315: FreeBSD Handbook:
1.5 flipk 316:
317: <ul>
318: <li> <a href="http://www.FreeBSD.org/handbook/ctm.html">
319: http://www.FreeBSD.org/handbook/ctm.html
320: </a>
321: </ul>
322:
1.13 ! naddy 323: There are also man pages for all of the ctm utilities.
1.5 flipk 324:
325: <p>
326: Summary:
327:
328: <p>
329:
330: <ul>
331: <li> Mailing lists:
332:
333: <ul>
1.13 ! naddy 334: <li><tt>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</tt>
! 335: <li><tt>echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org</tt>
! 336: <li><tt>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</tt>
! 337: <li><tt>echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org</tt>
! 338: <li><tt>echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org</tt>
! 339: <li><tt>echo subscribe ctm-log | mail majordomo@OpenBSD.org</tt>
1.5 flipk 340: </ul>
341:
1.13 ! naddy 342: <li> FTP (for base and relative deltas):
1.5 flipk 343:
344: <ul>
1.10 deraadt 345: <li> <a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm">
1.13 ! naddy 346: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm</a>
1.10 deraadt 347: <li> <a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm">
1.13 ! naddy 348: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm</a>
1.10 deraadt 349: <li> <a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm">
1.13 ! naddy 350: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm</a>
1.5 flipk 351: </ul>
352: </ul>
353:
354: <p>
1.13 ! naddy 355: Important notes and announcements about ctm will be posted to:
1.5 flipk 356: <ul>
357: <li> misc@OpenBSD.org
358: <li> announce@OpenBSD.org
359: </ul>
360:
361: <p>
362:
363: <H3>Acknowledgements</H3>
364:
1.13 ! naddy 365: <ul>
1.5 flipk 366: <li> Poul-Henning Kamp (phk@FreeBSD.org) for making ctm possible and helping
1.13 ! naddy 367: getting the delta generation running
1.5 flipk 368:
369: <li> Theo de Raadt (deraadt@theos.com) for making OpenBSD possible and
1.13 ! naddy 370: initially providing the resources needed for creating the ctm
! 371: deltas on cvs.OpenBSD.org (now they are generated elsewhere), and
1.10 deraadt 372: for the nameserver magic
373:
374: <li> Todd C. Miller (millert@OpenBSD.org) for setting up and
375: maintaining the ctm mailinglists
376:
377: <li> Bob Beck (beck@OpenBSD.org), Artur Grabowski (art@OpenBSD.org),
378: Magnus Holmberg (mho@OpenBSD.org) and
379: Wolfram Schneider (wosch@FreeBSD.org) for setting up the ftp space
380: for the bases and deltas on ctm.*OpenBSD.org
1.1 graichen 381:
1.13 ! naddy 382: <li> Thomas Graichen (graichen@OpenBSD.org) for starting ctm for
! 383: OpenBSD and maintaining it all those years
1.1 graichen 384:
1.13 ! naddy 385: <li> Wim Vandeputte (wvdeputte@OpenBSD.org) for hosting ctm now
1.1 graichen 386:
1.13 ! naddy 387: <li> ... and all the others who contributed indirectly
! 388: </ul>
1.1 graichen 389:
390:
1.13 ! naddy 391: <p>
! 392: For any problems, suggestions, reports and questions regarding ctm
! 393: contact the ctm maintainer
! 394: <a href="mailto:hgw@d1906.inka.de">Hans Günter Weigand</a>.
! 395: <p>
1.5 flipk 396: OpenBSD/CTM logo designed for the OpenBSD Project by
397: <a href="mailto:flipk@openbsd.org">Phillip F Knaack</a>.
1.12 jufi 398: <p>
399: <hr>
400: <a href="index.html"><img height="24" width="24" src="back.gif" border="0" alt="OpenBSD" /></a>
401: <a href="mailto:www@openbsd.org">www@openbsd.org</a>
1.13 ! naddy 402: <br>
! 403: <small>$OpenBSD: ctm.html,v 1.12 2000/11/17 19:11:33 jufi Exp $</small>
1.1 graichen 404:
1.12 jufi 405: </body>
406: </html>
![[BACK]](/icons/back.gif){kind=icon}
Return to ctm.html CVS log ![[TXT]](/icons/text.gif){kind=icon}
![[DIR]](/icons/dir.gif){kind=icon}
Up to [local] / www