Annotation of www/ctm.html, Revision 1.14
1.1 graichen 1: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
2: <HTML>
3: <HEAD>
4: <TITLE>CTM</TITLE>
5: </HEAD>
1.5 flipk 6: <BODY BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#23238E">
1.1 graichen 7:
1.5 flipk 8: <center>
1.13 naddy 9: <img alt="CTM for OpenBSD" src="images/bsdctm.gif" width="401" height="126">
1.5 flipk 10: </center>
1.1 graichen 11:
1.14 ! naddy 12: <H3>What is CTM?</H3>
1.1 graichen 13:
14: <P>
1.14 ! naddy 15: CTM is a system which was designed by Poul-Henning Kamp for making
1.5 flipk 16: changes to a source tree available on a daily basis by email.
1.14 ! naddy 17: This is a good way to stay up to date with the current source tree or
! 18: CVS repository if you have a slow Internet connection, for instance via modem.
1.1 graichen 19:
1.5 flipk 20: <P>
1.14 ! naddy 21: CTM uses very low bandwidth for distributing the changes to the tree,
! 22: and compresses them using <tt><b>gzip -9</b></tt>. In contrast to
! 23: <a href="cvsup.html">CVSup</a>, CTM is written in plain C language
! 24: and therefore available for every platform supported by OpenBSD.
! 25:
! 26: <p>
! 27: For <b>any</b> problems, suggestions, reports and questions regarding CTM
! 28: contact the CTM maintainer
! 29: <a href="mailto:hgw@d1906.inka.de">Hans Günter Weigand</a>.
! 30: <small>(Please do not ask other OpenBSD people for CTM support,
! 31: as they are busy doing other things. Thank you.)</small>
1.1 graichen 32:
1.5 flipk 33: <H3>How does it work?</H3>
1.1 graichen 34:
1.5 flipk 35: <P>
1.14 ! naddy 36: The basic idea of CTM is that you subscribe to a special mailing list
1.5 flipk 37: for a particular source tree. Each day you receive an email message
38: containing all the changes to that tree during that period.
1.14 ! naddy 39: One such set of differences is called a <b>delta</b>.
1.1 graichen 40:
1.5 flipk 41: <P>
1.13 naddy 42: You begin with a base set, against which later deltas are patched. To
1.14 ! naddy 43: begin using CTM you download the latest base set and all deltas
1.13 naddy 44: generated after that. Once you have processed those you can process
45: later deltas you receive from the mailing list. The base sets are split
1.14 ! naddy 46: into pieces of 1400kB each for easier downloading over bad lines.
1.1 graichen 47:
1.5 flipk 48: <P>
1.13 naddy 49: Base sets are generated once every 50 relative deltas, so you never
50: have to grab more than 50 deltas to catch up.
51:
52: <P>
1.14 ! naddy 53: There are currently two kinds of base set files. For quite a while we
1.13 naddy 54: were making <tt><b>tar.gz</b></tt> files manually, but we are
1.14 ! naddy 55: switching back to real CTM base deltas. The latter contain MD5
1.13 naddy 56: checksums and are generated automatically, thus are a bit safer. You
1.14 ! naddy 57: can distinguish these two kinds of base set by their filename. An
1.13 naddy 58: example for a <tt><b>tar.gz</b></tt> file is
1.14 ! naddy 59: <tt><b>OpenBSD-cvs.1450.tar.gz</b></tt>, whereas a CTM base delta
1.13 naddy 60: would be named <tt><b>OpenBSD-cvs.1500A.gz</b></tt>. See below for
61: detailed instructions.
1.1 graichen 62:
1.14 ! naddy 63: <H3>How much bandwidth does CTM use?</H3>
1.1 graichen 64:
1.5 flipk 65: <P>
1.13 naddy 66: You obtain the base set and all relative deltas up to the recent one
67: via ftp from the sites listed below. Sizes are:
1.5 flipk 68:
69: <ul>
1.13 naddy 70: <li>Approximately 115MB for OpenBSD-cvs.
71: <li>Approximately 37MB for OpenBSD-cvs-x11.
72: <li>Approximately 65MB for OpenBSD-src.
73: <li>Approximately 35MB for OpenBSD-src-x11.
74: <li>Approximately 2MB for OpenBSD-ports.
1.14 ! naddy 75: <br>Keep in mind that these are compressed sizes. The uncompressed
1.13 naddy 76: files are about five times as large.
1.14 ! naddy 77: <li>Relative deltas vary from 10kB to 500kB, and occasionally a couple of
1.13 naddy 78: megabytes.
1.14 ! naddy 79: <li>CTM always splits deltas into 100kB mail messages which are reassembled
1.13 naddy 80: automatically.
1.14 ! naddy 81: <li>If a delta is larger than 5MB, it is not sent via mail. Instead,
! 82: you will receive a notification to get the delta by ftp. Larger
! 83: CTM updates are rare, though, usually occurring only after a major
! 84: import of something like binutils, perl, gcc, etc, or applying a
! 85: CVS tag to the CVS repository.
1.5 flipk 86: </ul>
1.1 graichen 87:
1.14 ! naddy 88: <H3>How do I use CTM?</H3>
1.1 graichen 89:
90: <UL>
1.14 ! naddy 91: <LI>Subscribe to the CTM mailing list for the source set you want.
1.5 flipk 92: <p>
1.14 ! naddy 93: There are several mailing lists related to CTM. There is a list for
! 94: updates to the CVS repository, a list for the source tree, a list
1.13 naddy 95: for the X11 part of both, and one for the ports tree, plus
1.14 ! naddy 96: a list for log messages from the CTM delta generation runs.
1.5 flipk 97: <p>
98: The following commands will subscribe you to the appropriate lists:
1.13 naddy 99: <p>
1.14 ! naddy 100: For the CVS repository (without the X11 part):
1.13 naddy 101: <pre>
102: echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org
103: </pre>
1.14 ! naddy 104: For the X11 part of the CVS repository:
1.13 naddy 105: <pre>
106: echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org
107: </pre>
108: For the source tree (without the X11 part):
109: <pre>
110: echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org
111: </pre>
112: For the X11 part of the source tree:
113: <pre>
114: echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org
115: </pre>
116: For the ports tree:
117: <pre>
118: echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org
119: </pre>
1.14 ! naddy 120: For the CTM log list:
1.13 naddy 121: <pre>
122: echo subscribe ctm-log | mail majordomo@OpenBSD.org
123: </pre>
124:
125: <p>
1.14 ! naddy 126: <li>Install the CTM program from a package or from the ports
1.13 naddy 127: collection. You'll find the port at
128: <tt><b>/usr/ports/devel/ctm</b></tt>.
129:
130: </ul>
131: <b>Note:</b> The examples below just cover OpenBSD-cvs for clarity. The
132: other source sets are handled in the same fashion.
133:
134: <p><ul>
135:
136: <li>Download the base set from an FTP site. You'll find it in the
137: subdirectory <tt><b>base/OpenBSD-cvs</b></tt>. Get all files in
138: that directory. Once you have them all you can create the base set
1.14 ! naddy 139: file and unpack it with the following commands, depending on the type
! 140: of file you got:
1.13 naddy 141:
142: <pre>
143: cat split/OpenBSD-cvs.1500A.gz.* > OpenBSD-cvs.1500A.gz
144: cd target
145: ctm -v -v -v .../OpenBSD-cvs.1500A.gz
146: </pre>
147: <p>
148: or
149:
150: <pre>
151: cat split/OpenBSD-cvs.1500.tar.gz.* > OpenBSD-cvs.1500.tar.gz
152: cd target
153: tar -xzvf .../OpenBSD-cvs.1500.tar.gz
154: </pre>
155:
156: <p>
157: where <tt><b>split</b></tt> is the directory in which you have
158: placed the files you downloaded and <tt><b>target</b></tt> is the
1.14 ! naddy 159: place you have chosen for unpacking the source set.
1.5 flipk 160:
1.14 ! naddy 161: <p> It is not necessary to keep these base set files around, if
! 162: you're sure you've got a complete tree. But you can keep the base set
! 163: file(s) for backup purposes. However, be sure to note the number of
! 164: the base set before continuing.
1.5 flipk 165:
166: <p>
1.13 naddy 167: <li>You now need to get the deltas which have been generated since the
168: base set. Go to the FTP site in the subdirectory
169: <tt><b>OpenBSD-cvs</b></tt> and download every file whose number is
170: <em>larger</em> than the number of the base set you used. Then apply
171: these deltas:
1.5 flipk 172:
1.13 naddy 173: <pre>
174: cd target
175: ctm -v -v -v deltas/OpenBSD-cvs.*
176: </pre>
1.5 flipk 177:
1.13 naddy 178: <p>
179: where <tt><b>target</b></tt> is the directory where your source set
180: tree is, and <tt><b>deltas</b></tt> is where you have stored the
181: deltas.
1.5 flipk 182:
1.13 naddy 183: <p>
184: <LI>Wait until you get your first delta from the mailing list.
185: <p>
1.14 ! naddy 186: <LI>Unpack, assemble, and apply the CTM delta using <tt><b>ctm_rmail</b></tt>.
1.13 naddy 187: <p>
188:
189: <pre>
190: ctm_rmail -p /tmp -d deltas -b target folder
191: </pre>
1.5 flipk 192:
193: <p>
1.13 naddy 194: where <tt><b>folder</b></tt> is the mail folder containing the delta
1.14 ! naddy 195: mail. This will decode the CTM delta and save it in the
1.13 naddy 196: <tt><b>deltas</b></tt> directory. The delta will be a file of the
197: form <tt><b>OpenBSD-cvs.<var>XXXX</var>.gz</b></tt>, where <var>XXXX</var>
198: is the number of the delta. Then the delta will be applied to the
199: directory tree located at <tt><b>target</b></tt>, the place where you
1.14 ! naddy 200: unpacked the base set.
1.5 flipk 201:
202: <p>
1.13 naddy 203: Repeat this last step every time you receive a delta from the
204: mailing list. If you like you can automate this step using procmail
205: or any similar program.
206: </ul>
1.5 flipk 207:
208: <p>
1.14 ! naddy 209: The addresses of the FTP servers carrying CTM deltas are:
1.10 deraadt 210:
211: <ul>
212: <li><a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/">
213: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/
214: </a>
215: </ul>
1.5 flipk 216:
217: <ul>
1.10 deraadt 218: <li><a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/">
219: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/
220: </a>
221: </ul>
222:
223: <ul>
224: <li><a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/">
225: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/
1.5 flipk 226: </a>
227: </ul>
228:
1.14 ! naddy 229: <ul>
! 230: <li>An experimental server can be reached at
! 231: <a href="ftp://openbsd.rug.ac.be:/ftp/pub/OpenBSD-ctm/">
! 232: ftp://openbsd.rug.ac.be:/ftp/pub/OpenBSD-ctm/
! 233: </a>.
! 234: This server has a slightly different directory layout. Base sets are
! 235: available as big single files in the <tt><b>base</b></tt> directory.
! 236: The split files are in <tt><b>base-split</b></tt>.
! 237: </ul>
! 238:
1.5 flipk 239: <p>
1.14 ! naddy 240: <b>NOTE:</b> <a href="mailto:hgw@d1906.inka.de">I'm</a> very interested
! 241: in finding other sites around the world that would like to
! 242: mirror these deltas.
1.5 flipk 243: <p>
1.1 graichen 244:
1.14 ! naddy 245: <h3>Should I choose the source tree or the CVS repository?</h3>
1.1 graichen 246:
1.5 flipk 247: <P>
1.14 ! naddy 248: This depends on your diskspace, among other factors. Since CTM
! 249: doesn't deal very well with files which have been modified locally,
! 250: the "better" choice is probably the CVS repository. You can check out
! 251: your working copy of the source tree from your CVS repository then,
! 252: and keep your local modifications in your source tree. CVS is
! 253: smart enough to merge in your local changes. You can also use the
! 254: full range of CVS commands except <tt><b>cvs commit</b></tt>.
1.5 flipk 255:
256: <p>
1.14 ! naddy 257: The drawback, however, is the amount of diskspace it takes. A
! 258: checked-out source tree takes about 350MB or so, but if you have your
! 259: own CVS repository you need more than 1Gb for it, <b>plus</b> another
! 260: 350MB for the checked out tree.
1.13 naddy 261: This doesn't count the 100MB or more (depending on the architecture)
262: you will need to do a build, i.e. compile the sources.
1.5 flipk 263:
264: <p>
265: If you just get the source tree, you will need to deal with local
266: modifications. One way to do this is to use the <b>union</b> filesystem,
267: although there have been reports that the <b>union</b> filesystem can be
268: unstable if both the upper and lower layers reside on the same physical
1.6 todd 269: filesystem. If you place your upper and lower layers on separate filesystems
1.5 flipk 270: you should have no problems. A sample of this is as follows:
271:
272: <p>
1.14 ! naddy 273: Suppose your CTM-updated tree is in <tt><b>/usr/src-ctm</b></tt>, and
1.13 naddy 274: your real source tree, the one you make your modifications to and do
275: your builds from, is in <tt><b>/usr/src</b></tt>.
276: <tt><b>/usr/src</b></tt> should be initially an empty directory. The
277: following command will set up the union mount:
278:
279: <pre>
280: mount -t union -o -b /usr/src-ctm /usr/src
281: </pre>
282:
283: <p>
1.14 ! naddy 284: Modifications made to files in <tt><b>/usr/src</b></tt> will become a
! 285: file contained within <tt><b>/usr/src</b></tt>, hiding the one in
! 286: <tt><b>src-ctm</b></tt>. If changes are made via CTM to the
! 287: underlying <tt><b>src-ctm</b></tt> tree, those changes will <b>not</b>
! 288: be seen if there is a file in the upper layer hiding it.
1.5 flipk 289:
290: <p>
291: You should periodically unmount the
292: union and search for files which are local to the union filesystem.
1.13 naddy 293:
294: <pre>
295: umount /usr/src
296: find /usr/src -type f
297: </pre>
298:
299: <p>
300: The commands <tt><b>ls -W</b></tt> and <tt><b>rm -W</b></tt> will be
1.14 ! naddy 301: useful too, as objects named "whiteouts" in <tt><b>/usr/src</b></tt> will also hide
! 302: files in <tt><b>src-ctm</b></tt>.
1.13 naddy 303:
304: <p>
1.14 ! naddy 305: If you get the CVS repository, you can use the <tt><b>cvs checkout</b></tt>
! 306: command to check out a source tree from it. Each time you update
! 307: with CTM you can use the <tt><b>cvs update</b></tt> command to
! 308: update your source repository.
1.13 naddy 309:
310: <p>
1.14 ! naddy 311: To get an initial src repository:
1.13 naddy 312: <pre>
313: cd /usr
1.14 ! naddy 314: cvs -qd YOUR_CVS_REPOSITORY checkout -A src
1.13 naddy 315: </pre>
316: <p>
1.14 ! naddy 317: and after each CTM update:
1.13 naddy 318: <pre>
319: cd /usr/src
320: cvs -q update -PAd
321: </pre>
1.5 flipk 322:
1.14 ! naddy 323: <H3>How stable is CTM for OpenBSD?</H3>
1.5 flipk 324:
1.14 ! naddy 325: CTM has been in use for over four years for OpenBSD, and for FreeBSD
! 326: for an even longer period of time. CTM is very reliable and stable.
! 327:
! 328: <h3>What does the future hold for CTM?</h3>
! 329:
! 330: There are numerous (small) changes and optimizations possible.
! 331: Among them are:
! 332:
! 333: <ul>
! 334: <li> CTM never sents a delta bigger than 5MB via mail. This could be
! 335: changed in two ways: The maximum delta size could be increased,
! 336: or a queue could be installed, from which only a certain number
! 337: of messages are mailed out daily. I. e., if a delta is 6MB in
! 338: size, it is split into 96 mail messages. These are put into a
! 339: queue. On the first day, the first 50 messages are sent, on the
! 340: second day the remaining 46, plus the first 4 messages of the
! 341: following delta.
! 342:
! 343: <li> CTM could be run two times per day.
! 344:
! 345: </ul>
1.5 flipk 346:
347: <H3>More information</H3>
348:
1.14 ! naddy 349: If you want to learn more about CTM, a good place to start with is the
1.13 naddy 350: FreeBSD Handbook:
1.5 flipk 351:
352: <ul>
353: <li> <a href="http://www.FreeBSD.org/handbook/ctm.html">
354: http://www.FreeBSD.org/handbook/ctm.html
355: </a>
356: </ul>
357:
1.14 ! naddy 358: There are also man pages for all of the CTM utilities.
1.5 flipk 359:
1.14 ! naddy 360: <h3>Summary:</h3>
1.5 flipk 361:
362: <p>
363:
364: <ul>
365: <li> Mailing lists:
366:
367: <ul>
1.13 naddy 368: <li><tt>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</tt>
369: <li><tt>echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org</tt>
370: <li><tt>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</tt>
371: <li><tt>echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org</tt>
372: <li><tt>echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org</tt>
373: <li><tt>echo subscribe ctm-log | mail majordomo@OpenBSD.org</tt>
1.5 flipk 374: </ul>
375:
1.14 ! naddy 376: <li> FTP (for base sets and relative deltas):
1.5 flipk 377:
378: <ul>
1.10 deraadt 379: <li> <a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm">
1.13 naddy 380: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm</a>
1.10 deraadt 381: <li> <a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm">
1.13 naddy 382: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm</a>
1.10 deraadt 383: <li> <a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm">
1.13 naddy 384: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm</a>
1.5 flipk 385: </ul>
386: </ul>
387:
388: <p>
1.14 ! naddy 389: Important notes and announcements about CTM will be posted to:
1.5 flipk 390: <ul>
391: <li> misc@OpenBSD.org
392: <li> announce@OpenBSD.org
393: </ul>
394:
395: <p>
396:
397: <H3>Acknowledgements</H3>
398:
1.13 naddy 399: <ul>
1.14 ! naddy 400: <li> Poul-Henning Kamp (phk@FreeBSD.org) for making CTM possible and helping
1.13 naddy 401: getting the delta generation running
1.5 flipk 402:
403: <li> Theo de Raadt (deraadt@theos.com) for making OpenBSD possible and
1.14 ! naddy 404: initially providing the resources needed for creating the CTM
1.13 naddy 405: deltas on cvs.OpenBSD.org (now they are generated elsewhere), and
1.10 deraadt 406: for the nameserver magic
407:
408: <li> Todd C. Miller (millert@OpenBSD.org) for setting up and
1.14 ! naddy 409: maintaining the CTM mailinglists
1.10 deraadt 410:
411: <li> Bob Beck (beck@OpenBSD.org), Artur Grabowski (art@OpenBSD.org),
412: Magnus Holmberg (mho@OpenBSD.org) and
413: Wolfram Schneider (wosch@FreeBSD.org) for setting up the ftp space
1.14 ! naddy 414: for the base sets and deltas on ctm.*OpenBSD.org
1.1 graichen 415:
1.14 ! naddy 416: <li> Thomas Graichen (graichen@OpenBSD.org) for starting CTM for
1.13 naddy 417: OpenBSD and maintaining it all those years
1.1 graichen 418:
1.14 ! naddy 419: <li> Wim Vandeputte (wvdputte@OpenBSD.org) for hosting CTM now
1.1 graichen 420:
1.13 naddy 421: <li> ... and all the others who contributed indirectly
422: </ul>
1.1 graichen 423:
424:
1.13 naddy 425: <p>
1.5 flipk 426: OpenBSD/CTM logo designed for the OpenBSD Project by
427: <a href="mailto:flipk@openbsd.org">Phillip F Knaack</a>.
1.12 jufi 428: <p>
429: <hr>
430: <a href="index.html"><img height="24" width="24" src="back.gif" border="0" alt="OpenBSD" /></a>
431: <a href="mailto:www@openbsd.org">www@openbsd.org</a>
1.13 naddy 432: <br>
1.14 ! naddy 433: <small>$OpenBSD: ctm.html,v 1.13 2001/01/01 22:01:21 naddy Exp $</small>
1.1 graichen 434:
1.12 jufi 435: </body>
436: </html>
![[BACK]](/icons/back.gif){kind=icon}
Return to ctm.html CVS log ![[TXT]](/icons/text.gif){kind=icon}
![[DIR]](/icons/dir.gif){kind=icon}
Up to [local] / www