Annotation of www/ctm.html, Revision 1.11
1.1 graichen 1: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
2: <HTML>
3: <HEAD>
4: <TITLE>CTM</TITLE>
5: </HEAD>
1.5 flipk 6: <BODY BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#23238E">
1.1 graichen 7:
1.5 flipk 8: <center>
1.8 deraadt 9: <img alt="CTM for OpenBSD" src=images/bsdctm.gif width=401 height=126>
1.5 flipk 10: </center>
1.1 graichen 11:
1.5 flipk 12: <H3>What is ctm?</H3>
1.1 graichen 13:
14: <P>
1.5 flipk 15: Ctm is a system which was designed by Poul-Henning Kamp for making
16: changes to a source tree available on a daily basis by email.
17: This is a good way to stay up to date with the current source- or
18: cvs-tree if you have a bad internet connection - for instance via modem.
1.1 graichen 19:
1.5 flipk 20: <P>
1.6 todd 21: Ctm uses very low bandwidth for distributing the changes to the
1.5 flipk 22: tree, and compresses them using gzip -9.
1.1 graichen 23:
1.5 flipk 24: <H3>How does it work?</H3>
1.1 graichen 25:
1.5 flipk 26: <P>
27: The basic idea of ctm is that you subscribe to a special mailing list
28: for a particular source tree. Each day you receive an email message
29: containing all the changes to that tree during that period.
30: One set of differences is called a "delta".
1.1 graichen 31:
1.5 flipk 32: <P>
33: You begin with a base set against which later deltas are patched.
34: To begin using ctm you must grab the latest base set and all deltas
35: after that; once you have processed those you can process later
1.10 deraadt 36: deltas you receive on the mailing list. The base sets are split
37: into pieces of 1400k each for easier downloading over bad lines
38: and you can even put them onto floppies this way.
1.1 graichen 39:
1.5 flipk 40: <P>
1.9 espie 41: Bases are generated approximately once every 100 relative deltas, so you never
1.5 flipk 42: have to grab more than 100 deltas to catch up.
1.1 graichen 43:
1.5 flipk 44: <H3>How much bandwidth does ctm use?</H3>
1.1 graichen 45:
1.5 flipk 46: <P>
47: As mentioned above, the base set contains a full source tree in TAR format,
48: and is thus quite large. You must obtain the base set and all relative
49: deltas up to present via ftp from the site listed below. Sizes are:
50:
51: <ul>
1.10 deraadt 52: <li> approximately 115Mb for OpenBSD-cvs
53: <li> approximately 37Mb for OpenBSD-cvs-x11
54: <li> approximately 65Mb for OpenBSD-src
55: <li> approximately 35Mb for OpenBSD-src-x11
56: <li> approximately 2Mb for OpenBSD-ports
57: <li> keep in mind that that are compressed sizes - count about five times
58: of those sizes for the unpacked files
59: <li> relative deltas vary from 10k to 100k, and occasionally a couple of Mb
1.5 flipk 60: <li> ctm always splits deltas into 100k messages which are reassembled
61: automatically
62: </ul>
1.1 graichen 63:
1.5 flipk 64: <P>
1.6 todd 65: Larger ctm updates are rare, though, usually occurring only after a major
1.5 flipk 66: import of something like binutils, perl, etc.
1.1 graichen 67:
1.5 flipk 68: <H3>How do I use ctm?</H3>
1.1 graichen 69:
70: <UL>
1.5 flipk 71: <LI>Subscribe to the ctm mailing list for the source set you want.
72: <p>
73: There are several mailing lists related to ctm. There is a list for
1.10 deraadt 74: updates to the updates to cvs tree, a list for the source tree, a list
75: for the X11 part of both and one for the ports tree plus
76: a list for seeing log messages from the ctm delta generator runs.
1.5 flipk 77: <p>
78: The following commands will subscribe you to the appropriate lists:
79: <br><br><tt>
1.10 deraadt 80: For the cvs tree (without the X11 part):<br>
81: <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b><br>
82: For the X11 part of the cvs tree:<br>
83: <b>echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org</b><br>
84: For the source tree (without the X11 part): <br>
1.5 flipk 85: <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b><br>
1.10 deraadt 86: For the X11 part of the source tree: <br>
87: <b>echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org</b><br>
88: For the ports tree: <br>
89: <b>echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org</b><br>
1.5 flipk 90: For the ctm log list:<br>
91: <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b><br>
92: </tt><br>
93:
94: <LI>Wait until you get your first delta.
95:
96: <LI>Unpack and assemble the ctm delta using ctm_rmail.
97: <br>
98: <br>
99: <tt>
100: <b>ctm_rmail -p. -d. -b. folder</b>
101: </tt>
102: <br><br>
103: where <b>folder</b> is the mail folder containing the delta mail.
104: <br>
105: This will decode the ctm delta and place it in the working directory.
106: The delta will be a file of the form OpenBSD-XXX.YYYY.gz, where XXX is
107: either "src" or "cvs", and YYYY is the number of the delta.
108: <p>
109:
110: <li>Grab the base set from the ftp site below, in the subdirectory
1.10 deraadt 111: <b>base/OpenBSD-cvs</b> (for OpenBSD-cvs) or <b>base/OpenBSD-src</b>
112: (for OpenBSD-src). Grab all files in that directory, and once you have
1.5 flipk 113: them all you can create the base source tree with the following command:
114:
115: <br><br>
116: <tt>
117: <b>cd target <br>
1.7 graichen 118: cat /split/OpenBSD-* | tar zxf - <br>
1.5 flipk 119: </b></tt><br>
120:
1.7 graichen 121: where <b>split</b> is the directory in which you placed the files you
1.5 flipk 122: grabbed.
123:
124: <p>
125: It is no longer necessary to keep these files around, if you're sure
126: you've got a complete tree. However, be sure to note the number of the
127: base, before continuing.
128:
129: <p>
130: You now need to grab the deltas which have been generated since the base
1.10 deraadt 131: was generated. Go to the ftp site below in the subdirectory
132: <b>OpenBSD-cvs</b> or <b>OpenBSD-src</b> for instance and grab
133: every file whose number is <it>larger</it> than the number of the
134: base you used.
1.5 flipk 135:
136: <p>
1.10 deraadt 137: The addresses for the base and deltas are:
138:
139: <ul>
140: <li><a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/">
141: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/
142: </a>
143: </ul>
1.5 flipk 144:
145: <ul>
1.10 deraadt 146: <li><a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/">
147: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/
148: </a>
149: </ul>
150:
151: <ul>
152: <li><a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/">
153: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/
1.5 flipk 154: </a>
155: </ul>
156:
157: <p>
158: <ul>
159: <li><b>NOTE:</b> I'm very interested in finding other sites around the world
160: that would be interested in mirroring these deltas.
161: </ul>
162: <p>
163:
164: <LI> Run ctm.
165: <br>
166: <br>
167: <tt>
168: <b>cd target<br>
1.10 deraadt 169: ctm -v -v /deltas/OpenBSD-cvs.*</b><br></tt>
1.5 flipk 170: or<br><tt>
1.10 deraadt 171: where <b>target</b> is the directory where you want your cvs tree to be,
1.5 flipk 172: and <b>deltas</b> is where you have stored the deltas.
173: <p>
174: It will take some time to run all of the deltas, but the daily invocations
175: of <b>ctm_rmail</b> and <b>ctm</b> will not take nearly as long.
176: </ul>
1.1 graichen 177:
1.5 flipk 178: <h3>Should I choose the source or cvs tree?</h3>
1.1 graichen 179:
1.5 flipk 180: <P>
181: This depends on your diskspace, among other factors. Since ctm doesn't
182: yet deal with files which have been modified outside of the ctm process,
183: the "better" choice is probably the cvs-tree. Then you can check out
184: your working source tree from your cvs-tree, and keep your local mods
185: in your source tree, leaving only ctm to touch your cvs tree.
186:
187: <p>
188: The drawback, however, is the amount of diskspace it takes. A checked-out
1.10 deraadt 189: source tree takes about 300Mb or so, but if you have your own cvs tree you
190: need 400Mb or so for it, <b>plus</b> another 300Mb for the checked out tree.
1.5 flipk 191:
192: <p>
1.10 deraadt 193: This doesn't count the 100Mb or more you will need to do a build (depending
1.5 flipk 194: on the architecture).
195:
196: <p>
197: If you just get the source tree, you will need to deal with local
198: modifications. One way to do this is to use the <b>union</b> filesystem,
199: although there have been reports that the <b>union</b> filesystem can be
200: unstable if both the upper and lower layers reside on the same physical
1.6 todd 201: filesystem. If you place your upper and lower layers on separate filesystems
1.5 flipk 202: you should have no problems. A sample of this is as follows:
203:
204: <p>
205: Suppose your ctm-updated tree is in <b>/usr/src-ctm</b>, and your real
206: source tree, the one you make your modifications to and do your builds from,
207: is in <b>/usr/src</b>. <b>/usr/src</b> should be initially an empty
208: directory. The following command will set up the union mount:
209: <br><br>
210: <tt>
211: <b>mount -t union -o -b /usr/src-ctm /usr/src</b>
212: </tt>
213: <br><br>
214: Modifications made to files in /usr/src will become a file contained within
215: /usr/src, hiding the one in src-ctm. If changes are made via ctm to the
216: underlying src-ctm tree, those changes will <b>not</b> be seen if there is
217: a file in the upper layer hiding it.
218:
219: <p>
220: You should periodically unmount the
221: union and search for files which are local to the union filesystem.
222: <br><br>
223: <tt>
224: <b>umount /usr/src <br>
225: find /usr/src -type f</b> <br>
226: </tt>
227: <br><br>
228: The commands <b>ls -W</b> and <b>rm -W</b> will be useful too, as objects
229: named "whiteouts" in /usr/src will also hide files in src-ctm.
230:
231: <p>
232: If you get the cvs-tree, you can use the "cvs checkout" command to check
233: out a source tree from it, and each time you update with ctm you can use
234: the "cvs update" command to update your source tree.
235:
236: <br><br>
237: To get an initial src tree:<br>
238: <tt>
239: <b>cd /usr <br>
240: cvs -qd YOUR_CVS_TREE checkout src</b> <br><br>
241: </tt>
242: and after each ctm update:<br>
243: <tt>
244: <b>cd /usr/src <br>
245: cvs -q update -PAd</b> <br>
246: </tt>
247:
248: <H3>How stable is ctm for OpenBSD?</H3>
249:
250: Ctm has been in use for over a year for OpenBSD, and for FreeBSD for a
251: number of years. Ctm is quite reliable and stable.
252:
253: <H3>More information</H3>
254:
255: If you have problems with ctm, a good place to start is the FreeBSD
256: handbook:
257:
258: <ul>
259: <li> <a href="http://www.FreeBSD.org/handbook/ctm.html">
260: http://www.FreeBSD.org/handbook/ctm.html
261: </a>
262: </ul>
263:
264: There are also man pages for all of the ctm utilities included in the
265: source package. If you have OpenBSD, you should already have the man
266: pages, source, and binaries. If not you can obtain them from the same
267: ftp site as the deltas below.
268:
269: <p>
270: You can also contact the OpenBSD mailing lists or myself if you have
271: difficulties with ctm, although my free time tends to be rather limited.
272:
273: <p>
274: Summary:
275:
276: <p>
277:
278: <ul>
279: <li> Mailing lists:
280:
281: <ul>
1.10 deraadt 282: <li> <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b>
283: <li> <b>echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org</b>
1.5 flipk 284: <li> <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b>
1.10 deraadt 285: <li> <b>echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org</b>
286: <li> <b>echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org</b>
1.5 flipk 287: <li> <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b>
288: </ul>
289:
290: <li> FTP (for base and relative deltas; and OpenBSD has ctm in the source
291: tree, but if you don't have source or binaries, the source is here too):
292:
293: <ul>
1.10 deraadt 294: <li> <a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm">
295: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm
296: </a>
297: <li> <a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm">
298: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm
1.5 flipk 299: </a>
1.10 deraadt 300: <li> <a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm">
301: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm
1.5 flipk 302: </a>
303: </ul>
304: </ul>
305:
306: <p>
307: I will make any important notes and announcements about ctm in:
308: <ul>
309: <li> misc@OpenBSD.org
310: <li> announce@OpenBSD.org
311: </ul>
312:
313: <p>
314: <b>NOTE:</b> If you use ctm for OpenBSD, please send me email so I
315: know how many people are using it!
316:
317: <H3>Acknowledgements</H3>
318:
319: <li> Poul-Henning Kamp (phk@FreeBSD.org) for making ctm possible and helping
320: me getting the delta generation running.
321:
322: <li> Theo de Raadt (deraadt@theos.com) for making OpenBSD possible and
1.10 deraadt 323: initially giving me the resources i needed for creating the ctm
324: deltas on cvs.OpenBSD.org (now they are generated elsewhere) and
325: for the nameserver magic
326:
327: <li> Todd C. Miller (millert@OpenBSD.org) for setting up and
328: maintaining the ctm mailinglists
329:
330: <li> Bob Beck (beck@OpenBSD.org), Artur Grabowski (art@OpenBSD.org),
331: Magnus Holmberg (mho@OpenBSD.org) and
332: Wolfram Schneider (wosch@FreeBSD.org) for setting up the ftp space
333: for the bases and deltas on ctm.*OpenBSD.org
1.1 graichen 334:
1.5 flipk 335: <li> ... and all the others who contributed indirectly ..
336: </ul>
1.1 graichen 337:
1.5 flipk 338: <P>
339: Good luck!
1.1 graichen 340:
1.5 flipk 341: <P>
342: t
1.1 graichen 343:
1.5 flipk 344: <HR>
1.1 graichen 345:
346: <ADDRESS><A HREF="mailto:graichen@OpenBSD.org">Thomas Graichen</A></ADDRESS>
1.5 flipk 347: <br>
348: OpenBSD/CTM logo designed for the OpenBSD Project by
349: <a href="mailto:flipk@openbsd.org">Phillip F Knaack</a>.
1.1 graichen 350:
1.5 flipk 351: <P>
352: <!-- Created: Mon Oct 28 22:20:54 MET 1996 -->
353: <!-- hhmts start -->
1.11 ! wvdputte 354: $OpenBSD$
1.5 flipk 355: <!-- hhmts end -->
1.1 graichen 356:
357: </BODY>
358: </HTML>