Annotation of www/ctm.html, Revision 1.12
1.1 graichen 1: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
2: <HTML>
3: <HEAD>
4: <TITLE>CTM</TITLE>
5: </HEAD>
1.5 flipk 6: <BODY BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#23238E">
1.1 graichen 7:
1.5 flipk 8: <center>
1.8 deraadt 9: <img alt="CTM for OpenBSD" src=images/bsdctm.gif width=401 height=126>
1.5 flipk 10: </center>
1.1 graichen 11:
1.5 flipk 12: <H3>What is ctm?</H3>
1.1 graichen 13:
14: <P>
1.5 flipk 15: Ctm is a system which was designed by Poul-Henning Kamp for making
16: changes to a source tree available on a daily basis by email.
17: This is a good way to stay up to date with the current source- or
18: cvs-tree if you have a bad internet connection - for instance via modem.
1.1 graichen 19:
1.5 flipk 20: <P>
1.6 todd 21: Ctm uses very low bandwidth for distributing the changes to the
1.5 flipk 22: tree, and compresses them using gzip -9.
1.1 graichen 23:
1.5 flipk 24: <H3>How does it work?</H3>
1.1 graichen 25:
1.5 flipk 26: <P>
27: The basic idea of ctm is that you subscribe to a special mailing list
28: for a particular source tree. Each day you receive an email message
29: containing all the changes to that tree during that period.
30: One set of differences is called a "delta".
1.1 graichen 31:
1.5 flipk 32: <P>
33: You begin with a base set against which later deltas are patched.
34: To begin using ctm you must grab the latest base set and all deltas
35: after that; once you have processed those you can process later
1.10 deraadt 36: deltas you receive on the mailing list. The base sets are split
37: into pieces of 1400k each for easier downloading over bad lines
38: and you can even put them onto floppies this way.
1.1 graichen 39:
1.5 flipk 40: <P>
1.9 espie 41: Bases are generated approximately once every 100 relative deltas, so you never
1.5 flipk 42: have to grab more than 100 deltas to catch up.
1.1 graichen 43:
1.5 flipk 44: <H3>How much bandwidth does ctm use?</H3>
1.1 graichen 45:
1.5 flipk 46: <P>
47: As mentioned above, the base set contains a full source tree in TAR format,
48: and is thus quite large. You must obtain the base set and all relative
49: deltas up to present via ftp from the site listed below. Sizes are:
50:
51: <ul>
1.10 deraadt 52: <li> approximately 115Mb for OpenBSD-cvs
53: <li> approximately 37Mb for OpenBSD-cvs-x11
54: <li> approximately 65Mb for OpenBSD-src
55: <li> approximately 35Mb for OpenBSD-src-x11
56: <li> approximately 2Mb for OpenBSD-ports
57: <li> keep in mind that that are compressed sizes - count about five times
58: of those sizes for the unpacked files
59: <li> relative deltas vary from 10k to 100k, and occasionally a couple of Mb
1.5 flipk 60: <li> ctm always splits deltas into 100k messages which are reassembled
61: automatically
62: </ul>
1.1 graichen 63:
1.5 flipk 64: <P>
1.6 todd 65: Larger ctm updates are rare, though, usually occurring only after a major
1.5 flipk 66: import of something like binutils, perl, etc.
1.1 graichen 67:
1.5 flipk 68: <H3>How do I use ctm?</H3>
1.1 graichen 69:
70: <UL>
1.5 flipk 71: <LI>Subscribe to the ctm mailing list for the source set you want.
72: <p>
73: There are several mailing lists related to ctm. There is a list for
1.10 deraadt 74: updates to the updates to cvs tree, a list for the source tree, a list
75: for the X11 part of both and one for the ports tree plus
76: a list for seeing log messages from the ctm delta generator runs.
1.5 flipk 77: <p>
78: The following commands will subscribe you to the appropriate lists:
79: <br><br><tt>
1.10 deraadt 80: For the cvs tree (without the X11 part):<br>
81: <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b><br>
82: For the X11 part of the cvs tree:<br>
83: <b>echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org</b><br>
84: For the source tree (without the X11 part): <br>
1.5 flipk 85: <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b><br>
1.10 deraadt 86: For the X11 part of the source tree: <br>
87: <b>echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org</b><br>
88: For the ports tree: <br>
89: <b>echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org</b><br>
1.5 flipk 90: For the ctm log list:<br>
91: <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b><br>
92: </tt><br>
93:
94: <LI>Wait until you get your first delta.
95:
96: <LI>Unpack and assemble the ctm delta using ctm_rmail.
97: <br>
98: <br>
99: <tt>
100: <b>ctm_rmail -p. -d. -b. folder</b>
101: </tt>
102: <br><br>
103: where <b>folder</b> is the mail folder containing the delta mail.
104: <br>
105: This will decode the ctm delta and place it in the working directory.
106: The delta will be a file of the form OpenBSD-XXX.YYYY.gz, where XXX is
107: either "src" or "cvs", and YYYY is the number of the delta.
108: <p>
109:
110: <li>Grab the base set from the ftp site below, in the subdirectory
1.10 deraadt 111: <b>base/OpenBSD-cvs</b> (for OpenBSD-cvs) or <b>base/OpenBSD-src</b>
112: (for OpenBSD-src). Grab all files in that directory, and once you have
1.5 flipk 113: them all you can create the base source tree with the following command:
114:
115: <br><br>
116: <tt>
117: <b>cd target <br>
1.7 graichen 118: cat /split/OpenBSD-* | tar zxf - <br>
1.5 flipk 119: </b></tt><br>
120:
1.7 graichen 121: where <b>split</b> is the directory in which you placed the files you
1.5 flipk 122: grabbed.
123:
124: <p>
125: It is no longer necessary to keep these files around, if you're sure
126: you've got a complete tree. However, be sure to note the number of the
127: base, before continuing.
128:
129: <p>
130: You now need to grab the deltas which have been generated since the base
1.10 deraadt 131: was generated. Go to the ftp site below in the subdirectory
132: <b>OpenBSD-cvs</b> or <b>OpenBSD-src</b> for instance and grab
133: every file whose number is <it>larger</it> than the number of the
134: base you used.
1.5 flipk 135:
136: <p>
1.10 deraadt 137: The addresses for the base and deltas are:
138:
139: <ul>
140: <li><a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/">
141: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm/
142: </a>
143: </ul>
1.5 flipk 144:
145: <ul>
1.10 deraadt 146: <li><a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/">
147: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm/
148: </a>
149: </ul>
150:
151: <ul>
152: <li><a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/">
153: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm/
1.5 flipk 154: </a>
155: </ul>
156:
157: <p>
158: <ul>
159: <li><b>NOTE:</b> I'm very interested in finding other sites around the world
160: that would be interested in mirroring these deltas.
161: </ul>
162: <p>
163:
164: <LI> Run ctm.
165: <br>
166: <br>
167: <tt>
168: <b>cd target<br>
1.10 deraadt 169: ctm -v -v /deltas/OpenBSD-cvs.*</b><br></tt>
170: where <b>target</b> is the directory where you want your cvs tree to be,
1.5 flipk 171: and <b>deltas</b> is where you have stored the deltas.
172: <p>
173: It will take some time to run all of the deltas, but the daily invocations
174: of <b>ctm_rmail</b> and <b>ctm</b> will not take nearly as long.
175: </ul>
1.1 graichen 176:
1.5 flipk 177: <h3>Should I choose the source or cvs tree?</h3>
1.1 graichen 178:
1.5 flipk 179: <P>
180: This depends on your diskspace, among other factors. Since ctm doesn't
181: yet deal with files which have been modified outside of the ctm process,
182: the "better" choice is probably the cvs-tree. Then you can check out
183: your working source tree from your cvs-tree, and keep your local mods
184: in your source tree, leaving only ctm to touch your cvs tree.
185:
186: <p>
187: The drawback, however, is the amount of diskspace it takes. A checked-out
1.10 deraadt 188: source tree takes about 300Mb or so, but if you have your own cvs tree you
189: need 400Mb or so for it, <b>plus</b> another 300Mb for the checked out tree.
1.5 flipk 190:
191: <p>
1.10 deraadt 192: This doesn't count the 100Mb or more you will need to do a build (depending
1.5 flipk 193: on the architecture).
194:
195: <p>
196: If you just get the source tree, you will need to deal with local
197: modifications. One way to do this is to use the <b>union</b> filesystem,
198: although there have been reports that the <b>union</b> filesystem can be
199: unstable if both the upper and lower layers reside on the same physical
1.6 todd 200: filesystem. If you place your upper and lower layers on separate filesystems
1.5 flipk 201: you should have no problems. A sample of this is as follows:
202:
203: <p>
204: Suppose your ctm-updated tree is in <b>/usr/src-ctm</b>, and your real
205: source tree, the one you make your modifications to and do your builds from,
206: is in <b>/usr/src</b>. <b>/usr/src</b> should be initially an empty
207: directory. The following command will set up the union mount:
208: <br><br>
209: <tt>
210: <b>mount -t union -o -b /usr/src-ctm /usr/src</b>
211: </tt>
212: <br><br>
213: Modifications made to files in /usr/src will become a file contained within
214: /usr/src, hiding the one in src-ctm. If changes are made via ctm to the
215: underlying src-ctm tree, those changes will <b>not</b> be seen if there is
216: a file in the upper layer hiding it.
217:
218: <p>
219: You should periodically unmount the
220: union and search for files which are local to the union filesystem.
221: <br><br>
222: <tt>
223: <b>umount /usr/src <br>
224: find /usr/src -type f</b> <br>
225: </tt>
226: <br><br>
227: The commands <b>ls -W</b> and <b>rm -W</b> will be useful too, as objects
228: named "whiteouts" in /usr/src will also hide files in src-ctm.
229:
230: <p>
231: If you get the cvs-tree, you can use the "cvs checkout" command to check
232: out a source tree from it, and each time you update with ctm you can use
233: the "cvs update" command to update your source tree.
234:
235: <br><br>
236: To get an initial src tree:<br>
237: <tt>
238: <b>cd /usr <br>
239: cvs -qd YOUR_CVS_TREE checkout src</b> <br><br>
240: </tt>
241: and after each ctm update:<br>
242: <tt>
243: <b>cd /usr/src <br>
244: cvs -q update -PAd</b> <br>
245: </tt>
246:
247: <H3>How stable is ctm for OpenBSD?</H3>
248:
249: Ctm has been in use for over a year for OpenBSD, and for FreeBSD for a
250: number of years. Ctm is quite reliable and stable.
251:
252: <H3>More information</H3>
253:
254: If you have problems with ctm, a good place to start is the FreeBSD
255: handbook:
256:
257: <ul>
258: <li> <a href="http://www.FreeBSD.org/handbook/ctm.html">
259: http://www.FreeBSD.org/handbook/ctm.html
260: </a>
261: </ul>
262:
263: There are also man pages for all of the ctm utilities included in the
264: source package. If you have OpenBSD, you should already have the man
265: pages, source, and binaries. If not you can obtain them from the same
266: ftp site as the deltas below.
267:
268: <p>
269: You can also contact the OpenBSD mailing lists or myself if you have
270: difficulties with ctm, although my free time tends to be rather limited.
271:
272: <p>
273: Summary:
274:
275: <p>
276:
277: <ul>
278: <li> Mailing lists:
279:
280: <ul>
1.10 deraadt 281: <li> <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b>
282: <li> <b>echo subscribe OpenBSD-cvs-x11 | mail majordomo@OpenBSD.org</b>
1.5 flipk 283: <li> <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b>
1.10 deraadt 284: <li> <b>echo subscribe OpenBSD-src-x11 | mail majordomo@OpenBSD.org</b>
285: <li> <b>echo subscribe OpenBSD-ports-ctm | mail majordomo@OpenBSD.org</b>
1.5 flipk 286: <li> <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b>
287: </ul>
288:
289: <li> FTP (for base and relative deltas; and OpenBSD has ctm in the source
290: tree, but if you don't have source or binaries, the source is here too):
291:
292: <ul>
1.10 deraadt 293: <li> <a href="ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm">
294: ftp://ctm.OpenBSD.org:/pub/OpenBSD-ctm
295: </a>
296: <li> <a href="ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm">
297: ftp://ctm.ca.OpenBSD.org:/pub/OpenBSD-ctm
1.5 flipk 298: </a>
1.10 deraadt 299: <li> <a href="ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm">
300: ftp://ctm.se.OpenBSD.org:/pub/OpenBSD-ctm
1.5 flipk 301: </a>
302: </ul>
303: </ul>
304:
305: <p>
306: I will make any important notes and announcements about ctm in:
307: <ul>
308: <li> misc@OpenBSD.org
309: <li> announce@OpenBSD.org
310: </ul>
311:
312: <p>
313: <b>NOTE:</b> If you use ctm for OpenBSD, please send me email so I
314: know how many people are using it!
315:
316: <H3>Acknowledgements</H3>
317:
318: <li> Poul-Henning Kamp (phk@FreeBSD.org) for making ctm possible and helping
319: me getting the delta generation running.
320:
321: <li> Theo de Raadt (deraadt@theos.com) for making OpenBSD possible and
1.10 deraadt 322: initially giving me the resources i needed for creating the ctm
323: deltas on cvs.OpenBSD.org (now they are generated elsewhere) and
324: for the nameserver magic
325:
326: <li> Todd C. Miller (millert@OpenBSD.org) for setting up and
327: maintaining the ctm mailinglists
328:
329: <li> Bob Beck (beck@OpenBSD.org), Artur Grabowski (art@OpenBSD.org),
330: Magnus Holmberg (mho@OpenBSD.org) and
331: Wolfram Schneider (wosch@FreeBSD.org) for setting up the ftp space
332: for the bases and deltas on ctm.*OpenBSD.org
1.1 graichen 333:
1.5 flipk 334: <li> ... and all the others who contributed indirectly ..
335: </ul>
1.1 graichen 336:
1.5 flipk 337: <P>
338: Good luck!
1.1 graichen 339:
1.5 flipk 340: <P>
341: t
1.1 graichen 342:
1.5 flipk 343: <HR>
1.1 graichen 344:
345: <ADDRESS><A HREF="mailto:graichen@OpenBSD.org">Thomas Graichen</A></ADDRESS>
1.5 flipk 346: <br>
347: OpenBSD/CTM logo designed for the OpenBSD Project by
348: <a href="mailto:flipk@openbsd.org">Phillip F Knaack</a>.
1.12 ! jufi 349: <p>
! 350: <hr>
! 351: <a href="index.html"><img height="24" width="24" src="back.gif" border="0" alt="OpenBSD" /></a>
! 352: <a href="mailto:www@openbsd.org">www@openbsd.org</a>
! 353: </br>
! 354: <small>$OpenBSD$</small>
1.1 graichen 355:
1.12 ! jufi 356: </body>
! 357: </html>
1.1 graichen 358:
![[BACK]](/icons/back.gif){kind=icon}
Return to ctm.html CVS log ![[TXT]](/icons/text.gif){kind=icon}
![[DIR]](/icons/dir.gif){kind=icon}
Up to [local] / www