[BACK]Return to ctm.html CVS log [TXT][DIR] Up to [local] / www

Diff for /www/Attic/ctm.html between version 1.4 and 1.5

version 1.4, 1997/01/20 09:55:53 version 1.5, 1997/09/04 21:57:18
Line 3 
Line 3 
 <HEAD>  <HEAD>
    <TITLE>CTM</TITLE>     <TITLE>CTM</TITLE>
 </HEAD>  </HEAD>
 <BODY>  <BODY BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#23238E">
   
 <H1>CTM for OpenBSD </H1>  <center>
   <img alt="CTM for OpenBSD" src=bsdctm.gif width=401 height=126>
   </center>
   
 <P><FONT SIZE=+1>... get the current OpenBSD source- or cvs-tree daily  <H3>What is ctm?</H3>
 via email </FONT></P>  
   
 <P>  <P>
 <HR WIDTH="100%"></P>  Ctm is a system which was designed by Poul-Henning Kamp for making
   changes to a source tree available on a daily basis by email.
   This is a good way to stay up to date with the current source- or
   cvs-tree if you have a bad internet connection - for instance via modem.
   
 <H3>WHAT IS CTM ?</H3>  <P>
   Ctm uses very low bandwith for distributing the changes to the
   tree, and compresses them using gzip -9.
   
 <P>ctm is a system which was designed by Poul-Henning Kamp for making all  <H3>How does it work?</H3>
 the changes to the FreeBSD source- and cvs-tree (and some other trees -  
 like the FreeBSD ports-tree) available on a daily (or multiple time per  
 day) basis by email. this is a very good way to stay up to date with the  
 current source- or cvs-tree if you have a bad internet connection - for  
 instance via modem. </P>  
   
 <P>ctm uses only very low bandwith for distributing the changes to the  <P>
 tree - and it compresses these differences using gzip -9.</P>  The basic idea of ctm is that you subscribe to a special mailing list
   for a particular source tree. Each day you receive an email message
   containing all the changes to that tree during that period.
   One set of differences is called a &quot;delta&quot;.
   
 <H3>HOW DOES IT WORK ?</H3>  <P>
   You begin with a base set against which later deltas are patched.
   To begin using ctm you must grab the latest base set and all deltas
   after that; once you have processed those you can process later
   deltas you receive on the mailing list.
   
 <P>the basic idea of ctm is - that you subscribe to a special mailinglist  <P>
 to get the daily changes of the OpenBSD source- or cvs-tree and then you'll  Bases are generated once approximate each 100 relative deltas, so you never
 get a mail each day containing all the changes to the tree (maybe we increase  have to grab more than 100 deltas to catch up.
 the number of ctm files sent per day in the future) - one such set of differences  
 is called a &quot;delta&quot;.</P>  
   
 <P>if you received your first delta - you have to download all the delta's  <H3>How much bandwidth does ctm use?</H3>
 until the first delta you received by email - this always starts with a  
 big - so called &quot;base delta&quot; - which contains the full source  
 tree you start with. starting from this big one all other deltas only contain  
 the changes to the tree of the day of the delta generation. to avoid that  
 you have to get the base delta and then some hundred relative deltas a  
 new base delta is generated as a startpoint each 100 relative delta's -  
 so that you always have to get less than hundred relative deltas for starting  
 ctm :-)</P>  
   
 <H3>HOW MUCH BANDWIDTH DOES CTM USE ?</H3>  <P>
   As mentioned above, the base set contains a full source tree in TAR format,
   and is thus quite large.  You must obtain the base set and all relative
   deltas up to present via ftp from the site listed below.  Sizes are:
   
 <P>as said above the base delta contains the full source- or cvs-tree so  <ul>
 it is really big - you'll have to get it via ftp from the site below. here  <li> approximately 55Mb for base/OpenBSD-src.XXXX.tar.gz
 are the sizes of the current base deltas: ~ 45 mbyte for OpenBSD-src.0001.gz  <li> uncompressed, this is 250M or so
 - the base delta for the source-tree ~ 55 mbyte for OpenBSD-cvs.0001.gz  <li> approximately 80Mb for base/OpenBSD-cvs.XXXX.tar.gz
 - the base delta for the cvs-tree and the size of the relative deltas vary  <li> uncompressed, this is 370M or so
 from ~ 10 kbyte to ~ 100 kbyte usually and sometimes they are some (1,2  <li> relative deltas vary from 10k to 100k, and occasionally a couple of meg
 or 3) mbytes - but they are always split into 100 kbyte mails which will  <li> ctm always splits deltas into 100k messages which are reassembled
 automatically assembled from the ctm-programs - the bigger sizes come from       automatically
 bigger changes in the source- or cvs-tree (update of the binutils, import  </ul>
 of perl etc.)</P>  
   
 <H3>HOW DO I USE CTM ?</H3>  <P>
   Larger ctm updates are rare, though, usually occuring only after a major
   import of something like binutils, perl, etc.
   
 <P>ctm is very simple to use: </P>  <H3>How do I use ctm?</H3>
   
 <LI>subscribe to the ctm mailinglist you want</LI>  
   
 <UL>  <UL>
 <P><TT>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org for the  <LI>Subscribe to the ctm mailing list for the source set you want.
 source-tree</TT></P>  <p>
 </UL>     There are several mailing lists related to ctm.  There is a list for
      updates to the source tree, a list for updates to the cvs tree, a list
      for announcements where I'll post important announcements to ctm users,
      and a list for seeing log messages from the ctm delta generator runs.
      <p>
      The following commands will subscribe you to the appropriate lists:
   <br><br><tt>
   For the source tree: <br>
       <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b><br>
   For the cvs tree:<br>
       <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b><br>
   For the announce list:<br>
       <b>echo subscribe ctm-announce | mail majordomo@OpenBSD.org</b><br>
   For the ctm log list:<br>
       <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b><br>
   </tt><br>
   
 <P>or</P>  <LI>Wait until you get your first delta.
   
 <UL>  <LI>Unpack and assemble the ctm delta using ctm_rmail.
 <P><TT>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org for the  <br>
 cvs-tree</TT></P>  <br>
 </UL>  <tt>
       <b>ctm_rmail -p. -d. -b. folder</b>
   </tt>
   <br><br>
   where <b>folder</b> is the mail folder containing the delta mail.
   <br>
   This will decode the ctm delta and place it in the working directory.
   The delta will be a file of the form OpenBSD-XXX.YYYY.gz, where XXX is
   either "src" or "cvs", and YYYY is the number of the delta.
   <p>
   
 <P>you should also subscribe to</P>  <li>Grab the base set from the ftp site below, in the subdirectory
   <b>base-splitted/src</b> (for OpenBSD-src) or <b>base-splitted/cvs</b>
   (for OpenBSD-cvs).  Grab all files in that directory, and once you have
   them all you can create the base source tree with the following command:
   
 <UL>  <br><br>
 <P><TT>echo subscribe ctm-announce | mail majordomo@OpenBSD.org</TT></P>  <tt>
 </UL>      <b>cd target <br>
       cat /splitted/OpenBSD-* | tar zxf - <br>
   </b></tt><br>
   
 <P>on this list i'll post announcements for users of ctm and there is also  where <b>splitted</b> is the directory in which you placed the files you
 a list for somekind of a log of the ctm delta generation - so you can see  grabbed.
 what is all in the ctm delta and if the assembling of it succeeded - you  
 should really subscribe to this list if you use ctm using</P>  
   
 <UL>  <p>
 <P><TT>echo subscribe ctm-log | mail majordomo@OpenBSD.org</TT></P>  It is no longer necessary to keep these files around, if you're sure
 </UL>  you've got a complete tree.  However, be sure to note the number of the
   base, before continuing.
   
 <LI>wait until you get your first delta</LI>  <p>
   You now need to grab the deltas which have been generated since the base
   was generated.  Go to the ftp site below in the subdirectory <b>src</b> or
   <b>cvs</b> and grab every file whose number is <it>larger</it> than the
   number of the base you used.
   
 <LI>unpack and assemble the ctm delta using ctm_rmail (for the sources  <p>
 to all the ctm-programs see below)</LI>  The address of the base and deltas is:
   
 <UL>  <ul>
 <P><TT>ctm_rmail -p. -d. -b. the_mail_folder_containing_the_ctm_delta_mail</TT></P>  <li><a href="ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD/">
 </UL>       ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD/
   </a>
   </ul>
   
 <P>after that you should have the bare ctm delta there (something like  <p>
 OpenBSD-cvs.0015.gz or so)</P>  <ul>
   <li><b>NOTE:</b> I'm very interested in finding other sites around the world
      that would be interested in mirroring these deltas.
   </ul>
   <p>
   
 <LI>if you have this - get all the deltas beginning with 0001 up to your  <LI> Run ctm.
 first mail-received one via ftp from ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD  <br>
 also known as ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD for the tree you    <br>
 subscribed too - i'm very interested in sites all over the world who wan't    <tt>
 to mirror this or want to put up the ctm deltas there too</LI>       <b>cd target<br>
        ctm -v -v /deltas/OpenBSD-src.*</b><br></tt>
   or<br><tt>
        <b>cd target<br>
        ctm -v -v /deltas/OpenBSD-cvs.*</b><br>
     </tt><br>
   where <b>target</b> is the directory where you want your source tree to be,
   and <b>deltas</b> is where you have stored the deltas.
   <p>
   It will take some time to run all of the deltas, but the daily invocations
   of <b>ctm_rmail</b> and <b>ctm</b> will not take nearly as long.
   </ul>
   
 <LI>then run ctm (you can get the sources of ctm which should compile at  <h3>Should I choose the source or cvs tree?</h3>
 least on OpenBSD just fine from the same ftp adress - maybe you have to  
 tweak it a bit for other systems - you may need the libmd from the FreeBSD  
 sources there eventually)</LI>  
   
 <UL>  <P>
 <P><TT>cd where_you_want_your_source_tree_to_be</TT></P>  This depends on your diskspace, among other factors.  Since ctm doesn't
   yet deal with files which have been modified outside of the ctm process,
   the "better" choice is probably the cvs-tree.  Then you can check out
   your working source tree from your cvs-tree, and keep your local mods
   in your source tree, leaving only ctm to touch your cvs tree.
   
 <P><TT>ctm -v -v /where_your_ctm_deltas_are_stored/OpenBSD-src.*</TT></P>  <p>
 </UL>  The drawback, however, is the amount of diskspace it takes.  A checked-out
   source tree takes about 250MB or so, but if you have your own cvs tree you
   need 370MB or so for it, <b>plus</b> another 250MB for the checked out tree.
   
 <P>or</P>  <p>
   This doesn't count the 100MB or more you will need to do a build (depending
   on the architecture).
   
 <UL>  <p>
 <P><TT>cd where_you_want_your_cvs_tree_to_be</TT></P>  If you just get the source tree, you will need to deal with local
   modifications.  One way to do this is to use the <b>union</b> filesystem,
   although there have been reports that the <b>union</b> filesystem can be
   unstable if both the upper and lower layers reside on the same physical
   filesystem.  If you place your upper and lower layers on separate filesytems
   you should have no problems.  A sample of this is as follows:
   
 <P><TT>ctm -v -v /where_your_ctm_deltas_are_stored/OpenBSD-cvs.*</TT></P>  <p>
 </UL>  Suppose your ctm-updated tree is in <b>/usr/src-ctm</b>, and your real
   source tree, the one you make your modifications to and do your builds from,
   is in <b>/usr/src</b>.  <b>/usr/src</b> should be initially an empty
   directory.  The following command will set up the union mount:
   <br><br>
   <tt>
       <b>mount -t union -o -b /usr/src-ctm /usr/src</b>
   </tt>
   <br><br>
   Modifications made to files in /usr/src will become a file contained within
   /usr/src, hiding the one in src-ctm.  If changes are made via ctm to the
   underlying src-ctm tree, those changes will <b>not</b> be seen if there is
   a file in the upper layer hiding it.
   
 <LI>this will take some time for the base delta - and less time for the  <p>
 relative ones * now you can run ctm_rmail and ctm on every new ctm delta  You should periodically unmount the
 to get up to date</LI>  union and search for files which are local to the union filesystem.
   <br><br>
   <tt>
       <b>umount /usr/src <br>
       find /usr/src -type f</b> <br>
   </tt>
   <br><br>
   The commands <b>ls -W</b> and <b>rm -W</b> will be useful too, as objects
   named "whiteouts" in /usr/src will also hide files in src-ctm.
   
 <H3>WHAT SHOULD I CHOOSE - SOURCE- OR CVS-TREE VIA CTM ?</H3>  <p>
   If you get the cvs-tree, you can use the "cvs checkout" command to check
   out a source tree from it, and each time you update with ctm you can use
   the "cvs update" command to update your source tree.
   
 <P>this simply depends on your diskspace - the better is the cvs-tree -  <br><br>
 because you can make modifications to the local checked out copy (ctm can't  To get an initial src tree:<br>
 yet handle modifications in files it want to update) - but the drawback  <tt>
 is a _lot_ of diskspace - you need currently ~ 240 mbyte for the full cvs-tree      <b>cd /usr <br>
 and ~ 200 mbyte for the checked out source tree plus ~ 100 mbyte for the      cvs -qd YOUR_CVS_TREE checkout src</b> <br><br>
 obj files while &quot;make build&quot; if you have enough space then go  </tt>
 for the cvs-tree - then you simply have to check out a local copy of the  and after each ctm update:<br>
 source tree after the first ctm -v -v command using cd /usr cvs -q checkout  <tt>
 -PA src and after each new applied ctm relative delta you have to run cd      <b>cd /usr/src <br>
 /usr cvs -q update -PAd src to get your local copy up to date if you don't      cvs -q update -PAd</b> <br>
 have enough space so you can use ctm for getting the checked out copy of  </tt>
 the source tree - but then you have to think about the handling of local  
 modifications - at this point someone from the NetBSD people is using a  
 mixture of null and union mounts and some scripts for doing this - if i  
 remember right it was something like this /usr/src-local is null mounted  
 to /usr/src - /usr/src-ctm is union mounted to /usr/src so you have all  
 local modifications in /usr/src-local and you have to run a script which  
 does something like for all files in /usr/src-local diff them against the  
 /usr/src-ctm versions and apply this patch to a fresh ctm applied /usr/src-ctm  
 tree mounted in the above way against a fresh /usr/src-local directory  
 the union mount of /usr/src-ctm guaranties that the /usr/src-ctm tree is  
 not modified which is very importand for the operation of ctm all this  
 is just a rough idea - all in all - it is doable because someone from the  
 NetBSD people is definitely doing it in a way close to this one</P>  
   
 <H3>HOW STABLE IS CTM FOR OpenBSD ?</H3>  <H3>How stable is ctm for OpenBSD?</H3>
   
 <P>i will call this now more than a &quot;public beta&quot; state - i'm  Ctm has been in use for over a year for OpenBSD, and for FreeBSD for a
 using ctm now for more than two months to stay up to date with the OpenBSD  number of years.  Ctm is quite reliable and stable.
 cvs-tree without any problems - and also the FreeBSD people use ctm very  
 extensively for years know without any major problems i think (i also use  
 ctm for FreeBSD now for about half a year without any problems) so i think  
 you really can trust ctm - in the last weeks also some other people used  
 it and i haven't heard aboutany problems so far</P>  
   
 <H3>WHERE CAN I FIND MORE INFORMATION ABOUT CTM FOR OpenBSD ?</H3>  <H3>More information</H3>
   
 <P>here :-) - but you can take all the information from FreeBSD and you  If you have problems with ctm, a good place to start is the FreeBSD
 have the man-pages for the ctm-programs in the ctm source-package there  handbook:
 are some informations about the use and ideas of ctm in the FreeBSD handbook  
 - which you will find under http://www.FreeBSD.org/handbook/ctm.html</P>  
   
 <P>if you have problems with ctm you may also ask me - i'll try to help  <ul>
 you as far as i can and as far as i have enough time (which is a big problem  <li> <a href="http://www.FreeBSD.org/handbook/ctm.html">
 in the moment)</P>           http://www.FreeBSD.org/handbook/ctm.html
        </a>
   </ul>
   
 <H3>OK - IF YOU GOT HERE ...</H3>  There are also man pages for all of the ctm utilities included in the
   source package.  If you have OpenBSD, you should already have the man
   pages, source, and binaries.  If not you can obtain them from the same
   ftp site as the deltas below.
   
 <P>... and decided to use ctm from now on - here's the summary of adresses:  <p>
 mailinglists: </P>  You can also contact the OpenBSD mailing lists or myself if you have
   difficulties with ctm, although my free time tends to be rather limited.
   
 <LI><TT>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</TT></LI>  <p>
   Summary:
   
 <LI><TT>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</TT></LI>  <p>
   
 <LI><TT>echo subscribe ctm-announce | mail majordomo@OpenBSD.org</TT></LI>  <ul>
   <li> Mailing lists:
   
 <LI><TT>echo subscribe ctm-log | mail majordomo@OpenBSD.org</TT></LI>     <ul>
      <li> <b>echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org</b>
      <li> <b>echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org</b>
      <li> <b>echo subscribe ctm-announce | mail majordomo@OpenBSD.org</b>
      <li> <b>echo subscribe ctm-log | mail majordomo@OpenBSD.org</b>
      </ul>
   
 <P>ftp server:</P>  <li> FTP (for base and relative deltas; and OpenBSD has ctm in the source
        tree, but if you don't have source or binaries, the source is here too):
   
 <LI>ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD</LI>     <ul>
      <li> <a href="ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD">
            ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD
              </a>
      <li> <a href="ftp://ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD">
            ftp://ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD
              </a>
      </ul>
   </ul>
   
 <LI>ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD</LI>  <p>
   I will make any important notes and announcements about ctm in:
   <ul>
   <li> ctm-announce@OpenBSD.org
   <li> misc@OpenBSD.org
   <li> announce@OpenBSD.org
   </ul>
   
 <P>there you'll find the deltas and the sources to ctm (ctm.tar.gz - simply  <p>
 put it into /usr/src/usr.sbin and it should compile fine under OpenBSD)</P>  <b>NOTE:</b> If you use ctm for OpenBSD, please send me email so I
   know how many people are using it!
   
 <P>i'll also make any importnat notes and announcements about ctm in</P>  <H3>Acknowledgements</H3>
   
 <LI>ctm-announce@OpenBSD.org</LI>  <li> Poul-Henning Kamp (phk@FreeBSD.org) for making ctm possible and helping
        me getting the delta generation running.
   
 <LI>misc@OpenBSD.org</LI>  <li> Theo de Raadt (deraadt@theos.com) for making OpenBSD possible and
        giving me the resources i needed for creating the ctm
        deltas on cvs.OpenBSD.org
   
 <P>and the big ones in</P>  <li> Wolfram Schneider (wosch@FreeBSD.org) for setting up the ftp space
        for the deltas on ctm.OpenBSD.org
   
 <LI>announce@OpenBSD.org</LI>  <li> ... and all the others who contributed indirectly ..
   </ul>
   
 <P>and pleeese - send me a mail if you use ctm for OpenBSD so that i can  <P>
 see how many people are using it !</P>  Good luck!
   
 <H3>THANKS ...</H3>  <P>
   t
   
 <P>... go to: </P>  <HR>
   
 <LI>poul-henning kamp (phk@FreeBSD.org) for making ctm possible and helping  
 me getting the delta generation running</LI>  
   
 <LI>theo de raadt (deraadt@theos.com) for making OpenBSD possible :-) and  
 giving me all the ressources i needed for creating the ctm deltas on cvs.OpenBSD.org</LI>  
   
 <LI>wolfram schneider (wosch@FreeBSD.org) for setting up the ftp space  
 for the deltas on ctm.OpenBSD.org</LI>  
   
 <LI>... and all the others who contributed indirectly</LI>  
   
 <P>SO - GOOD LUCK WITH CTM FOR OpenBSD wishes</P>  
   
 <P>t</P>  
   
 <HR></LI>  
   
 <ADDRESS><A HREF="mailto:graichen@OpenBSD.org">Thomas Graichen</A></ADDRESS>  <ADDRESS><A HREF="mailto:graichen@OpenBSD.org">Thomas Graichen</A></ADDRESS>
   <br>
   OpenBSD/CTM logo designed for the OpenBSD Project by
      <a href="mailto:flipk@openbsd.org">Phillip F Knaack</a>.
   
 <P><!-- Created: Mon Oct 28 22:20:54 MET 1996 --><!-- hhmts start -->Last  <P>
 modified: Mon Oct 28 22:24:34 MET <!-- hhmts end --></P>  <!-- Created: Mon Oct 28 22:20:54 MET 1996 -->
   <!-- hhmts start -->
     $Id$
   <!-- hhmts end -->
   
 </BODY>  </BODY>
 </HTML>  </HTML>
Legend:
Removed from v.1.4  
changed lines
  Added in v.1.5