=================================================================== RCS file: /cvsrepo/anoncvs/cvs/www/Attic/ctm.html,v retrieving revision 1.4 retrieving revision 1.5 diff -c -r1.4 -r1.5 *** www/Attic/ctm.html 1997/01/20 09:55:53 1.4 --- www/Attic/ctm.html 1997/09/04 21:57:18 1.5 *************** *** 3,243 **** CTM ! !

CTM for OpenBSD

... get the current OpenBSD source- or cvs-tree daily ! via email

WHAT IS CTM ?

ctm is a system which was designed by Poul-Henning Kamp for making all ! the changes to the FreeBSD source- and cvs-tree (and some other trees - ! like the FreeBSD ports-tree) available on a daily (or multiple time per ! day) basis by email. this is a very good way to stay up to date with the ! current source- or cvs-tree if you have a bad internet connection - for ! instance via modem.

ctm uses only very low bandwith for distributing the changes to the ! tree - and it compresses these differences using gzip -9.

HOW DOES IT WORK ?

the basic idea of ctm is - that you subscribe to a special mailinglist ! to get the daily changes of the OpenBSD source- or cvs-tree and then you'll ! get a mail each day containing all the changes to the tree (maybe we increase ! the number of ctm files sent per day in the future) - one such set of differences ! is called a "delta".

if you received your first delta - you have to download all the delta's ! until the first delta you received by email - this always starts with a ! big - so called "base delta" - which contains the full source ! tree you start with. starting from this big one all other deltas only contain ! the changes to the tree of the day of the delta generation. to avoid that ! you have to get the base delta and then some hundred relative deltas a ! new base delta is generated as a startpoint each 100 relative delta's - ! so that you always have to get less than hundred relative deltas for starting ! ctm :-)

HOW MUCH BANDWIDTH DOES CTM USE ?

as said above the base delta contains the full source- or cvs-tree so ! it is really big - you'll have to get it via ftp from the site below. here ! are the sizes of the current base deltas: ~ 45 mbyte for OpenBSD-src.0001.gz ! - the base delta for the source-tree ~ 55 mbyte for OpenBSD-cvs.0001.gz ! - the base delta for the cvs-tree and the size of the relative deltas vary ! from ~ 10 kbyte to ~ 100 kbyte usually and sometimes they are some (1,2 ! or 3) mbytes - but they are always split into 100 kbyte mails which will ! automatically assembled from the ctm-programs - the bigger sizes come from ! bigger changes in the source- or cvs-tree (update of the binutils, import ! of perl etc.)

HOW DO I USE CTM ?

ctm is very simple to use:

subscribe to the ctm mailinglist you want

echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org for the ! source-tree

echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org for the ! cvs-tree

you should also subscribe to

echo subscribe ctm-announce | mail majordomo@OpenBSD.org

on this list i'll post announcements for users of ctm and there is also ! a list for somekind of a log of the ctm delta generation - so you can see ! what is all in the ctm delta and if the assembling of it succeeded - you ! should really subscribe to this list if you use ctm using

echo subscribe ctm-log | mail majordomo@OpenBSD.org

wait until you get your first delta

unpack and assemble the ctm delta using ctm_rmail (for the sources ! to all the ctm-programs see below)

ctm_rmail -p. -d. -b. the_mail_folder_containing_the_ctm_delta_mail

after that you should have the bare ctm delta there (something like ! OpenBSD-cvs.0015.gz or so)

if you have this - get all the deltas beginning with 0001 up to your ! first mail-received one via ftp from ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD ! also known as ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD for the tree you ! subscribed too - i'm very interested in sites all over the world who wan't ! to mirror this or want to put up the ctm deltas there too

then run ctm (you can get the sources of ctm which should compile at ! least on OpenBSD just fine from the same ftp adress - maybe you have to ! tweak it a bit for other systems - you may need the libmd from the FreeBSD ! sources there eventually)

cd where_you_want_your_source_tree_to_be

ctm -v -v /where_your_ctm_deltas_are_stored/OpenBSD-src.*

cd where_you_want_your_cvs_tree_to_be

ctm -v -v /where_your_ctm_deltas_are_stored/OpenBSD-cvs.*

this will take some time for the base delta - and less time for the ! relative ones * now you can run ctm_rmail and ctm on every new ctm delta ! to get up to date

WHAT SHOULD I CHOOSE - SOURCE- OR CVS-TREE VIA CTM ?

this simply depends on your diskspace - the better is the cvs-tree - ! because you can make modifications to the local checked out copy (ctm can't ! yet handle modifications in files it want to update) - but the drawback ! is a _lot_ of diskspace - you need currently ~ 240 mbyte for the full cvs-tree ! and ~ 200 mbyte for the checked out source tree plus ~ 100 mbyte for the ! obj files while "make build" if you have enough space then go ! for the cvs-tree - then you simply have to check out a local copy of the ! source tree after the first ctm -v -v command using cd /usr cvs -q checkout ! -PA src and after each new applied ctm relative delta you have to run cd ! /usr cvs -q update -PAd src to get your local copy up to date if you don't ! have enough space so you can use ctm for getting the checked out copy of ! the source tree - but then you have to think about the handling of local ! modifications - at this point someone from the NetBSD people is using a ! mixture of null and union mounts and some scripts for doing this - if i ! remember right it was something like this /usr/src-local is null mounted ! to /usr/src - /usr/src-ctm is union mounted to /usr/src so you have all ! local modifications in /usr/src-local and you have to run a script which ! does something like for all files in /usr/src-local diff them against the ! /usr/src-ctm versions and apply this patch to a fresh ctm applied /usr/src-ctm ! tree mounted in the above way against a fresh /usr/src-local directory ! the union mount of /usr/src-ctm guaranties that the /usr/src-ctm tree is ! not modified which is very importand for the operation of ctm all this ! is just a rough idea - all in all - it is doable because someone from the ! NetBSD people is definitely doing it in a way close to this one

HOW STABLE IS CTM FOR OpenBSD ?

i will call this now more than a "public beta" state - i'm ! using ctm now for more than two months to stay up to date with the OpenBSD ! cvs-tree without any problems - and also the FreeBSD people use ctm very ! extensively for years know without any major problems i think (i also use ! ctm for FreeBSD now for about half a year without any problems) so i think ! you really can trust ctm - in the last weeks also some other people used ! it and i haven't heard aboutany problems so far

WHERE CAN I FIND MORE INFORMATION ABOUT CTM FOR OpenBSD ?

here :-) - but you can take all the information from FreeBSD and you ! have the man-pages for the ctm-programs in the ctm source-package there ! are some informations about the use and ideas of ctm in the FreeBSD handbook ! - which you will find under http://www.FreeBSD.org/handbook/ctm.html

if you have problems with ctm you may also ask me - i'll try to help ! you as far as i can and as far as i have enough time (which is a big problem ! in the moment)

OK - IF YOU GOT HERE ...

... and decided to use ctm from now on - here's the summary of adresses: ! mailinglists:

echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org

echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org

echo subscribe ctm-announce | mail majordomo@OpenBSD.org

echo subscribe ctm-log | mail majordomo@OpenBSD.org

ftp server:

ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD

ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD

there you'll find the deltas and the sources to ctm (ctm.tar.gz - simply ! put it into /usr/src/usr.sbin and it should compile fine under OpenBSD)

i'll also make any importnat notes and announcements about ctm in

ctm-announce@OpenBSD.org

misc@OpenBSD.org

and the big ones in

announce@OpenBSD.org

and pleeese - send me a mail if you use ctm for OpenBSD so that i can ! see how many people are using it !

THANKS ...

... go to:

poul-henning kamp (phk@FreeBSD.org) for making ctm possible and helping - me getting the delta generation running

- -

theo de raadt (deraadt@theos.com) for making OpenBSD possible :-) and - giving me all the ressources i needed for creating the ctm deltas on cvs.OpenBSD.org

- -

wolfram schneider (wosch@FreeBSD.org) for setting up the ftp space - for the deltas on ctm.OpenBSD.org

- -

... and all the others who contributed indirectly

- -

SO - GOOD LUCK WITH CTM FOR OpenBSD wishes

- -

Thomas Graichen

Last ! modified: Mon Oct 28 22:24:34 MET

--- 3,329 ---- CTM ! ! ! CTM for OpenBSD

! !

What is ctm?

! Ctm is a system which was designed by Poul-Henning Kamp for making ! changes to a source tree available on a daily basis by email. ! This is a good way to stay up to date with the current source- or ! cvs-tree if you have a bad internet connection - for instance via modem. !

! Ctm uses very low bandwith for distributing the changes to the ! tree, and compresses them using gzip -9. !

How does it work?

! The basic idea of ctm is that you subscribe to a special mailing list ! for a particular source tree. Each day you receive an email message ! containing all the changes to that tree during that period. ! One set of differences is called a "delta". !

! You begin with a base set against which later deltas are patched. ! To begin using ctm you must grab the latest base set and all deltas ! after that; once you have processed those you can process later ! deltas you receive on the mailing list. !

! Bases are generated once approximate each 100 relative deltas, so you never ! have to grab more than 100 deltas to catch up. !

How much bandwidth does ctm use?

! As mentioned above, the base set contains a full source tree in TAR format, ! and is thus quite large. You must obtain the base set and all relative ! deltas up to present via ftp from the site listed below. Sizes are: !

approximately 55Mb for base/OpenBSD-src.XXXX.tar.gz !
uncompressed, this is 250M or so !
approximately 80Mb for base/OpenBSD-cvs.XXXX.tar.gz !
uncompressed, this is 370M or so !
relative deltas vary from 10k to 100k, and occasionally a couple of meg !
ctm always splits deltas into 100k messages which are reassembled ! automatically !

! Larger ctm updates are rare, though, usually occuring only after a major ! import of something like binutils, perl, etc. !

How do I use ctm?

Subscribe to the ctm mailing list for the source set you want. !
! There are several mailing lists related to ctm. There is a list for ! updates to the source tree, a list for updates to the cvs tree, a list ! for announcements where I'll post important announcements to ctm users, ! and a list for seeing log messages from the ctm delta generator runs. !
! The following commands will subscribe you to the appropriate lists: !

! For the source tree: ! echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org ! For the cvs tree: ! echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org ! For the announce list: ! echo subscribe ctm-announce | mail majordomo@OpenBSD.org ! For the ctm log list: ! echo subscribe ctm-log | mail majordomo@OpenBSD.org !
!
Wait until you get your first delta. !
Unpack and assemble the ctm delta using ctm_rmail. !
!
! ! ctm_rmail -p. -d. -b. folder ! !

! where folder is the mail folder containing the delta mail. !
! This will decode the ctm delta and place it in the working directory. ! The delta will be a file of the form OpenBSD-XXX.YYYY.gz, where XXX is ! either "src" or "cvs", and YYYY is the number of the delta. !
!
Grab the base set from the ftp site below, in the subdirectory ! base-splitted/src (for OpenBSD-src) or base-splitted/cvs ! (for OpenBSD-cvs). Grab all files in that directory, and once you have ! them all you can create the base source tree with the following command: !

! ! cd target ! cat /splitted/OpenBSD-* | tar zxf - !
! where splitted is the directory in which you placed the files you ! grabbed. !
! It is no longer necessary to keep these files around, if you're sure ! you've got a complete tree. However, be sure to note the number of the ! base, before continuing. !
! You now need to grab the deltas which have been generated since the base ! was generated. Go to the ftp site below in the subdirectory src or ! cvs and grab every file whose number is larger than the ! number of the base you used. !
! The address of the base and deltas is: !
- ! ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD/ ! !
!
!
- NOTE: I'm very interested in finding other sites around the world ! that would be interested in mirroring these deltas. !
!
!
Run ctm. !
!
! ! cd target ! ctm -v -v /deltas/OpenBSD-src.* ! or
! cd target ! ctm -v -v /deltas/OpenBSD-cvs.* !
! where target is the directory where you want your source tree to be, ! and deltas is where you have stored the deltas. !
! It will take some time to run all of the deltas, but the daily invocations ! of ctm_rmail and ctm will not take nearly as long. !

Should I choose the source or cvs tree?

! This depends on your diskspace, among other factors. Since ctm doesn't ! yet deal with files which have been modified outside of the ctm process, ! the "better" choice is probably the cvs-tree. Then you can check out ! your working source tree from your cvs-tree, and keep your local mods ! in your source tree, leaving only ctm to touch your cvs tree. !

! The drawback, however, is the amount of diskspace it takes. A checked-out ! source tree takes about 250MB or so, but if you have your own cvs tree you ! need 370MB or so for it, plus another 250MB for the checked out tree. !

! This doesn't count the 100MB or more you will need to do a build (depending ! on the architecture). !

! If you just get the source tree, you will need to deal with local ! modifications. One way to do this is to use the union filesystem, ! although there have been reports that the union filesystem can be ! unstable if both the upper and lower layers reside on the same physical ! filesystem. If you place your upper and lower layers on separate filesytems ! you should have no problems. A sample of this is as follows: !

! Suppose your ctm-updated tree is in /usr/src-ctm, and your real ! source tree, the one you make your modifications to and do your builds from, ! is in /usr/src. /usr/src should be initially an empty ! directory. The following command will set up the union mount: !

! ! mount -t union -o -b /usr/src-ctm /usr/src ! !

! Modifications made to files in /usr/src will become a file contained within ! /usr/src, hiding the one in src-ctm. If changes are made via ctm to the ! underlying src-ctm tree, those changes will not be seen if there is ! a file in the upper layer hiding it. !

! You should periodically unmount the ! union and search for files which are local to the union filesystem. !

! ! umount /usr/src ! find /usr/src -type f ! !

! The commands ls -W and rm -W will be useful too, as objects ! named "whiteouts" in /usr/src will also hide files in src-ctm. !

! If you get the cvs-tree, you can use the "cvs checkout" command to check ! out a source tree from it, and each time you update with ctm you can use ! the "cvs update" command to update your source tree. !

! To get an initial src tree:
! ! cd /usr ! cvs -qd YOUR_CVS_TREE checkout src ! ! and after each ctm update:
! ! cd /usr/src ! cvs -q update -PAd ! !

How stable is ctm for OpenBSD?

! Ctm has been in use for over a year for OpenBSD, and for FreeBSD for a ! number of years. Ctm is quite reliable and stable. !

More information

! If you have problems with ctm, a good place to start is the FreeBSD ! handbook: !

! http://www.FreeBSD.org/handbook/ctm.html ! !

! There are also man pages for all of the ctm utilities included in the ! source package. If you have OpenBSD, you should already have the man ! pages, source, and binaries. If not you can obtain them from the same ! ftp site as the deltas below. !

! You can also contact the OpenBSD mailing lists or myself if you have ! difficulties with ctm, although my free time tends to be rather limited. !

! Summary: !

Mailing lists: !
- echo subscribe OpenBSD-src | mail majordomo@OpenBSD.org !
- echo subscribe OpenBSD-cvs | mail majordomo@OpenBSD.org !
- echo subscribe ctm-announce | mail majordomo@OpenBSD.org !
- echo subscribe ctm-log | mail majordomo@OpenBSD.org !
!
FTP (for base and relative deltas; and OpenBSD has ctm in the source ! tree, but if you don't have source or binaries, the source is here too): !
- ! ftp://ctm.OpenBSD.org:/pub/bsd/CTM/OpenBSD ! !
- ! ftp://ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD ! !
!

! I will make any important notes and announcements about ctm in: !

ctm-announce@OpenBSD.org !
misc@OpenBSD.org !
announce@OpenBSD.org !

! NOTE: If you use ctm for OpenBSD, please send me email so I ! know how many people are using it! !

Acknowledgements

Poul-Henning Kamp (phk@FreeBSD.org) for making ctm possible and helping ! me getting the delta generation running. !

Theo de Raadt (deraadt@theos.com) for making OpenBSD possible and ! giving me the resources i needed for creating the ctm ! deltas on cvs.OpenBSD.org !

Wolfram Schneider (wosch@FreeBSD.org) for setting up the ftp space ! for the deltas on ctm.OpenBSD.org !

... and all the others who contributed indirectly .. ! !

! Good luck! !

! t !

Thomas Graichen

+
+ OpenBSD/CTM logo designed for the OpenBSD Project by + Phillip F Knaack. !

! ! ! $Id: ctm.html,v 1.5 1997/09/04 21:57:18 flipk Exp $ !