[OpenBSD]

Anonymous CVS

Table Of Contents

What is Anonymous CVS?

Anonymous CVS is a method of keeping your local copy of the OpenBSD source tree up to date with respect to changes made to current OpenBSD sources. In addition to following the bleeding edge of development, it is also possible to track the patches for errata of a release.

The major advantage of Anonymous CVS over other source code update techniques is that it works directly against a central source code repository or mirror. This means that you have the full set of CVS commands available to control merging and updating your changes with other source changes and for performing diffs, change histories and other queries against the central repository.

The OpenBSD Project currently has five main source repositories:

To summarize, the real strength of using Anonymous CVS is that it is a "tolerant" source code control system - it respects changes that you have made to your local sources and makes "best efforts" to update your entire source tree, rather than leaving you a list of arcane problems that have to be resolved before continuing.

What is CVS?

CVS is the source code control system used to manage the OpenBSD source tree. It implements a central repository for all officially released source code and changes, while permitting developers to maintain local copies of the source code with their working changes. There are two levels of source tree access:

The major strength of CVS is that it has the ability to perform intelligent merges of changes to the central repository with changes that you make to your local copy. This means that if you make a change to a module and perform an update, your changes are not "blown away", rather CVS makes best efforts to merge the changes made to the central sources with changes you've made to your local copy.

In the event that the changes can't be completely merged, CVS provides a "soft fallback", providing you with annotated changes to your local copy, preserving an unmodified copy of your version and continuing to update any other source modules you requested.

Getting Started Using Anonymous CVS

While you can download the entire source tree from an AnonCVS server, you can often save a lot of time and bandwidth by "preloading" your source tree with the source files from either the OpenBSD CD or from an FTP server. This is particularly true if you are running -stable, as relatively few files change between the -release and -stable.

To extract the source tree from the CD to /usr/src (assuming the CD is mounted on /mnt): 

    # cd /usr/src; tar xzf /mnt/src.tar.gz
    # cd /usr; tar xzf /mnt/XF4.tar.gz
    # tar xzf /mnt/ports.tar.gz
The source files for download from the FTP servers are separated into two files to minimize the time required to download for those wishing to work with only one part of the tree. The two files are sys.tar.gz, which contains the files used to create the kernel, and src.tar.gz which contains all the other "userland" utilities. In general, however, you will usually want both of them installed. Assuming the downloaded files, src.tar.gz and sys.tar.gz are in /usr: 
    # cd /usr/src
    # tar xzf ../sys.tar.gz
    # tar xzf ../src.tar.gz
    # cd /usr
    # tar xzf XF4.tar.gz
    # tar xzf ports.tar.gz

Not all people will wish to unpack all the file sets, but as the system must be kept in sync, you will generally need to set up all trees.

You can also just use cvs(1) to "checkout" the source repository for you. This is discussed in the next section.

After this, /usr/src will be a nice checkout area where all cvs(1) commands will work properly.

Using CVS to get and update your source tree

CVS was designed to be a simple way to retrieve and update your sources. You must first decide whether you want to track -current or a patch branch. The current tree has all of the up to the minute changes, whereas a patch branch contains a formal release plus the patches from the errata and lesser issues already applied. For more information on these "flavors" of OpenBSD, see here.

Once you have decided which tree to follow, you must choose which Anonymous CVS server you are going to use. A list of these servers is below.

Once you have chosen which Anonymous CVS Server you will use, you can start using cvs. For those of you who have CDs you can start with the CVS checkout that is on the CD by using the method above to get the sources onto your system. If you don't have a CD handy, use the method below to checkout the sources. This method puts the OpenBSD source tree into /usr/src. 

	# cd /usr; cvs checkout -P src

The above will checkout the current source tree. Many of you will only want the patch branch sources. To checkout a patch branch, you must specify a tag along with your command. Example: 

	# cd /usr; cvs checkout -P -rOPENBSD_3_9 src

Or OPENBSD_3_8 for 3.8, etc.

The OPENBSD_3_9 tag contains the release sources and errata already applied.

Available Anonymous CVS Servers

Please see the note about ssh vs. rsh below!

Note: If your server is listed on here with inaccurate or unknown information, please contact beck@openbsd.org

You may want to use traceroute(8) to find out which server is nearest you. Problems with a server should be reported to the maintainer of the server in question.

Getting crypto sources through cvs(1)

IMPORTANT NOTE: There are a few issues relating to cryptographic software that everyone should be aware of:

Example usages for cvs(1)

NOTICE: If you want to update a branch (such as a patch branch) to current, you would add the -A flag to cvs, but this flag is of little use otherwise. Some older versions of the OpenBSD documentation recommended use of this flag in many examples. We no longer recommend this flag unless absolutely necessary.

A sample use of an anoncvs server would be: 

$ cd /tmp
$ cvs -d anoncvs@anoncvs.ca.openbsd.org:/cvs get src/sys/arch/sparc
    [copies the files from the repository to your machine]
$ cd src/sys/arch/sparc
$ cvs log locore.s
    [shows the commit log for the chosen file]
$ cvs diff -bc -r1.1 -r1.5 locore.s
    [shows the changes between revisions 1.1 and rev 1.5]

In order to use a cvs ``pserver'' (a direct TCP connection instead of using ssh or rsh) you must login once: 

$ cvs -d :pserver:anoncvs@anoncvs.ca.openbsd.org:/cvs login
(Logging in to anoncvs@anoncvs1.ca.openbsd.org)
CVS password: anoncvs
    [This writes a line to ~/.cvspass (filename over-ridden by CVS_PASSFILE).]
    [An example line from my ~/.cvspass after typing 'blah' for the above    ]
    [password is:                                                            ]
    [:pserver:anoncvs@anoncvs5.usa.openbsd.org:/cvs Au'yc                    ]
    [After logging in ONCE every other use of the above CVSROOT will work.   ]
$ cvs -d :pserver:anoncvs@anoncvs.ca.openbsd.org:/cvs get ksrc-i386 ksrc-common
    [Allows you to retrieve ONLY that necessary to rebuild an i386 kernel.   ]

Here is how someone using anoncvs regularly would update his source tree:

To use ports, it is similar to src:

In the above example, -q is optional, only intended to minimize cvs's output. For those who like to see screenfulls of output, it can be omitted.

or to make a diff of a locally patched module (here cd.c) to include with a bug report: 

	# cd /usr
	# cvs diff -u src/sys/scsi/cd.c > /tmp/patch

The cvs(1) man page (included with the CVS sources) has much more information about how CVS can be used.

Warning: When using cvs you should take care that your current directory is either the root of the tree you are referencing or in a separate place such as /tmp. Some commands such as "get" can create an arbitrary sub-tree in the current directory, and a subsequent update will recursively flesh out this sub-tree.

The anoncvs service gives fledgling developers a chance to learn CVS operation and get thoroughly involved in the development process before getting "commit" access -- as a result of showing useful skills and high quality results they will naturally later be given developer access. As well, people providing patches can create their "diff"s relative to the CVS tree, which will ease integration.

Use rsh(1) or ssh(1)?

CVS supports three access methods between the CVS server and the CVS client:

NOTE: For users wishing to use rsh, you must first set the CVS_RSH environment variable to point to the rsh(1) program:

By default, OpenBSD's CVS client uses ssh ("secure shell": OpenSSH) to talk to the CVS server.

Many of the CVS sites no longer support rsh for security reasons. Local problems like firewalls or imperfect protocol emulators such as slirp may also hinder rsh usage. However, if rsh is desired, one must set the CVS_RSH environment variable to point to rsh (typically /usr/bin/rsh).

If local policy prevents outgoing connections to ssh's default port of 22, port 2022 may be used in its place. Note, however, that not all anoncvs servers accept ssh connections on this port. Furthermore, most anoncvs servers no longer accept the none cipher, as it is disabled in recent versions of ssh for security reasons. Also, do not be tempted to turn on compression: CVS already compresses.

One could specify something like the following in the $HOME/.ssh/config configuration file to avoid the pitfalls and restrictions mentioned above: 

	Host anoncvs.ca.openbsd.org
	    Compression no
	    Port 2022

CVS is a little noisy starting up; to quiet it a bit you may want to do this: 

	$ export CVS_CLIENT_PORT=-1

Mirroring the CVS repository via sup(1)

Users wishing to mirror the OpenBSD CVS tree itself may now do so from anoncvs.usa.openbsd.org or anoncvs3.usa.openbsd.org (these are different machines). Note that this is the cvs tree, not a checked out source tree. It is only useful if you want to be able to do fast cvs operations (diff, annotate, etc) or if you have multiple source trees and you only want to transfer new data once (you can then checkout a tree from your local cvs mirror).

A sample supfile would be: 

	cvs host=anoncvs.usa.openbsd.org hostbase=/ base=/home delete

which would mirror the cvs tree into /home/cvs with the sup data files ending up in /home/sup. The full OpenBSD cvs tree is currently about 2.5GB in size, and will, of course continue to grow.

Setting up an anoncvs mirror

If you wish to setup a new anoncvs mirror site and make it available to the general public, please contact the anoncvs maintainer. Anoncvs mirrors require about 2.2GB of disk, and use up to 32MB of swap per anoncvs user (assuming the user does a large operation; while smaller operations use fewer resources, anoncvs still makes much more of an impact than ftp or sup). Such anoncvs machines should have excellent network connectivity for the area they are expected to serve. A document which describes the setup of anoncvs servers is available.

Final notes

After upgrading your source tree, you should read the comments at the top of /usr/src/Makefile before attempting a build. Also, you should build a new kernel before doing a make build if possible. In some cases it may be necessary to rebuild and install the config utility before you can build the kernel. If config GENERIC fails this is probably the case.

It is important to note that upgrading from a release to the current tree by rebuilding the sources can be rather difficult due to dependencies that are often not obvious. Therefore, it is suggested that you first install the latest snapshot before attempting a tree build from source.

OpenBSD www@openbsd.org
$OpenBSD: anoncvs.html,v 1.247 2006/05/01 08:52:01 steven Exp $