Return to INSTALL.linux CVS log

Up to [local] / src / etc / etc.i386

Typo; rd@thrush.com

$OpenBSD: INSTALL.linux,v 1.10 2001/04/18 14:22:20 aaron Exp $

   Linux + OpenBSD: it's possible

   by Marc Espie --  espie@cvs.OpenBSD.org

It is perfectly possible to have Linux and OpenBSD on the same disk.
As of this writing (OpenBSD 2.5 & linux 2.2.3), both can read and write
other partitions, though Linux's support of BSD filesystems is still
experimental. Please note that 2.0 linux kernels, and most 2.1 kernels
don't know how to handle OpenBSD partitions (other BSD partitions are type
A5 and differ significantly from OpenBSD partitions--type A6).

You can even install OpenBSD from an ext2fs partition (choose install from
disk... ext2fs does not appear in the choices, but `default' it is).

If you are starting from scratch, it is better to install Linux first.
Since you are going to use several OSes, you need a way to multi-boot, and
Linux's lilo fits the bill fine.

IMPORTANT: don't forget about lilo. You can't uninstall linux from this
disk without *first* restoring the MBR to an un-liloed state and making
*dead* sure OpenBSD boots as a default.

If you want to grab space from a Windows/DOS partition, use fips.
Fips20 knows all about FAT32, so windows 95 is no longer a problem.

Other sources of information, especially concerning other BSD systems,
must be taken with a healthy dose of skepticism. OpenBSD definitely
differs:
- disklabels can hold up to 16 partitions,
- type is A6, not A5,
- the `special' partitions in the disklabel are only a (root), b (swap),
and c (whole disk).

Planning & Good advice
----------------------
If you are starting on a new machine, be prepared to throw your
installation away. It is generally the case that you will install the
machine, play with it for a week/a month, and find out that you don't like
the setup, and then start over.

Write down any interesting information you find out during your first 
installation.  Don't do too many things to your box during the first month,
as you will lose these while reinstalling, unless you can do backups 
conveniently.

Do you really need to have a dual-boot machines ? Most people don't need
both Linux and OpenBSD.  Once you're satisfied with OpenBSD, you may find
out you just want to erase Linux... 

Try to find out what your precise needs are, locate partitions whose size
may change next to each other, as far as possible... Put partitions whose
contents are unimportant (or whose backups are always up-to-date) next to
the frontier between OpenBSD and Linux. For instance, it's usually a good
idea to locate the swap area such that you can grow or shrink it. Keep in
mind that exceptional usage (very large, temporary swaps) can use a
temporary file instead of a partition, under both OpenBSD (svnd) and Linux.

First principles
----------------
OpenBSD doesn't only use the MBR partitions (the ones mapped in the Master
Boot Record) for booting. Afterwards, it trusts some bsd specific 
information called the disklabel, which is another completely distinct 
description of your hard disk. It does not even have to be consistent with 
the usual DOS partitions information.
[OpenBSD requires a primary MBR partition for booting, anything else is
officially unsupported.]

Throughout this document, we will distinguish between MBR partitions and
disklabel partitions whenever this is necessary.

The safest way to deal with things is to allocate one primary MBR partition for
OpenBSD, type A6, that you will partition further with disklabel.

If you can, it is even safer to devote a full disk to OpenBSD: this limits
the number of mistakes you can do. Admittedly there are some cases where
this isn't a option (my machine is a laptop... I have to cope with the
hard disk I have), or where this can be slower (SCSI disk setups will
yield better throughput with swap interleaved among the disks).

Prime rule: *ALWAYS* use the disk partitioning tools that go with the OS.
They know more about its internal workings than you do. So use linux fdisk
for linux partitions, don't let it touch the OpenBSD disklabel, and
reciprocally.

DOS and BIOS and all the problems of the world
----------------------------------------------
Due to historical accident, your machine resident `Operating System',
also known as the BIOS, can only access hard-disks up to cylinder 1024.
Various lying tricks are used, so that your whole disk is usually
accessible to the BIOS, except for very large disks (>8Gb).

fdisk is usually going to give you reliable information: anything that is
before cylinder 1024 can be accessed through the BIOS.

When you first boot up OpenBSD, the kernel will detect your hardware,
and give you a message such as
  wd0 at wdc0 drive 0: <TOSHIBA MK4006MAV>
  wd0: 3909MB, 7944 cyl, 16 head, 63 sec, 512 bytes/sec, 8007552 sec total
  wd0: using 16-sector 16-bit pio transfers, lba addressing
don't panic. This is just the real disk geometry. Trust fdisk on this one.

If fdisk shows you more than 1024 cylinders, you will have to cram OpenBSD
into that. Actually, it's enough that the disklabel partition used for
booting fits within the first 1024 cylinders (a:), so if you can get your
OpenBSD partition to start within 1024 cylinders, just get a small enough
a:, and you're in the clear.  (You can get by with a: a bit under 20Mb,
BTW, just enough for /bin /sbin, a kernel and /etc).

Mapping your disk
-----------------
Starting from Linux, get a grasp of your partitions. Use df to check which
is what, then fdisk to get the actual setup of the disk. 

Write down the setup onto a bit of paper, in case you make a mistake further
down.  It can come in very handy.

Disk /dev/hda: 128 heads, 63 sectors, 993 cylinders
Units = cylinders of 8064 * 512 bytes

   Device Boot    Start      End   Blocks   Id  System
/dev/hda1             1      211   850720+   6  DOS 16-bit >=32M
/dev/hda2           212      273   249984   83  Linux native
/dev/hda3           274      992  2899008   a6  OpenBSD

The + at the end of the DOS line is because linux fdisk is brain-damaged
and wants to write output in 1024-sized chunks, so this stands for
`850720 blocks and a half'

Older flavors of linux fdisk won't recognize a6 as OpenBSD.

As you can see, my linux setup is very small. I have enough to check how
things such as gcc work on linux, but my machine is definitely an
OpenBSD developer's box.

Get the display to sectors with u, and jot down the corresponding
information as well:

Disk /dev/hda: 128 heads, 63 sectors, 993 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start      End   Blocks   Id  System
/dev/hda1            63  1701503   850720+   6  DOS 16-bit >=32M
/dev/hda2       1701504  2201471   249984   83  Linux native
/dev/hda3       2201472  7999487  2899008   a6  OpenBSD
--
Okay, finally switch to expert mode, and write the corresponding data.

Disk /dev/hda: 128 heads, 63 sectors, 993 cylinders

Nr AF  Hd Sec  Cyl  Hd Sec  Cyl   Start    Size ID
 1 00   1   1    0 127  63  210      63 1701441 06
 2 00   0   1  211 127  63  272 1701504  499968 83
 3 00   0   1  273 127  63  991 2201472 5798016 a6
 4 00   0   0    0   0   0    0       0       0 00


Note that this is STILL the same data. The good point about this last
display is that it is almost what you're going to see in OpenBSD fdisk,
usually ! (I think there may be some very weird cases where this won't
match)

There are some differences though, mostly because Linux fdisk has made
some rather confusing choices:
- in simple mode it starts numbering cylinders at 1... whereas 
everything else starts from 0.
- in simple mode it shows blocks of 1024 bytes, which makes for half-blocks
(marked with a +) and sizes halved from the real block size.
- in expert mode it shows extended partitions offset from the start 
of the extended partition. 
- the hd/sec/cyl is a confusing order, as the sector number is computed
from cyl/hd/sec, in that order.
- it never shows and doesn't care about the real disk geometry.

You will notice that I don't have a linux swap partition visible. My
linux setup currently uses the OpenBSD swap area.

Before starting to install OpenBSD, now would be a good time to check the
INSTALL.pt document...  Especially note the alignment restriction of 
partitions (first sector of a partition must be at head 0, sector 1 of a
cylinder). This is enforced by Linux' fdisk.

The other point to note is that extended partitions are actually linked
lists. This will show up in OpenBSD's fdisk.

Your clock and OpenBSD
----------------------
OpenBSD expects your hardware clock to be in universal time, and uses 
time zones to give you local time. With Linux, this depends... 
most distributions use a small program called hwclock to set up the 
system time from the hardware clock when booting... there is a --utc 
option if your hardware clock is in universal time, but this is not 
always what happens by default.

For instance, on a redhat system, up to 5.2, this happens in 
/etc/rc.d/rc.sysinit which loads an /etc/sysconfig/clock that defines a 
variable called UTC, and then proceeds calling hwclock.
- ensure UTC is set to true,
- adjust your hardware clock from the system time if necessary, e.g.,
hwclock --systohc --utc.

Normally, this is one of the choices that the Linux installation program
lets you do: set your hardware clock to GMT.

The Linux partition table and OpenBSD
-------------------------------------
There used to be a problem with Linux's rc: it always mounts all file systems 
even in single-user mode.  The 2.2 kernels fix that in a handy way: the
partition recorded in the MBR is scanned for a disklabel, and marked with 
a ! if one is found. Then, the rest of the disk is scanned, before 
coming back to the disklabel itself. That way, changes to the 
OpenBSD disklabel won't affect the setup of the rest of the disk.

Anyhow, you may want to check that you can still boot from a Linux kernel
which doesn't know about disklabels.  The long term solution is to fix your
inittab and rc scripts to make deadly sure that single-user boot will work
-- preferably by moving disk mounts to multi-user.

The OpenBSD installation
------------------------
If you've got the space, you can install from your ext2fs partitions. This
is what I did, a long time in the past, as I had a slip connection to the 
rest of the world, and the OpenBSD install floppy does not include slip.

REMEMBER TO BACKUP ALL IMPORTANT DATA ON YOUR DISK BEFORE DOING THE 
INSTALLATION !!!

So you cp floppy*.fs /dev/fd0, then reboot from the floppy.

First, the booter loads, then there is a boot prompt. Five seconds later,
the kernel and the ram disk image are loaded, and the kernel is run.
After a while, through a few more prompts, you get to fdisk and you can
enter the new partition into the MBR.

This is what the fdisk dump looked like after my changes:
Disk: wd0       geometry: 992/128/63 [7999488 sectors]
Offset: 0       Signatures: 0xAA55,0x0
         Starting        Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
-------------------------------------------------------------------------
 0: 06    0   1   1 -  210 127  63 [        63 -    1701441] DOS > 32MB  
 1: 83  211   0   1 -  272 127  63 [   1701504 -     499968] Linux files*
 2: A6  273   0   1 -  991 127  63 [   2201472 -    5798016] OpenBSD     
 3: 00    0   0   0 -    0   0   0 [         0 -          0] unused      

fdisk geometry tells me that I needn't worry about cylinder 1024. 
This matches very  closely with what linux fdisk saw.

If you had extended partitions, it would be a trifle bit harder:
you just follow the extended partition links using select, jot down 
whatever you need, add the OpenBSD partition into the MBR to look like 
you want it to, and save everything.

The * at the end of partition #1 means that the system normally boots under
Linux. In reality, lilo takes control and disregards this completely.

After you leave fdisk, you get to the interesting part: the disklabel
itself. If all goes well, OpenBSD synthesizes a nice disklabel out of what
it can deduce from the disk, including the ext2fs partitions.

There are only a few subtleties to take care of:
- initially, you can ONLY edit the disklabel part that matches the
OpenBSD partition that was declared in the MBR (what you just entered in
fdisk, the `slice' from FreeBSD lingo). Most simple installation don't
need to edit more than that, but you can use b 0 * to unlock the whole
disk (this is a bad idea in most cases).
- your real disk geometry becomes more relevant. The Berkeley fast file system 
can't use partial cylinder groups, hence BSD partitions should start 
on cylinder boundaries, as any remaining sectors  will be lost anyway.
(Actually, what's important is the disk geometry that disklabel gives you.
Trust it on that). In my case, sectors/cylinder=1008 and bytes/sector=512, 
so the granularity of disklabel partitions is 504 Kb.
- units for size and offset can be given as sectors (default) or cylinders.

After edition, this is what my disklabel looks like:
# editing 

# using MBR partition 2: type A6 off 2201472 (0x219780) size 5798016 (0x587880)
# /dev/rwd0c:
type: ESDI
disk: 
label: TOSHIBA MK4006M
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 7944
total sectors: 8007552
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # milliseconds
track-to-track seek: 0  # milliseconds
drivedata: 0 

16 partitions:
#        size   offset    fstype   [fsize bsize   cpg]
  a:   104832  2201472    4.2BSD     1024  8192    16   # (Cyl. 2184 - 2287)
  b:   274176  2306304      swap                        # (Cyl. 2288 - 2559)
  c:  8007552        0    unused        0     0         # (Cyl.    0 - 7943)
  d:   613872  2580480    4.2BSD     1024  8192    16   # (Cyl. 2560 - 3168)
  e:   920304  4846464    4.2BSD     1024  8192    16   # (Cyl. 4808 - 5720)
  f:   628992  4217472    4.2BSD     1024  8192    16   # (Cyl. 4184 - 4807)
  g:   204624  5766768    4.2BSD     1024  8192    16   # (Cyl. 5721 - 5923)
  h:  1073520  5971392    4.2BSD     1024  8192    16   # (Cyl. 5924 - 6988)
  i:   962640  7044912    4.2BSD     1024  8192    16   # (Cyl. 6989 - 7943)
  j:  1701441       63     MSDOS                        # (Cyl.    0*- 1687)
  k:   499968  1701504    ext2fs                        # (Cyl. 1688 - 2183)
  l:  1023120  3194352    4.2BSD     1024  8192    16   # (Cyl. 3169 - 4183)

Things that check, more or less automatically:
- this disklabel is saved in MBR partition #2 (basic DOS partition 2), 
as expected.
- all the BSD partitions proper are aligned on a cylinder boundary (ie no '*').
the root partition begins at the precise same offset the corresponding DOS
partition begins, and it extends for the same length.
- the ext2fs partitions have the exact same layout under the OpenBSD
disklabel.
- the swap partition is very large. It gets used as mfs for my tmp
directories.

And here is the corresponding /etc/fstab:
/dev/wd0a       /          ffs    rw    1       1
/dev/wd0d       /usr       ffs    ro    1       2
/dev/wd0l       /usr/local ffs    ro    1       2
/dev/wd0e       /home      ffs    rw    1       2
/dev/wd0g       /var       ffs    rw    1       2
/dev/wd0h       /big       ffs    ro    1       2
/dev/wd0f       /usr/obj   ffs    rw    1       2
/dev/wd0i       /vbig      ffs    rw    1       2
/dev/wd0j       /dos       msdos  rw    1       2
/dev/wd0k       /linux     ext2fs rw    
/dev/wd0b       /tmp       mfs    rw    

One point that is somewhat laborious is that the disklabel -E mode
(which you are currently using) tends to move partitions around to ensure
that ALL defined partitions are contiguous. For that reason, it is better
if you don't have to use b 0 *, otherwise partitions will be moved around to
remove holes, without regard for the rigid MBR partitioning.
ext2fs and DOS partitions  should be recognized and positioned 
automatically if all goes well. 

Once the disklabel is written to disk, the installation proceeds as usual.

ext2fs partitions are perfectly usable from OpenBSD. 

Booting
-------
First time I booted my system back, I did not get into OpenBSD as expected... 
I plain forgot I had installed lilo in the master boot block, and lilo 
does not heed the active partition flag. The fix was rather simple: from
the Linux system, I just had to edit lilo.conf to add the OpenBSD entry:

boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=500
other=/dev/hda3
        label=bsd
        table=/dev/hda
image=/boot/vmlinuz-2.2.3
        label=linux
        vga=4
        root=/dev/hda2
        read-only
image=/boot/vmlinuz-2.0.36-0.7
        label=linux.orig
        root=/dev/hda2
        read-only
other=/dev/hda1
        label=dos
        table=/dev/hda

More details: I've kept the original redhat installation as
vmlinuz-2.0.36 because I'm paranoid, but the real setup uses only
bsd, linux, and dos.

Rerun lilo (DON'T FORGET THAT STEP), and voila, OpenBSD is able to boot!

Linux and OpenBSD partitions
----------------------------
The 2.2 kernel does incorporate my patch for the correct handling of
OpenBSD partitions. You will probably need to reconfigure and rebuild
your linux kernel to recognize BSD disklabels... Here is how it shows up
on my box:

Partition check:
 hda: hda1 hda2 hda3! < hda5 hda6 hda7 hda8 hda9 hda10 hda11 hda12 hda13 hda14 
hda15 >

- the disklabel is detected early, but handled later.
- disklabel handling should remove duplicates: all partitions that are present
as both MBR and disklabel partition should get removed silently. (this does
not seem to work as advertised presently).
- the remaining partitions are checked for consistency.

and here is my linux fstab:

/dev/hda2               /                       ext2    defaults        1 1
/dev/hda6               swap                    swap    defaults        0 0
/dev/hda5               /bsd                    ufs     ufstype=44bsd   1 2
/dev/hda7               /bsd/usr                ufs     ufstype=44bsd   1 2
/dev/hda8               /home                   ufs     ufstype=44bsd   1 2
/dev/hda9               /bsd/usr/obj            ufs     ufstype=44bsd   1 2
/dev/hda1               /dos                    vfat    defaults        1 2
/dev/fd0                /mnt/floppy             ext2    noauto          0 0
/dev/cdrom              /mnt/cdrom              iso9660 noauto,ro    

2.2 kernels also include a working UFS, though you may run into problems when
writing to ufs partitions.   Note the ufstype=44bsd. If you forget that
in your mounts, it will fail. Depending upon your installation, you may
get a failure message, or you will have to dig through /var/log/ to find
it.

Running Linux binaries under OpenBSD
------------------------------------
You just have to recompile your BSD kernel with COMPAT_LINUX, and set up
/emul/linux as explained in compat_linux(1). If you run GENERIC, you don't
have to recompile anything, as this is the default setup.

It's a good idea to mount your Linux file system under another point, then
make symbolic links so that you can control what gets used precisely.

As you have a complete linux system, don't bother with the ports 
emul/linux_lib entry: it's only a set of Linux libraries for people who 
don't have a Linux system running.

A small detail that may cause problems: uname still says `OpenBSD', even
under Linux compatibility.  The reason behind that is that we don't want
netscape to tell it was run from a Linux box, when it is used under
OpenBSD.

Some programs, for instance maple, do depend on uname answering `Linux'.
For maple, this is straightforward: you just have to fudge
/usr/local/maple/bin/maple.system.type to check OpenBSD in the same
class with Linux.

Similar shell scripts are easy to fix.  Binary programs that don't run
suid can be coerced by using LD_PRELOAD.

As a rule, this should be achieved on a program-by-program basis. 
The more networking programs that do tell they're running under OpenBSD,
the merrier !

A word of caution: brain-damaged linux installations
----------------------------------------------------
I just reinstalled the linux side of my laptop using redhat 5.2.  This
CD does insist on having TWO linux partitions: one root partition, and one
swap partition (even though I have 32 Mb of memory, largely enough for
the installation). Since it uses a 2.0.36 kernel, it does NOT handle BSD
disklabels, so I couldn't tell it to use my swap area (I have this bad
feeling that distributed 2.2 boot kernels won't include BSD disklabel
handling anyway). Instead, I had to back my last OpenBSD partition up, 
fiddle with my fdisk setup to feed the last cylinder as a swap partition
to their brain dead install CD. Then fetch the latest kernel source to the
linux side, and recompile to get a fully working linux setup. Finally,
reset the fdisk/disklabel to its normal state, and get the backed-up
partition to its normal location.

Another word of caution: getting enough rope to hang yourself
-------------------------------------------------------------
One previous version of this document got into much nastier details, and
gave installation instructions that were thoroughly dangerous.
- various tools may yield distinct views of your disk. These will match,
most often, but not always. Don't get confused if various fdisk, disklabel
utilities don't yield the same information. Generally, sector sizes and
offsets should match.
- try to keep various OSes segregated to their areas. Don't depend on
OpenBSD information to be correct for your linux setup and vice-versa.
Some weird problem with the brain-damaged PC architecture may force us to
tweak things so that OpenBSD will work seamlessly everywhere. Compatibility
with weird tricks is not a priority.

This being said, you will have noticed by now that the OpenBSD disklabel is
just another description of your hard-disk. It is almost completely
independent from the MBR description (it just has to be on the right sector
for the boot process to find it). You can get into trouble if things don't
match. Let it live within its MBR partition, unless you're completely sure
you know what you are doing, and don't expect there will always be someone
to get you out of trouble. If your setup is really too weird, no-one can help.

As far as the boot process goes, I think lilo allows you to boot from ANY
partition recorded in the MBR, including extended partitions. The only 
limitation is that the next stage bootstrap MUST take place entirely within
the first 1024 cylinders of the disk, as seen by the BIOS.  OpenBSD
MBR partitions that extend beyond cylinder 1024 are no problem, as long as
the disklabel root (a) partition doesn't extend beyond cylinder 1024.

Since Windows, OpenBSD, and linux all have that limitation, the easiest way 
is to start with Windows partitions (entirely within the first 1024
cylinders), follow with the linux boot partition (still within the first
1024 cylinders), then the OpenBSD area (which can span the 1024 cylinders
boundary, as long as a lives within the limit), and the remaining linux
partitions.

Weirder setups are unwarranted.  Several bsd on the same disk MAY be
possible, but will be harder to manage:
- it is better if disklabels match,
- linux will obey the first disklabel it finds, try to ensure this is
OpenBSD disklabel, it can describe more partitions than the others,
- other BSD may get confused with each other data. Normally, the A5/A6
split ensures that Net/Free won't get mixed up with OpenBSD,
- FreeBSD and NetBSD will probably get confused with each other,
- if you have a 1024 cylinder limit, all boot areas must stay within the
1024 cylinder boundary, so only one of the BSD may span that limit, apart
from very, very nasty tricks.

Finally, how much disk space do you have anyway ?  Do you really need to
cram that many OSes on the same disk ? Put them on separate disks rather.
If you reach that stage, you'd better be ready to hack at the linux kernel
to recognize several disklabels, for instance, or generally know what
you're doing.