OpenBSD CVS

CVS log for src/usr.sbin/vmd/vmd.h


[BACK] Up to [local] / src / usr.sbin / vmd

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.125 / (download) - annotate - [select for diffs], Tue Feb 20 21:40:37 2024 UTC (3 months, 2 weeks ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5, HEAD
Changes since 1.124: +11 -2 lines
Diff to previous 1.124 (colored)

Utilize separate threads for RX and TX in vmd(8)'s vionet.

This commit adds multithreading to allow both virtqueues to be
processed in parallel along with additional synchronization primitives
to protect device configuration state. Allowing RX and TX to operate
independently reduces overall network latency for guests and helps
alleviate the TX side dominating cpu time.

Tested with help from phessler@, kn@, and mlarkin@. ok mlarkin@.

Revision 1.124 / (download) - annotate - [select for diffs], Thu Jul 27 09:27:43 2023 UTC (10 months, 1 week ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_7_4_BASE, OPENBSD_7_4
Changes since 1.123: +5 -1 lines
Diff to previous 1.123 (colored)

vmd(8): fix verbose logging in child processes.

The introduction of exec for vm's and fork+exec for virtio block
and network devices missed passing the log verbosity in argv. Add
the "-v" arguments based on current vverbosity at time of exec.

ok brynet@, mlarkin@

Revision 1.123 / (download) - annotate - [select for diffs], Thu Jul 13 18:31:59 2023 UTC (10 months, 3 weeks ago) by dv
Branch: MAIN
Changes since 1.122: +12 -12 lines
Diff to previous 1.122 (colored)

vmd(8): pull validation into local prefix parser.

Validation for local prefixes, both inet and inet6, was scattered
around. To make it even more confusing, vmd was using generic address
parsing logic from prior network daemons. vmd doesn't need to parse
addresses other than when parsing the local prefix settings in
vm.conf and no runtime parsing is needed.

This change merges parsing and validation based on vmd's specific
needs for local prefixes (e.g. reserving enough bits for vm id and
network interface id encoding in an ipv4 address). In addition, it
simplifies the struct from a generic address struct to one focused
on just storing the v4 and v6 prefixes and masks. This cleans up an
unused TAILQ struct member that isn't used by vmd and was leftover
copy-pasta from those prior daemons.

The address parsing that vmd uses is also updated to using the
latest logic in bgpd(8).

ok mlarkin@

Revision 1.122 / (download) - annotate - [select for diffs], Sat May 13 23:15:28 2023 UTC (12 months, 3 weeks ago) by dv
Branch: MAIN
Changes since 1.121: +5 -8 lines
Diff to previous 1.121 (colored)

vmm(4)/vmd(8): switch to anonymous shared mappings.

While splitting out emulated virtio network and block devices into
separate processes, I originally used named mappings via shm_mkstemp(3).
While this functionally achieved the desired result, it had two
unintended consequences:

1) tearing down a vm process and its child processes required
excessive locking as the guest memory was tied into the VFS layer.

2) it was observed by mlarkin@ that actions in other parts of the
VFS layer could cause some of the guest memory to flush to storage,
possibly filling /tmp.

This commit adds a new vmm(4) ioctl dedicated to allowing a process
request the kernel share a mapping of guest memory into its own vm
space. This requires an open fd to /dev/vmm (requiring root) and
both the "vmm" and "proc" pledge(2) promises. In addition, the caller
must know enough about the original memory ranges to reconstruct them
to make the vm's ranges.

Tested with help from Mischa Peters.

ok mlarkin@

Revision 1.121 / (download) - annotate - [select for diffs], Fri Apr 28 19:46:42 2023 UTC (13 months, 1 week ago) by dv
Branch: MAIN
Changes since 1.120: +5 -2 lines
Diff to previous 1.120 (colored)

vmd(8)/vmctl(8): allow vm owners to override boot kernel.

vmd allows non-root users to "own" a vm defined in vm.conf(5). While
the user can start/stop the vm, if they break their filesystem they
have no means of booting recovery media like a ramdisk kernel.

This change opens the provided boot kernel via vmctl and passes the
file descriptor through the control channel to vmd. The next boot
of the vm will use the provided file descriptor as boot kernel/bios.
Subsequent boots (e.g. a reboot) will return to using behavior
defined in vm.conf or the default bios image.

ok mlarkin@

Revision 1.120 / (download) - annotate - [select for diffs], Thu Apr 27 22:47:27 2023 UTC (13 months, 1 week ago) by dv
Branch: MAIN
Changes since 1.119: +19 -2 lines
Diff to previous 1.119 (colored)

vmd(8): introduce multi-process model for virtio devices.

Isolate virtio network and block device emulation in dedicated
processes, forked and exec'd from the vm process. This allows for
tightening pledge promises to just "stdio".

Communication between the vcpu's and these devices now occurs via
imsg channels, which adds the benefit of not always blocking the
vcpu thread while emulating the device.

With this commit, it's possible that vmd is the first open source
hypervisor that *defaults* to a multi-process device emulation
model without requiring any additional configuration from the
operator.

Testing help from phessler@ and Mischa Peters.

ok mlarkin@

Revision 1.119 / (download) - annotate - [select for diffs], Wed Apr 26 16:12:21 2023 UTC (13 months, 1 week ago) by mlarkin
Branch: MAIN
Changes since 1.118: +3 -1 lines
Diff to previous 1.118 (colored)

Fix vmd after vmm MI/MD split

Revision 1.118 / (download) - annotate - [select for diffs], Tue Apr 25 12:46:13 2023 UTC (13 months, 2 weeks ago) by dv
Branch: MAIN
Changes since 1.117: +9 -4 lines
Diff to previous 1.117 (colored)

vmm(4)/vmd(8): pull struct members out of vmm ioctl create struct.

The object sent to vmm(4) contained file paths and details the
kernel does not need for cpu virtualization as device emulation is
in userland. Effectively, "pull up" the struct members from the
vm_create_params struct to the parent vmop_create_params struct.

This allows us to clean up some of vmd(8) and simplify things for
switching to having vmctl(8) open the "kernel" file (SeaBIOS, bsd.rd,
etc.) to allow users to boot recovery ramdisk kernels.

ok mlarkin@

Revision 1.117 / (download) - annotate - [select for diffs], Sun Apr 23 12:11:37 2023 UTC (13 months, 2 weeks ago) by dv
Branch: MAIN
Changes since 1.116: +6 -1 lines
Diff to previous 1.116 (colored)

vmd(8): teach vmm process how to exec.

Use execvp(2) to launch vm children with new address spaces.
Consequently, introduces use of unveil(2) into the vmm and vm
processes.

This imposes the requirement of launching vmd with absolute paths,
similar to sshd(8).

ok mlarkin@

Revision 1.116 / (download) - annotate - [select for diffs], Sun Apr 16 12:47:26 2023 UTC (13 months, 3 weeks ago) by dv
Branch: MAIN
Changes since 1.115: +2 -1 lines
Diff to previous 1.115 (colored)

vmd(8): clean up fd closing in vmm process.

Some mild tidying of fd closing in the vmm process in prep for
landing parts of my fork+exec diff.

With input from guenther@ on the nuances of if/when EINTR may happen
in a call to close(2).

ok mlarkin@

Revision 1.115 / (download) - annotate - [select for diffs], Sun Apr 2 02:04:10 2023 UTC (14 months, 1 week ago) by dv
Branch: MAIN
Changes since 1.114: +2 -2 lines
Diff to previous 1.114 (colored)

vmd(8): migrate vmd_vm.vm_ttyname to char array.

Other structs use a fixed length array already. This allows a vmd_vm
object to be transmitted over an ipc channel, too.

Additionally, solves a segfault caused by a strlcpy(3) in an error
path.

ok mlarkin@

Revision 1.114 / (download) - annotate - [select for diffs], Sat Jan 28 14:40:53 2023 UTC (16 months, 1 week ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_7_3_BASE, OPENBSD_7_3
Changes since 1.113: +21 -10 lines
Diff to previous 1.113 (colored)

Move some header definitions from vmm(4) to vmd(8).

Part of an ongoing effort to move userland-specific information out
of a kernel header and directly into vmd(8). No functional change.

ok mlarkin@

Revision 1.113 / (download) - annotate - [select for diffs], Sat Jan 14 20:55:55 2023 UTC (16 months, 3 weeks ago) by dv
Branch: MAIN
Changes since 1.112: +2 -1 lines
Diff to previous 1.112 (colored)

Only open /dev/vmm once in vmd(8).

Have the parent process open /dev/vmm and send the fd to the vmm
child process. Only the vmm process and its resulting children
(guest vms) need it for ioctl calls.

ok kn@

Revision 1.112 / (download) - annotate - [select for diffs], Fri Dec 23 19:25:22 2022 UTC (17 months, 2 weeks ago) by dv
Branch: MAIN
Changes since 1.111: +2 -1 lines
Diff to previous 1.111 (colored)

vmd(8): implement zero-copy operations on virtqueues.

The original virtio device implementation relied on allocating a
buffer on heap, copying the virtqueue from the guest, mutating the
copy, and then overwriting the virtqueue in the guest.

While the approach worked, it was both complex and added extra
overhead. On older hardware, switching to the zero-copy approach
can show a noticeable performance improvement for vionet devices.
An added benefit is this diff also reduces the amount of code in
vmd, which is always a welcome change.

In addition, change to talking about the queue pfn and not "address"
as the virtio-pci spec has drivers provide a 32-bit value representing
the physical page number of the location in guest memory, not the
linear address.

Original idea from dlg@ while working on re-adding async task queues.

ok dlg@, tested by many

Revision 1.111 / (download) - annotate - [select for diffs], Mon Oct 31 14:02:11 2022 UTC (19 months, 1 week ago) by dv
Branch: MAIN
Changes since 1.110: +1 -23 lines
Diff to previous 1.110 (colored)

vmd(8): remove unfinished user accounting.

User accounting and enforcement was never finished. tedu the thing
until someone wants to pick it up and finish it.

Originally found by Matthew Martin.

ok mlarkin@, kn@. input from tb@.

Revision 1.110 / (download) - annotate - [select for diffs], Tue Sep 13 10:28:19 2022 UTC (20 months, 3 weeks ago) by martijn
Branch: MAIN
CVS Tags: OPENBSD_7_2_BASE, OPENBSD_7_2
Changes since 1.109: +20 -1 lines
Diff to previous 1.109 (colored)

Add (partial) support for agentx in vmd.

Metrics can be found under mib-2.236 and VM-MIB (RFC7666).

Stress tested by and happy noises from Mischa Peters
OK dv@

Revision 1.109 / (download) - annotate - [select for diffs], Tue May 3 21:39:18 2022 UTC (2 years, 1 month ago) by dv
Branch: MAIN
Changes since 1.108: +2 -2 lines
Diff to previous 1.108 (colored)

vmm/vmd/vmctl: standardize memory units to bytes

At different points in the vm lifecycle vmm(4), vmctl(8), and vmd(8)
refer to a vm's memory range sizes in either bytes or megabytes.
This is needlessly complex.

Switch to using bytes everywhere and adjust types and constants
accordingly. While this makes it possible to specify vm's with
memory in fractions of megabytes, the logic requiring whole
megabyte values remains.

Feedback from deraadt@, mlarkin@, and Matthew Martin.

ok mlarkin@

Revision 1.108 / (download) - annotate - [select for diffs], Tue Jan 4 15:22:53 2022 UTC (2 years, 5 months ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_1_BASE, OPENBSD_7_1
Changes since 1.107: +1 -2 lines
Diff to previous 1.107 (colored)

Remove unused imsg type IMSG_VMDOP_RECEIVE_VM_RESPONSE.
OK benno@ dv@

Revision 1.107 / (download) - annotate - [select for diffs], Mon Nov 29 05:17:35 2021 UTC (2 years, 6 months ago) by deraadt
Branch: MAIN
Changes since 1.106: +3 -2 lines
Diff to previous 1.106 (colored)

mostly avoid sys/param.h with a local nitems()
ok mlarkin

Revision 1.106 / (download) - annotate - [select for diffs], Wed Sep 1 11:08:21 2021 UTC (2 years, 9 months ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_7_0_BASE, OPENBSD_7_0
Changes since 1.105: +5 -9 lines
Diff to previous 1.105 (colored)

remove unused functions and cleanup vmd.h

Discussed with mlarkin@. These functions were implemented but never
used. While in vmd.h, fix the order to match current vmd(8) reality.

Revision 1.105 / (download) - annotate - [select for diffs], Sun Apr 11 14:12:42 2021 UTC (3 years, 1 month ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE, OPENBSD_6_9
Changes since 1.104: +2 -2 lines
Diff to previous 1.104 (colored)

Correct a comment: reference the correct file

Revision 1.104 / (download) - annotate - [select for diffs], Mon Apr 5 18:09:48 2021 UTC (3 years, 2 months ago) by dv
Branch: MAIN
Changes since 1.103: +1 -5 lines
Diff to previous 1.103 (colored)

Support booting from compressed kernel images.

The bsd.rd ramdisk now ships gzip'd on amd64. Use libz in base to
transparently handle decompression of any compressed kernel images.

Patch from Josh Rickmar.

ok kn@

Revision 1.103 / (download) - annotate - [select for diffs], Mon Mar 29 23:37:01 2021 UTC (3 years, 2 months ago) by dv
Branch: MAIN
Changes since 1.102: +14 -1 lines
Diff to previous 1.102 (colored)

Propagate host-side tap(4) lladdr to guest vm process to allow unicast dhcp
and bootp renewals with vmd(8)'s built-in dhcp server. Previous behavior
ignored did not intercept these packets and instead transmitted them.

This should make vmd(8)'s dhcp behave more as a true dhcp server should and
allows it to work properly with the new dhcpleased(8) attempting a renewal.

OK mlarkin@

Revision 1.102 / (download) - annotate - [select for diffs], Fri Mar 19 09:29:33 2021 UTC (3 years, 2 months ago) by kn
Branch: MAIN
Changes since 1.101: +1 -3 lines
Diff to previous 1.101 (colored)

Remove booting from kernels in raw/qcow2 images

Diff and (slightly tweaked) text below from
Dave Voutila < dave at sisu dot io >, thanks!

--
Since 6.7 switched to FFS2 as the default filesystem for new installs,
the ability for vmd(8) to load a kernel and boot.conf from a disk image
directly (without SeaBIOS) has been broken.

A diff from tb to add FFS2 support never mdae it into the tree.

On 5th Jan 2021, new ramdisks for amd64 have started shipping gzipped,
breaking the ability to load the bsd.rd directly as a kernel image for a vmd
guest without first uncompressing the image.

Using BIOS works, the FFS2 change happend ten months ago and few if any have
complained about the breakage.  vmctl(8) is still vague about supporting it
per its man page and one still has to pass the disk image twice as a "-b"
and "-d" argument to boot an OpenBSD guest *without* BIOS.

Josh Rickmar reported the gzip issue on bugs@ and provided patches to add
support for compressed ramdisks and kernel images.  The easiest way to do so
is to drop support for FFS images since they require a call to fmemopen(3)
while all the other logic uses fopen(3)/fdopen(3) calls and a file
descriptor.  It is much easier to get thsoe patches merged if they don't
have to account for extracting files from disk images.
--

No objections anyone
"Removing it makes sense" reyk (who wrote the FFS module)
OK mlarkin

Revision 1.101 / (download) - annotate - [select for diffs], Wed Sep 23 19:18:18 2020 UTC (3 years, 8 months ago) by martijn
Branch: MAIN
CVS Tags: OPENBSD_6_8_BASE, OPENBSD_6_8
Changes since 1.100: +0 -14 lines
Diff to previous 1.100 (colored)

Revert agentx support for now, we're too close to release.

requested by deraadt@

Revision 1.100 / (download) - annotate - [select for diffs], Wed Sep 23 15:52:06 2020 UTC (3 years, 8 months ago) by martijn
Branch: MAIN
Changes since 1.99: +15 -1 lines
Diff to previous 1.99 (colored)

Add support for agentx to vmd.

This is based around VM-MIB from RFC7666,but  does not export the full
spec. People more knowledgeable of vmd are encouraged to expand on this.

Revision 1.99 / (download) - annotate - [select for diffs], Sun Jun 28 16:52:45 2020 UTC (3 years, 11 months ago) by pd
Branch: MAIN
Changes since 1.98: +19 -1 lines
Diff to previous 1.98 (colored)

vmd(8): Eliminate libevent state corruption

libevent functions for com, pic and rtc are now only called on event_thread.
vcpu exit handlers send messages on a dev pipe and callbacks on these events do
the event management (event_add, evtimer_add, etc).  Previously, libevent state
was mutated by two threads, event_thread, that runs all the callbacks and the
vcpu thread when running exit handlers.  This could have lead to libevent state
corruption.

Patch from Dave Voutila <dave@sisu.io>

ok claudio@
tested by abieber@ and brynet@

Revision 1.98 / (download) - annotate - [select for diffs], Thu Dec 12 03:53:38 2019 UTC (4 years, 5 months ago) by pd
Branch: MAIN
CVS Tags: OPENBSD_6_7_BASE, OPENBSD_6_7
Changes since 1.97: +8 -1 lines
Diff to previous 1.97 (colored)

vmd: start vms defined in vm.conf in a staggered fashion

This addresses 'thundering herd' problem when a lot of
vms are configured in vm.conf.  A lot of vms booting in parallel can
overload the host and also mess up tsc calibration in openbsd guests as
it uses PIT which doesn't fire reliably if the host is overloaded.

We default to starting vms with parallelism of ncpuonline and a delay 30 seconds
between batches.  This is configurable in vm.conf.

ok mlarkin@ (also addressed comments from cheloha@)

Revision 1.97 / (download) - annotate - [select for diffs], Sat Sep 7 09:11:14 2019 UTC (4 years, 9 months ago) by tobhe
Branch: MAIN
CVS Tags: OPENBSD_6_6_BASE, OPENBSD_6_6
Changes since 1.96: +2 -2 lines
Diff to previous 1.96 (colored)

Remove unused VMD_DISK_INVALID message type and mark it obsolete.

ok mlarkin@

Revision 1.96 / (download) - annotate - [select for diffs], Wed Aug 14 07:34:49 2019 UTC (4 years, 9 months ago) by anton
Branch: MAIN
Changes since 1.95: +2 -1 lines
Diff to previous 1.95 (colored)

Improve the error message when supplying an invalid template to vmctl
start. Favoring 'invalid template' over 'permission denied' should give
the user a better hint on what went wrong.

ok kn@ mlarkin@

Revision 1.95 / (download) - annotate - [select for diffs], Wed Jul 17 05:51:07 2019 UTC (4 years, 10 months ago) by pd
Branch: MAIN
Changes since 1.94: +2 -2 lines
Diff to previous 1.94 (colored)

vmm/vmd:  Fix migration with pvclock

Implement VMM_IOC_READVMPARAMS and VMM_IOC_WRITEVMPARAMS ioctls to read and
write pvclock state.

reads ok mlarkin@

Revision 1.94 / (download) - annotate - [select for diffs], Sat May 11 23:07:46 2019 UTC (5 years ago) by jasper
Branch: MAIN
Changes since 1.93: +2 -1 lines
Diff to previous 1.93 (colored)

report vm state through 'vmctl status'; whereas previously this would display the state of
the vcpu (which is why it got removed), it now actually reports the correct state
(running, stopped, disabled, paused, etc)

ok ccardenas@ mlarkin@

Revision 1.93 / (download) - annotate - [select for diffs], Sat May 11 19:58:02 2019 UTC (5 years ago) by jasper
Branch: MAIN
Changes since 1.92: +2 -1 lines
Diff to previous 1.92 (colored)

add missing comment about VM_STATE_SHUTDOWN; as discussed with ccardenas@

Revision 1.92 / (download) - annotate - [select for diffs], Sat May 11 19:55:14 2019 UTC (5 years ago) by jasper
Branch: MAIN
Changes since 1.91: +9 -8 lines
Diff to previous 1.91 (colored)

track the state of the vm (running, paused, etc) using a single bitfield instead of
a handful of separate variables. this will makes it easier for vmd to report
and check on the individual vm states

no functional change intended

ok ccardenas@ mlarkin@

Revision 1.91 / (download) - annotate - [select for diffs], Fri May 10 18:11:27 2019 UTC (5 years, 1 month ago) by jasper
Branch: MAIN
Changes since 1.90: +1 -2 lines
Diff to previous 1.90 (colored)

remove receive_vm prototype for the function does not exist (anymore)

ok pd@

Revision 1.90 / (download) - annotate - [select for diffs], Wed Feb 20 07:00:25 2019 UTC (5 years, 3 months ago) by mlarkin
Branch: MAIN
CVS Tags: OPENBSD_6_5_BASE, OPENBSD_6_5
Changes since 1.89: +2 -2 lines
Diff to previous 1.89 (colored)

vmd(8): initialize guest %drX registers to power-on defaults on launch

Initializes the %drX registers to power on defaults, and bump the VM
send/recieve header to reflect same

discussed with deraadt@

Revision 1.89 / (download) - annotate - [select for diffs], Mon Dec 10 21:30:33 2018 UTC (5 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.88: +2 -2 lines
Diff to previous 1.88 (colored)

Implement the fw_cfg interface basics and use it to set the bootorder
if a bootdevice was forced. This implements both the pure IO port interface
and also the new DMA interface, a few direct commands are implemented which
are needed but in general the "file" interface should be used. There is no
write support for the guest. Tested against the latest vmm-firmware port.
This requires also a -current kernel to pass the IO ports to vmd(8).
OK mlarkin@ ccardenas@

Revision 1.88 / (download) - annotate - [select for diffs], Thu Dec 6 09:20:06 2018 UTC (5 years, 6 months ago) by claudio
Branch: MAIN
Changes since 1.87: +6 -1 lines
Diff to previous 1.87 (colored)

Make it possible to define the bootdevice in vmd. This information is used
currently only when booting a OpenBSD kernel. If VMBOOTDEV_NET is used the
internal dhcp server will pass "auto_install" as boot file to the client and
the boot loader passes the MAC of the first interface to the kernel to indicate
PXE booting. Adding boot order support to SeaBIOS is not yet implemented.
Ok ccardenas@

Revision 1.87 / (download) - annotate - [select for diffs], Tue Dec 4 08:15:09 2018 UTC (5 years, 6 months ago) by claudio
Branch: MAIN
Changes since 1.86: +2 -1 lines
Diff to previous 1.86 (colored)

Introduce IMSG_VMDOP_WAIT_VM_REQUEST a control message that registers a
vmctl peerid that should be informed when the VM is stopped (like when the
guest does a shutdown). Uses the same logic as using the VMOP_WAIT flag on
IMSG_VMDOP_TERMINATE_VM_REQUEST.
Ok ccardenas@, reyk@

Revision 1.86 / (download) - annotate - [select for diffs], Mon Nov 26 05:44:46 2018 UTC (5 years, 6 months ago) by ori
Branch: MAIN
Changes since 1.85: +10 -1 lines
Diff to previous 1.85 (colored)

Keep a list of known vms, and reuse the VM IDs.

This means that when using '-L', the IP addresses of the VMs are stable.

ok reyk@

Revision 1.85 / (download) - annotate - [select for diffs], Wed Nov 21 12:31:47 2018 UTC (5 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.84: +21 -6 lines
Diff to previous 1.84 (colored)

Add support for "local inet6" interfaces.

ok & test ccardenas@, additional review from kn@

Revision 1.84 / (download) - annotate - [select for diffs], Fri Oct 19 10:12:39 2018 UTC (5 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.83: +2 -2 lines
Diff to previous 1.83 (colored)

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format.  This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in.  The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@

Revision 1.83 / (download) - annotate - [select for diffs], Mon Oct 15 10:35:41 2018 UTC (5 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.82: +10 -1 lines
Diff to previous 1.82 (colored)

Prevent VM reboot loops by rate-limiting the interval a VM can reboot.

This looping has been experienced by people who run VMs with a broken
kernel or boot loader that trigger a very fast reboot loop (triple
fault) of a VM that ends up using a lot of CPU and resources on the
host.  Some fixes in vmm(4) and vmd(8) helped to avoid such conditions
but it can still occur if something is wrong in the guest VM itself.

If the VM restarts after less than VM_START_RATE_SEC (6) seconds, we
increment the limit counter.  After VM_START_RATE_LIMIT (3) of suchs
fast reboots the VM is stopped.

There are only very few people who intentionally want to reboot-loop a
VM very quickly (many times within a second); mostly for fuzzing.
They will have to recompile and adjust the stated #defines in the code
as we don't have a config option to disable it.

OK mlarkin@

Revision 1.82 / (download) - annotate - [select for diffs], Mon Oct 8 16:32:01 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
CVS Tags: OPENBSD_6_4_BASE, OPENBSD_6_4
Changes since 1.81: +8 -4 lines
Diff to previous 1.81 (colored)

Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image.  Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl.  To use it:

	vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@

Revision 1.81 / (download) - annotate - [select for diffs], Mon Oct 1 09:31:15 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.80: +4 -1 lines
Diff to previous 1.80 (colored)

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)

Revision 1.80 / (download) - annotate - [select for diffs], Fri Sep 28 12:35:32 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.79: +5 -3 lines
Diff to previous 1.79 (colored)

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@

Revision 1.79 / (download) - annotate - [select for diffs], Sun Sep 9 04:09:32 2018 UTC (5 years, 9 months ago) by ccardenas
Branch: MAIN
Changes since 1.78: +6 -1 lines
Diff to previous 1.78 (colored)

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf.  The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
    disable
    memory 2G
    disk "/home/user/vmm/current.qc2" format "qcow2"
    interface { switch "external" }
}

or

vm "current" {
    disable
    memory 2G
    disk "/home/user/vmm/current.raw" format "raw"
    interface { switch "external" }
}

is equivlanet to

vm "current" {
    disable
    memory 2G
    disk "/home/user/vmm/current.raw"
    interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.

Revision 1.78 / (download) - annotate - [select for diffs], Sun Jul 15 14:36:54 2018 UTC (5 years, 10 months ago) by reyk
Branch: MAIN
Changes since 1.77: +23 -1 lines
Diff to previous 1.77 (colored)

Track resources and enforce cpu/memory/interface limits for non-root users.

The limits are currently hard-coded and undocumented (4 CPUs/VMs, 2G
memory, 8 interfaces) but will be configurable in an upcoming diff.
These limits are tracked in total usage; for example, a user will be
able to run up to 4 VMs with 512M of memory or a single VM with 2G.

OK ccardenas@ mlarkin@

Revision 1.77 / (download) - annotate - [select for diffs], Fri Jul 13 10:26:57 2018 UTC (5 years, 10 months ago) by reyk
Branch: MAIN
Changes since 1.76: +5 -2 lines
Diff to previous 1.76 (colored)

Check the disk/kernel/cdrom file permissions after openening the fd.

This prevents time of TOCTOU attacks for instances.

OK mlarkin@

Revision 1.76 / (download) - annotate - [select for diffs], Fri Jul 13 08:42:49 2018 UTC (5 years, 10 months ago) by reyk
Branch: MAIN
Changes since 1.75: +20 -10 lines
Diff to previous 1.75 (colored)

Add "allow instance" option.

This allows users to create VM instances and change desired options,
for example a user can be allowed to run a VM with all the
pre-configured options but specify an own disk image.

(mlarkin@ was fine with iterating over it)

OK ccardenas@

Revision 1.75 / (download) - annotate - [select for diffs], Thu Jul 12 12:04:49 2018 UTC (5 years, 10 months ago) by reyk
Branch: MAIN
Changes since 1.74: +3 -1 lines
Diff to previous 1.74 (colored)

Allow to use configured/running VMs as templates for other VM instances.

This introduces new grammar and the -t optional in vmctl start.

(For now, only root can create VM instances; but it is planned to allow
users to create their own VMs based on permissions and quota.)

OK ccardenas@ mlarkin@ jmc@

Revision 1.74 / (download) - annotate - [select for diffs], Wed Jul 11 13:19:47 2018 UTC (5 years, 10 months ago) by reyk
Branch: MAIN
Changes since 1.73: +4 -2 lines
Diff to previous 1.73 (colored)

Add -w option to vmctl stop to wait for completion of VM termination.

Use it in /etc/rc.d/vmd accordingly.

OK sthen@

Revision 1.73 / (download) - annotate - [select for diffs], Wed Jul 11 09:35:44 2018 UTC (5 years, 10 months ago) by reyk
Branch: MAIN
Changes since 1.72: +2 -1 lines
Diff to previous 1.72 (colored)

Add -f option to vmctl stop to forcefully kill a VM.

This also fixes a bug in vmm_sighdlr where it might have missed
forwarding the TERMINATE_EVENT to the vmd parent after a VM child
died, leading to an abandoned VM in the vmd parent process.

OK ccardenas@ mlarkin@ benno@ kn@

Revision 1.72 / (download) - annotate - [select for diffs], Tue Jul 10 20:43:15 2018 UTC (5 years, 11 months ago) by reyk
Branch: MAIN
Changes since 1.71: +1 -7 lines
Diff to previous 1.71 (colored)

vmd already had DEBUG/DPRINTF, there is no need for VMD_DEBUG/dprintf

Replace all occurences of dprintf with DPRINTF (defined in proc.h).

Revision 1.71 / (download) - annotate - [select for diffs], Tue Jul 10 16:15:51 2018 UTC (5 years, 11 months ago) by reyk
Branch: MAIN
Changes since 1.70: +3 -3 lines
Diff to previous 1.70 (colored)

Tweak debug log messages

- Turn tracing messages into DPRINTF (only compiled with DEBUG).

- Pass __func__ to vm_stop and vm_remove: this way we can track who
called the function in the async context.  It replaces the manual
log_debug in front of each vm_stop/vm_remove.  This debug logging
trick can be removed in the future once we are more confident about
it.

OK ccardenas@ mlarkin@

Revision 1.70 / (download) - annotate - [select for diffs], Mon Jul 9 08:43:09 2018 UTC (5 years, 11 months ago) by mlarkin
Branch: MAIN
Changes since 1.69: +2 -2 lines
Diff to previous 1.69 (colored)

vmd(8): stash device IRQ in the device struct

ok kettenis

Revision 1.69 / (download) - annotate - [select for diffs], Tue Jun 26 10:00:08 2018 UTC (5 years, 11 months ago) by reyk
Branch: MAIN
Changes since 1.68: +3 -2 lines
Diff to previous 1.68 (colored)

Add "socket owner" to allow changing the owner of the vmd control socket.

This allows to open vmctl control or console access to other users
that are not in group wheel.  Access for non-root users still defaults
to read-only actions unless you change the owner (user/group) of each
individual VM.

Requested by Mischa Peters

OK mlarkin@

Revision 1.68 / (download) - annotate - [select for diffs], Fri Apr 27 12:15:10 2018 UTC (6 years, 1 month ago) by mlarkin
Branch: MAIN
Changes since 1.67: +2 -2 lines
Diff to previous 1.67 (colored)

vmd(8): implement vmd side of ELCR registers

ok guenther

Revision 1.67 / (download) - annotate - [select for diffs], Wed Jan 3 05:39:56 2018 UTC (6 years, 5 months ago) by ccardenas
Branch: MAIN
CVS Tags: OPENBSD_6_3_BASE, OPENBSD_6_3
Changes since 1.66: +7 -1 lines
Diff to previous 1.66 (colored)

Add initial CD-ROM support to VMD via vioscsi.

* Adds 'cdrom' keyword to vm.conf(5) and '-r' to vmctl(8)
* Support various sized ISOs (Limitation of 4G ISOs on Linux guests)
* Known working guests: OpenBSD (primary), Alpine Linux (primary),
  CentOS 6 (secondary), Ubuntu 17.10 (secondary).
  NOTE: Secondary indicates some issue(s) preventing full/reliable
  functionality outside the scope of the vioscsi work.
* If the attached disks are non-bootable (i.e. empty), SeaBIOS (vmd's
  default BIOS) will boot from CD-ROM.

ok mlarkin@, jca@

Revision 1.66 / (download) - annotate - [select for diffs], Sat Nov 11 02:50:08 2017 UTC (6 years, 6 months ago) by mlarkin
Branch: MAIN
Changes since 1.65: +1 -3 lines
Diff to previous 1.65 (colored)

update switch handling in vmd(8). vmd now gets switch information (rdomain,
etc) from underlying switch interface instead of handling this on its
own.

Diff from carlos cardenas, Thanks!

ok reyk@

Revision 1.65 / (download) - annotate - [select for diffs], Mon Oct 30 03:37:33 2017 UTC (6 years, 7 months ago) by mlarkin
Branch: MAIN
Changes since 1.64: +2 -2 lines
Diff to previous 1.64 (colored)

vmd no longer creates bridges by default. users should create bridges in
/etc/hostname.bridge* files, and specify which bridge to use for a given
virtual switch in vm.conf.

diff from Carlos Cardenas, thanks

Revision 1.64 / (download) - annotate - [select for diffs], Mon Sep 11 23:32:34 2017 UTC (6 years, 8 months ago) by dlg
Branch: MAIN
CVS Tags: OPENBSD_6_2_BASE, OPENBSD_6_2
Changes since 1.63: +5 -1 lines
Diff to previous 1.63 (colored)

add functions to provide direct access to guest memory as vmd addresses

iovec_mem() populates an iovec array based on guest physical
addresses. this allows the use of things like readv and writev for
moving data between the guest and a disk image file without having
to bounce the memory.

vaddr_mem() provides a vmd usable pointer based on a guests physical
address. this makes it possible to directly reference things like
virtio rings without having to bounce that memory either. however,
it assumes that a contiguous range of guest physical memory will
sit in a single vm memory range. mlarkin@ says this is right.

ok mlarkin@

Revision 1.63 / (download) - annotate - [select for diffs], Mon Sep 11 23:25:05 2017 UTC (6 years, 8 months ago) by dlg
Branch: MAIN
Changes since 1.62: +5 -1 lines
Diff to previous 1.62 (colored)

bring SET, CLR, and ISSET macros in

they make bit fiddling easier to read in later diffs.

ok mlarkin@

Revision 1.62 / (download) - annotate - [select for diffs], Fri Sep 8 07:08:49 2017 UTC (6 years, 9 months ago) by mlarkin
Branch: MAIN
Changes since 1.61: +2 -1 lines
Diff to previous 1.61 (colored)

better VM termination handling.

diff provided by Carlos Cardenas, thanks

Revision 1.61 / (download) - annotate - [select for diffs], Thu Aug 31 06:23:37 2017 UTC (6 years, 9 months ago) by mlarkin
Branch: MAIN
Changes since 1.60: +2 -1 lines
Diff to previous 1.60 (colored)

vmd/vmctl: disallow use of block or character devices as disks in VMs.

These don't work today and present the user with a confusing error
message if an attempt is made to use them. This commit detects attempts
to use block or character devices and if detected, presents the user with
a better message.

ok jasper

From Carlos Cardenas, thanks!

Revision 1.60 / (download) - annotate - [select for diffs], Sun Aug 20 21:15:32 2017 UTC (6 years, 9 months ago) by pd
Branch: MAIN
Changes since 1.59: +11 -2 lines
Diff to previous 1.59 (colored)

vmd: Allow only upward migration

This restricts receiving vms from hosts with more cpu features.

Tested on
broadwell -> skylake (works)
skylake -> broadwell (don't work)

ok mlarkin@

Revision 1.59 / (download) - annotate - [select for diffs], Fri Aug 18 07:01:29 2017 UTC (6 years, 9 months ago) by mlarkin
Branch: MAIN
Changes since 1.58: +5 -1 lines
Diff to previous 1.58 (colored)

distinguish errors during vm launch and return different error codes to
vmctl so that it can display proper errors for "disk missing" and "bios
missing" situations. Expands on an earlier idea/diff from jasper.

ok jasper@, pd@, tedu@

Revision 1.58 / (download) - annotate - [select for diffs], Sat Jul 15 05:05:36 2017 UTC (6 years, 10 months ago) by pd
Branch: MAIN
Changes since 1.57: +17 -1 lines
Diff to previous 1.57 (colored)

Add vmctl send and vmctl receive

ok reyk@ and mlarkin@

Revision 1.57 / (download) - annotate - [select for diffs], Sun Jul 9 00:51:40 2017 UTC (6 years, 11 months ago) by pd
Branch: MAIN
Changes since 1.56: +6 -1 lines
Diff to previous 1.56 (colored)

vmd/vmctl: Add ability to pause / unpause vms

With help from Ashwin Agrawal

ok reyk@ mlarkin@

Revision 1.56 / (download) - annotate - [select for diffs], Mon Jun 12 13:41:24 2017 UTC (6 years, 11 months ago) by deraadt
Branch: MAIN
Changes since 1.55: +2 -2 lines
Diff to previous 1.55 (colored)

device path should be PATH_MAX.  Any NAME_MAX without +1 is suspect
to begin with anyways.

Revision 1.55 / (download) - annotate - [select for diffs], Tue May 30 17:56:47 2017 UTC (7 years ago) by tedu
Branch: MAIN
Changes since 1.54: +2 -2 lines
Diff to previous 1.54 (colored)

split vioblk read/write functions into start and finish as prep for
async io operations. ok mlarkin

Revision 1.54 / (download) - annotate - [select for diffs], Thu May 4 19:41:58 2017 UTC (7 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.53: +2 -2 lines
Diff to previous 1.53 (colored)

Report command failure back to vmctl reload, reset, load, log verbose.

OK mlarkin@

Revision 1.53 / (download) - annotate - [select for diffs], Thu May 4 08:26:06 2017 UTC (7 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.52: +7 -2 lines
Diff to previous 1.52 (colored)

Add support for rdomains.

This allows to configure VM interfaces and switches in individual rdomains.

OK mlarkin@

Revision 1.52 / (download) - annotate - [select for diffs], Fri Apr 21 07:03:26 2017 UTC (7 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.51: +22 -6 lines
Diff to previous 1.51 (colored)

Add global configuration option "local prefix" to change prefix for -L.

The default prefix is 100.64.0.0/10 from RFC6598.

Requested by sthen@ chris@
OK mlarkin@

Revision 1.51 / (download) - annotate - [select for diffs], Wed Apr 19 15:38:32 2017 UTC (7 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.50: +45 -2 lines
Diff to previous 1.50 (colored)

Add support for dynamic "NAT" interfaces (-L/local interface).

When a local interface is configured, vmd configures a /31 address on
the tap(4) interface of the host and provides another IP in the same
subnet via DHCP (BOOTP) to the VM.  vmd runs an internal BOOTP server
that replies with IP, gateway, and DNS addresses to the VM.  The
built-in server only ever responds to the VM on the inside and cannot
leak its DHCP responses to the outside.

Thanks to Uwe Werler, Josh Grosse, and some others for testing!

OK deraadt@

Revision 1.50 / (download) - annotate - [select for diffs], Thu Apr 6 18:07:13 2017 UTC (7 years, 2 months ago) by reyk
Branch: MAIN
Changes since 1.49: +4 -3 lines
Diff to previous 1.49 (colored)

Do not expose vmm(4) VM IDs to the user, use vmd(8)'s IDs instead.

Each VM has two IDs: one from the kernel (vmm) and a different one
from userland (vmd).  The vmm ID is not consistent and incremented on
every boot during runtimg of the host system.  The vmd ID remains the
same during the lifetime of a configured VM, even after reboots.
Configured VMs will even get and keep their IDs when the configuration
is loaded.  This is more what users expect.

Pointed out and tested by otto@

OK deraadt@

Revision 1.49 / (download) - annotate - [select for diffs], Sat Mar 25 16:28:25 2017 UTC (7 years, 2 months ago) by reyk
Branch: MAIN
CVS Tags: OPENBSD_6_1_BASE, OPENBSD_6_1
Changes since 1.48: +2 -1 lines
Diff to previous 1.48 (colored)

Boot using BIOS from /etc/firmware/vmm-bios by default.

Instead of using the internal "vmboot", VMs will now be booted using
the external BIOS firmware in /etc/firmware/vmm-bios (which is subject
to a LGPLv3 license).  Direct booting of OpenBSD kernels or
non-default BIOS images is still supported for now using the -b/boot
option that is replacing the -k/kernel option.

As requested by Theo, vmd(8) fails if neither the default BIOS is
found nor a kernel has been specified in the VM configuration.  The
"vmm" BIOS has to be installed using fw_update(1), which will be done
automatically in most cases where the OpenBSD can fetch it after
install/upgrade.

OK mlarkin@

Revision 1.48 / (download) - annotate - [select for diffs], Wed Mar 15 18:06:18 2017 UTC (7 years, 2 months ago) by reyk
Branch: MAIN
Changes since 1.47: +2 -1 lines
Diff to previous 1.47 (colored)

Improve vmmci(4) shutdown and reboot.

This change handles various cases to power off the VM, even if it is
unresponsive, stuck in ddb, or when the shutdown was initiated from
the VM guest side.  Usage of timeout and VM ACKs make sure that the VM
is really turned off at some point.

OK mlarkin@

Revision 1.47 / (download) - annotate - [select for diffs], Thu Mar 2 07:33:37 2017 UTC (7 years, 3 months ago) by reyk
Branch: MAIN
Changes since 1.46: +4 -1 lines
Diff to previous 1.46 (colored)

Add "locked lladdr" option to prevent VMs from spoofing MAC addresses.

This is especially useful when multiple VMs share a switch, the
implementation is independent from the underlying switch or bridge.

no objections mlarkin@

Revision 1.46 / (download) - annotate - [select for diffs], Wed Mar 1 18:00:50 2017 UTC (7 years, 3 months ago) by reyk
Branch: MAIN
Changes since 1.45: +5 -1 lines
Diff to previous 1.45 (colored)

Split vmm.c into two files: vm.c for the VM child, vmm.c for the parent

As discussed with mlarkin@, it makes it easier to maintain the file.

OK mlarkin@

Revision 1.45 / (download) - annotate - [select for diffs], Wed Mar 1 07:43:33 2017 UTC (7 years, 3 months ago) by reyk
Branch: MAIN
Changes since 1.44: +11 -3 lines
Diff to previous 1.44 (colored)

Add "owner" option to set a user/group ownership for pre-configured VMs

This allows matching users to start or stop VMs that they "own" and to
access the console accordingly.

OK mlarkin@

Revision 1.44 / (download) - annotate - [select for diffs], Mon Feb 27 14:37:58 2017 UTC (7 years, 3 months ago) by reyk
Branch: MAIN
Changes since 1.43: +4 -1 lines
Diff to previous 1.43 (colored)

Replace openpty(3) with local function that uses pre-opened /dev/ptm fd

This allows more flexibility for upcoming changes and better pledge.
We also didn't use half of the features of libutil's openpty function.
Additionally, make sure that the ttys are closed correctly on shutdown.

OK gilles@

Revision 1.43 / (download) - annotate - [select for diffs], Fri Jan 13 19:21:16 2017 UTC (7 years, 4 months ago) by edd
Branch: MAIN
Changes since 1.42: +3 -1 lines
Diff to previous 1.42 (colored)

Make it possible to remove VMs from vmd(8)'s internal queue.

The semantics agreed with reyk@ are:

 * ad-hoc created vms, created  with `vmctl start`, are removed once stopped.
 * Stopped VMs defined in a config file are flushed before a `vmctl reload`.

OK reyk@

Revision 1.42 / (download) - annotate - [select for diffs], Fri Jan 13 14:50:56 2017 UTC (7 years, 4 months ago) by reyk
Branch: MAIN
Changes since 1.41: +5 -2 lines
Diff to previous 1.41 (colored)

Add host side of vmmci(4) to vmd(8).

It currently uses the device to request graceful shutdown of a VM on
"vmctl stop myvm" but will be extended for reboot and a other edge cases.

OK mlarkin@

Revision 1.41 / (download) - annotate - [select for diffs], Wed Jan 11 22:38:10 2017 UTC (7 years, 4 months ago) by reyk
Branch: MAIN
Changes since 1.40: +2 -1 lines
Diff to previous 1.40 (colored)

Add imsg communication channel between vmd and invividual VMs.
For now, this is only used to forward "log verbose|brief" requests,
but it will be used for better things later.

OK mlarkin@

Revision 1.40 / (download) - annotate - [select for diffs], Wed Dec 14 21:17:25 2016 UTC (7 years, 5 months ago) by reyk
Branch: MAIN
Changes since 1.39: +6 -1 lines
Diff to previous 1.39 (colored)

Allow to start disabled and pre-configured VMs by name, "vmctl start foo".

With testing from Jon Bernard

OK mlarkin@

Revision 1.39 / (download) - annotate - [select for diffs], Wed Dec 14 06:59:12 2016 UTC (7 years, 5 months ago) by reyk
Branch: MAIN
Changes since 1.38: +2 -1 lines
Diff to previous 1.38 (colored)

If a VM terminates with the result EAGAIN, close all fds except the
pty and re-send it to the vmm monitor process.  With additional
changes in vmm.c, this will allow perform a cold reboot of VM.

With testing and feedback from Jon Bernard
OK mlarkin@

Revision 1.38 / (download) - annotate - [select for diffs], Sat Nov 26 20:03:42 2016 UTC (7 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.37: +15 -3 lines
Diff to previous 1.37 (colored)

Implement basic support for boot.conf(8) on the disk image.

Like the real boot loader, load and parse hd0a:/etc/boot.conf from the
first disk and fall back to /bsd.  Not all boot loader options are
supported, but it at least does set device, set image, and boot -acds
(eg. for booting single-user).

For example, it can now boot install60.fs that includes a boot.conf
with "set image /6.0/amd64/bsd.rd":
	vmctl start install -c -d install60.fs -d OpenBSD.img

This pseudo-bootloader is only needed without BIOS and could
potentially be replaced in the future.

OK mlarkin@

Revision 1.37 / (download) - annotate - [select for diffs], Sat Nov 26 19:49:11 2016 UTC (7 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.36: +2 -1 lines
Diff to previous 1.36 (colored)

If -m/memory is not specified, use 512M by default.

Default value picked with mlarkin - not too small and not too large.

OK mlarkin@

Revision 1.36 / (download) - annotate - [select for diffs], Thu Nov 24 07:58:55 2016 UTC (7 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.35: +7 -1 lines
Diff to previous 1.35 (colored)

Add support for booting the kernel from the disk image.

This make the kernel/-k argument optional and, if not specified, tries
to find the /bsd kernel in the primary hd0a partition of the first
disk image itself.  It doesn't support hd0a:/etc/boot.conf yet, and it
is no BIOS or full boot loader, but it makes booting and handling of
VMs a bit easier - booting an external kernel is still supported.

The UFS file system code ufs.c is directly from libsa which is also
used by the real boot loader.  The code compiles with a few signedness
warning which will be fixed separately.

OK mlarkin@

Revision 1.35 / (download) - annotate - [select for diffs], Tue Nov 22 11:31:38 2016 UTC (7 years, 6 months ago) by edd
Branch: MAIN
Changes since 1.34: +5 -1 lines
Diff to previous 1.34 (colored)

Insert disabled VMs into vmd(8)'s queues and allow vmctl(8) to display them.

Tested by Jon Bernard and reyk@.

OK reyk@, no objections mlarkin@.

Thanks

Revision 1.34 / (download) - annotate - [select for diffs], Fri Nov 4 15:16:44 2016 UTC (7 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.33: +2 -2 lines
Diff to previous 1.33 (colored)

Pass the internal vmid or 0 to vm_register() instead of changing it
once again after setting the next available id.

Suggested by edd@

Revision 1.33 / (download) - annotate - [select for diffs], Fri Nov 4 15:07:26 2016 UTC (7 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.32: +5 -4 lines
Diff to previous 1.32 (colored)

Update the config/register/get VM methods to match the config_set/get
style that is used in other places.  Also keep the vmid from the parent.

OK edd@

Revision 1.32 / (download) - annotate - [select for diffs], Sat Oct 29 14:56:05 2016 UTC (7 years, 7 months ago) by edd
Branch: MAIN
Changes since 1.31: +7 -4 lines
Diff to previous 1.31 (colored)

Separate parsing vms and switches from starting them in vmd(8).

Brings us one step closer to having disabled by default vms is vm.conf(5),
which can be started with vmctl(8).

Input, testing and OK reyk@. Thanks.

Revision 1.31 / (download) - annotate - [select for diffs], Mon Oct 17 16:26:20 2016 UTC (7 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.30: +2 -1 lines
Diff to previous 1.30 (colored)

Add the option to specify an interface group per virtual switch as well;
this group will be added to all VM tap(4) interfaces in the switch.

Tested by martijn@

Revision 1.30 / (download) - annotate - [select for diffs], Sat Oct 15 14:02:11 2016 UTC (7 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.29: +6 -2 lines
Diff to previous 1.29 (colored)

Allow to add an interface to an interface group; with the group keyword.

Requested and tested by martijn@

Revision 1.29 / (download) - annotate - [select for diffs], Wed Oct 12 19:10:03 2016 UTC (7 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.28: +2 -2 lines
Diff to previous 1.28 (colored)

Fix functionality and semantics of vmctl load/reload/reset.

OK rzalamena@

Revision 1.28 / (download) - annotate - [select for diffs], Thu Oct 6 18:48:41 2016 UTC (7 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.27: +2 -1 lines
Diff to previous 1.27 (colored)

Terminate VMs on shutdown of vmd instead of leaving them running as
undead VM processes.

OK mlarkin@

Revision 1.27 / (download) - annotate - [select for diffs], Wed Oct 5 17:30:13 2016 UTC (7 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.26: +45 -6 lines
Diff to previous 1.26 (colored)

Add support for enhanced networking configuration and virtual switches.
See vm.conf(5) for more details.

OK mlarkin@

Revision 1.26 / (download) - annotate - [select for diffs], Tue Oct 4 17:17:30 2016 UTC (7 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.25: +20 -5 lines
Diff to previous 1.25 (colored)

Add a new "priv" process that is responsible for ioctls and restricted
operations that aren't allowed under pledge.  This is a companion to
the "vmd" process that runs as root but with pledge.

With the "priv" process, each new tap(4) interface now gets a
description to indicate the vm, eg. "vm1-if0-myvm".  For network
configuration will be done by vmd/priv later.

OK mlarkin@

Revision 1.25 / (download) - annotate - [select for diffs], Thu Sep 29 22:42:04 2016 UTC (7 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.24: +2 -2 lines
Diff to previous 1.24 (colored)

Implement fork+exec for vmd, using the same framework from httpd etc.

No objections from mlarkin@ sunil@

Revision 1.24 / (download) - annotate - [select for diffs], Fri Sep 2 16:23:40 2016 UTC (7 years, 9 months ago) by stefan
Branch: MAIN
Changes since 1.23: +4 -1 lines
Diff to previous 1.23 (colored)

Move event handling to separate thread

That allows the main thread to handle the termination
of a VM on reboot/shutdown or normal exit. The way it works
is that VCPUs that terminate for some reason communicate this
to a main thread that takes care of shutdown.

ok mlarkin@

Revision 1.23 / (download) - annotate - [select for diffs], Thu Sep 1 14:48:09 2016 UTC (7 years, 9 months ago) by mlarkin
Branch: MAIN
Changes since 1.22: +1 -3 lines
Diff to previous 1.22 (colored)


Add a set of emulated legacy devices (PIT, PIC, RTC)

discussed with stefan and deraadt

Revision 1.22 / (download) - annotate - [select for diffs], Fri Jul 29 16:36:51 2016 UTC (7 years, 10 months ago) by stefan
Branch: MAIN
Changes since 1.21: +6 -2 lines
Diff to previous 1.21 (colored)

Allow starting a VM again after it was terminated

If a VM exits, terminate it and remove it from the list of
available VMs. That allows a VM with name `foo' to be restarted
after it has exited.

This changes structures shared between vmd and vmctl. You need to
rebuild vmctl also.

ok mlarkin@

Revision 1.21 / (download) - annotate - [select for diffs], Sat Jul 9 09:06:22 2016 UTC (7 years, 11 months ago) by stefan
Branch: MAIN
CVS Tags: OPENBSD_6_0_BASE, OPENBSD_6_0
Changes since 1.20: +2 -1 lines
Diff to previous 1.20 (colored)

Prepare vionet to be handled asynchronously to the VCPU thread

This splits the handling of received data into a separate function
that can later be called in parallel to the VCPU thread instead of
handling received packets on VCPU exits only.

It also makes virtq accesses in the rx path safe to run in parallel
to the VCPU thread: the last index into the 'avail' ring the driver
has notified to the host is kept track of. It also makes sure that
the host only writes back to the 'avail' ring instead of modifying
the whole receive virtq.

While there, describe what virtio_vq_info and virtio_io_cfg are used
for, as suggested by mlarkin@

ok mlarkin@

Revision 1.20 / (download) - annotate - [select for diffs], Mon Apr 4 17:13:54 2016 UTC (8 years, 2 months ago) by stefan
Branch: MAIN
Changes since 1.19: +3 -3 lines
Diff to previous 1.19 (colored)

Directly use physical addresses from ELF header for kernel loading.

This allows us to remove the 'do_mask' parameters in read_mem and
write_mem as well as the address mask operaton itself.

ok mlarkin@

Revision 1.19 / (download) - annotate - [select for diffs], Sun Mar 13 13:11:47 2016 UTC (8 years, 2 months ago) by stefan
Branch: MAIN
Changes since 1.18: +3 -3 lines
Diff to previous 1.18 (colored)

Introduce memory ranges to support VMs with >= 4G RAM

Kernel bits:
- When creating a VM, a list of memory ranges has to be specified,
  similar to the BIOS memory map. This is necessary for VMs with
  RAM sizes approaching 4G because we'll need PCI MMIO space in
  the higher parts of the 32 bit address space.

vmctl and vmd bits:
- Construct appropriate memory ranges to create a VM with a given
  RAM size
- Construct a corresponding BIOS memory map from the memory ranges
  and update the boot params page accordingly.
- Make sure that all variables that represent guest physical addresses
  match the address width of the target CPU instead of using uint32_t.
- Fix some integer promotion glitches that actually restricted VM
  RAM size to 2G.

This changes the VM create ioctl interface, so update your kernel,
vmd, and vmctl.

ok mlarkin@

Revision 1.18 / (download) - annotate - [select for diffs], Sat Jan 16 08:55:40 2016 UTC (8 years, 4 months ago) by stefan
Branch: MAIN
CVS Tags: OPENBSD_5_9_BASE, OPENBSD_5_9
Changes since 1.17: +3 -3 lines
Diff to previous 1.17 (colored)

vmd(8) sometimes attempts page-crossing data copies between the host
and guest. The readpage/writepage ioctls of vmm(4) do not support this
and they return EINVAL on such attempts since recently.

Avoid page-crossing guest memory accesses by changing read_page() and
write_page() into read_mem() and write_mem() that can copy arbitrary
lengths of data between host<->guest without page-crossing accesses.

This also allows us to remove page-wise copy-loops in a few places.

ok mlarkin@

Revision 1.17 / (download) - annotate - [select for diffs], Sat Jan 2 15:05:21 2016 UTC (8 years, 5 months ago) by benno
Branch: MAIN
Changes since 1.16: +2 -2 lines
Diff to previous 1.16 (colored)

fix typo in comment, found by Michal Mazurek, thanks

Revision 1.16 / (download) - annotate - [select for diffs], Fri Dec 11 10:16:53 2015 UTC (8 years, 5 months ago) by reyk
Branch: MAIN
Changes since 1.15: +6 -1 lines
Diff to previous 1.15 (colored)

The vmctl "id" argument can now be a number of or a vm name, eg.
vmctl stop 3
vmctl stop "openbsd.vm"

Revision 1.15 / (download) - annotate - [select for diffs], Sun Dec 6 21:02:51 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.14: +2 -1 lines
Diff to previous 1.14 (colored)

Prevent running a VM with the same name multiple times - multiple
instances of the same configuration will be handled in a different way
later.  It is also not a good idea to use the same writeable disk
with multiple VMs at the same time.

As discussed with mlarkin@

Revision 1.14 / (download) - annotate - [select for diffs], Sun Dec 6 19:37:15 2015 UTC (8 years, 6 months ago) by mlarkin
Branch: MAIN
Changes since 1.13: +2 -6 lines
Diff to previous 1.13 (colored)


remove some now-unused enums

ok reyk@

Revision 1.13 / (download) - annotate - [select for diffs], Sun Dec 6 10:30:41 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.12: +13 -13 lines
Diff to previous 1.12 (colored)

spacing

Revision 1.12 / (download) - annotate - [select for diffs], Sun Dec 6 02:26:14 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.11: +7 -2 lines
Diff to previous 1.11 (colored)

Print the TTY in the vmctl status output.

Revision 1.11 / (download) - annotate - [select for diffs], Sun Dec 6 01:58:21 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.10: +6 -4 lines
Diff to previous 1.10 (colored)

When a new vm is created with VMM_IOC_CREATE, the kernel assigns a
unique id to it.  This happens in the vm child process and has to be
communicated to the parent processes to track the vm.  Knowing the vm
id in the parent and vmm processes also allows to remove vm from the
daemons list on terminate requests later.

Revision 1.10 / (download) - annotate - [select for diffs], Thu Dec 3 23:32:32 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.9: +6 -2 lines
Diff to previous 1.9 (colored)

Re-add the "load" and "reload" commands to vmctl: Instead of parsing
the configuration in vmctl directly, it now sends a (re)load request
to vmd.  The reload also resets the existing configuration status -
this doesn't do much difference yet but a future change will compare
if a specified VM is already running.  "load" will allow to add
configuration, while "reload" resets the state before loading.

Revision 1.9 / (download) - annotate - [select for diffs], Thu Dec 3 16:11:32 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.8: +9 -2 lines
Diff to previous 1.8 (colored)

Add support for an optional vm.conf(5) file in vmd.  This will replace
vmm.conf(5) in vmmctl.  For a short time, both vmd and vmmctl will
support a configuration file, but vmmctl will be changed to send
"load" requests to vmd instead of loading and parsing the file
directly.

Revision 1.8 / (download) - annotate - [select for diffs], Thu Dec 3 13:27:14 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.7: +3 -2 lines
Diff to previous 1.7 (colored)

prepare config_getvm() for parse.y

Revision 1.7 / (download) - annotate - [select for diffs], Thu Dec 3 08:42:11 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.6: +7 -7 lines
Diff to previous 1.6 (colored)

spacing

Revision 1.6 / (download) - annotate - [select for diffs], Wed Dec 2 23:33:43 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.5: +8 -2 lines
Diff to previous 1.5 (colored)

send the tty name to vmmctl and print it as a result.

Revision 1.5 / (download) - annotate - [select for diffs], Wed Dec 2 22:19:11 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.4: +44 -11 lines
Diff to previous 1.4 (colored)

Split the fully privileged parent into two processes "parent" and
"vmm" with reduced privileges:
- the "parent" opens fds (disks, ifs, etc.) but runs as root but pledged as
  "stdio rpath wpath proc tty sendfd".
- the "vmm" process handles the creation and supervision of vm processes,
  and the primary communication with the vmm(4) subsystem.  It runs as _vmd
  in the chroot but does not use pledge, as the vmm ioctls are not allowed
  by any pledge model yet.
With this change, vmd starts to track the configuration state of VMs
in vmd and will allow other things later (like terminating a vm by
name, moving the configuration parser to vmd, ...).  More incremental
changes will follow.

Revision 1.4 / (download) - annotate - [select for diffs], Wed Dec 2 09:14:25 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.3: +24 -26 lines
Diff to previous 1.3 (colored)

Start tweaking vmd's privsep and daemon model by splitting the main
process into multiple parts and adopting the "proc.c"-style from other
daemons.  This allows to further reduce the privileges, to give better
pledge(2), and to add some upcoming changes.

"please do" mlarkin@, deraadt@

Revision 1.3 / (download) - annotate - [select for diffs], Mon Nov 23 13:04:49 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.2: +25 -2 lines
Diff to previous 1.2 (colored)

Add support for logging to stderr or syslog, and to run vmd in
foreground with -d.

OK mlarkin@ jung@

Revision 1.2 / (download) - annotate - [select for diffs], Sun Nov 22 21:51:32 2015 UTC (8 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.1: +1 -0 lines
Diff to previous 1.1 (colored)

Add $ Ids

Revision 1.1 / (download) - annotate - [select for diffs], Sun Nov 22 20:20:32 2015 UTC (8 years, 6 months ago) by mlarkin
Branch: MAIN


vmd(8) - virtual machine daemon.

There is still a lot to be done, and fixed, in these userland components
but I have received enough "it works, commit it" emails that it's time
to finish those things in tree.

discussed with many, tested by many.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.