OpenBSD CVS

CVS log for src/usr.sbin/vmd/vioqcow2.c


[BACK] Up to [local] / src / usr.sbin / vmd

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.24 / (download) - annotate - [select for diffs], Thu Sep 14 15:25:43 2023 UTC (8 months, 3 weeks ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5, OPENBSD_7_4_BASE, OPENBSD_7_4, HEAD
Changes since 1.23: +42 -2 lines
Diff to previous 1.23 (colored)

vmd(8)/vioblk: use zero-copy approach & vectored io.

The original version of the virtio block device dynamically allocated
buffers to hold intermediate data when reading or writing to the
underlying disk fd(s). Since vioblk drivers may chain multiple
segments together, this leads to overly complex logic and on
read(2)/write(2) call per data segment.

Additionally, the virtio block logic in vmd didn't handle segments
that weren't block aligned (e.g. 512 bytes). If a guest provided
unaligned segments, garbage will be read or written.

Since virtio descriptors mimic iovec structures, this changes vmd's
device emulation to use that model. (This is how other hypervisors
emulate virtio devices.) This allows for zero-copy semantics using
iovec's, reducing memcpy and multiple read/write syscalls per io
transaction.

Testing by phessler@ and mlarkin@. OK mlarkin@.

Revision 1.23 / (download) - annotate - [select for diffs], Sun May 28 05:28:50 2023 UTC (12 months, 1 week ago) by asou
Branch: MAIN
Changes since 1.22: +2 -2 lines
Diff to previous 1.22 (colored)

Used number of bytes instead of number of 512 byte sectors.

ok dv@

Revision 1.22 / (download) - annotate - [select for diffs], Fri Apr 28 18:52:22 2023 UTC (13 months, 1 week ago) by dv
Branch: MAIN
Changes since 1.21: +1 -2 lines
Diff to previous 1.21 (colored)

Remove unneeded header includes in vmd.

No functional change. virtio block/networking emulation do not need
to know about vmm or any kernel types.

Revision 1.21 / (download) - annotate - [select for diffs], Thu Apr 27 22:47:27 2023 UTC (13 months, 1 week ago) by dv
Branch: MAIN
Changes since 1.20: +4 -5 lines
Diff to previous 1.20 (colored)

vmd(8): introduce multi-process model for virtio devices.

Isolate virtio network and block device emulation in dedicated
processes, forked and exec'd from the vm process. This allows for
tightening pledge promises to just "stdio".

Communication between the vcpu's and these devices now occurs via
imsg channels, which adds the benefit of not always blocking the
vcpu thread while emulating the device.

With this commit, it's possible that vmd is the first open source
hypervisor that *defaults* to a multi-process device emulation
model without requiring any additional configuration from the
operator.

Testing help from phessler@ and Mischa Peters.

ok mlarkin@

Revision 1.20 / (download) - annotate - [select for diffs], Fri May 20 22:06:47 2022 UTC (2 years ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_7_3_BASE, OPENBSD_7_3, OPENBSD_7_2_BASE, OPENBSD_7_2
Changes since 1.19: +3 -3 lines
Diff to previous 1.19 (colored)

use correct logging or error calls

ftruncate sets errno, so use fatal() and not fatalx().

Checking for under-reads of read(2) isn't going to set errno so
use errx().

ok abieber@

Revision 1.19 / (download) - annotate - [select for diffs], Fri May 13 16:46:34 2022 UTC (2 years ago) by dv
Branch: MAIN
Changes since 1.18: +3 -3 lines
Diff to previous 1.18 (colored)

fix typos in comments

No functional change. Pointed out by Martin Vahlensieck.

Revision 1.18 / (download) - annotate - [select for diffs], Wed May 4 23:17:25 2022 UTC (2 years, 1 month ago) by dv
Branch: MAIN
Changes since 1.17: +3 -5 lines
Diff to previous 1.17 (colored)

vmctl(8)/vmd(8): convert disk sizes from MB to bytes

Continue converting other parts to storing data in bytes instead
of MB. In this case, the logic for disk sizes was being scaled.

This fixes issues reported by Martin Vahlensieck where vmctl could
no longer create disks larger than 7 MiB after previous commits to
change storing memory sizes as bytes.

While this keeps the vm memory limit check in vmctl's size parser,
it skips the limit check for disks. The error messages adjust
accordingly and this removes the double error message logging.

Update comments and function types accordingly.

ok marlkin@

Revision 1.17 / (download) - annotate - [select for diffs], Tue Jan 4 15:21:40 2022 UTC (2 years, 5 months ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_1_BASE, OPENBSD_7_1
Changes since 1.16: +1 -3 lines
Diff to previous 1.16 (colored)

Another -Wunused-but-set-variable fix.
Based on input from dv@

Revision 1.16 / (download) - annotate - [select for diffs], Wed Jun 16 16:55:02 2021 UTC (2 years, 11 months ago) by dv
Branch: MAIN
CVS Tags: OPENBSD_7_0_BASE, OPENBSD_7_0
Changes since 1.15: +7 -9 lines
Diff to previous 1.15 (colored)

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@

Revision 1.15 / (download) - annotate - [select for diffs], Thu Apr 29 23:27:10 2021 UTC (3 years, 1 month ago) by dv
Branch: MAIN
Changes since 1.14: +3 -23 lines
Diff to previous 1.14 (colored)

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@

Revision 1.14 / (download) - annotate - [select for diffs], Mon Oct 19 19:06:49 2020 UTC (3 years, 7 months ago) by naddy
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE, OPENBSD_6_9
Changes since 1.13: +8 -2 lines
Diff to previous 1.13 (colored)

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@

Revision 1.13 / (download) - annotate - [select for diffs], Thu Jan 10 19:21:02 2019 UTC (5 years, 4 months ago) by deraadt
Branch: MAIN
CVS Tags: OPENBSD_6_8_BASE, OPENBSD_6_8, OPENBSD_6_7_BASE, OPENBSD_6_7, OPENBSD_6_6_BASE, OPENBSD_6_6, OPENBSD_6_5_BASE, OPENBSD_6_5
Changes since 1.12: +3 -2 lines
Diff to previous 1.12 (colored)

an alloca() snuck into the tree.  We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region.  So use malloc() instead.

Revision 1.8.2.1 / (download) - annotate - [select for diffs], Thu Nov 29 17:25:31 2018 UTC (5 years, 6 months ago) by bluhm
Branch: OPENBSD_6_4
Changes since 1.8: +5 -5 lines
Diff to previous 1.8 (colored) next main 1.9 (colored)

Fix qcow2 disk images for data sizes greater than 4 gigs.
We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:
        disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);
This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.
from ori@

OpenBSD 6.4 errata 008

Revision 1.12 / (download) - annotate - [select for diffs], Mon Nov 26 10:39:30 2018 UTC (5 years, 6 months ago) by reyk
Branch: MAIN
Changes since 1.11: +143 -2 lines
Diff to previous 1.11 (colored)

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@

Revision 1.11 / (download) - annotate - [select for diffs], Sat Nov 24 04:51:55 2018 UTC (5 years, 6 months ago) by ori
Branch: MAIN
Changes since 1.10: +70 -116 lines
Diff to previous 1.10 (colored)

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk

Revision 1.10 / (download) - annotate - [select for diffs], Wed Oct 24 05:19:03 2018 UTC (5 years, 7 months ago) by ori
Branch: MAIN
Changes since 1.9: +5 -5 lines
Diff to previous 1.9 (colored)

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

	disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.

Revision 1.9 / (download) - annotate - [select for diffs], Fri Oct 19 10:12:39 2018 UTC (5 years, 7 months ago) by reyk
Branch: MAIN
Changes since 1.8: +41 -10 lines
Diff to previous 1.8 (colored)

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format.  This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in.  The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@

Revision 1.8 / (download) - annotate - [select for diffs], Mon Oct 8 16:32:01 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
CVS Tags: OPENBSD_6_4_BASE
Branch point for: OPENBSD_6_4
Changes since 1.7: +55 -34 lines
Diff to previous 1.7 (colored)

Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image.  Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl.  To use it:

	vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@

Revision 1.7 / (download) - annotate - [select for diffs], Mon Oct 1 17:34:56 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.6: +1 -2 lines
Diff to previous 1.6 (colored)

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@

Revision 1.6 / (download) - annotate - [select for diffs], Mon Oct 1 09:31:15 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.5: +3 -2 lines
Diff to previous 1.5 (colored)

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)

Revision 1.5 / (download) - annotate - [select for diffs], Fri Sep 28 12:35:32 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.4: +7 -6 lines
Diff to previous 1.4 (colored)

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@

Revision 1.4 / (download) - annotate - [select for diffs], Fri Sep 28 08:29:06 2018 UTC (5 years, 8 months ago) by reyk
Branch: MAIN
Changes since 1.3: +8 -5 lines
Diff to previous 1.3 (colored)

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.

Revision 1.3 / (download) - annotate - [select for diffs], Wed Sep 19 04:29:21 2018 UTC (5 years, 8 months ago) by ccardenas
Branch: MAIN
Changes since 1.2: +41 -22 lines
Diff to previous 1.2 (colored)

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@

Revision 1.2 / (download) - annotate - [select for diffs], Tue Sep 11 04:06:32 2018 UTC (5 years, 8 months ago) by ccardenas
Branch: MAIN
Changes since 1.1: +14 -13 lines
Diff to previous 1.1 (colored)

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.

Revision 1.1 / (download) - annotate - [select for diffs], Sun Sep 9 04:09:32 2018 UTC (5 years, 9 months ago) by ccardenas
Branch: MAIN

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf.  The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
    disable
    memory 2G
    disk "/home/user/vmm/current.qc2" format "qcow2"
    interface { switch "external" }
}

or

vm "current" {
    disable
    memory 2G
    disk "/home/user/vmm/current.raw" format "raw"
    interface { switch "external" }
}

is equivlanet to

vm "current" {
    disable
    memory 2G
    disk "/home/user/vmm/current.raw"
    interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.