OpenBSD CVS

CVS log for src/usr.sbin/rpki-client/rsync.c


[BACK] Up to [local] / src / usr.sbin / rpki-client

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.50 / (download) - annotate - [select for diffs], Fri Mar 22 03:38:12 2024 UTC (2 months, 1 week ago) by job
Branch: MAIN
CVS Tags: HEAD
Changes since 1.49: +2 -2 lines
Diff to previous 1.49 (colored)

Replace protocol literal strings and strlen() calls with defined constants

OK tb@ claudio@

Revision 1.49 / (download) - annotate - [select for diffs], Mon Feb 26 20:37:27 2024 UTC (3 months ago) by job
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5
Changes since 1.48: +2 -1 lines
Diff to previous 1.48 (colored)

Also download SPLs via rsync

OK tb@

Revision 1.48 / (download) - annotate - [select for diffs], Fri Nov 24 14:05:47 2023 UTC (6 months, 1 week ago) by job
Branch: MAIN
Changes since 1.47: +2 -1 lines
Diff to previous 1.47 (colored)

Require files to be of a minimum size in the RRDP & RSYNC transports

Picked 100 bytes as a minimum, to accommodate future signature schemes
(such as the smaller P-256) and small files like empty CRLs.

With and OK claudio@ tb@

Revision 1.47 / (download) - annotate - [select for diffs], Thu Nov 23 13:01:15 2023 UTC (6 months, 1 week ago) by job
Branch: MAIN
Changes since 1.46: +2 -2 lines
Diff to previous 1.46 (colored)

Don't set directory modtimes to match the source

When syncing against remote repositories, the modtimes of the
remote directories is irrelevant. In the RRDP protocol the directory
modtimes aren't signalled either. This should save some IOPS.

OK tb@

Revision 1.46 / (download) - annotate - [select for diffs], Wed Dec 28 21:30:18 2022 UTC (17 months ago) by jmc
Branch: MAIN
CVS Tags: OPENBSD_7_4_BASE, OPENBSD_7_4, OPENBSD_7_3_BASE, OPENBSD_7_3
Changes since 1.45: +2 -2 lines
Diff to previous 1.45 (colored)

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Revision 1.45 / (download) - annotate - [select for diffs], Tue Nov 29 20:26:22 2022 UTC (18 months ago) by job
Branch: MAIN
Changes since 1.44: +1 -2 lines
Diff to previous 1.44 (colored)

Only include assert.h if we call assert()

OK tb@

Revision 1.44 / (download) - annotate - [select for diffs], Wed Nov 2 12:43:02 2022 UTC (18 months, 4 weeks ago) by job
Branch: MAIN
Changes since 1.43: +2 -1 lines
Diff to previous 1.43 (colored)

Add support for draft-ietf-sidrops-signed-tal-12

Add support validation of Signed Objects containing Trust Anchor Keys
(TAKs - aka 'Signed TALs'). Signed TALs provide a mechanism for RIRs
to distribute and sign the next Trust Anchor with the current Trust
Anchor. This might be an improvement over visiting RIR websites and
copy+pasting TAL data by hand.

OK tb@

Revision 1.43 / (download) - annotate - [select for diffs], Fri Sep 2 17:39:51 2022 UTC (20 months, 4 weeks ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_2_BASE, OPENBSD_7_2
Changes since 1.42: +1 -2 lines
Diff to previous 1.42 (colored)

extra newline

Revision 1.42 / (download) - annotate - [select for diffs], Fri Sep 2 13:04:16 2022 UTC (20 months, 4 weeks ago) by claudio
Branch: MAIN
Changes since 1.41: +128 -87 lines
Diff to previous 1.41 (colored)

Rework the rsync proc code. Use a proper queue of requests and enforce
the limit on that queue instead of stopping to read new messages.
This is needed to implement an abort request.
"There is not enough RB_TREE in this diff" tb@

Revision 1.41 / (download) - annotate - [select for diffs], Tue Aug 9 09:02:26 2022 UTC (21 months, 3 weeks ago) by claudio
Branch: MAIN
Changes since 1.40: +2 -2 lines
Diff to previous 1.40 (colored)

Make the http code respect MAX_CONN_TIMEOUT and fail connects once they
hit this timeout. This is in line with the rsync code.
OK tb@ job@

Revision 1.40 / (download) - annotate - [select for diffs], Mon Aug 8 15:22:31 2022 UTC (21 months, 3 weeks ago) by job
Branch: MAIN
Changes since 1.39: +2 -2 lines
Diff to previous 1.39 (colored)

Unify the maximum idle IO timeout for RSYNC & HTTPS

OK claudio@

Revision 1.39 / (download) - annotate - [select for diffs], Mon Aug 8 14:10:10 2022 UTC (21 months, 3 weeks ago) by job
Branch: MAIN
Changes since 1.38: +2 -1 lines
Diff to previous 1.38 (colored)

Set rsync connection timeout to 15 seconds.

OK sthen@

Revision 1.38 / (download) - annotate - [select for diffs], Tue May 24 09:20:49 2022 UTC (2 years ago) by claudio
Branch: MAIN
Changes since 1.37: +9 -9 lines
Diff to previous 1.37 (colored)

Introduce MAX_HTTP_REQUESTS and MAX_RSYNC_REQUESTS.
These just replace MAX_CONNECTIONS and MAX_RSYNC_PROCESSES to be more unified.
OK tb@

Revision 1.37 / (download) - annotate - [select for diffs], Wed Apr 20 15:38:24 2022 UTC (2 years, 1 month ago) by deraadt
Branch: MAIN
Changes since 1.36: +2 -2 lines
Diff to previous 1.36 (colored)

more whitespace cleanups

Revision 1.36 / (download) - annotate - [select for diffs], Wed Apr 20 15:31:48 2022 UTC (2 years, 1 month ago) by tb
Branch: MAIN
Changes since 1.35: +2 -2 lines
Diff to previous 1.35 (colored)

Fix various annoying whitespace errors.

Revision 1.35 / (download) - annotate - [select for diffs], Mon Apr 11 18:59:23 2022 UTC (2 years, 1 month ago) by claudio
Branch: MAIN
Changes since 1.34: +4 -2 lines
Diff to previous 1.34 (colored)

Refactor on how the subprocesses are started.

Move the unveil and pledges to the actuall subprocesses and put all the
common code to start these into process_start(). Reduces the lenght of
main() a fair bit.
OK tb@

Revision 1.34 / (download) - annotate - [select for diffs], Mon Apr 4 13:47:58 2022 UTC (2 years, 1 month ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_1_BASE, OPENBSD_7_1
Changes since 1.33: +11 -19 lines
Diff to previous 1.33 (colored)

Change from a dynamic allocation for the process list to a static
array because the maximum size. The number of processes was already
limited by stopping to poll for new commands but this enforces it
even more. With this remove the FIXME comment since it is no longer
true.
OK tb@

Revision 1.33 / (download) - annotate - [select for diffs], Thu Mar 31 12:00:00 2022 UTC (2 years, 2 months ago) by job
Branch: MAIN
Changes since 1.32: +2 -1 lines
Diff to previous 1.32 (colored)

Sync & permit ASPA objects to appear on Manifests

OK tb@ claudio@

Revision 1.32 / (download) - annotate - [select for diffs], Thu Jan 13 11:50:29 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.31: +37 -3 lines
Diff to previous 1.31 (colored)

Implement but don't use code to use rsync's --compare-dest feature.
One gotcha is that the path passed to --compare-dest needs to be relative
to the dst directory. rsync_fixup_dest() will prepend the necessary ../
for that by counting number of '/' in dst.
OK tb@

Revision 1.31 / (download) - annotate - [select for diffs], Wed Dec 22 09:35:14 2021 UTC (2 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.30: +7 -6 lines
Diff to previous 1.30 (colored)

Replace two questionable size_t types. For the repo id use a unsigned int
and for the roa maxlength use unsigned char (like the prefixlen in struct
ip_addr).
With input and OK job@

Revision 1.23.2.1 / (download) - annotate - [select for diffs], Tue Nov 9 13:41:19 2021 UTC (2 years, 6 months ago) by benno
Branch: OPENBSD_6_9
Changes since 1.23: +47 -28 lines
Diff to previous 1.23 (colored) next main 1.24 (colored)

rpki-client(8) should handle CA misbehaviours as soft-errors.

This is a merge of usr.sbin/rpki-client and usr.bin/rsync from current
and includes all commits in rpki-client 7.5 up to Tue Nov 9 11:03:40
2021 and to openrsync up to Wed Nov 3 14:42:13 2021, including:

* Make rpki-client more resilient regarding untrusted input:
  - fail repository synchronisation after 15min runtime
  - limit the number of publication points per TAL
  - don't allow DOCTYPE definitions in RRDP XML files
  - fix detection of HTTP redirect loops.
* limit the number of concurrent rsync processes.
* fix CRLF in tal files.

This is patches/6.9/common/021_rpki.patch.sig

Revision 1.25.4.1 / (download) - annotate - [select for diffs], Tue Nov 9 13:40:32 2021 UTC (2 years, 6 months ago) by benno
Branch: OPENBSD_7_0
Changes since 1.25: +26 -20 lines
Diff to previous 1.25 (colored) next main 1.26 (colored)

rpki-client(8) should handle CA misbehaviours as soft-errors.

This is a merge of usr.sbin/rpki-client and usr.bin/rsync from current
and includes all commits in rpki-client 7.5 up to Tue Nov 9 11:03:40
2021 and to openrsync up to Wed Nov 3 14:42:13 2021, including:

* Make rpki-client more resilient regarding untrusted input:
  - fail repository synchronisation after 15min runtime
  - limit the number of publication points per TAL
  - don't allow DOCTYPE definitions in RRDP XML files
  - fix detection of HTTP redirect loops.
* limit the number of concurrent rsync processes.
* fix CRLF in tal files.

This is patches/7.0/common/004_rpki.patch.sig

Revision 1.30 / (download) - annotate - [select for diffs], Wed Nov 3 14:59:37 2021 UTC (2 years, 6 months ago) by claudio
Branch: MAIN
Changes since 1.29: +7 -3 lines
Diff to previous 1.29 (colored)

Limit the number of rsync processes being spawned by stopping to accept
new requests when over the limit. Use a generous limit of 16.
OK deraadt@

Revision 1.29 / (download) - annotate - [select for diffs], Thu Oct 28 13:50:29 2021 UTC (2 years, 7 months ago) by job
Branch: MAIN
Changes since 1.28: +5 -1 lines
Diff to previous 1.28 (colored)

Don't fetch files larger than 2MB

OK claudio@

Revision 1.28 / (download) - annotate - [select for diffs], Sat Oct 23 20:01:16 2021 UTC (2 years, 7 months ago) by claudio
Branch: MAIN
Changes since 1.27: +3 -3 lines
Diff to previous 1.27 (colored)

Rename io_buf_new to io_new_buffer and io_buf_close to io_close_buffer.
With this the write functions are all of the form io_xyz_buffer.
Remove some prototypes of functions I forgot to remove in previous commit.
OK benno@

Revision 1.27 / (download) - annotate - [select for diffs], Sat Oct 23 16:06:04 2021 UTC (2 years, 7 months ago) by claudio
Branch: MAIN
Changes since 1.26: +13 -7 lines
Diff to previous 1.26 (colored)

Finnally move away from blocking reads in rpki-client. The code was a
mish mash of poll, non-blocking writes and blocking reads. Using the
introduced ibuf size header in io_buf_new()/io_buf_close() the read
side can be changed to pull in a full ibuf and only start the un-marshal
once all data has been read.
OK benno@

Revision 1.26 / (download) - annotate - [select for diffs], Fri Oct 22 11:13:06 2021 UTC (2 years, 7 months ago) by claudio
Branch: MAIN
Changes since 1.25: +10 -18 lines
Diff to previous 1.25 (colored)

First step of cleanup in the io land. Introduce io_buf_new() and
io_buf_close(). These function will inject a size of the the buffer
at the beginning of the buffer and will allow the read size to be
switched to proper async IO.
OK benno@

Revision 1.25 / (download) - annotate - [select for diffs], Wed Sep 1 12:26:26 2021 UTC (2 years, 9 months ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_0_BASE
Branch point for: OPENBSD_7_0
Changes since 1.24: +9 -3 lines
Diff to previous 1.24 (colored)

RPKI only cares about *.{cer,crl,gbr,mft,roa} files. Use rsync --include
and --exclude to only fetch those files from the CA repositories.
OK job@

Revision 1.24 / (download) - annotate - [select for diffs], Mon Apr 19 17:04:35 2021 UTC (3 years, 1 month ago) by deraadt
Branch: MAIN
Changes since 1.23: +17 -10 lines
Diff to previous 1.23 (colored)

code review results in KNF, and moving local variables into lowest scope
ok claudio

Revision 1.23 / (download) - annotate - [select for diffs], Thu Apr 1 11:04:30 2021 UTC (3 years, 2 months ago) by job
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE
Branch point for: OPENBSD_6_9
Changes since 1.22: +2 -1 lines
Diff to previous 1.22 (colored)

Abate superfluous lines from remote servers

OK claudio@

Revision 1.22 / (download) - annotate - [select for diffs], Thu Mar 18 15:47:10 2021 UTC (3 years, 2 months ago) by claudio
Branch: MAIN
Changes since 1.21: +8 -3 lines
Diff to previous 1.21 (colored)

Fail in rsync_base_uri() if the strdup calls fail. Do not bubble this
error upwards since a NULL return represents a bad-URI.
Diff originally from tb@

Revision 1.21 / (download) - annotate - [select for diffs], Thu Mar 4 14:24:17 2021 UTC (3 years, 2 months ago) by claudio
Branch: MAIN
Changes since 1.20: +3 -3 lines
Diff to previous 1.20 (colored)

Use the same way to error out in out of memory situation.
Just use 'err(1, NULL);' there is no need to include the type of function
that failed since it is still impossible to locate the right call.
Just use a debugger in that case.
OK tb@ deraadt@

Revision 1.20 / (download) - annotate - [select for diffs], Thu Mar 4 14:02:34 2021 UTC (3 years, 2 months ago) by claudio
Branch: MAIN
Changes since 1.19: +1 -2 lines
Diff to previous 1.19 (colored)

Remove NOTREACHED marker, it should be obvious when the code is:
	exit(rc);
	/* NOTREACHED */

Revision 1.19 / (download) - annotate - [select for diffs], Tue Feb 23 14:25:29 2021 UTC (3 years, 3 months ago) by claudio
Branch: MAIN
Changes since 1.18: +1 -8 lines
Diff to previous 1.18 (colored)

Adjust pledge() and unveil() calls for proc_rsync() a bit. Since the
mkdir was moved to the main process there is no need for access to .
in the rsync process.
OK job@ deraadt@

Revision 1.18 / (download) - annotate - [select for diffs], Fri Feb 19 08:14:49 2021 UTC (3 years, 3 months ago) by claudio
Branch: MAIN
Changes since 1.17: +2 -11 lines
Diff to previous 1.17 (colored)

Move the mkpath() call from the rsync path to the main process. This allows
to drop cpath from the rsync proc pledge (down to "stdio proc exec").
This will also make work easier with the upcoming http fetcher.
OK tb@

Revision 1.17 / (download) - annotate - [select for diffs], Tue Feb 16 08:52:00 2021 UTC (3 years, 3 months ago) by claudio
Branch: MAIN
Changes since 1.16: +21 -81 lines
Diff to previous 1.16 (colored)

Rework the repository handling. Split the handling of trust anchors into
ta_lookup() while regular repositories (to fetch .mft files) are handled
by repo_lookup(). Also the cache directory layout changed; moving the
trust anchors to ./ta/{tal basename}/ the other repositories end up in
./rsync/
OK tb@

Revision 1.16 / (download) - annotate - [select for diffs], Wed Feb 3 09:29:22 2021 UTC (3 years, 3 months ago) by claudio
Branch: MAIN
Changes since 1.15: +2 -2 lines
Diff to previous 1.15 (colored)

Use mkpath() == -1 to check for failure. No functional change.

Revision 1.15 / (download) - annotate - [select for diffs], Tue Feb 2 18:35:38 2021 UTC (3 years, 3 months ago) by claudio
Branch: MAIN
Changes since 1.14: +10 -18 lines
Diff to previous 1.14 (colored)

Adjust the repository handling a bit. Instead of storing host/module pairs
store repo (rsync URI) and local (the local path to the repository).
Simplifies the the rsync handling a fair bit.
OK deraadt@

Revision 1.14 / (download) - annotate - [select for diffs], Tue Jan 12 09:22:11 2021 UTC (3 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.13: +1 -3 lines
Diff to previous 1.13 (colored)

rsync is using buffered output now, so remove this FIXME comment

Revision 1.13 / (download) - annotate - [select for diffs], Fri Jan 8 08:09:07 2021 UTC (3 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.12: +33 -11 lines
Diff to previous 1.12 (colored)

Start using the ibuf API (ibuf_dynamic, ibuf_add, ibuf_close) for writing
data between processes. This completely decouples the write side.
rpki-client can't really use the imsg framework but it can use the ibuf
bits wich imsg is built on.
OK benno@ job@

Revision 1.12 / (download) - annotate - [select for diffs], Mon Dec 21 11:35:55 2020 UTC (3 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.11: +3 -1 lines
Diff to previous 1.11 (colored)

Now that a NULL string is marshalled as NULL again we can drop some
extra has_xyz integers to indicate if the following buffer is present
or not. At the same time sprinkle some asserts for strings which must
be not NULL.
OK tb@

Revision 1.11 / (download) - annotate - [select for diffs], Wed Dec 2 15:31:15 2020 UTC (3 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.10: +11 -5 lines
Diff to previous 1.10 (colored)

Remove the last users of io_*_write functions that call io_simple_write()
internally. This is a step in direction of more async aware io in rpki-client.
Now everything uses a buffer which is then written.
OK tb@

Revision 1.10 / (download) - annotate - [select for diffs], Tue Nov 24 17:54:57 2020 UTC (3 years, 6 months ago) by job
Branch: MAIN
Changes since 1.9: +3 -1 lines
Diff to previous 1.9 (colored)

Kill connection if rsync server stalls

OK deraadt@

Revision 1.9 / (download) - annotate - [select for diffs], Sat Sep 12 15:46:48 2020 UTC (3 years, 8 months ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_6_8_BASE, OPENBSD_6_8
Changes since 1.8: +1 -3 lines
Diff to previous 1.8 (colored)

Include openssl/x509.h in extern.h since it uses a few of the typedefs from
there in structs and prototypes. Remove the openssl/ssl.h and other strange
openssl includes in the .c files that don't use openssl specific functions.
OK beck@ and tb@

Revision 1.8 / (download) - annotate - [select for diffs], Sat Sep 12 10:02:01 2020 UTC (3 years, 8 months ago) by claudio
Branch: MAIN
Changes since 1.7: +235 -1 lines
Diff to previous 1.7 (colored)

Move the proc_rsync and with that the rsync processing into rsync.c
main.c is too crowded
OK deraadt@

Revision 1.7 / (download) - annotate - [select for diffs], Thu Oct 31 08:36:43 2019 UTC (4 years, 7 months ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_6_7_BASE, OPENBSD_6_7
Changes since 1.6: +1 -3 lines
Diff to previous 1.6 (colored)

Handle the TAL files in the master process and pass them as buffer to the
parser process. This way the parser never needs to read outside of the
cache directory which makes the unveil simpler. Additionally rsync_uri_parse
no longer needs to know about .tal files so there is now no chance to sneak
in a .tal file later on.
OK deraadt@

Revision 1.6 / (download) - annotate - [select for diffs], Wed Jun 19 16:30:37 2019 UTC (4 years, 11 months ago) by deraadt
Branch: MAIN
CVS Tags: OPENBSD_6_6_BASE, OPENBSD_6_6
Changes since 1.5: +1 -1 lines
Diff to previous 1.5 (colored)

use $OpenBSD$ headers

Revision 1.5 / (download) - annotate - [select for diffs], Wed Jun 19 04:21:43 2019 UTC (4 years, 11 months ago) by deraadt
Branch: MAIN
Changes since 1.4: +4 -4 lines
Diff to previous 1.4 (colored)

indentation adjustments, in particular near warn statements
ok claudio

Revision 1.4 / (download) - annotate - [select for diffs], Wed Jun 19 02:02:28 2019 UTC (4 years, 11 months ago) by deraadt
Branch: MAIN
Changes since 1.3: +3 -3 lines
Diff to previous 1.3 (colored)

swap comparisons

Revision 1.3 / (download) - annotate - [select for diffs], Mon Jun 17 15:08:08 2019 UTC (4 years, 11 months ago) by deraadt
Branch: MAIN
Changes since 1.2: +2 -2 lines
Diff to previous 1.2 (colored)

system includes first, always.

Revision 1.2 / (download) - annotate - [select for diffs], Mon Jun 17 15:04:59 2019 UTC (4 years, 11 months ago) by deraadt
Branch: MAIN
Changes since 1.1: +1 -2 lines
Diff to previous 1.1 (colored)

Don't do -portable in base.  It is better done outside the tree.
Imagine if we did it throughout the tree, how many copies of strlcpy
would we have, and how much time would all the configure shell scripts
and includes take?  It would be ludicrous.

Revision 1.1.1.1 / (download) - annotate - [select for diffs] (vendor branch), Mon Jun 17 14:31:31 2019 UTC (4 years, 11 months ago) by job
Branch: job
CVS Tags: job_20190617
Changes since 1.1: +0 -0 lines
Diff to previous 1.1 (colored)

Import Kristaps Dzonsons' RPKI validator into the tree

rpki-client(1) is an implementation of the Resource Public Key
Infrastructure (RPKI), specified by RFC 6480. The client is responsible
for downloading, validating and converting Route Origin Authorisations
(ROAs) into Validated ROA Payloads (VRPs). The client's output (VRPs)
can be used by bgpd(8) to perform BGP Origin Validation (RFC 6811).

The current rpki-client(1) version depends on the CMS functions in
OpenSSL, this of course needs to be addressed urgently.

Thanks to NetNod, IIS.SE, SUNET & 6connect for supporting this effort!

OK deraadt@

Revision 1.1 / (download) - annotate - [select for diffs], Mon Jun 17 14:31:31 2019 UTC (4 years, 11 months ago) by job
Branch: MAIN

Initial revision

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.