OpenBSD CVS

CVS log for src/usr.sbin/rpki-client/repo.c


[BACK] Up to [local] / src / usr.sbin / rpki-client

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.59 / (download) - annotate - [select for diffs], Thu May 30 12:33:15 2024 UTC (2 days, 6 hours ago) by claudio
Branch: MAIN
CVS Tags: HEAD
Changes since 1.58: +9 -7 lines
Diff to previous 1.58 (colored)

Properly setup the fts_state in the FTS_D case.

At fts_level 1 the state needs to be fully reset since we most probably
exited from a directory at level 1 and entered a new dir at level 1.
Without this empty directories remained since the fts_state.type
was wrong for those entries.
Noticed by job@, OK tb@

Revision 1.58 / (download) - annotate - [select for diffs], Mon May 20 15:51:43 2024 UTC (12 days, 3 hours ago) by claudio
Branch: MAIN
Changes since 1.57: +17 -10 lines
Diff to previous 1.57 (colored)

Instead of tracking certificates by SKI track them by an internal identifier.

The certificate SKI is not strictly unique so using it as a unique id is
problematic. It is also not really needed to do that since in theory we
already know the path (but this got lost in the privsep communication).
So add a cert id and pass this id back and forth between main process and
the parser. With this id we can lookup the authentication chain in the
parser and this even works with multiple paths to the same resource.
Since we no longer lookup by SKI the valid_aki_ski function is replaced
by find_issuer() which does the lookup by certid.

The loop protection is now extended to allow each TAL to reach each file
once but still triggers if a file is reaccessed by the tree of a TAL.

In filemode the lookup now uses an AIA uri based lookup tree. Again this
replaces the SKI based lookups from before.

Done together with tb@
OK tb@ job@

Revision 1.57 / (download) - annotate - [select for diffs], Sun Apr 21 19:27:44 2024 UTC (5 weeks, 5 days ago) by claudio
Branch: MAIN
Changes since 1.56: +1 -2 lines
Diff to previous 1.56 (colored)

P-256 support is experimental so require -x to enable it.

Also clean up the externs a little bit by moving experimental and noop
to extern.h.
Reminded by and OK tb@

Revision 1.56 / (download) - annotate - [select for diffs], Mon Apr 8 14:02:13 2024 UTC (7 weeks, 5 days ago) by tb
Branch: MAIN
Changes since 1.55: +4 -1 lines
Diff to previous 1.55 (colored)

Fix capping of VAPs

The previous approach introduced a cap, but it might not always be hit as
intended (I missed this on review). Fix this to check the cap after merging
an ASPA into an already existing VAP. Also free the list of providers since
nothing should be looking at it anymore.

Count VAPs that hit the limit with a new overflowed counter. There are
still a few aspects of the accounting that probably aren't entirely right.
This will be fixed at another point. It's just statistics after all.

with/ok claudio, ok job

Revision 1.55 / (download) - annotate - [select for diffs], Fri Mar 22 03:38:12 2024 UTC (2 months, 1 week ago) by job
Branch: MAIN
Changes since 1.54: +8 -6 lines
Diff to previous 1.54 (colored)

Replace protocol literal strings and strlen() calls with defined constants

OK tb@ claudio@

Revision 1.54 / (download) - annotate - [select for diffs], Mon Feb 26 15:40:33 2024 UTC (3 months ago) by job
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5
Changes since 1.53: +13 -1 lines
Diff to previous 1.53 (colored)

Track the number of new files moving from 'staging' to 'validated cache'

The OpenMetrics output shows per-repository counters for new files
added, the main process and JSON output emit the sum of all new files.

OK claudio@

Revision 1.53 / (download) - annotate - [select for diffs], Thu Feb 22 12:49:42 2024 UTC (3 months, 1 week ago) by job
Branch: MAIN
Changes since 1.52: +25 -1 lines
Diff to previous 1.52 (colored)

Add support for RPKI Signed Prefix Lists

Signed Prefix List are a CMS protected content type for use with the
RPKI to carry the complete list of prefixes which an Autonomous System
may originate to all or any of its routing peers. The validation of a
Signed Prefix List confirms that the holder of the listed ASN produced
the object, and that this list is a current, accurate and complete
description of address prefixes that may be announced into the routing
system originated by this AS.

https://datatracker.ietf.org/doc/html/draft-ietf-sidrops-rpki-prefixlist

with and OK claudio@ tb@

Revision 1.52 / (download) - annotate - [select for diffs], Sat Feb 3 14:30:47 2024 UTC (3 months, 3 weeks ago) by job
Branch: MAIN
Changes since 1.51: +1 -3 lines
Diff to previous 1.51 (colored)

Refactor handling of stale manifests

No need to hoist a staleness indicator through the whole process and
count it explicitly.

OK tb@

Revision 1.51 / (download) - annotate - [select for diffs], Thu Jul 20 05:18:31 2023 UTC (10 months, 1 week ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_4_BASE, OPENBSD_7_4
Changes since 1.50: +6 -4 lines
Diff to previous 1.50 (colored)

Do not set O_EXCL on open() when a published file is on the withdraw list.

If during the process of applying RRDP deltas a object is published then
withdrawn and later published again the current code fails. This is because
published elements use O_EXCL in open() but in the case above the previous
file is still around since withdraws are delayed.

Problem noticed by job@
OK tb@

Revision 1.50 / (download) - annotate - [select for diffs], Thu Jun 29 14:33:35 2023 UTC (11 months ago) by tb
Branch: MAIN
Changes since 1.49: +2 -8 lines
Diff to previous 1.49 (colored)

rpki-client: fix vap_pas stats

A small mistake in a diff broke the counters. Make them AFI agnostic and
adjust ometric output.

guidance & ok claudio

Revision 1.49 / (download) - annotate - [select for diffs], Thu Jun 29 14:09:42 2023 UTC (11 months ago) by claudio
Branch: MAIN
Changes since 1.48: +2 -2 lines
Diff to previous 1.48 (colored)

In rrdp_session_save() stop the loop over deltas once MAX_RRDP_DELTAS
have been processed.
With and OK tb@

Revision 1.48 / (download) - annotate - [select for diffs], Fri Jun 23 11:36:24 2023 UTC (11 months, 1 week ago) by claudio
Branch: MAIN
Changes since 1.47: +106 -36 lines
Diff to previous 1.47 (colored)

Improve detection of RRDP session desynchronization

According to RFC 8182, a given session_id and serial number represent an
immutable record of the state of the Repository Server at a certain
point in time.

Add a check to the RRDP notification file processing to compare whether
the delta hashes associated to previously seen serials are different in
newly fetched notification files. Fall back to a snapshot if a difference
is detected, because such a mutation is a strong desynchronization
indicator.

Idea from Ties de Kock (RIPE NCC).
Based on a diff by job@
With and OK job@ tb@

Revision 1.47 / (download) - annotate - [select for diffs], Tue May 30 16:02:28 2023 UTC (12 months ago) by job
Branch: MAIN
Changes since 1.46: +27 -3 lines
Diff to previous 1.46 (colored)

Fixup file modification timestamps to optimize failover from RRDP to RSYNC

In the RSYNC protocol a file's last modification time and its size are
used to determine whether sending a (partial) copy over the wire is needed.
Previously, when RRDP data structures are serialized to disk, the mtime of
files in DIR_VALID ended up being UTIME_NOW.

Thus, the mtimes of files obtained through RRDP will never match the mtimes
of the same files available through RSYNC - causing each and every file to
be added to the file transfer list.

Instead, use the internal timestamps of RPKI files as the last modified
timestamp. Specifically, for Signed Objects (ROAs, MFTs, GBRs, TAKs, ASPAs)
the CMS signing-time, for .cer files the X.509 notBefore, and for .crl files
the CRL lastUpdate. This results in a surprising optimization for the number
files which have to be transfered.

OK claudio@

Revision 1.46 / (download) - annotate - [select for diffs], Thu May 25 12:49:39 2023 UTC (12 months, 1 week ago) by claudio
Branch: MAIN
Changes since 1.45: +10 -6 lines
Diff to previous 1.45 (colored)

Fix repo_cleanup_entry() state machine so that the repository lookups
are done when the full repo path is available. Without this all repo
lookups returned NULL and the code did not work as intended.
OK tb@

Revision 1.45 / (download) - annotate - [select for diffs], Tue May 16 17:01:31 2023 UTC (12 months, 2 weeks ago) by claudio
Branch: MAIN
Changes since 1.44: +12 -2 lines
Diff to previous 1.44 (colored)

RRDP snapshots should encode publish elements only once. If encountered
fail the transfer and fall back to rsync.

When more than one publish element for the same file exist the RP does
not know which one to choose. Lets fail the RRDP transfer in this case
and fall back to rsync. CA that publish a file more than once are buggy
and need to be fixed.
OK job@ tb@

Revision 1.44 / (download) - annotate - [select for diffs], Wed Apr 26 16:32:41 2023 UTC (13 months ago) by claudio
Branch: MAIN
Changes since 1.43: +210 -174 lines
Diff to previous 1.43 (colored)

Improve accounting by tracking things by repo and tal.
This fixes some wrong accounting for repositories that are referenced
from more than one TAL. It changes the ometric lable output a little bit
since there are repository metrics that no longer include the 'name' label.
OK tb@

Revision 1.43 / (download) - annotate - [select for diffs], Thu Mar 30 15:29:15 2023 UTC (14 months ago) by claudio
Branch: MAIN
Changes since 1.42: +22 -1 lines
Diff to previous 1.42 (colored)

Add the protocol used to sync the repository to the open-metric output.
OK tb@

Revision 1.42 / (download) - annotate - [select for diffs], Wed Mar 29 17:03:29 2023 UTC (14 months ago) by claudio
Branch: MAIN
Changes since 1.41: +4 -4 lines
Diff to previous 1.41 (colored)

Instead of storing the short TAL name as repouri use the first TAL URI
for that. This is more in line with the other uses of repouri.
OK tb@

Revision 1.41 / (download) - annotate - [select for diffs], Wed Dec 28 21:30:18 2022 UTC (17 months ago) by jmc
Branch: MAIN
CVS Tags: OPENBSD_7_3_BASE, OPENBSD_7_3
Changes since 1.40: +6 -6 lines
Diff to previous 1.40 (colored)

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Revision 1.40 / (download) - annotate - [select for diffs], Thu Dec 15 12:02:29 2022 UTC (17 months, 2 weeks ago) by claudio
Branch: MAIN
Changes since 1.39: +158 -17 lines
Diff to previous 1.39 (colored)

Rework statistic collection to be per repository and add metric output option

Many statistic values are now accounted by repository via repo_stat_inc()
At end of the run sum_stats() accumulates these stats per TAL and globally.
The new output file metrics is written when the -m output flag is specified.
The metrics file is written in OpenMetrics format (with a few tweaks to
allow node_exporter to parse the file as well). The ometric code is a copy
from bgpctl(8) and should be kept in sync.
OK tb@

Revision 1.39 / (download) - annotate - [select for diffs], Fri Sep 2 21:56:45 2022 UTC (20 months, 4 weeks ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_2_BASE, OPENBSD_7_2
Changes since 1.38: +25 -6 lines
Diff to previous 1.38 (colored)

Introduce a deadline timer that aborts all repository syncs.
With this rpki-client has a chance to still finish and produce an output
even when a CA is excessivly slow and holds back progress.
With and OK benno@ tb@ and job@

Revision 1.38 / (download) - annotate - [select for diffs], Fri Sep 2 19:10:37 2022 UTC (20 months, 4 weeks ago) by claudio
Branch: MAIN
Changes since 1.37: +16 -2 lines
Diff to previous 1.37 (colored)

Use the abort commands when a repo timeout happens. This is cleaner
then just failing the repo fetch but leaving the backends running.
OK tb@

Revision 1.37 / (download) - annotate - [select for diffs], Fri Sep 2 15:09:19 2022 UTC (20 months, 4 weeks ago) by job
Branch: MAIN
Changes since 1.36: +9 -9 lines
Diff to previous 1.36 (colored)

Move mkpath logic after checking for 'noop' to prevent creation of directories in -n mode

OK claudio@

Revision 1.36 / (download) - annotate - [select for diffs], Tue Aug 30 12:45:13 2022 UTC (21 months ago) by claudio
Branch: MAIN
Changes since 1.35: +14 -1 lines
Diff to previous 1.35 (colored)

In RRDP mode it is possible that a file in the validated cache dir is
no longer valid while the newer file in the .rrdp directory is not (yet)
valid. In this case do not copy the old file over the newer file.
RRDP has no method to identiy inconsistent repos (unlike rsync) so if
the wrong file is put in .rrdp it is not noticed appart from the
verification process failing for that file.
OK tb@ job@

Revision 1.35 / (download) - annotate - [select for diffs], Sun Jul 17 10:32:45 2022 UTC (22 months, 2 weeks ago) by jsg
Branch: MAIN
Changes since 1.34: +2 -2 lines
Diff to previous 1.34 (colored)

infromation -> information

Revision 1.34 / (download) - annotate - [select for diffs], Sun May 15 16:43:34 2022 UTC (2 years ago) by tb
Branch: MAIN
Changes since 1.33: +4 -7 lines
Diff to previous 1.33 (colored)

More KNF and whitespace fixes.

Revision 1.33 / (download) - annotate - [select for diffs], Wed Apr 20 15:31:48 2022 UTC (2 years, 1 month ago) by tb
Branch: MAIN
Changes since 1.32: +3 -3 lines
Diff to previous 1.32 (colored)

Fix various annoying whitespace errors.

Revision 1.32 / (download) - annotate - [select for diffs], Mon Apr 4 16:02:54 2022 UTC (2 years, 1 month ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_1_BASE, OPENBSD_7_1
Changes since 1.31: +101 -67 lines
Diff to previous 1.31 (colored)

Adjust cache cleanup to the deficits of RRDP. Unlike rysnc, RRDP has no
method to sync the cache. It just depends on all deltas to work but has
no method to check if the result is consistent with the source.
Because of this do not unlink files which were fetched via RRDP. Instead
move them back to their .rrdp directory and hope the will be cleaned up.
This should help to keep the cache coherent in some edge cases.
OK tb@

Revision 1.31 / (download) - annotate - [select for diffs], Mon Feb 14 14:47:49 2022 UTC (2 years, 3 months ago) by job
Branch: MAIN
Changes since 1.30: +5 -5 lines
Diff to previous 1.30 (colored)

Draw a bit more attention to transport switches between RRDP, RSYNC, and cache

OK claudio@ tb@

Revision 1.30 / (download) - annotate - [select for diffs], Wed Feb 2 15:13:00 2022 UTC (2 years, 3 months ago) by claudio
Branch: MAIN
Changes since 1.29: +3 -1 lines
Diff to previous 1.29 (colored)

When a RRDP fetch failed not only clear the filesystem but also flush
the to delete list stored this RRDP node.
Noticed by Job on console.rpki-client.org with the help of idnic.net
OK tb@

Revision 1.29 / (download) - annotate - [select for diffs], Fri Jan 28 14:11:27 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.28: +2 -2 lines
Diff to previous 1.28 (colored)

Do not skip .rrdp cache cleanup if rrdp is off (option -R). The sync
via rsync alters the cache and rrdp can only recover by downloading
a snapshot. By doing the cleanup here it will make sure this happens.
Noticed by job@
OK tb@

Revision 1.28 / (download) - annotate - [select for diffs], Wed Jan 26 13:57:56 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.27: +76 -56 lines
Diff to previous 1.27 (colored)

Change the repository layout by removing the valid directory. Everything
that was in that directory is now in the chachedir root. The rsync and
rrdp directories are now .rsync/ and .rrdp/. The ta/ directory still
remains because TAs are special.
Idea and most of the diff from job@
OK tb@ and benno@

Revision 1.27 / (download) - annotate - [select for diffs], Mon Jan 24 15:50:34 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.26: +2 -2 lines
Diff to previous 1.26 (colored)

When rename fails show the source filename and not the destination.
The error should be more helpful thisway.
OK tb@

Revision 1.26 / (download) - annotate - [select for diffs], Sun Jan 23 12:09:24 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.25: +9 -24 lines
Diff to previous 1.25 (colored)

Handle EINTR the same way in all poll loops. In all cases restart the
poll loop. In the main process move the timeout handling for repositories
into a single function that does the timeouts and the calculation of the
timeout in one go.
OK tb@

Revision 1.25 / (download) - annotate - [select for diffs], Fri Jan 14 15:00:23 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.24: +279 -220 lines
Diff to previous 1.24 (colored)

Introduce a validated cache which holds all the files that have
successfully been verified by rpki-client.

With this the rsync and rrdp directories are more of a temporary storage
location. New files are downloaded there and then moved to the valid
directory at the end. In -n mode only the valid directory is looked at with
the exception of the ta directory holding the trust anchors.
A file can now be in two different locations so adjust all the code paths
that open files to check both locations.
One nice side-effect of this is that the RRDP handling in the main process
got simplified. There is no longer the need for temporary RRDP directories.
OK tb@

Revision 1.24 / (download) - annotate - [select for diffs], Thu Jan 13 14:57:02 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.23: +77 -79 lines
Diff to previous 1.23 (colored)

Move some functions around to reduce diff with an upcoming change.
OK tb@

Revision 1.23 / (download) - annotate - [select for diffs], Thu Jan 13 13:46:03 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.22: +4 -3 lines
Diff to previous 1.22 (colored)

Alter valid_filehash() to take a file descriptor instead of a path.
This is needed so that callers can allow a file to be in multiple
locations. Also move mft_check() from mft.c to parser.c.
OK tb@

Revision 1.22 / (download) - annotate - [select for diffs], Thu Jan 13 13:18:41 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.21: +69 -24 lines
Diff to previous 1.21 (colored)

Implement a RRDP_CLEAR message that instructs the parent to cleanup
the rrdp directory. This is used before a snapshot download to ensure
that the snapshot is applied to a clean repo.
Similar cleanup happens if the transfer fails. In that case remove the
temp directory contents only.
This uses a new function remove_contents() to remove everything below
a base directory (a bit like rm -r X/*).
OK tb@

Revision 1.21 / (download) - annotate - [select for diffs], Thu Jan 13 11:47:44 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.20: +4 -1 lines
Diff to previous 1.20 (colored)

Add the same entityq_flush() logic in ta_lookup() as it is done in
repo_lookup(). This fixes -n mode.
OK tb@

Revision 1.20 / (download) - annotate - [select for diffs], Tue Jan 11 13:06:07 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.19: +56 -30 lines
Diff to previous 1.19 (colored)

Change the way the parser accesses files. It now builds the file path
based on information from the repository, a local path and the filename.
This simplifies some code both in the main process and the parser.
For this to work repositories are passed to the parser before any other
entity of this repository is passed. Struct entity is extended to include
the repoid and the path along the file(name).
Input and OK tb@ & job@

Revision 1.19 / (download) - annotate - [select for diffs], Tue Jan 4 18:16:09 2022 UTC (2 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.18: +4 -4 lines
Diff to previous 1.18 (colored)

Fix -n mode by setting repository states to REPO_DONE instead of the
more correct REPO_FAILED. The problem is that with REPO_FAILED the repo
no longer matches a subsequent rrdp_get lookup and as a result the repo
uses rsync which wrong in this case.
OK benno@ tb@

Revision 1.18 / (download) - annotate - [select for diffs], Wed Dec 29 11:35:23 2021 UTC (2 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.17: +65 -69 lines
Diff to previous 1.17 (colored)

Merge multiple similar code bits together into repo_done(). Which currently
finds all repos that point to the passed backend and flushes their queues.
Also make sure that REPO_FAILED is set in cases where no sync happened and
also fall back from rrdp to rsync in any case of failure (including errors
merging the rrdp repo itself).
OK tb@

Revision 1.17 / (download) - annotate - [select for diffs], Wed Dec 22 09:35:14 2021 UTC (2 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.16: +18 -18 lines
Diff to previous 1.16 (colored)

Replace two questionable size_t types. For the repo id use a unsigned int
and for the roa maxlength use unsigned char (like the prefixlen in struct
ip_addr).
With input and OK job@

Revision 1.16 / (download) - annotate - [select for diffs], Tue Dec 21 16:16:15 2021 UTC (2 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.15: +4 -11 lines
Diff to previous 1.15 (colored)

Simplify code a bit. There is only one TA per TAL and so only one
ta_lookup(). Implementing the talrepocnt limiter there makes little
sense and gains us nothing.
OK job@

Revision 1.15 / (download) - annotate - [select for diffs], Tue Dec 7 12:46:47 2021 UTC (2 years, 5 months ago) by claudio
Branch: MAIN
Changes since 1.14: +22 -23 lines
Diff to previous 1.14 (colored)

Use one common function to build the base directory path for repositories.
OK benno@

Revision 1.14 / (download) - annotate - [select for diffs], Thu Nov 25 14:03:40 2021 UTC (2 years, 6 months ago) by job
Branch: MAIN
Changes since 1.13: +4 -3 lines
Diff to previous 1.13 (colored)

Replace MAX_REPO_TIMEOUT with repo_timeout, which is set to 1/4th of
timeout, or if timeout is disabled set to 24 hours.

OK claudio@

Revision 1.13 / (download) - annotate - [select for diffs], Thu Nov 25 12:55:34 2021 UTC (2 years, 6 months ago) by claudio
Branch: MAIN
Changes since 1.12: +3 -1 lines
Diff to previous 1.12 (colored)

Check for possible negative diff timeout. Clamp it to 0 in that case.
poll with negative timeout noticed by job@
OK job@

Revision 1.12 / (download) - annotate - [select for diffs], Mon Nov 15 16:32:15 2021 UTC (2 years, 6 months ago) by claudio
Branch: MAIN
Changes since 1.11: +26 -25 lines
Diff to previous 1.11 (colored)

Adjust how the repository count limit works. Instead of failing hard just
fall back to a possible cache and try to validate what is available.
This still limits the number of repositories fetched but allows valid
repositories to finish with the available data.
OK job@

Revision 1.5.2.1 / (download) - annotate - [select for diffs], Tue Nov 9 13:41:19 2021 UTC (2 years, 6 months ago) by benno
Branch: OPENBSD_6_9
Changes since 1.5: +293 -139 lines
Diff to previous 1.5 (colored) next main 1.6 (colored)

rpki-client(8) should handle CA misbehaviours as soft-errors.

This is a merge of usr.sbin/rpki-client and usr.bin/rsync from current
and includes all commits in rpki-client 7.5 up to Tue Nov 9 11:03:40
2021 and to openrsync up to Wed Nov 3 14:42:13 2021, including:

* Make rpki-client more resilient regarding untrusted input:
  - fail repository synchronisation after 15min runtime
  - limit the number of publication points per TAL
  - don't allow DOCTYPE definitions in RRDP XML files
  - fix detection of HTTP redirect loops.
* limit the number of concurrent rsync processes.
* fix CRLF in tal files.

This is patches/6.9/common/021_rpki.patch.sig

Revision 1.9.4.1 / (download) - annotate - [select for diffs], Tue Nov 9 13:40:32 2021 UTC (2 years, 6 months ago) by benno
Branch: OPENBSD_7_0
Changes since 1.9: +113 -11 lines
Diff to previous 1.9 (colored) next main 1.10 (colored)

rpki-client(8) should handle CA misbehaviours as soft-errors.

This is a merge of usr.sbin/rpki-client and usr.bin/rsync from current
and includes all commits in rpki-client 7.5 up to Tue Nov 9 11:03:40
2021 and to openrsync up to Wed Nov 3 14:42:13 2021, including:

* Make rpki-client more resilient regarding untrusted input:
  - fail repository synchronisation after 15min runtime
  - limit the number of publication points per TAL
  - don't allow DOCTYPE definitions in RRDP XML files
  - fix detection of HTTP redirect loops.
* limit the number of concurrent rsync processes.
* fix CRLF in tal files.

This is patches/7.0/common/004_rpki.patch.sig

Revision 1.11 / (download) - annotate - [select for diffs], Tue Nov 9 11:03:39 2021 UTC (2 years, 6 months ago) by claudio
Branch: MAIN
Changes since 1.10: +92 -6 lines
Diff to previous 1.10 (colored)

Limit the number of publication points under a given TAL.
Introduce an additional timeout for each publication point.

The limits are large enough to accomodate normal operating levels.

With and OK benno@ job@ tb@ beck@ deraadt@

Revision 1.10 / (download) - annotate - [select for diffs], Thu Nov 4 17:35:09 2021 UTC (2 years, 6 months ago) by claudio
Branch: MAIN
Changes since 1.9: +22 -6 lines
Diff to previous 1.9 (colored)

Instead of creating a struct repo for each unique caRepository URI
use the rsync URI (a base version of caRepository) and the notify URI
to identify repositories. If both rsync URI and notify URI are the same
then the repo is the same. The notify URI is optional and can be NULL
so the lookup needs to be a bit careful.
This reduces the number of struct repos from 26k to around 50.
OK tb@

Revision 1.9 / (download) - annotate - [select for diffs], Thu Aug 12 15:27:15 2021 UTC (2 years, 9 months ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_7_0_BASE
Branch point for: OPENBSD_7_0
Changes since 1.8: +30 -19 lines
Diff to previous 1.8 (colored)

Rework the logic for fetching TA files a bit. Move the logic that checks
if all possible sources have been used to ta_fetch() this simplifies the
logic in the rsync and http done cases.
Additionally when -R is used, skip https URI and only connect via rsync.
If the TAL file has no rsync URI then the system will fall back to the
cache and not try https instead.
This makes -R a only use rsync flag.
With and OK job@

Revision 1.8 / (download) - annotate - [select for diffs], Mon Jun 14 10:01:23 2021 UTC (2 years, 11 months ago) by claudio
Branch: MAIN
Changes since 1.7: +97 -70 lines
Diff to previous 1.7 (colored)

Change errors for file manipulations from err(3) to warn(3) and adjust the
surrounding code to handle these errors gracefully. When rpki-client runs
out of space it will keep on running including the cleanup at the end.
This way the temporary and old files are removed hopfully giving back some
space. This affects mainly RRDP and the initial fetch of TA files.
OK job@

Revision 1.7 / (download) - annotate - [select for diffs], Tue May 4 08:16:36 2021 UTC (3 years ago) by claudio
Branch: MAIN
Changes since 1.6: +55 -37 lines
Diff to previous 1.6 (colored)

Similar to rsync repo delay deletes for rrdp repos to after processing
all files. Now RRDP has a weak sync mechanism and therefor not all
unreferenced files can be removed, instead look at the list of removes
which were part of the delta processing and remove those files.
Warn and keep the file if a delta specifies to remove a file which is
actually still referenced.
OK benno@

Revision 1.6 / (download) - annotate - [select for diffs], Mon Apr 19 17:04:35 2021 UTC (3 years, 1 month ago) by deraadt
Branch: MAIN
Changes since 1.5: +4 -8 lines
Diff to previous 1.5 (colored)

code review results in KNF, and moving local variables into lowest scope
ok claudio

Revision 1.5 / (download) - annotate - [select for diffs], Tue Apr 13 13:35:59 2021 UTC (3 years, 1 month ago) by claudio
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE
Branch point for: OPENBSD_6_9
Changes since 1.4: +5 -1 lines
Diff to previous 1.4 (colored)

Before adding a file to the temporary rrdp repo remove it from the
deleted filepath set. A file can only be in one set (deleted or added)
but not on both.
OK benno@

Revision 1.4 / (download) - annotate - [select for diffs], Wed Apr 7 14:19:31 2021 UTC (3 years, 1 month ago) by claudio
Branch: MAIN
Changes since 1.3: +27 -12 lines
Diff to previous 1.3 (colored)

When merging a repo even files to delete can be part of the temporary
work dir. So unlink can return an ENOENT error for the main repo. In
which case the temp dir should be tried.
Refactor this code a bit since there is no way rrdp_filename() should
fail in this part of the code.
OK tb@

Revision 1.3 / (download) - annotate - [select for diffs], Fri Apr 2 05:16:29 2021 UTC (3 years, 2 months ago) by tb
Branch: MAIN
Changes since 1.2: +5 -5 lines
Diff to previous 1.2 (colored)

fix typo + some whitespace

Revision 1.2 / (download) - annotate - [select for diffs], Thu Apr 1 16:50:23 2021 UTC (3 years, 2 months ago) by deraadt
Branch: MAIN
Changes since 1.1: +2 -2 lines
Diff to previous 1.1 (colored)

spelling

Revision 1.1 / (download) - annotate - [select for diffs], Thu Apr 1 16:04:48 2021 UTC (3 years, 2 months ago) by claudio
Branch: MAIN

Initial commit of RRDP (The RPKI Repository Delta Protocol - RFC8182) support
in rpki-client. For now it is off by default.

All XML processing is done in its own process with minimal pledge rights.
It uses the already present https process to fetch the xml files and uses
the master porcess to handle the file IO into the repositories.
RRDP data is stored in the cache under ./rrdp/ and the first directory
is the SHA256 hash of the notify URI.

Fetching snapshots and deltas works to bring the cache up to date.
If something goes wrong rpki-client will fall back to rsync.

RRDP was implemented by Nils Fisher and integrated into rpki-client by myself.
"Time to get it in" deraadt@

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.