OpenBSD CVS

CVS log for src/usr.sbin/smtpd/smtpd.h


[BACK] Up to [local] / src / usr.sbin / smtpd

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.685 / (download) - annotate - [select for diffs], Tue May 28 07:10:30 2024 UTC (4 days, 12 hours ago) by op
Branch: MAIN
CVS Tags: HEAD
Changes since 1.684: +2 -1 lines
Diff to previous 1.684 (colored)

actually honour the services supported by the proc tables

ok gilles@

Revision 1.684 / (download) - annotate - [select for diffs], Tue May 7 12:10:06 2024 UTC (3 weeks, 4 days ago) by op
Branch: MAIN
Changes since 1.683: +4 -2 lines
Diff to previous 1.683 (colored)

change the smtpd table protocol

Using imsg for the "proc" table (external programs) has proven quite
painful in practice since a lot of smtpd internals (structs, enums,
etc..) have to be kept in sync with the various tables implementations.

Instead, a filter-like protocol for tables decouples the implementations
and allows to write and test tables easily.

The new text-based transport protocol is documented in the (added)
smtpd-tables(7) manpage.

The old imsg protocol is no longer supported and existing tables have to
be converted.  In particular, users of opensmtpd-extras tables will need
install the new opensmtpd-table-* packages.

With lots of suggestions and improvements from gilles and a tweak
from Philipp (philipp+openbsd [at] bureaucracy [dot] de), thanks!

ok gilles

Revision 1.683 / (download) - annotate - [select for diffs], Sat Mar 2 22:40:28 2024 UTC (2 months, 4 weeks ago) by op
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5
Changes since 1.682: +2 -2 lines
Diff to previous 1.682 (colored)

bump version to 7.5.0

Revision 1.682 / (download) - annotate - [select for diffs], Sun Feb 11 09:24:26 2024 UTC (3 months, 2 weeks ago) by op
Branch: MAIN
Changes since 1.681: +4 -1 lines
Diff to previous 1.681 (colored)

unify smtpd and makemap table parser

These are supposed to parse the same file format but have subtle
difference in the handling of comments, continuation lines and escaping.

Converge both to the simpler smtpd parser which doesn't handle
continuation lines nor escaping, and support comments only at the start
of the line.

improvements and ok millert@

Revision 1.681 / (download) - annotate - [select for diffs], Fri Feb 2 22:02:12 2024 UTC (3 months, 4 weeks ago) by gilles
Branch: MAIN
Changes since 1.680: +2 -1 lines
Diff to previous 1.680 (colored)

there's no good reason to allow smtpd to execute custom command set by root
in a .forward file so disallow custom commands and file reading, only allow
setting forward addresses and users.

as root is no longer allowed to run any MDA but mbox, we can be stricter on
the setup of the MDA process and refuse to exec anything that's not an mbox
dispatcher.

tested by op@ who edited a root envelope to simulate an exploit injecting a
custom command in a root envelope, smtpd refused to exec.

ok millert@ and op@

Revision 1.680 / (download) - annotate - [select for diffs], Wed Jan 3 08:11:15 2024 UTC (4 months, 4 weeks ago) by op
Branch: MAIN
Changes since 1.679: +4 -2 lines
Diff to previous 1.679 (colored)

relax ORCPT syntax validation

We expected the ORCPT parameter to be a valid rfc822 address.  This is
wrong on multiple levels:

 - any other IANA-registered "addr-type" can be used
 - the parameter may be encoded and we didn't decode it prior validation
 - RFC3461 explicitly states that "[..] the address associated with the
   ORCPT keyword is NOT constrained to conform to the syntax rules for
   that 'addr-type'".

Instead, just validate the xtext and preserve the ORCPT value as-is.

Issue originally reported by Tim Kuijsten, Tassilo Philipp and others.

ok millert@

Revision 1.679 / (download) - annotate - [select for diffs], Wed Nov 8 08:46:35 2023 UTC (6 months, 3 weeks ago) by op
Branch: MAIN
Changes since 1.678: +3 -1 lines
Diff to previous 1.678 (colored)

RFC 7505 ("Null MX") handling

mail delivery will not be attempted if a domain advertises a single MX
record with preference 0 and a zero-length label.

based on an initial diff from Philipp (philipp+openbsd [at] bureaucracy
[dot] de), thanks!

ok jung@

Revision 1.678 / (download) - annotate - [select for diffs], Fri Sep 29 18:30:14 2023 UTC (8 months ago) by op
Branch: MAIN
CVS Tags: OPENBSD_7_4_BASE, OPENBSD_7_4
Changes since 1.677: +2 -2 lines
Diff to previous 1.677 (colored)

bump version to 7.4.0

Revision 1.677 / (download) - annotate - [select for diffs], Sat Jun 17 08:32:48 2023 UTC (11 months, 2 weeks ago) by op
Branch: MAIN
Changes since 1.676: +2 -2 lines
Diff to previous 1.676 (colored)

bump version to 7.3.0

Revision 1.676 / (download) - annotate - [select for diffs], Wed May 31 16:51:46 2023 UTC (12 months ago) by op
Branch: MAIN
Changes since 1.675: +2 -1 lines
Diff to previous 1.675 (colored)

add missing include of time.h

spotted after a report on OpenSMTPD-portable.  While here include
sys/time.h in smtpd.h, as noted in event_init(3), since it includes
event.h.

ok millert@

Revision 1.675 / (download) - annotate - [select for diffs], Thu May 25 11:37:58 2023 UTC (12 months, 1 week ago) by op
Branch: MAIN
Changes since 1.674: +1 -4 lines
Diff to previous 1.674 (colored)

remove two unused defines

last PROC_COUNT use was removed with the switch to fork+exec by eric@ in
2016, CA_FILE with the removal of cert.c two years ago.

ok tb@, kn@

Revision 1.674 / (download) - annotate - [select for diffs], Fri Feb 18 16:57:36 2022 UTC (2 years, 3 months ago) by millert
Branch: MAIN
CVS Tags: OPENBSD_7_3_BASE, OPENBSD_7_3, OPENBSD_7_2_BASE, OPENBSD_7_2, OPENBSD_7_1_BASE, OPENBSD_7_1
Changes since 1.673: +9 -3 lines
Diff to previous 1.673 (colored)

Revert changes to use the new libtls signer api
There are bugs in the new libtls signer that can lead to a crash.
OK tb@ jsing@

Revision 1.673 / (download) - annotate - [select for diffs], Sat Feb 12 18:22:04 2022 UTC (2 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.672: +4 -10 lines
Diff to previous 1.672 (colored)

use new libtls signer api

ok tb@

Revision 1.672 / (download) - annotate - [select for diffs], Thu Feb 10 14:59:35 2022 UTC (2 years, 3 months ago) by millert
Branch: MAIN
Changes since 1.671: +2 -3 lines
Diff to previous 1.671 (colored)

Do not verify the cert or CA for a relay using opportunistic TLS.
If a relay is not explicitly configured to use TLS but the remote
side supports STARTTLS, we will try to use it.  However, in this
case we should not verify the cert or CA (which may be self-signed).
This restores the relay behavior before the switch to libtls was made.
There is no change if the relay is explicitly configured to use TLS.
OK eric@

Revision 1.671 / (download) - annotate - [select for diffs], Wed Sep 22 17:12:34 2021 UTC (2 years, 8 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_7_0_BASE, OPENBSD_7_0
Changes since 1.670: +2 -2 lines
Diff to previous 1.670 (colored)

bump version to 7.0.0

Revision 1.670 / (download) - annotate - [select for diffs], Wed Jul 28 19:39:51 2021 UTC (2 years, 10 months ago) by benno
Branch: MAIN
Changes since 1.669: +3 -2 lines
Diff to previous 1.669 (colored)

add format attribute to vaararg functions.
millert@ thinks its useful.

Revision 1.669 / (download) - annotate - [select for diffs], Mon Jun 14 17:58:16 2021 UTC (2 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.668: +9 -3 lines
Diff to previous 1.668 (colored)

add required headers for smtpd.h and remove unnecessary ones in other files.

ok jung@

Revision 1.668 / (download) - annotate - [select for diffs], Wed Apr 21 07:54:10 2021 UTC (3 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.667: +1 -19 lines
Diff to previous 1.667 (colored)

unplug unused certificate verification code, now that this is done by libtls.

ok tb@ millert@

Revision 1.667 / (download) - annotate - [select for diffs], Sun Apr 11 07:18:08 2021 UTC (3 years, 1 month ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE, OPENBSD_6_9
Changes since 1.666: +1 -6 lines
Diff to previous 1.666 (colored)

do not build unused code and remove uneeded dependency on libm.

ok tb@

Revision 1.666 / (download) - annotate - [select for diffs], Sat Apr 10 06:44:18 2021 UTC (3 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.665: +2 -2 lines
Diff to previous 1.665 (colored)

bump smtpd version

Revision 1.665 / (download) - annotate - [select for diffs], Fri Apr 9 16:43:43 2021 UTC (3 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.664: +3 -1 lines
Diff to previous 1.664 (colored)

allow to specify tls ciphers and protocols on listeners

ok tb@

Revision 1.664 / (download) - annotate - [select for diffs], Wed Mar 31 19:09:19 2021 UTC (3 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.663: +4 -3 lines
Diff to previous 1.663 (colored)

turn log_trace() into a macro to prevent evaluating the format string
parameters when tracing is not enabled.

ok millert@

Revision 1.663 / (download) - annotate - [select for diffs], Wed Mar 31 17:47:16 2021 UTC (3 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.662: +3 -1 lines
Diff to previous 1.662 (colored)

allow to specify tls protocols and ciphers on relay actions

ok espie@ sthen@ tb@

Revision 1.662 / (download) - annotate - [select for diffs], Fri Mar 5 12:37:32 2021 UTC (3 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.661: +7 -1 lines
Diff to previous 1.661 (colored)

Start porting smtpd to libtls.

Note that it changes the way SNI works: The certificate to use is now
selected by looking at the names found in the certificates themselves,
rather than the names of the pki entries in the configuration file.
The set of certificates for a tls listener must be defined explicitly by
using the pki listener option multiple times.

ok tb@

Revision 1.661 / (download) - annotate - [select for diffs], Tue Jan 19 09:16:20 2021 UTC (3 years, 4 months ago) by claudio
Branch: MAIN
Changes since 1.660: +2 -2 lines
Diff to previous 1.660 (colored)

Do the KAME embedded scope fixup in the two places where getifaddrs() is
used. With this there should be no more embedded scopes left and therefor
in6addr_to_text() can be removed. getnameinfo() will just do the right
thing now.
OK eric@

Revision 1.660 / (download) - annotate - [select for diffs], Thu Dec 31 08:27:15 2020 UTC (3 years, 5 months ago) by martijn
Branch: MAIN
Changes since 1.659: +6 -6 lines
Diff to previous 1.659 (colored)

Rename the pony process to dispatcher and klondike to crypto.

From gilles@
OK millert@ giovanni@

Revision 1.659 / (download) - annotate - [select for diffs], Wed Sep 23 19:11:50 2020 UTC (3 years, 8 months ago) by martijn
Branch: MAIN
CVS Tags: OPENBSD_6_8_BASE, OPENBSD_6_8
Changes since 1.658: +1 -22 lines
Diff to previous 1.658 (colored)

Revert agentx support for now, we're too close to release.

requested by deraadt@

Revision 1.658 / (download) - annotate - [select for diffs], Wed Sep 23 18:01:27 2020 UTC (3 years, 8 months ago) by martijn
Branch: MAIN
Changes since 1.657: +23 -2 lines
Diff to previous 1.657 (colored)

Add support for agentx to smtpd.

This is based around NETWORK-SERVICES-MIB from RFC2788 and MTA-MIB from
RFC2789, but does not export the full spec. Hopefully this will expand in
the future.

People who want to use this against net-snmp (currently the only option
known to me at the time of writing) may want to add -I -mta_sendmail to the
flags, so net-snmp doesn't throw garbage into the mib-2.28 subtree.

Revision 1.657 / (download) - annotate - [select for diffs], Wed Sep 16 11:19:42 2020 UTC (3 years, 8 months ago) by martijn
Branch: MAIN
Changes since 1.656: +3 -1 lines
Diff to previous 1.656 (colored)

Add the admd keyword. This can be used by filters interested in the
Authentication-Results header.

OK giovanni@

Revision 1.656 / (download) - annotate - [select for diffs], Wed Apr 8 07:30:44 2020 UTC (4 years, 1 month ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_6_7_BASE, OPENBSD_6_7
Changes since 1.655: +2 -2 lines
Diff to previous 1.655 (colored)

bump smtpd version

Revision 1.655 / (download) - annotate - [select for diffs], Mon Feb 24 17:49:23 2020 UTC (4 years, 3 months ago) by millert
Branch: MAIN
Changes since 1.654: +2 -2 lines
Diff to previous 1.654 (colored)

Bump version to 6.6.4 for errata and to match -portable.

Revision 1.621.2.1 / (download) - annotate - [select for diffs], Mon Feb 24 16:20:06 2020 UTC (4 years, 3 months ago) by millert
Branch: OPENBSD_6_5
Changes since 1.621: +4 -3 lines
Diff to previous 1.621 (colored) next main 1.622 (colored)

OpenBSD 6.5 errata 031, February 24, 2020:

An out of bounds read in smtpd allows an attacker to inject arbitrary
commands into the envelope file which are then executed as root.
Separately, missing privilege revocation in smtpctl allows arbitrary
commands to be run with the _smtpq group.

Revision 1.641.2.1 / (download) - annotate - [select for diffs], Mon Feb 24 16:19:32 2020 UTC (4 years, 3 months ago) by millert
Branch: OPENBSD_6_6
Changes since 1.641: +4 -3 lines
Diff to previous 1.641 (colored) next main 1.642 (colored)

OpenBSD 6.6 errata 021, February 24, 2020:

An out of bounds read in smtpd allows an attacker to inject arbitrary
commands into the envelope file which are then executed as root.
Separately, missing privilege revocation in smtpctl allows arbitrary
commands to be run with the _smtpq group.

Revision 1.654 / (download) - annotate - [select for diffs], Mon Feb 24 16:16:08 2020 UTC (4 years, 3 months ago) by millert
Branch: MAIN
Changes since 1.653: +4 -3 lines
Diff to previous 1.653 (colored)

Fix two security vulnerabilities discovered by Qualys.
An out of bounds read in smtpd allows an attacker to inject arbitrary
commands into the envelope file which are then executed as root.
Separately, missing privilege revocation in smtpctl allows arbitrary
commands to be run with the _smtpq group.

Revision 1.653 / (download) - annotate - [select for diffs], Mon Feb 3 15:41:22 2020 UTC (4 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.652: +2 -1 lines
Diff to previous 1.652 (colored)

now that mail.local(8) relies on lockspool(1) for mailbox locking, have the
mailbox created by smtpd for mbox before privileges are dropped then we can
call mail.local(8) with the recipient privileges.

ok millert@

Revision 1.652 / (download) - annotate - [select for diffs], Fri Jan 31 22:01:20 2020 UTC (4 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.651: +6 -2 lines
Diff to previous 1.651 (colored)

introduce mda_mbox() to handle mbox delivery in its own code path, and make
it use execle() since we know all parameters and don't need command line to
be parsed.

ok millert@ and jung@

Revision 1.651 / (download) - annotate - [select for diffs], Thu Jan 30 13:10:06 2020 UTC (4 years, 4 months ago) by solene
Branch: MAIN
Changes since 1.650: +2 -2 lines
Diff to previous 1.650 (colored)

Bump smtpd version after recent changes

ok gilles@

Revision 1.650 / (download) - annotate - [select for diffs], Wed Jan 8 01:41:11 2020 UTC (4 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.649: +10 -1 lines
Diff to previous 1.649 (colored)

allow using the session username in builtin filters when available

Revision 1.649 / (download) - annotate - [select for diffs], Sat Dec 21 10:40:20 2019 UTC (4 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.648: +5 -2 lines
Diff to previous 1.648 (colored)

add FILTER_SUBSYSTEM_SMTP_OUT to filter_subsystem enum and add filter name
to struct dispatcher_remote, this will reduce the smtp-out reporting diff

Revision 1.648 / (download) - annotate - [select for diffs], Sat Dec 21 10:23:37 2019 UTC (4 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.647: +2 -2 lines
Diff to previous 1.647 (colored)

do not pass rdns, fcrdns, ss_src and ss_dest with IMSG_FILTER_SMTP_BEGIN,
but gather the information from the link-connect reporting event instead.
this removes redundant code and makes it easier to prepare for smtp-out.

Revision 1.647 / (download) - annotate - [select for diffs], Wed Dec 18 10:00:39 2019 UTC (4 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.646: +2 -2 lines
Diff to previous 1.646 (colored)

give a better name to a couple functions and struct fields related to
filters, no functional change

Revision 1.646 / (download) - annotate - [select for diffs], Wed Dec 18 07:57:52 2019 UTC (4 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.645: +3 -1 lines
Diff to previous 1.645 (colored)

teach relay action how to do domain-based relay host, this allows declaring
a single relay action with a mapping of relay hosts per domain.

ok eric@

Revision 1.645 / (download) - annotate - [select for diffs], Sat Dec 14 16:24:52 2019 UTC (4 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.644: +2 -1 lines
Diff to previous 1.644 (colored)

introduce a bypass keyword so that builtin filters can bypass processing of
a phase when a condition is met

suggested by several people including jung@, ok jung@

Revision 1.644 / (download) - annotate - [select for diffs], Thu Dec 12 22:10:47 2019 UTC (4 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.643: +15 -9 lines
Diff to previous 1.643 (colored)

filter protocol has an initial handshake within which smtpd tells filters
about a few global configuration informations. this makes smtpd tell proc
filters for which subsystem they are registered allowing them to register
only events that are relevant.

Revision 1.643 / (download) - annotate - [select for diffs], Mon Nov 25 14:18:33 2019 UTC (4 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.642: +2 -1 lines
Diff to previous 1.642 (colored)

store smtp session username in envelope and allow ruleset to match specific
users or mailaddr:

match auth "gilles@openbsd.org" [...]
match auth "@openbsd.org" [...]

ok eric@

Revision 1.642 / (download) - annotate - [select for diffs], Sun Nov 3 23:58:51 2019 UTC (4 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.641: +2 -2 lines
Diff to previous 1.641 (colored)

6.6.0 -> 6.6.1

Revision 1.641 / (download) - annotate - [select for diffs], Mon Sep 30 08:31:41 2019 UTC (4 years, 8 months ago) by martijn
Branch: MAIN
CVS Tags: OPENBSD_6_6_BASE
Branch point for: OPENBSD_6_6
Changes since 1.640: +1 -2 lines
Diff to previous 1.640 (colored)

Allow maildir and mbox MDAs to tempfail on situations that might be
resolved over time.

While here remove mkdirs component from utils.c, which isn't used anywhere.

OK gilles@, millert@

Revision 1.640 / (download) - annotate - [select for diffs], Sun Sep 29 10:03:49 2019 UTC (4 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.639: +3 -1 lines
Diff to previous 1.639 (colored)

SRS uses base64 encoding for the checksum, however while this is ok when we
only have MTA in the loop, some implementations like Dovecot's LMTP dislike
finding '/' in an e-mail address. Since checksum is meant to be verified at
the MX that generated the SRS encoding, use alternate rfc354 base64 encode,
swapping '/' with '_' and '+' with '-'.

ok eric@ millert@

Revision 1.639 / (download) - annotate - [select for diffs], Fri Sep 20 17:46:05 2019 UTC (4 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.638: +13 -1 lines
Diff to previous 1.638 (colored)

teach smtpd how to do SRS so hosts that act as forwarders don't break SPF.
this basic implementation does SRS0/SRS1 encoding/decoding, validating time
and checksums.

with insight from semarie@, ok eric@ and millert@

Revision 1.638 / (download) - annotate - [select for diffs], Thu Sep 19 07:35:36 2019 UTC (4 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.637: +2 -2 lines
Diff to previous 1.637 (colored)

unescape / and ^ in the general delivery case, they only need to be for
maildir

ok eric@

Revision 1.637 / (download) - annotate - [select for diffs], Wed Sep 18 11:26:30 2019 UTC (4 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.636: +3 -2 lines
Diff to previous 1.636 (colored)

Implement server certificate validation in smtp(1).
Check certificate against MX name in smtpd(8) mta.

ok gilles@

Revision 1.636 / (download) - annotate - [select for diffs], Wed Sep 11 04:19:19 2019 UTC (4 years, 8 months ago) by martijn
Branch: MAIN
Changes since 1.635: +5 -3 lines
Diff to previous 1.635 (colored)

Sprinkle around some __attribute__((__format__ (printf(...))).

OK gilles@

Revision 1.635 / (download) - annotate - [select for diffs], Fri Sep 6 08:23:56 2019 UTC (4 years, 8 months ago) by martijn
Branch: MAIN
Changes since 1.634: +5 -1 lines
Diff to previous 1.634 (colored)

Add support for filter-reports. These allow filters to send freetext
reports to other filters/report handlers.
Builtin filters work via the new "report" keyword.
Proc filters can send reports via:
"report|<seconds.<microseconds>|smtp-in|<reqid>|<message>"
Subscribing to these reports can be done via the the "filter-report"
keyword.
The reports themselves contain the usual elements followed by:
<type>|<name>|<message>
Type can be builtin or proc.
Name is the process name for type proc and the filter name for type builtin.

OK gilles@

Revision 1.634 / (download) - annotate - [select for diffs], Wed Sep 4 07:28:27 2019 UTC (4 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.633: +3 -1 lines
Diff to previous 1.633 (colored)

introduce the 'junk' builtin filter action which marks a session or
transaction as junked when a filter matches. this with the maildir
junk option allows classifying messages in Spam folder instead of
rejecting/disconnecting.

ok semarie@, eric@, martijn@

Revision 1.633 / (download) - annotate - [select for diffs], Wed Aug 28 15:50:36 2019 UTC (4 years, 9 months ago) by martijn
Branch: MAIN
Changes since 1.632: +5 -1 lines
Diff to previous 1.632 (colored)

Add a link-greeting report. This allows us to get the active domain name
in use for the current request.

OK gilles@

Revision 1.632 / (download) - annotate - [select for diffs], Fri Aug 23 07:09:52 2019 UTC (4 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.631: +2 -1 lines
Diff to previous 1.631 (colored)

res_hnok() is too lenient wrt to acceptable domain name in mail addresses.
replace it with a valid_domainname() check that implements something closer
to RFC 5321, but still usable in real-life.

ok gilles@ millert@

Revision 1.631 / (download) - annotate - [select for diffs], Sat Aug 10 16:07:02 2019 UTC (4 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.630: +2 -1 lines
Diff to previous 1.630 (colored)

this introduces experimental proxy v2 support which is fairly isolated to a
single proxy.c file, importing it to work in tree

initial work from Antoine Kaufmann <toni@famkaufmann.info>

Revision 1.630 / (download) - annotate - [select for diffs], Sat Aug 10 13:38:01 2019 UTC (4 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.629: +2 -2 lines
Diff to previous 1.629 (colored)

bump version

Revision 1.629 / (download) - annotate - [select for diffs], Fri Jul 26 06:30:13 2019 UTC (4 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.628: +4 -4 lines
Diff to previous 1.628 (colored)

even though RSET can be issued outside a tx, RFC states it's noop outside a
transaction so rename link-reset to tx-reset and only issue the smtp report
when a reset _actually_ has a side-effect.

note that rset is implicit on a message commit or rollback, so tx-reset get
issued even though there was no explicit RSET. the filters are MUCH simpler
to write when you don't need to track every event that can reset a tx :-)

Revision 1.628 / (download) - annotate - [select for diffs], Thu Jul 11 21:40:03 2019 UTC (4 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.627: +4 -1 lines
Diff to previous 1.627 (colored)

introduce link-auth to the smtp reporting stream so that filters may know
if a link has been authenticated successfully or not and for which user

Revision 1.627 / (download) - annotate - [select for diffs], Thu Jul 11 21:04:59 2019 UTC (4 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.626: +3 -3 lines
Diff to previous 1.626 (colored)

modify link-identify so it reports if HELO or EHLO was used

Revision 1.626 / (download) - annotate - [select for diffs], Thu Jul 11 20:58:54 2019 UTC (4 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.625: +4 -1 lines
Diff to previous 1.625 (colored)

introduce link-reset to let smtpd report resets happening in a session

Revision 1.625 / (download) - annotate - [select for diffs], Thu Jun 27 05:14:49 2019 UTC (4 years, 11 months ago) by martijn
Branch: MAIN
Changes since 1.624: +4 -1 lines
Diff to previous 1.624 (colored)

Allow filters to log information through stderr. This simplifies and
unifies the way filters need to get their logging to the right location.

Log-messages are read line by line and are logged at LOG_ERR level via
the lookup process.

OK gilles@

Revision 1.624 / (download) - annotate - [select for diffs], Fri Jun 14 19:55:25 2019 UTC (4 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.623: +5 -6 lines
Diff to previous 1.623 (colored)

simplify the runq interface:

- remove (unused) per-job callback
- rename runq_schedule() to runq_schedule_at() and runq_delay()
  to runq_schedule()
- remove unused runq_next()

ok sunil@ gilles@

Revision 1.623 / (download) - annotate - [select for diffs], Thu Jun 13 11:45:35 2019 UTC (4 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.622: +4 -1 lines
Diff to previous 1.622 (colored)

extend the resolver interface to delegate res_query() calls to the lka.

ok gilles@ sunil@

Revision 1.622 / (download) - annotate - [select for diffs], Wed Jun 5 06:40:13 2019 UTC (4 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.621: +4 -3 lines
Diff to previous 1.621 (colored)

a long long time ago, in a galaxy quite close actually, reyk@ introduced an
RSA privsep engine to isolate private keys in the ca process. ECDSA support
in smtpd is become a frequent request so here's an ECDSA privsep engine and
the code required for smtpd to load ECDSA certificates and use them.

Revision 1.621 / (download) - annotate - [select for diffs], Mon Apr 8 08:22:32 2019 UTC (5 years, 1 month ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_6_5_BASE
Branch point for: OPENBSD_6_5
Changes since 1.620: +1 -21 lines
Diff to previous 1.620 (colored)

remove unused declarations

ok gilles@

Revision 1.620 / (download) - annotate - [select for diffs], Thu Feb 28 13:23:32 2019 UTC (5 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.619: +2 -2 lines
Diff to previous 1.619 (colored)

bump smtpd version

on behalf of gilles@

Revision 1.619 / (download) - annotate - [select for diffs], Wed Jan 30 21:31:48 2019 UTC (5 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.618: +1 -2 lines
Diff to previous 1.618 (colored)

revert previous commit, i wasn't happy with it and it probably came from a
misunderstanding.

Revision 1.618 / (download) - annotate - [select for diffs], Wed Jan 30 14:21:01 2019 UTC (5 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.617: +2 -1 lines
Diff to previous 1.617 (colored)

don't be too strict with .forward permissions, it's ok to process it if the
group has write access, it's not ok if the world has write access.

ok eric@

Revision 1.617 / (download) - annotate - [select for diffs], Sat Jan 5 09:48:32 2019 UTC (5 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.616: +2 -3 lines
Diff to previous 1.616 (colored)

revert this change, it was committed by accident

Revision 1.616 / (download) - annotate - [select for diffs], Sat Jan 5 09:43:39 2019 UTC (5 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.615: +6 -2 lines
Diff to previous 1.615 (colored)

introduce smtp 'timeout' reporting event to notify filters that a timeout
occured during the smtp session

Revision 1.615 / (download) - annotate - [select for diffs], Fri Dec 28 15:09:28 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.614: +2 -1 lines
Diff to previous 1.614 (colored)

introduce table_dump() and tweak format

ok gilles@

Revision 1.614 / (download) - annotate - [select for diffs], Fri Dec 28 11:40:29 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.613: +3 -3 lines
Diff to previous 1.613 (colored)

remove the tag workaround for table_create() and table_find(),
now that static tables handle their updates internally.

ok gilles@

Revision 1.613 / (download) - annotate - [select for diffs], Fri Dec 28 11:13:58 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.612: +1 -4 lines
Diff to previous 1.612 (colored)

remove unused members in struct table

Revision 1.612 / (download) - annotate - [select for diffs], Thu Dec 27 15:41:50 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.611: +4 -1 lines
Diff to previous 1.611 (colored)

extract subaddress from last resolved node, not from dest or rcpt address
which was incorrect and can lead to ambiguous cases, this will affect the
people who were using subaddresses within aliases themselves AND expected
deliveries to a maildir subdir of the recipient user.

ok eric@

Revision 1.611 / (download) - annotate - [select for diffs], Thu Dec 27 14:23:41 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.610: +3 -1 lines
Diff to previous 1.610 (colored)

introduce dump() and add() table methods, only implemented for static tables.

ok gilles@

Revision 1.610 / (download) - annotate - [select for diffs], Thu Dec 27 09:30:29 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.609: +3 -3 lines
Diff to previous 1.609 (colored)

pass the table pointer to the lookup()/fecth() methods

ok gilles@

Revision 1.609 / (download) - annotate - [select for diffs], Thu Dec 27 08:57:03 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.608: +2 -2 lines
Diff to previous 1.608 (colored)

change the close() method to take the table pointer

ok gilles

Revision 1.608 / (download) - annotate - [select for diffs], Thu Dec 27 08:08:06 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.607: +2 -2 lines
Diff to previous 1.607 (colored)

Make the backend open method return an int to report success.
The implementation is responsible for setting the handle pointer
as needed.

ok gilles@

Revision 1.607 / (download) - annotate - [select for diffs], Wed Dec 26 20:13:43 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.606: +3 -3 lines
Diff to previous 1.606 (colored)

reorder parameters for consistency

Revision 1.606 / (download) - annotate - [select for diffs], Wed Dec 26 15:55:09 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.605: +2 -1 lines
Diff to previous 1.605 (colored)

introduce a table_match() function to check for a key in a table

ok gilles@

Revision 1.605 / (download) - annotate - [select for diffs], Wed Dec 26 14:15:12 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.604: +5 -5 lines
Diff to previous 1.604 (colored)

get rid of the unused dict argument in table lookup and fetch api.

ok gilles@

Revision 1.604 / (download) - annotate - [select for diffs], Wed Dec 26 11:53:02 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.603: +2 -1 lines
Diff to previous 1.603 (colored)

move the table backend name in the backend struct.
remove unused function.

ok gilles@

Revision 1.603 / (download) - annotate - [select for diffs], Sun Dec 23 16:37:53 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.602: +1 -40 lines
Diff to previous 1.602 (colored)

remove dead code

ok gilles@

Revision 1.602 / (download) - annotate - [select for diffs], Sun Dec 23 15:53:24 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.601: +3 -5 lines
Diff to previous 1.601 (colored)

Simplify the table backend interface: lookup results are returned
as strings, and parsing is handled by the upper layer.

ok gilles@

Revision 1.601 / (download) - annotate - [select for diffs], Sat Dec 22 13:09:05 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.600: +7 -1 lines
Diff to previous 1.600 (colored)

introduce 'rcpt-to' builtin filter, can only be used on 'rcpt-to' hook

Revision 1.600 / (download) - annotate - [select for diffs], Sat Dec 22 12:31:40 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.599: +7 -1 lines
Diff to previous 1.599 (colored)

introduce 'mail-from' builtin filter, can be applied on the 'mail-from',
'rcpt-to', 'data' and 'commit' phases.

Revision 1.599 / (download) - annotate - [select for diffs], Sat Dec 22 12:17:16 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.598: +7 -1 lines
Diff to previous 1.598 (colored)

introduce 'helo' builtin filter, can be used on any hook but 'connect'

Revision 1.598 / (download) - annotate - [select for diffs], Sat Dec 22 08:54:02 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.597: +3 -2 lines
Diff to previous 1.597 (colored)

introduce new matching criteria 'from rdns' to match sessions based on rDNS
of the client, works with literal and tables, both string and regex:

	match from rdns "mx1.poolp.org" for any action blahblah

Revision 1.597 / (download) - annotate - [select for diffs], Fri Dec 21 21:35:29 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.596: +11 -1 lines
Diff to previous 1.596 (colored)

since we already support regex lookups in tables for builtin filters, let's
also support regex lookups in match rule criterias performing table lookups

ok millert@

Revision 1.596 / (download) - annotate - [select for diffs], Fri Dec 21 17:04:46 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.595: +15 -8 lines
Diff to previous 1.595 (colored)

implement some additional builtin filters:
check-src-{table,regex}, check-rdns-{table,regex}

make sure that these builtins may be used at all phases

Revision 1.595 / (download) - annotate - [select for diffs], Fri Dec 21 14:33:52 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.594: +25 -8 lines
Diff to previous 1.594 (colored)

bring in new grammar for filters, allowing filter chains and plugging of
different filters & chains on different interfaces.

in this diff, proc filters are still disabled as they're missing on very
important piece of logic.

ok eric@

Revision 1.594 / (download) - annotate - [select for diffs], Thu Dec 13 17:08:10 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.593: +17 -15 lines
Diff to previous 1.593 (colored)

in event reports, use a struct timeval instead of time_t since we want more
than second precision

discussed with eric@

Revision 1.593 / (download) - annotate - [select for diffs], Wed Dec 12 21:28:39 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.592: +1 -34 lines
Diff to previous 1.592 (colored)

remove unused prototypes

Revision 1.592 / (download) - annotate - [select for diffs], Wed Dec 12 21:27:49 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.591: +4 -1 lines
Diff to previous 1.591 (colored)

add tx-data reporting event

Revision 1.591 / (download) - annotate - [select for diffs], Tue Dec 11 13:40:30 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.590: +6 -6 lines
Diff to previous 1.590 (colored)

do some imsg renaming to make them more clear

Revision 1.590 / (download) - annotate - [select for diffs], Tue Dec 11 13:35:23 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.589: +1 -30 lines
Diff to previous 1.589 (colored)

remove unused imsg names

Revision 1.589 / (download) - annotate - [select for diffs], Tue Dec 11 13:29:52 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.588: +32 -1 lines
Diff to previous 1.588 (colored)

factor smtp-in and smtp-out reporting code

Revision 1.588 / (download) - annotate - [select for diffs], Tue Dec 11 11:29:44 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.587: +5 -1 lines
Diff to previous 1.587 (colored)

report filter responses to smtp

Revision 1.587 / (download) - annotate - [select for diffs], Tue Dec 11 08:40:56 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.586: +6 -1 lines
Diff to previous 1.586 (colored)

generate an event when a helo name identifies a link

Revision 1.586 / (download) - annotate - [select for diffs], Tue Dec 11 07:25:57 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.585: +7 -1 lines
Diff to previous 1.585 (colored)

Improve the cert_*() interface. Use the return value to tell whether
the request is pending (waiting for an async event) or not.  Success
or failure is always reported through the callback function.

ok gilles@

Revision 1.585 / (download) - annotate - [select for diffs], Sun Dec 9 18:05:20 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.584: +4 -1 lines
Diff to previous 1.584 (colored)

add check-fcrdns builtin filter

ok eric@

Revision 1.584 / (download) - annotate - [select for diffs], Sun Dec 9 17:37:15 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.583: +2 -2 lines
Diff to previous 1.583 (colored)

no longer pass rdns in all filtering requests, they can be retrieved from
the filter session.

Revision 1.583 / (download) - annotate - [select for diffs], Sun Dec 9 16:37:51 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.582: +2 -2 lines
Diff to previous 1.582 (colored)

add client and listener address, as well as client rDNS and FCrDNS lookup
result to the filter_session structure upon filter session allocation. it
will allow me to simplify all filter hooks.

Revision 1.582 / (download) - annotate - [select for diffs], Sat Dec 8 08:01:15 2018 UTC (5 years, 5 months ago) by sunil
Branch: MAIN
Changes since 1.581: +4 -4 lines
Diff to previous 1.581 (colored)

Use correct RFC 3464 specified values for Action field in a DSN.
error -> failed
success -> delivered

This fixes DSN parsing for Mailman. Issue reported by Cristiano
Costa on misc@opensmtpd.org.

While here, rename enums to reflect the intent and properly handle
envelope ascii load/dump to understand change in the values.

Suggestions and ok gilles@

Revision 1.581 / (download) - annotate - [select for diffs], Fri Dec 7 08:05:59 2018 UTC (5 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.580: +14 -1 lines
Diff to previous 1.580 (colored)

Refactor certificate initialization and verification.
Factorize code duplicated in smtp_session.c and mta_session.c
Implement a simple callback interface, with proper request management
and simplified imsg protocol.

Only add the necessary parts for now.
Exisiting code path will be adapted later.

input from gilles@ sunil@
ok gilles@

Revision 1.580 / (download) - annotate - [select for diffs], Thu Dec 6 16:05:04 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.579: +7 -4 lines
Diff to previous 1.579 (colored)

link-connect event report had an empty fcrdns field, but now that eric@ has
plugged fcrdns in the smtp_session we can fill the field with a value

Revision 1.579 / (download) - annotate - [select for diffs], Thu Dec 6 15:32:06 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.578: +11 -1 lines
Diff to previous 1.578 (colored)

introduce tx-mail and tx-rcpt report events

Revision 1.578 / (download) - annotate - [select for diffs], Thu Dec 6 13:57:06 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.577: +2 -1 lines
Diff to previous 1.577 (colored)

allow passing data lines to proc filters

ok eric@

Revision 1.577 / (download) - annotate - [select for diffs], Thu Dec 6 12:32:11 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.576: +2 -2 lines
Diff to previous 1.576 (colored)

in mda variables expansions, do not consider empty strings as errors since
an empty %{sender} is really a mailer-daemon and not an error

reported and initial diff by Lauri Tirkkonen <lotheac@iki.fi>
commit is a revised version of the diff based on a discussion with eric@

Revision 1.576 / (download) - annotate - [select for diffs], Thu Dec 6 12:09:50 2018 UTC (5 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.575: +12 -4 lines
Diff to previous 1.575 (colored)

bring the first bits of DATA filtering plumbing but bypass it for now

ok eric@

Revision 1.575 / (download) - annotate - [select for diffs], Fri Nov 30 15:33:40 2018 UTC (5 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.574: +34 -13 lines
Diff to previous 1.574 (colored)

prepare for smtp-out reporting and while at it, make a few changes to the
report format

Revision 1.574 / (download) - annotate - [select for diffs], Thu Nov 29 12:48:16 2018 UTC (5 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.573: +2 -1 lines
Diff to previous 1.573 (colored)

introduce FILTER_COMMIT which will allow taking a decision at DATA commit
time, unusable yet but necessary for the upcoming serie of diffs.

ok eric@

Revision 1.573 / (download) - annotate - [select for diffs], Thu Nov 8 13:21:00 2018 UTC (5 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.572: +8 -5 lines
Diff to previous 1.572 (colored)

when reporting tx events, report tx id
when reporting tx commit, report data size
report tx-envelope events

Revision 1.572 / (download) - annotate - [select for diffs], Sat Nov 3 14:39:46 2018 UTC (5 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.571: +2 -2 lines
Diff to previous 1.571 (colored)

only apply filter rules to filtered interfaces

Revision 1.571 / (download) - annotate - [select for diffs], Sat Nov 3 13:56:49 2018 UTC (5 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.570: +5 -2 lines
Diff to previous 1.570 (colored)

check-rdns builtin filter, to be improved

Revision 1.570 / (download) - annotate - [select for diffs], Sat Nov 3 13:47:46 2018 UTC (5 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.569: +3 -2 lines
Diff to previous 1.569 (colored)

bring plumbing for proc filters

ok millert@, eric@, jung@

Revision 1.569 / (download) - annotate - [select for diffs], Sat Nov 3 13:42:24 2018 UTC (5 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.568: +49 -1 lines
Diff to previous 1.568 (colored)

bring plumbing for builtin filters

ok millert@, eric@, jung@

Revision 1.568 / (download) - annotate - [select for diffs], Fri Nov 2 17:20:22 2018 UTC (5 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.567: +3 -3 lines
Diff to previous 1.567 (colored)

report rDNS in link connect event

Revision 1.567 / (download) - annotate - [select for diffs], Fri Nov 2 16:50:23 2018 UTC (5 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.566: +3 -3 lines
Diff to previous 1.566 (colored)

pass struct sockaddr_storage instead of ss_to_text() in reporting

Revision 1.566 / (download) - annotate - [select for diffs], Thu Nov 1 14:48:49 2018 UTC (5 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.565: +36 -1 lines
Diff to previous 1.565 (colored)

teach smtp process how to report smtp events to lka and teach lka how to
report these events to a proc

ok millert@

Revision 1.565 / (download) - annotate - [select for diffs], Thu Nov 1 10:47:46 2018 UTC (5 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.564: +2 -1 lines
Diff to previous 1.564 (colored)

introduce K_REGEX table type and table_regex_match(), unused for now

ok eric@

Revision 1.564 / (download) - annotate - [select for diffs], Thu Nov 1 10:13:25 2018 UTC (5 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.563: +17 -2 lines
Diff to previous 1.563 (colored)

allow smtpd to fork processes at startup and maintain a socketpair with
them.

ok jung@, eric@

Revision 1.563 / (download) - annotate - [select for diffs], Wed Oct 31 16:32:12 2018 UTC (5 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.562: +2 -1 lines
Diff to previous 1.562 (colored)

add helper valid_smtp_response() to be used in upcoming commits

Revision 1.562 / (download) - annotate - [select for diffs], Mon Sep 24 16:14:34 2018 UTC (5 years, 8 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_6_4_BASE, OPENBSD_6_4
Changes since 1.561: +2 -1 lines
Diff to previous 1.561 (colored)

Allow to use the "tls" keyword on any relay action to force TLS, with
strict certificate validation.  The "no-verify" becomes optional.

ok gilles@ millert@ semarie@

Revision 1.561 / (download) - annotate - [select for diffs], Wed Sep 19 05:31:12 2018 UTC (5 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.560: +2 -3 lines
Diff to previous 1.560 (colored)

rename the ill-named "flags" member to "as_host" in domain structure.
remove yet another useless relay flag while there.

ok gilles@

Revision 1.560 / (download) - annotate - [select for diffs], Mon Sep 17 12:16:27 2018 UTC (5 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.559: +1 -2 lines
Diff to previous 1.559 (colored)

simplify code path for backup relay and remove useless flag

ok gilles@

Revision 1.559 / (download) - annotate - [select for diffs], Sat Sep 8 10:05:07 2018 UTC (5 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.558: +7 -5 lines
Diff to previous 1.558 (colored)

use symbolic integer values for the different tls options when relaying,
rather than a confusing set of flags.

ok gilles@

Revision 1.558 / (download) - annotate - [select for diffs], Tue Sep 4 13:04:42 2018 UTC (5 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.557: +7 -1 lines
Diff to previous 1.557 (colored)

upon mda failure, smtpd would assume tempfail and retry. this is at odds
with the other MTA which assume a permfail unless the exit status is one
of a specific set. make smtpd honour the same exit statuses as postfix.

note that all errors that occur before the user mda is executed (fork, pipe
and related) are still considered tempfail, only errors coming from the mda
itself are handled as permfail.

this commit is a temporary solution as i believe the SIGCHLD handler is way
more complex than it should be and we'll simplify it after 6.4 is out.

ok eric@

Revision 1.557 / (download) - annotate - [select for diffs], Fri Aug 31 07:28:27 2018 UTC (5 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.556: +1 -3 lines
Diff to previous 1.556 (colored)

switch to improved incoming message parser:
- simpler interface not using callbacks
- no hard-coded line length
- avoid unnecessary string copy

ok gilles@

Revision 1.556 / (download) - annotate - [select for diffs], Wed Jul 25 16:00:48 2018 UTC (5 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.555: +15 -3 lines
Diff to previous 1.555 (colored)

Implement a generic interface to forward resolver queries to the lka
process.  Use it for the reverse lookups required by smtp and mta.

Until now, DNS-related lookups were implemented using ad-hoc IMSGs
between the lka and other processes. It turns out to be confusing and
difficult to maintain/extend.  So we want to replace this with a better
set of IMSGs matching the standard resolver interface.

ok gilles@

Revision 1.555 / (download) - annotate - [select for diffs], Mon Jun 18 18:19:14 2018 UTC (5 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.554: +2 -1 lines
Diff to previous 1.554 (colored)

simplify parse_config() further so it no longer has any side effect outside
of parse.y, there's still work to be done but it's now able to run twice if
we want (we don't) without failing due to some global side-effect.

ok millert@

Revision 1.554 / (download) - annotate - [select for diffs], Mon Jun 18 18:14:39 2018 UTC (5 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.553: +2 -2 lines
Diff to previous 1.553 (colored)

split smtp_accept() in two parts: the accept part, the session init part,
while at it allow smtp_session() to receive a pre-allocated struct io

ok millert@
diff contributed by Antoine Kaufmann

Revision 1.553 / (download) - annotate - [select for diffs], Sat Jun 16 19:41:26 2018 UTC (5 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.552: +7 -7 lines
Diff to previous 1.552 (colored)

rework the table API so that it takes a struct smtpd * context in parameter
of functions creating, looking up or destroying tables.

this is a first step in cleaning up parse.y so it doesn't have side effects
outside of parse_config(), bringing nothing but making code cleaner.

ok millert@

Revision 1.552 / (download) - annotate - [select for diffs], Thu Jun 7 16:28:14 2018 UTC (5 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.551: +3 -6 lines
Diff to previous 1.551 (colored)

remove unused flags and obsolete comments

ok gilles@

Revision 1.551 / (download) - annotate - [select for diffs], Wed Jun 6 12:00:26 2018 UTC (5 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.550: +1 -7 lines
Diff to previous 1.550 (colored)

remove fields that are found in struct dispatcher from struct relayhost

ok gilles@

Revision 1.550 / (download) - annotate - [select for diffs], Tue Jun 5 11:34:21 2018 UTC (5 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.549: +1 -6 lines
Diff to previous 1.549 (colored)

remove struct relayhost from struct envelope.

ok gilles@

Revision 1.549 / (download) - annotate - [select for diffs], Mon Jun 4 15:57:46 2018 UTC (5 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.548: +4 -2 lines
Diff to previous 1.548 (colored)

add support for mda wrappers allowing postmaster to define command wrappers
that will be executed (with recipient privileges) before calling the users'
mail delivery agent

ok eric@

Revision 1.548 / (download) - annotate - [select for diffs], Sun Jun 3 14:04:06 2018 UTC (6 years ago) by gilles
Branch: MAIN
Changes since 1.547: +5 -1 lines
Diff to previous 1.547 (colored)

split forkmda() in two:
- forkmda() creates the process that will be used for the delivery and does
  the switching of privileges then calls mda_unpriv()
- mda_unpriv() runs with privileges of the recipient, it expands variables,
  sets up environment and executes the mda

ok millert@ and eric@

Revision 1.547 / (download) - annotate - [select for diffs], Fri Jun 1 12:24:16 2018 UTC (6 years ago) by eric
Branch: MAIN
Changes since 1.546: +2 -1 lines
Diff to previous 1.546 (colored)

Require a valid certificate by default when relaying through a smarthost.
Add "tls no-verify" relay option to disable it.

suggested and initial diff by semarie@.

ok gilles@

Revision 1.546 / (download) - annotate - [select for diffs], Thu May 31 21:06:12 2018 UTC (6 years ago) by gilles
Branch: MAIN
Changes since 1.545: +6 -6 lines
Diff to previous 1.545 (colored)

remove 'where' parameter from all x*() functions in utils.c, it doesn't
really help us with anything, propagate the change in codebase

ok millert@

Revision 1.545 / (download) - annotate - [select for diffs], Tue May 29 21:05:52 2018 UTC (6 years ago) by eric
Branch: MAIN
Changes since 1.544: +1 -2 lines
Diff to previous 1.544 (colored)

remove unused function

ok gilles@

Revision 1.544 / (download) - annotate - [select for diffs], Tue May 29 20:43:07 2018 UTC (6 years ago) by eric
Branch: MAIN
Changes since 1.543: +2 -2 lines
Diff to previous 1.543 (colored)

no need to parse and dump the relayhost in the lookup process.

ok gilles@

Revision 1.543 / (download) - annotate - [select for diffs], Tue May 29 18:16:14 2018 UTC (6 years ago) by gilles
Branch: MAIN
Changes since 1.542: +2 -1 lines
Diff to previous 1.542 (colored)

provide mail user agents with the same environments as Postfix

ok millert@, eric@

Revision 1.542 / (download) - annotate - [select for diffs], Thu May 24 12:22:00 2018 UTC (6 years ago) by gilles
Branch: MAIN
Changes since 1.541: +2 -2 lines
Diff to previous 1.541 (colored)

bump version, this will be a big release when OpenBSD 6.4 is released :-)

Revision 1.541 / (download) - annotate - [select for diffs], Thu May 24 11:38:24 2018 UTC (6 years ago) by gilles
Branch: MAIN
Changes since 1.540: +113 -95 lines
Diff to previous 1.540 (colored)

switch smtpd to new grammar

ok eric@

Revision 1.540 / (download) - annotate - [select for diffs], Mon May 14 15:23:05 2018 UTC (6 years ago) by gilles
Branch: MAIN
Changes since 1.539: +1 -4 lines
Diff to previous 1.539 (colored)

kill corrupt / uncorrupt queue mechanism as it has never been usable and it
will be made irrelevant when the new config comes up soon

ok eric@

Revision 1.539 / (download) - annotate - [select for diffs], Thu Apr 26 20:57:59 2018 UTC (6 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.538: +4 -1 lines
Diff to previous 1.538 (colored)

sync log.h with other daemons

ok gilles@

Revision 1.538 / (download) - annotate - [select for diffs], Wed Mar 14 22:25:21 2018 UTC (6 years, 2 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_6_3_BASE, OPENBSD_6_3
Changes since 1.537: +2 -2 lines
Diff to previous 1.537 (colored)

bump minor version just to be sure it makes release :-)

ok gilles@

Revision 1.537 / (download) - annotate - [select for diffs], Tue Jan 2 19:11:06 2018 UTC (6 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.536: +2 -2 lines
Diff to previous 1.536 (colored)

we haven't updated the version in a while despite many commits which is
confusing for people running the portable version

Revision 1.536 / (download) - annotate - [select for diffs], Fri Sep 8 16:51:22 2017 UTC (6 years, 8 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_6_2_BASE, OPENBSD_6_2
Changes since 1.535: +1 -21 lines
Diff to previous 1.535 (colored)

remove more filter-related cruft

ok gilles@

Revision 1.535 / (download) - annotate - [select for diffs], Sun Aug 13 11:10:30 2017 UTC (6 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.534: +1 -13 lines
Diff to previous 1.534 (colored)

bypass the filter code for incoming smtp sessions.
experimental support for filters has been removed from the config
parser already, and we want to get rid of the remaining code.

ok gilles@

Revision 1.534 / (download) - annotate - [select for diffs], Fri Aug 4 14:38:49 2017 UTC (6 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.533: +2 -2 lines
Diff to previous 1.533 (colored)

the PURGE_EVERYTHING flag used to purge config bits was inaccurate

ok eric@

Revision 1.533 / (download) - annotate - [select for diffs], Thu Jul 27 18:48:30 2017 UTC (6 years, 10 months ago) by sunil
Branch: MAIN
Changes since 1.532: +5 -2 lines
Diff to previous 1.532 (colored)

smtpctl(8): Use an int to determine mode instead of __progname.

Ok millert@ gilles@

Revision 1.532 / (download) - annotate - [select for diffs], Fri May 26 21:30:00 2017 UTC (7 years ago) by gilles
Branch: MAIN
Changes since 1.531: +6 -1 lines
Diff to previous 1.531 (colored)

move variables expansion out of lka_session into their own file, this is a
mechanical diff to simplify a bit the lka code and prepare for moving
variables outside of the lookup process into the chrooted mda process.
no functional change for now.

ok eric@

Revision 1.531 / (download) - annotate - [select for diffs], Mon May 22 13:43:15 2017 UTC (7 years ago) by gilles
Branch: MAIN
Changes since 1.530: +7 -1 lines
Diff to previous 1.530 (colored)

- filters are currently broken, do not allow using them until we're done

Revision 1.530 / (download) - annotate - [select for diffs], Mon Feb 13 12:23:47 2017 UTC (7 years, 3 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_6_1_BASE, OPENBSD_6_1
Changes since 1.529: +3 -2 lines
Diff to previous 1.529 (colored)

allow negation of authenticated keyword:
	accept ! authenticated [...]

ok sunil@, jung@

Revision 1.529 / (download) - annotate - [select for diffs], Fri Feb 3 08:23:46 2017 UTC (7 years, 3 months ago) by guenther
Branch: MAIN
Changes since 1.528: +2 -1 lines
Diff to previous 1.528 (colored)

Stop assuming that in_{addr,port}_t are typedefed in <sys/types.h> and
instead pull in <netinet/in.h> or <arpa/inet.h> when those are needed.

ok florian@ beck@ millert@

Revision 1.528 / (download) - annotate - [select for diffs], Mon Jan 9 09:53:23 2017 UTC (7 years, 4 months ago) by reyk
Branch: MAIN
Changes since 1.527: +2 -2 lines
Diff to previous 1.527 (colored)

smtpd joins the 7 other daemons that share the same log.c file.

The only major difference was the "log_trace" concept that is only
used by smtpd - move it from log.c into util.c and make it a local
concept.  This also needed to rename the global "verbose" variable to
"tracing" in a few places.

OK krw@ gilles@ eric@

Revision 1.527 / (download) - annotate - [select for diffs], Wed Nov 30 18:38:32 2016 UTC (7 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.526: +1 -4 lines
Diff to previous 1.526 (colored)

remove unused iobuf helpers

Revision 1.526 / (download) - annotate - [select for diffs], Wed Nov 30 11:52:48 2016 UTC (7 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.525: +3 -1 lines
Diff to previous 1.525 (colored)

make struct io opaque:

- move struct io definition to ioev.c
- replace io_init/io_clear with io_new/io_free
- allocate an iobuf for each new io internally
- use struct io pointer in the rest of the code
- remove remaining uses of iobuf_*

ok gilles@ sunil@

Revision 1.525 / (download) - annotate - [select for diffs], Fri Nov 25 09:21:21 2016 UTC (7 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.524: +2 -1 lines
Diff to previous 1.524 (colored)

assign an id to each rule in the ruleset, first step towards an MTA layer
and scheduler simplification

ok eric@

Revision 1.524 / (download) - annotate - [select for diffs], Thu Nov 17 17:34:55 2016 UTC (7 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.523: +3 -1 lines
Diff to previous 1.523 (colored)

Add io api functions for dealing with buffered data, as wrapper around
their iobuf counterparts.

ok gilles@ sunil@

Revision 1.523 / (download) - annotate - [select for diffs], Sun Sep 4 09:33:49 2016 UTC (7 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.522: +3 -4 lines
Diff to previous 1.522 (colored)

Remove the "smtpctl stop" command.
The daemon is stopped with kill(1).

ok gilles@

Revision 1.522 / (download) - annotate - [select for diffs], Sat Sep 3 16:06:26 2016 UTC (7 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.521: +2 -2 lines
Diff to previous 1.521 (colored)

get rid of the type-checking system on internal messages.
bump all imsg protocol versions since message format changed.

ok gilles@ sunil@

Revision 1.521 / (download) - annotate - [select for diffs], Thu Sep 1 10:54:25 2016 UTC (7 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.520: +1 -2 lines
Diff to previous 1.520 (colored)

remove noop function

ok sunil@

Revision 1.520 / (download) - annotate - [select for diffs], Thu Sep 1 10:07:20 2016 UTC (7 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.519: +1 -9 lines
Diff to previous 1.519 (colored)

get rid of the imsg buffer usage profiling code.

ok gilles@ jung@ sunil@

Revision 1.519 / (download) - annotate - [select for diffs], Wed Aug 31 21:49:01 2016 UTC (7 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.518: +1 -2 lines
Diff to previous 1.518 (colored)

Remove dead code. queue_flow_control() has never been used and is
probably a bad idea.

ok gilles@

Revision 1.518 / (download) - annotate - [select for diffs], Wed Aug 31 15:24:04 2016 UTC (7 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.517: +3 -1 lines
Diff to previous 1.517 (colored)

introduce "authenticated" parameter so rules may apply to authenticated
sessions specifically

ok eric@, sunil@, jung@

Revision 1.517 / (download) - annotate - [select for diffs], Wed Aug 31 10:18:08 2016 UTC (7 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.516: +3 -1 lines
Diff to previous 1.516 (colored)

allow overriding the subaddressing delimiter with subaddressing-delimiter
keyword, the default is still +

ok eric@, sunil@

Revision 1.516 / (download) - annotate - [select for diffs], Sun Jul 24 16:04:53 2016 UTC (7 years, 10 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_6_0_BASE, OPENBSD_6_0
Changes since 1.515: +2 -2 lines
Diff to previous 1.515 (colored)

bump version

ok deraadt@

Revision 1.515 / (download) - annotate - [select for diffs], Sat May 28 21:21:20 2016 UTC (8 years ago) by eric
Branch: MAIN
Changes since 1.514: +11 -9 lines
Diff to previous 1.514 (colored)

Implement the fork+exec pattern in smtpd.

The parent process forks child processes and re-exec each of them with
an additional "-x <proc>" argument.  During the early setup phase, the
parent process sends ipc socket pairs to interconnect the child
processes as needed, and it passes the queue encryption key to the
queue if necessary. When this is done, all processes have their
environment set as in the fork-only case, and they can start doing
their work as before.

ok gilles@ jung@

Revision 1.514 / (download) - annotate - [select for diffs], Fri Mar 25 15:06:58 2016 UTC (8 years, 2 months ago) by krw
Branch: MAIN
Changes since 1.513: +1 -8 lines
Diff to previous 1.513 (colored)

Nuke session_socket_blockmode() and session_socket_linger(). Use
the identical io_set_blocking() and io_set_linger().

Since both are always called to turn off blocking or lingering,
nuke the parameter and associated enum in favour of "just doing the
right thing".

While passing remove the unneeded last parameter to the remaining
fcntl(F_GETFL).

Finally, rename the functions to io_set_nonblocking() and
io_set_nolinger() for clarity.

No functional change.

Started with a sweep of fcntl() usage inspired by guenther@.

ok gilles@

Revision 1.513 / (download) - annotate - [select for diffs], Sun Feb 21 15:17:25 2016 UTC (8 years, 3 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_9_BASE, OPENBSD_5_9
Changes since 1.512: +2 -2 lines
Diff to previous 1.512 (colored)

bump version

Revision 1.512 / (download) - annotate - [select for diffs], Sat Feb 13 20:43:07 2016 UTC (8 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.511: +4 -1 lines
Diff to previous 1.511 (colored)

handle enqueuer socket as a regular listener that can be configured with
"listen on socket". this simplifies a bit of code, removes some special
cases and will allow attaching filters & masking source just as on lo0.

diff from Peter Bisroev <peter@int19h.net>
ok gilles@, jung@

Revision 1.511 / (download) - annotate - [select for diffs], Fri Feb 5 19:21:04 2016 UTC (8 years, 3 months ago) by jung
Branch: MAIN
Changes since 1.510: +1 -3 lines
Diff to previous 1.510 (colored)

remove no longer relevant ifndef

suggested by gilles

Revision 1.510 / (download) - annotate - [select for diffs], Wed Jan 27 12:46:03 2016 UTC (8 years, 4 months ago) by sunil
Branch: MAIN
Changes since 1.509: +8 -1 lines
Diff to previous 1.509 (colored)

Check imsg data length before use.

Ok jung@ gilles@ eric@

Revision 1.509 / (download) - annotate - [select for diffs], Mon Jan 4 13:30:20 2016 UTC (8 years, 4 months ago) by jung
Branch: MAIN
Changes since 1.508: +2 -3 lines
Diff to previous 1.508 (colored)

switch to /usr/local/libexec when looking for -extras and drop loop iterating
paths

this effectively reverts table.c r1.21 which was mainly introduced for a smooth
transition in -current

ok gilles

Revision 1.508 / (download) - annotate - [select for diffs], Sun Dec 13 11:06:13 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.507: +2 -2 lines
Diff to previous 1.507 (colored)

smtpd is no longer 5.4.6

Revision 1.507 / (download) - annotate - [select for diffs], Sun Dec 13 09:52:44 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.506: +2 -2 lines
Diff to previous 1.506 (colored)

refactor a bit to move the SNI handling away from smtp_session into smtp

ok sunil@, jung@

Revision 1.506 / (download) - annotate - [select for diffs], Sat Dec 12 17:16:56 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.505: +4 -2 lines
Diff to previous 1.505 (colored)

allow overriding the default cipher-suite

ok jung@, sunil@, millert@

Revision 1.505 / (download) - annotate - [select for diffs], Sat Dec 12 12:22:26 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.504: +3 -1 lines
Diff to previous 1.504 (colored)

add bits so local enqueuer can run filters when they are enabled

Revision 1.504 / (download) - annotate - [select for diffs], Sat Dec 12 10:52:22 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.503: +2 -1 lines
Diff to previous 1.503 (colored)

expose foreground_log in smtpd.h so filters can inherit it

Revision 1.503 / (download) - annotate - [select for diffs], Sat Dec 12 10:03:33 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.502: +4 -4 lines
Diff to previous 1.502 (colored)

sender and recipient are mail addresses, not pathnames, use proper define

Revision 1.502 / (download) - annotate - [select for diffs], Sat Dec 12 10:00:59 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.501: +3 -3 lines
Diff to previous 1.501 (colored)

use smtpd specific define for table name sizes

Revision 1.501 / (download) - annotate - [select for diffs], Sat Dec 12 09:58:33 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.500: +3 -3 lines
Diff to previous 1.500 (colored)

pki name and ca name must match a hostname, use HOST_NAME_MAX

Revision 1.500 / (download) - annotate - [select for diffs], Sat Dec 12 09:09:40 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.499: +8 -6 lines
Diff to previous 1.499 (colored)

do not limit usernames to LOGIN_NAME_MAX in places where "virtual users"
may be used, in such places an email address may be specified.

Revision 1.499 / (download) - annotate - [select for diffs], Sat Dec 12 09:00:22 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.498: +4 -5 lines
Diff to previous 1.498 (colored)

rename an smtpd specific define

Revision 1.498 / (download) - annotate - [select for diffs], Sat Dec 12 08:35:19 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.497: +4 -1 lines
Diff to previous 1.497 (colored)

prepare some imsg structures for upcoming diff to support wildcard ca

Revision 1.497 / (download) - annotate - [select for diffs], Fri Dec 11 21:44:01 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.496: +8 -2 lines
Diff to previous 1.496 (colored)

rename field member + whitespaces

Revision 1.496 / (download) - annotate - [select for diffs], Fri Dec 11 07:51:38 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.495: +17 -1 lines
Diff to previous 1.495 (colored)

add filter.c prototypes, unused for now

Revision 1.495 / (download) - annotate - [select for diffs], Fri Dec 11 07:48:33 2015 UTC (8 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.494: +3 -1 lines
Diff to previous 1.494 (colored)

prepare smtpd.h for masquerading

Revision 1.494 / (download) - annotate - [select for diffs], Mon Dec 7 12:29:19 2015 UTC (8 years, 5 months ago) by sunil
Branch: MAIN
Changes since 1.493: +3 -1 lines
Diff to previous 1.493 (colored)

Merge makemap(8) into smtpctl(8).

Ok gilles@, jung@

Revision 1.493 / (download) - annotate - [select for diffs], Thu Dec 3 21:11:33 2015 UTC (8 years, 6 months ago) by jung
Branch: MAIN
Changes since 1.492: +4 -1 lines
Diff to previous 1.492 (colored)

introduce limit session keyword replacing fixed values

original diff from Renaud Allard

ok gilles

Revision 1.492 / (download) - annotate - [select for diffs], Tue Dec 1 18:22:30 2015 UTC (8 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.491: +2 -1 lines
Diff to previous 1.491 (colored)

add received-auth parameter to listener to identify authenticated sessions
in locally appended Received header when enabled

ok millert@, jung@

Revision 1.491 / (download) - annotate - [select for diffs], Tue Dec 1 09:39:07 2015 UTC (8 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.490: +2 -1 lines
Diff to previous 1.490 (colored)

add IMSG_SMTP_CHECK_SENDER in preparation for another diff

Revision 1.490 / (download) - annotate - [select for diffs], Tue Dec 1 09:33:17 2015 UTC (8 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.489: +3 -2 lines
Diff to previous 1.489 (colored)

prepare the ground for the CA certificate handling refactor, this commit
adds the parse.y bit + structures & members needed but does not make use
of it yet

Revision 1.489 / (download) - annotate - [select for diffs], Mon Nov 30 14:27:25 2015 UTC (8 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.488: +4 -1 lines
Diff to previous 1.488 (colored)

add ca_name field to some structures in preparation for a larger
diff to refactor handling of CA certificates

Revision 1.488 / (download) - annotate - [select for diffs], Mon Nov 30 14:13:03 2015 UTC (8 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.487: +3 -2 lines
Diff to previous 1.487 (colored)

when looking up tables, start in /usr/local/libexec before /usr/libexec, so
ports/packages can be installed in the proper place

ok jung@

Revision 1.487 / (download) - annotate - [select for diffs], Mon Nov 30 12:49:35 2015 UTC (8 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.486: +9 -9 lines
Diff to previous 1.486 (colored)

mechanical rename of some IMSG constants

ok sunil@, ok jung@

Revision 1.486 / (download) - annotate - [select for diffs], Mon Nov 30 12:26:55 2015 UTC (8 years, 6 months ago) by sunil
Branch: MAIN
Changes since 1.485: +3 -1 lines
Diff to previous 1.485 (colored)

While delivering to lmtp or mda, accept optional "as user" parameter
whose privileges would be used instead of the default.

Ok gilles@ jung@

Revision 1.485 / (download) - annotate - [select for diffs], Mon Nov 23 21:50:12 2015 UTC (8 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.484: +2 -1 lines
Diff to previous 1.484 (colored)

allow table API to lookup for mailaddr mappings

ok sunil@, ok jung@

Revision 1.484 / (download) - annotate - [select for diffs], Thu Nov 5 09:14:31 2015 UTC (8 years, 6 months ago) by sunil
Branch: MAIN
Changes since 1.483: +4 -2 lines
Diff to previous 1.483 (colored)

Implement smtpctl uncorrupt <msgid>

"uncorrupt" moves envelopes from corrupt bucket back to the queue
for further discovery by the daemon.

After correcting the corrupt envelopes, admin could now...

# smtpctl uncorrupt msgid
# smtpctl discover msgid

to schedule the messages.

Ok gilles@

Revision 1.483 / (download) - annotate - [select for diffs], Thu Oct 29 10:25:36 2015 UTC (8 years, 7 months ago) by sunil
Branch: MAIN
Changes since 1.482: +16 -3 lines
Diff to previous 1.482 (colored)

Implement smtpctl discover <evpid|msgid>.

discover subcommand schedules envelopes manually moved to the queue.
It triggers a queue walk searching for envelopes with the given id,
schedules them and informs the user number of envelopes scheduled.
Admins no longer would need to restart the daemon to discover
manually moved messages.

Ok gilles@

Revision 1.482 / (download) - annotate - [select for diffs], Wed Oct 28 14:30:03 2015 UTC (8 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.481: +17 -1 lines
Diff to previous 1.481 (colored)

masquerade and senders map require being able to lookup mailaddr structures
in tables, prepare for such features by bringing the helpers + smtpd.h part

ok millert@

Revision 1.481 / (download) - annotate - [select for diffs], Wed Oct 28 07:25:30 2015 UTC (8 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.480: +3 -2 lines
Diff to previous 1.480 (colored)

aliases support resolving to maildir:/path

ok sunil@ millert@

Revision 1.480 / (download) - annotate - [select for diffs], Tue Oct 27 20:14:19 2015 UTC (8 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.479: +1 -2 lines
Diff to previous 1.479 (colored)

aliases_virtual_check() has been unneeded for a while

ok jung@, ok sunil@, ok millert@

Revision 1.479 / (download) - annotate - [select for diffs], Wed Oct 21 16:44:28 2015 UTC (8 years, 7 months ago) by jsing
Branch: MAIN
Changes since 1.478: +2 -2 lines
Diff to previous 1.478 (colored)

Only enable SSL_VERIFY_PEER when the verify option is set on a listener.

Always enabling SSL_VERIFY_PEER unnecessarily increases the number of
messages/bytes in the TLS handshake and increases our attack surface,
since we request and then process client certificates.

ok gilles@

Revision 1.478 / (download) - annotate - [select for diffs], Sat Oct 17 22:24:36 2015 UTC (8 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.477: +2 -1 lines
Diff to previous 1.477 (colored)

mailaddr_match() allows comparing two struct mailaddr taking into account
catchall and +-tags

ok millert@ and jung@ for util.c

Revision 1.477 / (download) - annotate - [select for diffs], Wed Oct 14 22:16:38 2015 UTC (8 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.476: +2 -2 lines
Diff to previous 1.476 (colored)

smtpd in tree is no longer neither 5.4.4, nor 5.4.5, bump SMTPD_VERSION

Revision 1.476 / (download) - annotate - [select for diffs], Fri Oct 9 14:37:38 2015 UTC (8 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.475: +2 -2 lines
Diff to previous 1.475 (colored)

turn our local enqueuer setgid _smtpq and restrict access to offline queue,
the enqueuer will revoke group and regain real gid right after mkstemp.

this would have prevented the symlink/hardlink attacks against offline, and
it will avoid having to deal with new ways users can mess with it.

ok eric@, ok millert@

Revision 1.475 / (download) - annotate - [select for diffs], Mon Sep 7 15:36:53 2015 UTC (8 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.474: +3 -1 lines
Diff to previous 1.474 (colored)

when bypassing the enqueuer, insert Message-Id header if none was found and
the client has connected from a loopback interface.

ok millert@ eric@

Revision 1.473.2.1 / (download) - annotate - [select for diffs], Sun Apr 19 20:30:33 2015 UTC (9 years, 1 month ago) by gilles
Branch: OPENBSD_5_7
Changes since 1.473: +2 -2 lines
Diff to previous 1.473 (colored) next main 1.474 (colored)

Incorrect logic in smtpd(8) can lead to unexpected client disconnect, invalid
certificate in SNI negotiation or server crash.

spotted by Edwin Torok

Revision 1.468.4.1 / (download) - annotate - [select for diffs], Sun Apr 19 20:30:06 2015 UTC (9 years, 1 month ago) by gilles
Branch: OPENBSD_5_6
Changes since 1.468: +2 -2 lines
Diff to previous 1.468 (colored) next main 1.469 (colored)

Incorrect logic in smtpd(8) can lead to unexpected client disconnect, invalid
certificate in SNI negotiation or server crash.

spotted by Edwin Torok

Revision 1.450.2.1 / (download) - annotate - [select for diffs], Sun Apr 19 20:29:43 2015 UTC (9 years, 1 month ago) by gilles
Branch: OPENBSD_5_5
Changes since 1.450: +2 -2 lines
Diff to previous 1.450 (colored) next main 1.451 (colored)

Incorrect logic in smtpd(8) can lead to unexpected client disconnect, invalid
certificate in SNI negotiation or server crash.

spotted by Edwin Torok

Revision 1.474 / (download) - annotate - [select for diffs], Sun Apr 19 20:29:12 2015 UTC (9 years, 1 month ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_8_BASE, OPENBSD_5_8
Changes since 1.473: +2 -2 lines
Diff to previous 1.473 (colored)

Incorrect logic in smtpd(8) can lead to unexpected client disconnect, invalid
certificate in SNI negotiation or server crash.

spotted by Edwin Torok

Revision 1.473 / (download) - annotate - [select for diffs], Tue Jan 20 17:37:54 2015 UTC (9 years, 4 months ago) by deraadt
Branch: MAIN
CVS Tags: OPENBSD_5_7_BASE
Branch point for: OPENBSD_5_7
Changes since 1.472: +38 -38 lines
Diff to previous 1.472 (colored)

use <limits.h> comprehensively.  For now try to push <> includes to
each .c file, and out of the .h files.  To avoid overinclude.
ok gilles, in principle.  If this has been done right, -portable should
become easier to maintain.

Revision 1.472 / (download) - annotate - [select for diffs], Wed Dec 24 13:53:39 2014 UTC (9 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.471: +2 -2 lines
Diff to previous 1.471 (colored)

bump version

Revision 1.471 / (download) - annotate - [select for diffs], Sun Dec 14 15:26:56 2014 UTC (9 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.470: +1 -2 lines
Diff to previous 1.470 (colored)

these are no longer used, remove

Revision 1.470 / (download) - annotate - [select for diffs], Sun Nov 16 19:07:50 2014 UTC (9 years, 6 months ago) by bluhm
Branch: MAIN
Changes since 1.469: +2 -1 lines
Diff to previous 1.469 (colored)

Convert the logic in yyerror().  Instead of creating a temporary
format string, create a temporary message.
OK doug@

Revision 1.469 / (download) - annotate - [select for diffs], Wed Oct 15 08:09:02 2014 UTC (9 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.468: +4 -1 lines
Diff to previous 1.468 (colored)

when From, To and Cc headers present users without domains, append the
listener hostname to avoid smtpd relaying a header that will be rewritten
by the destination MX.

ok eric@

Revision 1.468 / (download) - annotate - [select for diffs], Thu Jul 10 14:45:02 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_6_BASE
Branch point for: OPENBSD_5_6
Changes since 1.467: +5 -5 lines
Diff to previous 1.467 (colored)

Improve the scheduler, better and simpler.

- Get rid of the scheduler_batch structure. The scheduler can now return
  envelopes of different types in a single run, interlacing them to avoid
  batch effects.

- Ask for an acknowledgement from the queue when removing or expiring
  an envelope to benefit from the inflight envelope limitation mechanism.
  This ensures that the scheduler always keeps sending envelopes at a rate
  that the queue can sustain in all cases.

- Limit the number of envelopes in a holdq.  When a holdq is full,
  new envelopes are put back in the pending queue instead, with a
  shorter retry time.

- Plumbing for proc-ified schedulers.

imsg version bump. smtpctl stop before updating.

ok gilles@

Revision 1.467 / (download) - annotate - [select for diffs], Wed Jul 9 12:44:54 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.466: +2 -1 lines
Diff to previous 1.466 (colored)

add a "no-dsn" listener option to disable DSN extension.

Revision 1.466 / (download) - annotate - [select for diffs], Wed Jul 9 09:53:37 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.465: +2 -1 lines
Diff to previous 1.465 (colored)

config parser improvements:

- fail if the same option is specified multiple times on a listener
- prompt for queue encryption key after config parsing, not during.
- add ip addresses to localnames table
- prepare for filters

Revision 1.465 / (download) - annotate - [select for diffs], Tue Jul 8 21:58:33 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.464: +9 -7 lines
Diff to previous 1.464 (colored)

update filter configuration parsing (not plugged yet)

Revision 1.464 / (download) - annotate - [select for diffs], Tue Jul 8 20:14:46 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.463: +2 -26 lines
Diff to previous 1.463 (colored)

get rid of mfa leftovers

Revision 1.463 / (download) - annotate - [select for diffs], Tue Jul 8 15:45:32 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.462: +3 -2 lines
Diff to previous 1.462 (colored)

various queue improvements:

- add a "close" hook to the backend API.
- improve the sync() pattern in queue_fs: only sync at commit
  time and not for every envelope creation
- various fixes to the experimental external queue API.

Revision 1.462 / (download) - annotate - [select for diffs], Tue Jul 8 13:49:09 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.461: +11 -10 lines
Diff to previous 1.461 (colored)

Update the table API: lookup functions can take an optional parameters
dictionnary (currently not set). While there, add a helper for forking
external backends, and remove unused table functions.

ok gilles@

Revision 1.461 / (download) - annotate - [select for diffs], Sun May 4 16:38:19 2014 UTC (10 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.460: +2 -2 lines
Diff to previous 1.460 (colored)

Create a new default RSA engine instead of patching the existing one
if none is available.  Fixes SSL/TLS and a possible fatalx() on
machines without a default RSA engine.

Thanks to Bjorn Ketelaars for reporting and testing.

ok gilles@ (for the relayd part)

Revision 1.460 / (download) - annotate - [select for diffs], Thu May 1 15:50:20 2014 UTC (10 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.459: +5 -2 lines
Diff to previous 1.459 (colored)

Move RSA keys from "lka" to a new dedicated "ca" process because lka
is handling some async requests and shouldn't be busy with sync RSA.

ok gilles@

Revision 1.459 / (download) - annotate - [select for diffs], Wed Apr 30 09:17:29 2014 UTC (10 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.458: +2 -1 lines
Diff to previous 1.458 (colored)

when using maildir, do not create automatically create folders to match tag
in email address (ie: gilles+tag => ~/Maildir/.tag), instead use the folder
if it already exists and deliver to the mail Maildir otherwise.

ok eric@ and chl@

Revision 1.458 / (download) - annotate - [select for diffs], Wed Apr 30 08:23:43 2014 UTC (10 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.457: +3 -1 lines
Diff to previous 1.457 (colored)

The RSA engine (used by pony) has to wait for a response from the
privileged process (lka) and receive the imsgs in a while loop
synchronously.  But the lka also sends other imsgs (DNS etc.) that can
still be queued up in the buffer when waiting for the RSA response.
This only happens under load with many concurrent connections.  For
now, we just call the pony imsg handler for non-RSA imsgs that are
already in the buffer.

ok gilles@ eric@ blambert@

Revision 1.457 / (download) - annotate - [select for diffs], Tue Apr 29 19:13:14 2014 UTC (10 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.456: +15 -7 lines
Diff to previous 1.456 (colored)

Implement RSA privilege separation for OpenSMTPD, based on my previous
implementation for relayd(8).  The smtpd(8) pony processes (mta
client, smtp server) don't keep the private keys in memory but send
their private key operations as imsgs to the "lookup"/mta process.
It's worth mentioning that this prevents acidental private key leakage
as it could have been caused by "Heartbleed".

ok gilles@

Revision 1.456 / (download) - annotate - [select for diffs], Tue Apr 29 12:18:27 2014 UTC (10 years, 1 month ago) by reyk
Branch: MAIN
Changes since 1.455: +2 -2 lines
Diff to previous 1.455 (colored)

Remove unused arguments from ssl_smtp_init()

ok gilles@

Revision 1.455 / (download) - annotate - [select for diffs], Sat Apr 19 16:55:15 2014 UTC (10 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.454: +2 -2 lines
Diff to previous 1.454 (colored)

certs are looked up by hostname, the size of the buffer should use the
max hostname len, not max pathname len as before

Revision 1.454 / (download) - annotate - [select for diffs], Wed Apr 9 18:55:19 2014 UTC (10 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.453: +2 -4 lines
Diff to previous 1.453 (colored)

Zap the mfa process. It is not currently doing anything, and content filtering
will be done at session level anyway.

ok gilles@

Revision 1.453 / (download) - annotate - [select for diffs], Wed Apr 9 12:47:23 2014 UTC (10 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.452: +1 -2 lines
Diff to previous 1.452 (colored)

remove useless define for banner

ok gilles@

Revision 1.452 / (download) - annotate - [select for diffs], Fri Apr 4 16:10:42 2014 UTC (10 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.451: +129 -113 lines
Diff to previous 1.451 (colored)

Merge the mda, mta and smtp processes into a single unprivileged
process managing message reception, delivery and transfer.  Mostly
mechanical, but very intrusive as it required to rewamp all IMSG to
fix ambiguities.

with and ok gilles@

Revision 1.451 / (download) - annotate - [select for diffs], Sat Mar 22 09:41:28 2014 UTC (10 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.450: +2 -1 lines
Diff to previous 1.450 (colored)

disable the imsg buffers profiling code unless requested, this will prevent
all processes from waking up every second

Revision 1.450 / (download) - annotate - [select for diffs], Mon Feb 17 13:33:56 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_5_BASE
Branch point for: OPENBSD_5_5
Changes since 1.449: +3 -2 lines
Diff to previous 1.449 (colored)

new "smtpctl show status" command to show if mta/mda/smtp are currently running or paused.

Revision 1.449 / (download) - annotate - [select for diffs], Mon Feb 10 09:28:05 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.448: +2 -2 lines
Diff to previous 1.448 (colored)

tweak usage() and bump version.

Revision 1.448 / (download) - annotate - [select for diffs], Tue Feb 4 15:44:06 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.447: +37 -3 lines
Diff to previous 1.447 (colored)

Add support for DSN and Enhanced Status Code

Revision 1.447 / (download) - annotate - [select for diffs], Tue Feb 4 15:22:39 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.446: +6 -4 lines
Diff to previous 1.446 (colored)

Allow the admin to pause relaying to a specific domain:
 - smtpctl pause mta from <source> for <domain>
 - smtpctl resume mta from <source> for <domain>
 - smtpctl show mta paused

Revision 1.446 / (download) - annotate - [select for diffs], Tue Feb 4 14:56:03 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.445: +6 -5 lines
Diff to previous 1.445 (colored)

internal improvements and cleanups

- get rid of the whole penalty thing for failed envelopes in the mta and scheduler.
- do not disable routes on smtp errors
- try to schedule all types of envelopes on each scheduler frame.

Revision 1.445 / (download) - annotate - [select for diffs], Tue Feb 4 13:44:41 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.444: +16 -16 lines
Diff to previous 1.444 (colored)

pki code cleanup

- rename "struct ssl" and "cert" to "struct pki" and "cert" to "pki_name"
- inherit pki conf on fork instead of passing it through imsg at startup
- implement SNI on smtp listeners

Revision 1.444 / (download) - annotate - [select for diffs], Tue Feb 4 10:38:06 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.443: +10 -1 lines
Diff to previous 1.443 (colored)

extend allowed charset for email address, escape all potentially dangerous ones.

Revision 1.443 / (download) - annotate - [select for diffs], Tue Feb 4 09:50:31 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.442: +3 -1 lines
Diff to previous 1.442 (colored)

add base64_encode/base64_decode helpers

Revision 1.442 / (download) - annotate - [select for diffs], Tue Feb 4 09:05:06 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.441: +1 -2 lines
Diff to previous 1.441 (colored)

get rid of fdlimit()

Revision 1.441 / (download) - annotate - [select for diffs], Fri Dec 6 14:26:25 2013 UTC (10 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.440: +2 -2 lines
Diff to previous 1.440 (colored)

now at 5.4.1

Revision 1.440 / (download) - annotate - [select for diffs], Thu Dec 5 10:06:32 2013 UTC (10 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.439: +1 -5 lines
Diff to previous 1.439 (colored)

move defines around

Revision 1.439 / (download) - annotate - [select for diffs], Tue Dec 3 09:06:26 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.438: +4 -3 lines
Diff to previous 1.438 (colored)

Rework the envelope flushing loops in mta to avoid sending all delivery
notifications in one go to the queue. Simplify code in the process.

Revision 1.438 / (download) - annotate - [select for diffs], Sat Nov 30 10:11:57 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.437: +4 -1 lines
Diff to previous 1.437 (colored)

do not hardcode scheduler batch size, and reduce default limit to avoid
hammering effects.

Revision 1.437 / (download) - annotate - [select for diffs], Thu Nov 28 13:13:56 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.436: +3 -1 lines
Diff to previous 1.436 (colored)

limit the number of envelopes to recall in the hoststat cache.

Revision 1.436 / (download) - annotate - [select for diffs], Wed Nov 20 09:22:42 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.435: +9 -2 lines
Diff to previous 1.435 (colored)

Rework the mda and scheduler to use the holdq mechanism instead of
tempfail for limiting the number of pending deliveries to the same
user.  This allows to reach optimal delivery time even in case of
burst, while keeping the number of inflight envelopes low.

Revision 1.435 / (download) - annotate - [select for diffs], Tue Nov 19 10:01:20 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.434: +2 -1 lines
Diff to previous 1.434 (colored)

Add a limit on the number of inflight envelopes.  The scheduler suspends
scheduling of mta/mda envelopes until the number of inflight envelopes
falls below that line.

Revision 1.434 / (download) - annotate - [select for diffs], Mon Nov 18 12:24:26 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.433: +2 -1 lines
Diff to previous 1.433 (colored)

Allow overriding the local ca

Revision 1.433 / (download) - annotate - [select for diffs], Wed Nov 6 10:01:29 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.432: +23 -18 lines
Diff to previous 1.432 (colored)

Much much improved config parser and related changes.
Simplify code and do not impose an order on conditions and rule options.

Format changes that may require smtpd.conf update for some setups:

- SSL certificates are no longer automatically loaded, but must be
  explicitely declared using the "pki" keyword.
- "certificate" option becomes "pki" in listener and accept rules.
- "ssl://" becomes "secure://" in relay via rules.
- "helo" becomes "hostnames" in relay rules

New features:

- accept rules do not need an explicit action, in which case alias table
  or .forward must provide one.
- new "forward-only" action to force relaying and reject rcpts that expand
  as local delivery.
- "!" (negation) modifier on rule matching conditions.
- new "recipient" rule matching condition.
- new "verify" option on listeners and relay rules to reject invalid
  certificates.

Other changes:

- remember the helo name advertised on incoming mail and use it for sending
  bounces.
- bump envelope version (existing envelopes are updated on-the-fly).

Revision 1.432 / (download) - annotate - [select for diffs], Wed Oct 30 21:37:48 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.431: +5 -2 lines
Diff to previous 1.431 (colored)

add "smtpctl show relays" and "smtpctl show hosts" commands

Revision 1.431 / (download) - annotate - [select for diffs], Tue Oct 29 17:04:46 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.430: +2 -1 lines
Diff to previous 1.430 (colored)

add missing heloname field for relayhost.
differenciate relays with different helotable/heloname.
improve code a bit.

Revision 1.430 / (download) - annotate - [select for diffs], Tue Oct 29 14:30:05 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.429: +2 -1 lines
Diff to previous 1.429 (colored)

use "/etc/mail/mailname" instead of "/etc/mailname" and make it a define.

Revision 1.429 / (download) - annotate - [select for diffs], Tue Oct 29 11:23:58 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.428: +2 -2 lines
Diff to previous 1.428 (colored)

Report mta sessions errors on the route rather than on the MX.
If a route has too many of these errors, disable it for a while.
Reset the error counter for a route when it is re-enabled or when
it could establish a connection successfully.

Revision 1.428 / (download) - annotate - [select for diffs], Sun Oct 27 17:47:53 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.427: +11 -4 lines
Diff to previous 1.427 (colored)

Implement a feedback mechanism which allows the mta to "hold" envelopes
in the scheduler when it has too many tasks for a given relay.  The
envelopes are put on a wait queue, and are not scheduled again until
the mta "releases" some envelopes from that queue.

It prevents from having too many inflight envelopes, which are out of reach
for the admin.

Revision 1.427 / (download) - annotate - [select for diffs], Sun Oct 27 11:01:47 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.426: +7 -3 lines
Diff to previous 1.426 (colored)

Make the filter infrastructure move forward.
This is a work-in-progress and it's not supposed to be useable for now.

Revision 1.426 / (download) - annotate - [select for diffs], Sun Oct 27 07:56:25 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.425: +3 -1 lines
Diff to previous 1.425 (colored)

Create the control socket in the parent process to abort early if
another smtpd instance is running.  Close the inherited socket in
every forked process but control.

Revision 1.425 / (download) - annotate - [select for diffs], Sat Oct 26 20:32:48 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.424: +1 -37 lines
Diff to previous 1.424 (colored)

Simplify code for loading and dumping envelopes.  Makes it much easier
to deal with automatic upgrade between envelope versions at load time.

Revision 1.424 / (download) - annotate - [select for diffs], Fri Oct 25 21:31:23 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.423: +2 -2 lines
Diff to previous 1.423 (colored)

local enqueuer improvements:

- parse the whole input before trying to establish the connection
  to the local socket: fixes timeout problems when reading the output
  of a long running program.

- use sendmail(8)-like exit status.

Revision 1.423 / (download) - annotate - [select for diffs], Fri Jul 19 22:22:39 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_4_BASE, OPENBSD_5_4
Changes since 1.422: +2 -2 lines
Diff to previous 1.422 (colored)

We are basically at 5.4 now

Revision 1.422 / (download) - annotate - [select for diffs], Fri Jul 19 21:34:31 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.421: +1 -53 lines
Diff to previous 1.421 (colored)

Implement a scheduler_proc backend

Revision 1.421 / (download) - annotate - [select for diffs], Fri Jul 19 21:14:52 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.420: +87 -32 lines
Diff to previous 1.420 (colored)

Many MTA improvements:

- Better transient error handling logic: failing destinations are
  automatically disabled for a while.  When a destination is active
  again, ask the scheduler to retry previous envelopes immediatly.
- More informative error report when all routes fail for a mail.
- Implement a "smtpctl show hoststats" command to get the latest stat
  message per MX domain.
- Implement a "smtpctl show routes" command to show the state the
  currently known routes to remote MXs.
- Implement a "smtpctl resume route" command to re-enable a route that
  has been disabled.
- Do not hardcode limits
- Minor code improvements

Revision 1.420 / (download) - annotate - [select for diffs], Fri Jul 19 20:37:07 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.419: +5 -19 lines
Diff to previous 1.419 (colored)

Assorted queue improvements:
- cleanup the internal queue backend API and get rid of the QOP_* thing.
- implement a queue_proc backend
- rename queue_fsqueue.c to queue_fs
- enable support for queue encryption
- add an envelope cache
- better logging and error reporting

Revision 1.419 / (download) - annotate - [select for diffs], Fri Jul 19 19:53:33 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.418: +2 -13 lines
Diff to previous 1.418 (colored)

Add a table_proc backend for delegating table lookups to another
process.  Stop building experimental table_sqlite and table_ldap as
they will be provided as external backends.

Revision 1.418 / (download) - annotate - [select for diffs], Fri Jul 19 16:02:00 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.417: +2 -3 lines
Diff to previous 1.417 (colored)

Move the filter infrastructure forward.

Revision 1.417 / (download) - annotate - [select for diffs], Fri Jul 19 15:14:23 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.416: +11 -4 lines
Diff to previous 1.416 (colored)

scheduler improvements:
- implement suspend/resume scheduling for individual envelopes or message,
  with the associated smtpctl commands.
- allow the mta to request immediate scheduling of an envelope.
- on temporary failures a penalty can be given to further delay the next try.

Revision 1.416 / (download) - annotate - [select for diffs], Fri Jul 19 13:41:23 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.415: +9 -1 lines
Diff to previous 1.415 (colored)

New implementation for smtpctl and the command line parser.  Allows
richer syntax, and makes the code way simpler to follow and extend
with new commands.

Revision 1.415 / (download) - annotate - [select for diffs], Fri Jul 19 11:14:08 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.414: +5 -5 lines
Diff to previous 1.414 (colored)

Get rid of env->sc_pw and env->sc_pwqueue.  Early queue initialization
now happens in queue_init(), and backends take the queue passwd as
parameter in their init function.

Remove useless SMTPD_FILTER_USER while there.

Revision 1.414 / (download) - annotate - [select for diffs], Fri Jul 19 08:12:19 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.413: +2 -1 lines
Diff to previous 1.413 (colored)

Introduce expand string modifiers

Revision 1.413 / (download) - annotate - [select for diffs], Fri Jul 19 07:49:08 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.412: +1 -3 lines
Diff to previous 1.412 (colored)

Remove useless sc_pid from struct smtpd.

Revision 1.412 / (download) - annotate - [select for diffs], Fri Jul 19 07:37:29 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.411: +1 -2 lines
Diff to previous 1.411 (colored)

get rid of sa_set_port() and its awfully contorted implementation

Revision 1.411 / (download) - annotate - [select for diffs], Tue Jun 4 08:42:00 2013 UTC (11 years ago) by eric
Branch: MAIN
Changes since 1.410: +2 -2 lines
Diff to previous 1.410 (colored)

we are at 5.3.3 now.

Revision 1.410 / (download) - annotate - [select for diffs], Fri May 24 17:03:14 2013 UTC (11 years ago) by eric
Branch: MAIN
Changes since 1.409: +114 -116 lines
Diff to previous 1.409 (colored)

sync with OpenSMTPD 5.3.2

ok gilles@

Revision 1.409 / (download) - annotate - [select for diffs], Fri Apr 12 18:22:49 2013 UTC (11 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.408: +6 -9 lines
Diff to previous 1.408 (colored)

replace MAX_LINE_SIZE and SMTP_LINE_MAX with SMTPD_MAXLINESIZE for
consistency and clarity.  Remove useless and confusing extra byte in
a few arrays based on this define.

ok gilles@

Revision 1.408 / (download) - annotate - [select for diffs], Wed Mar 6 21:42:40 2013 UTC (11 years, 2 months ago) by sthen
Branch: MAIN
Changes since 1.407: +5 -1 lines
Diff to previous 1.407 (colored)

as done in ospf{,6}d/relayd, sync yyerror in various other daemons with
that from bgpd, so that it logs to syslog when daemonized.

Revision 1.407 / (download) - annotate - [select for diffs], Fri Feb 15 22:43:21 2013 UTC (11 years, 3 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_3_BASE, OPENBSD_5_3
Changes since 1.406: +3 -1 lines
Diff to previous 1.406 (colored)

add missing bits for lmtp support (from Ashish SHUKLA).

ok gilles@

Revision 1.406 / (download) - annotate - [select for diffs], Thu Feb 14 13:11:40 2013 UTC (11 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.405: +2 -1 lines
Diff to previous 1.405 (colored)

- log smtpd version at startup

ok eric@

Revision 1.405 / (download) - annotate - [select for diffs], Thu Feb 14 12:30:49 2013 UTC (11 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.404: +8 -5 lines
Diff to previous 1.404 (colored)

- smtpctl trace expand, enables tracing of aliases expansion
- replace "users" keyword with "userbase" when providing alternate userbase
- disambiguise expansion nodes when expanding across domains and userbases
- allow use of '=' instead of '=>' when declaring a mapping

ok eric@

Revision 1.404 / (download) - annotate - [select for diffs], Sun Feb 10 15:01:16 2013 UTC (11 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.403: +2 -7 lines
Diff to previous 1.403 (colored)

When getting the next batch of envelope to schedule, use an array to
store envelope ids, rather than a dynamic list.

ok gilles@

Revision 1.403 / (download) - annotate - [select for diffs], Tue Feb 5 15:23:40 2013 UTC (11 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.402: +3 -3 lines
Diff to previous 1.402 (colored)

unbreak broken smtpctl table update

fix by eric and I, ok eric@

Revision 1.402 / (download) - annotate - [select for diffs], Thu Jan 31 18:34:43 2013 UTC (11 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.401: +2 -1 lines
Diff to previous 1.401 (colored)

assorted fixes spotted by Coverity.
some log message updates.

ok gilles@

Revision 1.401 / (download) - annotate - [select for diffs], Thu Jan 31 18:24:47 2013 UTC (11 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.400: +2 -5 lines
Diff to previous 1.400 (colored)

do not need to tweak the socket sndbuf, now that the envelopes are passed
in compressed form. reduce the default size for envelope messages.

ok gilles@

Revision 1.400 / (download) - annotate - [select for diffs], Mon Jan 28 16:40:22 2013 UTC (11 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.399: +15 -6 lines
Diff to previous 1.399 (colored)

use a stripped-down mta_envelope structure in the mta process.
reduces memory footprint by a great deal when relaying lots of messages.

ok gilles@

Revision 1.399 / (download) - annotate - [select for diffs], Mon Jan 28 11:09:53 2013 UTC (11 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.398: +6 -5 lines
Diff to previous 1.398 (colored)

- introduce 'smtpctl trace lookup' to trace lookup process
- improve logging of the transfer process

trace by me, logging by eric

Revision 1.398 / (download) - annotate - [select for diffs], Sat Jan 26 09:37:23 2013 UTC (11 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.397: +677 -527 lines
Diff to previous 1.397 (colored)

Sync with our smtpd repo:

* first bricks of ldap and sqlite support (not finished but both working)
* new table API to replace map API, all lookups are done through tables
* improved handling of temporary errors throughout the daemon
* improved scheduler and mta logic: connection reuse, optimizes batches
* improved queue: more tolerant to admin errors, new layout, less disk-IO
* improved memory usage under high load
* SSL certs/keys isolated to lookup process to avoid facing network
* VIRTUAL support improved, fully virtual setups possible now
* runtime tracing of processes through smtpctl trace
* ssl_privsep.c sync-ed with relayd
* ssl.c no longer contains smtpd specific interfaces
* smtpd-specific ssl bits moved to ssl_smtpd.c
* update mail address in copyright

FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE.

smtpd.conf(5) simplified, it will require adaptations

ok eric@

Revision 1.397 / (download) - annotate - [select for diffs], Fri Nov 23 09:25:44 2012 UTC (11 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.396: +94 -90 lines
Diff to previous 1.396 (colored)

Replace the qwalk API (to retreive on disk envelopes at runtime) with
a simple QOP_WALK queue operation. Some knf and formating fixes while
there.

ok gilles@

Revision 1.396 / (download) - annotate - [select for diffs], Tue Nov 20 09:47:46 2012 UTC (11 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.395: +19 -2 lines
Diff to previous 1.395 (colored)

Allow "smtpctl show queue" to run in "online" mode if the smtpd server
is running.  The scheduler sends the runtime state of each envelope to
the queue process which loads the envelope, fills the runtime bits and
sends the envelope back to the client. Iteration over the envelope set
happens in small chunks to make the request interruptible and to allow
the server to keep doing its job in the meantime.

Adpat "smtpctl schedule-all" to schedule the messages one by one using
the same iteration mechanism.

Document "smtpctl monitor" and "smtpctl show queue".

ok gilles@

Revision 1.395 / (download) - annotate - [select for diffs], Mon Nov 12 14:58:53 2012 UTC (11 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.394: +30 -17 lines
Diff to previous 1.394 (colored)

Cleanups and improvements:

* Log more events (especially client session) and use a better scheme
  for that: each messages is prefixed with a token to easily identify
  its class:
    - info/warn/debug: general server messages
    - smtp-in: smtp client connections
    - relay: status update for relayed messages
    - delivery: status update for local deliveries

* Implement "smtpctl monitor" to display updates of selected internal
  counters.

* When reloading the on-disk queue at startup do not commit a message
  if no envelope was submitted for that message.

* Remove unused stuff in the config parser.

ok gilles@

Revision 1.394 / (download) - annotate - [select for diffs], Fri Nov 2 19:30:57 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.393: +2 -0 lines
Diff to previous 1.393 (colored)

Add a "kick counter" that gets incremented on each command, to detect
clients that don't do their best to do something useful, and just hog
the session.  When that kick counter reaches the limit, the client is
disconnected.  The counter is reset after the first HELO/EHLO command,
after tls is established, after a succesful authentication, and after
a message is accepted. It is decremented when a RCPT is accepted.

ok gilles@

Revision 1.393 / (download) - annotate - [select for diffs], Fri Nov 2 16:02:33 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.392: +1 -0 lines
Diff to previous 1.392 (colored)

Consistency and robustness improvements in mda:

- Introduce a mda_getlastline function(); improve the code to avoid
  useless allocations and string formatting; make it return the last
  line with content (skip trailing empty lines if found).
- Add a mechanism by which the mda can request the parent to abort a
  local delivery by killing the process.
- Use ioev/iobuf for draining data to the delivery process.
- Make sure to catch all transient errors and make them result in a
  tempfail rather than calling fatal().
- Make sure that the envelope status is properly set for all failures.
- Stop using SMTP response codes; it makes no sense in this context.

ok gilles@

Revision 1.392 / (download) - annotate - [select for diffs], Fri Nov 2 14:46:43 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.391: +4 -3 lines
Diff to previous 1.391 (colored)

Make counters more informative in the scheduler:

- Change the scheduler backend API a bit: commit() and rollback()
  API calls return the number of envelopes added or canceled; put
  the number of envelopes in the structure returned by batch().

- Properly report the number of incoming, registered, removed and
  expired envelopes, as well as the outcome of deliveries.

ok gilles@

Revision 1.391 / (download) - annotate - [select for diffs], Sun Oct 28 08:46:26 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.390: +2 -1 lines
Diff to previous 1.390 (colored)

Limit the number of messages that can be enqueued on a single SMTP
connection, and the number of recipients in each of them.

ok gilles@ chl@

Revision 1.390 / (download) - annotate - [select for diffs], Tue Oct 16 12:02:23 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.389: +2 -2 lines
Diff to previous 1.389 (colored)

MAX_RULEBUFFER_LEN is too small, bump it.

discussed with gilles@

Revision 1.389 / (download) - annotate - [select for diffs], Sun Oct 14 11:58:23 2012 UTC (11 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.388: +15 -2 lines
Diff to previous 1.388 (colored)

introduce map_file.c which will deprecate map_stdio.c

The idea is to have a file-backed map but to have smtpd(8) cache the maps
so that it cannot be partially read if edited while mail is received. The
file is read and converted to a static map (map_static.c), changes aren't
visible to smtpd until an explicit: smtpctl update map  which reads file,
builds a new static map and invalidates the former.

partial-read issue discussed with beck@ and halex@
idea to convert internally to a static map by eric@

diff ok eric@ and chl@

Revision 1.388 / (download) - annotate - [select for diffs], Fri Oct 12 08:51:02 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.387: +3 -3 lines
Diff to previous 1.387 (colored)

Extend the "retry" field to 16 bits.  The new quadratic retry formula
makes the maximum retry delay a bit to small on 8 bits.

ok gilles@ chl@

Revision 1.387 / (download) - annotate - [select for diffs], Thu Oct 11 21:43:11 2012 UTC (11 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.386: +2 -2 lines
Diff to previous 1.386 (colored)

- map_create() takes a map_src not a map_kind

ok eric@ and chl@

Revision 1.386 / (download) - annotate - [select for diffs], Thu Oct 11 21:14:32 2012 UTC (11 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.385: +4 -4 lines
Diff to previous 1.385 (colored)

- replace "from all" and "for all" with "from any" and "for any"

ok eric@, chl@

Revision 1.385 / (download) - annotate - [select for diffs], Wed Oct 10 20:29:46 2012 UTC (11 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.384: +3 -3 lines
Diff to previous 1.384 (colored)

F_BACKUP and ROUTE_BACKUP must be sync-ed for now, otherwise smtpd won't
work as a backup MX ...

bug experienced by todd, verified and analyzed by eric

Revision 1.384 / (download) - annotate - [select for diffs], Wed Oct 10 19:39:11 2012 UTC (11 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.383: +2 -1 lines
Diff to previous 1.383 (colored)

teach smtpctl how to display envelopes and messages using their id.
this allows an admin to inspect the queue without having to manually
extract bucket and find the path to an envelope or message.

diff by Sunil Nimmagadda <sunil@poolp.org>

ok eric@, chl@ and I

Revision 1.383 / (download) - annotate - [select for diffs], Wed Oct 10 18:02:37 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.382: +3 -1 lines
Diff to previous 1.382 (colored)

For each alias node, mark if it has been expanded from an alias map or
from a .forward file.  Local deliveries for files and filters expanded
from an alias map are run as user _smtpd.

issue reported by tood@

ok gilles@ todd@

Revision 1.382 / (download) - annotate - [select for diffs], Tue Oct 9 20:33:02 2012 UTC (11 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.381: +4 -2 lines
Diff to previous 1.381 (colored)

- allow a listen statement to impose tls on its clients;
- make listen statements impose authentication if 'auth' is specified and
	to make it optional if 'auth-optional' is specified;
- sync documentation accordingly

with ideas and input from beck@ and halex@, ok eric@

Revision 1.381 / (download) - annotate - [select for diffs], Mon Oct 8 20:35:16 2012 UTC (11 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.380: +2 -1 lines
Diff to previous 1.380 (colored)

disk space is cheap but we still want to limit the default size of a body
to a sane default for everyone.

Revision 1.380 / (download) - annotate - [select for diffs], Sun Oct 7 17:21:37 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.379: +5 -1 lines
Diff to previous 1.379 (colored)

Implement a simple wait queue API.  The idea is to allow multiple "waiters"
to wait on the same "tag" for a deferred result.

A waiter is a callback and a void *argument.  The first waiter (the one for
which waitq_wait() returns true) is supposed to run some code that leads to
waitq_run() being run, which will destroy that waitq and call all callbacks
in turn.

Not used at the moment, but will be soon.

ok gilles@ chl@

Revision 1.379 / (download) - annotate - [select for diffs], Sun Oct 7 15:46:38 2012 UTC (11 years, 7 months ago) by chl
Branch: MAIN
Changes since 1.378: +3 -1 lines
Diff to previous 1.378 (colored)

convert iobuf_queue()'s to iobuf_fqueue(). (idea from gilles@)
introduce iobuf_xinit() and iobuf_xfqueue(). (idea from eric@)

ok gilles@

Revision 1.378 / (download) - annotate - [select for diffs], Wed Oct 3 19:42:16 2012 UTC (11 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.377: +6 -2 lines
Diff to previous 1.377 (colored)

we reintroduced a bug that was fixed 2 years ago with the aliases rewrite:

During the entire expansion process, a username may be larger than
MAXLOGNAME because it may be an alias going through another expansion.
We should use a buffer that's large enough to fit a mailaddr user-part so
we avoid hitting a truncation check leading to a fatal().

ok eric@, ok chl@

Revision 1.377 / (download) - annotate - [select for diffs], Wed Oct 3 17:58:03 2012 UTC (11 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.376: +3 -2 lines
Diff to previous 1.376 (colored)

disallow root deliveries for "deliver to filename" and "deliver to mda"
rules, we only allow them for mbox and maildir though users should really
create a root alias ...

discussed with eric@ and chl@, ok both

Revision 1.376 / (download) - annotate - [select for diffs], Sun Sep 30 14:28:16 2012 UTC (11 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.375: +7 -1 lines
Diff to previous 1.375 (colored)

- add decision to the rule so that we can actually perform a reject match
  ie:

	reject from 192.168.1.0/24 for domain "openbsd.org"
	accept from 192.168.0.0/16 for domain "openbsd.org" deliver to mbox

it was documented but not working.

ok eric@ & chl@

Revision 1.375 / (download) - annotate - [select for diffs], Sat Sep 29 11:02:41 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.374: +6 -3 lines
Diff to previous 1.374 (colored)

some mfa_session cleanups.

- move mfa_session() prototype to smtpd.h
- make mfa session use a tree
- make static functions static
- merge mfa_session_init() into mfa_session()

ok chl@

Revision 1.374 / (download) - annotate - [select for diffs], Sat Sep 29 10:35:01 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.373: +1 -2 lines
Diff to previous 1.373 (colored)

finally remove rule member from struct envelope.

"wow!" gilles@

Revision 1.373 / (download) - annotate - [select for diffs], Sat Sep 29 10:32:08 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.372: +1 -2 lines
Diff to previous 1.372 (colored)

Remove support for "as user" for local deliveries.
It's not documented and not implemented.

ok gilles@

Revision 1.372 / (download) - annotate - [select for diffs], Fri Sep 28 17:28:30 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.371: +1 -20 lines
Diff to previous 1.371 (colored)

some smtpd.{c,h} cleanups:

- move struct child to smtpd.c
- make it use a tree keyed on the pid
- change child_add to take the title directly as a const char *
- remove useless child_lookup() and child_del()
- remove CHILD_INVALID

ok chl@ gilles@

Revision 1.371 / (download) - annotate - [select for diffs], Fri Sep 28 13:40:21 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.370: +1 -10 lines
Diff to previous 1.370 (colored)

Move mda_session to mda.c, and make it use a tree instead of a list,
but still use uint32_t keys since ithe key is used as peerid in msg.

ok gilles@

Revision 1.370 / (download) - annotate - [select for diffs], Fri Sep 28 12:00:09 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.369: +4 -12 lines
Diff to previous 1.369 (colored)

smtpd.h/control.c cleanups:

- move session_socket_* prototypes under util.c
- move struct ctl_conn in control.c
- make static functions static
- remove unused functions
- call unlink() in control_shutdown()
- make control_close() take a ctl_conn * instead of a fd

ok chl@ gilles@

Revision 1.369 / (download) - annotate - [select for diffs], Thu Sep 27 18:57:25 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.368: +17 -13 lines
Diff to previous 1.368 (colored)

clarify the alias expansion code.

The session manages a list of nodes to process. A node has a link to the
parent node from which it has been expanded, and a link to the rule that
led to its creation. Depending on its type and the associated rule, each
node is either "expanded" to create new nodes or "submitted" to create a
final envelope.  Nodes which have already been seen, either processed or
not, are discarded to avoid loops.

The expansion process is bootstrapped by creating an EXPAND_ADDRESS node
from the original dest, with no rule and no parent.  It is done when all
nodes have been expanded or if an error occurs before.  The expand depth
is limited 5 levels.  The whole expansion fails if the limit is reached.

While there, make sure that only one .forward file is queried at a time,
and only append the subfolder tag in the maildir case.

Fixe issues with some virtual map setups where the dest would get mixed
up, and make the whole expansion process generally easier to follow.

ok chl@ gilles@

Revision 1.368 / (download) - annotate - [select for diffs], Wed Sep 26 19:52:20 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.367: +8 -10 lines
Diff to previous 1.367 (colored)

Stop using the delivery_data union (field "to") in delivery_mda.
It's confusing and not necessary as it's only used for "buffer".
Instead, just add a "buffer" member in the structure and rename
"as_user" to "user".

The delivery_data union becomes an anonymous union in expandnode,
which is the only other place where it's used.

ok gilles@

Revision 1.367 / (download) - annotate - [select for diffs], Tue Sep 25 17:38:55 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.366: +2 -2 lines
Diff to previous 1.366 (colored)

make const arguments const, and static functions static.

ok chl@

Revision 1.366 / (download) - annotate - [select for diffs], Fri Sep 21 19:37:08 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.365: +3 -2 lines
Diff to previous 1.365 (colored)

Do not pass the username to forwards_get() which does not have to care about
this.  Instead, set the username on the expand context, and copy it on the
expand nodes as they are inserted.

ok gilles@

Revision 1.365 / (download) - annotate - [select for diffs], Fri Sep 21 16:40:20 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.364: +12 -10 lines
Diff to previous 1.364 (colored)

wrap expandtree into a "struct expand".

ok gilles@

Revision 1.364 / (download) - annotate - [select for diffs], Fri Sep 21 13:23:07 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.363: +1 -18 lines
Diff to previous 1.363 (colored)

move struct lka_session definition in lka_session.c

ok gilles@

Revision 1.363 / (download) - annotate - [select for diffs], Fri Sep 21 12:33:32 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.362: +2 -1 lines
Diff to previous 1.362 (colored)

Add a log_envelope() function that log envelope status in a uniform way.
It automagically adds an rcpt=<user@domain> field if "dest" differs from
the original "rcpt". The function takes an "extra" parameter that allows
to add some specific info depending on the context.

ok gilles@

Revision 1.362 / (download) - annotate - [select for diffs], Fri Sep 21 10:22:29 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.361: +16 -9 lines
Diff to previous 1.361 (colored)

Move ruleset_match() prototype to smtpd.h and make the envelope const.
Adapt a lot of functions in chain to use const args where required.

ok gilles@

Revision 1.361 / (download) - annotate - [select for diffs], Thu Sep 20 14:28:57 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.360: +7 -7 lines
Diff to previous 1.360 (colored)

constify parameters that are supposed to be const.

ok gilles@

Revision 1.360 / (download) - annotate - [select for diffs], Wed Sep 19 18:20:36 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.359: +1 -2 lines
Diff to previous 1.359 (colored)

Remove DF_ENQUEUE flag. It is mostly unused and logically broken.
Ignore it in existing envelopes until it gets completely dropped.
Change "smtpctl show queue" to display the address family of the
envelope source instead of the ENQUEUE flag.

ok gilles@

Revision 1.359 / (download) - annotate - [select for diffs], Wed Sep 19 11:57:35 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.358: +1 -4 lines
Diff to previous 1.358 (colored)

remove IS_RELAY and IS_MAILBOX macros.

ok gilles@

Revision 1.358 / (download) - annotate - [select for diffs], Wed Sep 19 10:10:30 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.357: +2 -4 lines
Diff to previous 1.357 (colored)

Remove aliases_exists() and aliases_virtual_exists(). The corresponding
*_get() functions can be called directly.

ok gilles@

Revision 1.357 / (download) - annotate - [select for diffs], Wed Sep 19 09:06:35 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.356: +5 -14 lines
Diff to previous 1.356 (colored)

start cleaning the expansion code:

- change expandtree_* prefix to expand_ for better readability and
  because the structure might change at some point
- rename <>_free_nodes() to <>_free()
- remove unused <>_remove_node()
- refcounting has no purpose at all; just remove it as well as the
  decrement/increment functions, and replace the latter with <>_insert
- expandnode flags is only used to know if it's been processed or not,
  don't make it a flag but a simple field with clear name.

ok gilles@ chl@

Revision 1.356 / (download) - annotate - [select for diffs], Tue Sep 18 14:23:01 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.355: +2 -1 lines
Diff to previous 1.355 (colored)

- add xmemdup() helper.
- remove useless block in switch.

ok gilles@

Revision 1.355 / (download) - annotate - [select for diffs], Tue Sep 18 13:42:39 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.354: +3 -7 lines
Diff to previous 1.354 (colored)

simple lka cleanups:

- fix lka* function prototypes in smtpd.h
- make static functions static
- merge lka_session_init() into lka_session()
- make lka_session.c use tree.c to store sessions

ok gilles@

Revision 1.354 / (download) - annotate - [select for diffs], Tue Sep 18 13:13:43 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.353: +2 -3 lines
Diff to previous 1.353 (colored)

this structure is not useful and ill-named. remove it.

ok gilles@

Revision 1.353 / (download) - annotate - [select for diffs], Tue Sep 18 12:13:22 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.352: +1 -2 lines
Diff to previous 1.352 (colored)

remove C_NET. it's not used and there is no plan for it at the moment.

ok gilles@

Revision 1.352 / (download) - annotate - [select for diffs], Mon Sep 17 20:19:18 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.351: +3 -1 lines
Diff to previous 1.351 (colored)

Add map_create() and map_add() helpers.  Simplify the config parser by a
great deal.

While there, rename the default "localhost" map to "<localhost>" to make
it look more internal, and create a single "<anyhost>" map referenced by
"from all" rules, instead of creating a dynamic one for each of them.

ok gilles@ chl@

Revision 1.351 / (download) - annotate - [select for diffs], Sun Sep 16 16:54:55 2012 UTC (11 years, 8 months ago) by chl
Branch: MAIN
Changes since 1.350: +1 -2 lines
Diff to previous 1.350 (colored)

now that log_imsg() is only used in smtpd.c, set it as static.

ok gilles@

Revision 1.350 / (download) - annotate - [select for diffs], Sun Sep 16 15:55:55 2012 UTC (11 years, 8 months ago) by chl
Branch: MAIN
Changes since 1.349: +5 -5 lines
Diff to previous 1.349 (colored)

silent warnings

reported by ajacoutot@

ok gilles@ ajacoutot@

Revision 1.349 / (download) - annotate - [select for diffs], Sun Sep 16 11:53:57 2012 UTC (11 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.348: +2 -2 lines
Diff to previous 1.348 (colored)

replace BSD-licensed mkdir_p() with ISC-licensed mkdirs(), this allows us
to avoid a dual-licensed util.c for no reason

ok chl@

Revision 1.348 / (download) - annotate - [select for diffs], Fri Sep 14 19:22:04 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.347: +3 -6 lines
Diff to previous 1.347 (colored)

Remove s_ssl from the smtp session since it is duplicated in the io struct.
Change ssl_session_init to ssl_smtp_init and make it simpler:  only create
an SSL* from the SSL_CTX* passed as parameter, so it does not have to know
about the struct session itself.  Kill some dead prototypes while there.

ok chl@ gilles@

Revision 1.347 / (download) - annotate - [select for diffs], Tue Sep 11 08:37:52 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.346: +2 -2 lines
Diff to previous 1.346 (colored)

Rework the scheduler internals.  Fix some scheduling loop issues and
handle envelope scheduling/expiration better.

ok gilles@

Revision 1.346 / (download) - annotate - [select for diffs], Sat Sep 1 16:25:27 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.345: +1 -2 lines
Diff to previous 1.345 (colored)

remove unused flag

Revision 1.345 / (download) - annotate - [select for diffs], Sat Sep 1 16:09:14 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.344: +1 -13 lines
Diff to previous 1.344 (colored)

- remove crypto_backend
- remove support for encrypted queue, it will be reintroduced later after
  pouring more thinking into it

if you had it enabled, flush your queue before updating

Revision 1.344 / (download) - annotate - [select for diffs], Thu Aug 30 21:59:46 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.343: +2 -2 lines
Diff to previous 1.343 (colored)

change crypto_setup() prototype to use const char * instead of uint8_t *

while there do some KNF:
- change 8 spaces to tab
- add/remove some missing/extra space after if's

ok gilles@

Revision 1.343 / (download) - annotate - [select for diffs], Thu Aug 30 19:33:25 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.342: +5 -5 lines
Diff to previous 1.342 (colored)

switch compress_backend to use FILE * instead of file descriptors, like
crypto_backend

ok gilles@

Revision 1.342 / (download) - annotate - [select for diffs], Thu Aug 30 18:25:44 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.341: +3 -2 lines
Diff to previous 1.341 (colored)

- import latest aldap.[ch] and ber.[ch] from ypldap
- revive map_ldap.c by updating it to the current API

diff by Mathieu Masson who played puzzle with an oooold changeset of mine,
this import is to let us work on it in tree, it won't work as is.

idea ok eric@ and chl@

Revision 1.341 / (download) - annotate - [select for diffs], Wed Aug 29 16:26:17 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.340: +13 -3 lines
Diff to previous 1.340 (colored)

Introduce the crypto_backend API and provide support for... encrypted queue
using the new API. By default, OpenSMTPD does not provide queue encryption,
but it can be enabled with "queue encryption [args]" and will transparently
encrypt/decrypt envelopes/messages as they hit the queue.

By default, it will use Blowfish in CBC mode with a different random IV for
each envelope and message. User provided key is expanded using sha256 but a
different cipher and digest may be specified in smtpd.conf

Queue encryption is compatible with compression and if both options are set
it will do them in correct order and transparently.

tested by chl@, a few users and myself
ok chl@ and I

Revision 1.340 / (download) - annotate - [select for diffs], Tue Aug 28 14:03:49 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.339: +1 -7 lines
Diff to previous 1.339 (colored)

remove encrypt.c prototypes leftovers

ok gilles@

Revision 1.339 / (download) - annotate - [select for diffs], Sun Aug 26 11:52:48 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.338: +2 -1 lines
Diff to previous 1.338 (colored)

- define ZLIB_BUFFER_SIZE instead of hardcoding 8192
- check gzdopen() failure
- call gzclose() whenever a failure occurs after gzdopen()
- simplify slightly some checks in compress/uncompress
- create PATH_TEMPORARY in /var/spool/smtpd, chmod 700, owned by _smtpd
- compress_zlib should use PATH_TEMPORARY instread of /tmp as we're
	chrooted and this will otherwise lead to a fatal()

ok chl@

Revision 1.338 / (download) - annotate - [select for diffs], Sat Aug 25 23:35:09 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.337: +28 -1 lines
Diff to previous 1.337 (colored)

Add compress_backend, allowing compression of messages and envelopes in the queue.
To use it, just add "queue compress" in smtpd.conf. For now, only zlib is used.

lots of feedback from eric@ and gilles@

ok eric@ gilles@

Revision 1.337 / (download) - annotate - [select for diffs], Sat Aug 25 22:03:26 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.336: +1 -2 lines
Diff to previous 1.336 (colored)

- add myself to the copyright in control.c, i've done quite a few changes
there in the last few years ;-)
- get rid of availdesc(): getdtablecount() is so much more reliable
- get rid of env->sc_maxconn, we can be much smarter with getdtablecount()
	and getdtablesize()
- disable accept when we hit the control process fd reserve
- disable accept when we fail
- enable accept when we're back below the limit

this is not the full fd exhaustion diff, i'll merge changes from relayd
tomorrow, this was only required to get rid of the env->sc_maxconn and
availdesc() mess

"reads alright" eric@

Revision 1.336 / (download) - annotate - [select for diffs], Sat Aug 25 21:33:33 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.335: +2 -1 lines
Diff to previous 1.335 (colored)

- stop accepting clients if we hit our fd reserve limit (or if we fail)
- resume if we go below the fd reserve

with feedback and ok eric@

Revision 1.335 / (download) - annotate - [select for diffs], Sat Aug 25 15:39:11 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.334: +1 -2 lines
Diff to previous 1.334 (colored)

- offline enqueue does not need to use the user_backend API, it relies on
	system users ... use getpwuid() instead of ub->getbyuid()
- since that was the only caller, get rid of user_backend->getbyuid()

this is the first step towards removing the user_backend API and making
user lookups available through the maps API (yes, virtual user support ;)

ok eric@, ok chl@

Revision 1.334 / (download) - annotate - [select for diffs], Sat Aug 25 11:38:18 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.333: +2 -1 lines
Diff to previous 1.333 (colored)

- introduce TRACE_PROFILING
- when smtpd starts with -T profiling it will log_trace() some prof. info
- when smtpd starts with -T profstat, it will push them to stats API with
	type STAT_TIMESPEC under key profiling.imsg.*

with this diff we can get live profiling of events with a very minimal
overhead :-)

ok chl@, ok eric@

Revision 1.333 / (download) - annotate - [select for diffs], Sat Aug 25 10:23:12 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.332: +31 -9 lines
Diff to previous 1.332 (colored)

- introduce struct stat_value
- statistics can now have a type (counter, timestamp, timeval, timespec and
  possibly others in the future)
- stat_increment() / stat_decrement() now take an increment/decrement value
  and are at the moment only of type counter
- stat_set() now takes a stat_value
- provide helpers to convert raw values to stat_value

ok eric@, ok chl@

while at it fix a rq_queue_dump() call using a bogus timestamp in scheduler
ramqueue.

Revision 1.332 / (download) - annotate - [select for diffs], Fri Aug 24 13:21:56 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.331: +1 -2 lines
Diff to previous 1.331 (colored)

In envelope ascii dump/load:
- remove loading of evpid.
- don't dump the msgid
- ignore msgid at load
- remove now unused functions ascii_{dump,load}_uint{32,64}_hex()

With inputs from eric@ and gilles@

ok gilles@ eric@

Revision 1.331 / (download) - annotate - [select for diffs], Fri Aug 24 13:13:13 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.330: +2 -5 lines
Diff to previous 1.330 (colored)

Don't pass struct envelope pointer in queue backend API, instead use envelope id and
an envelope ascii buffer.

ok eric@ gilles@

Revision 1.330 / (download) - annotate - [select for diffs], Fri Aug 24 12:29:50 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.329: +1 -3 lines
Diff to previous 1.329 (colored)

Remove the rq_host and rq_batch structures from the ramqueue scheduler.
The scheduler should only allow admin to schedule specific envelopes by
id, or msgid. More advanced scheduling (per host/route/whatever) should
be achieved using smtpctl schedule-id and proper filtering on the queue,
or using ad-hoc scheduler backend and tools.

ok gilles@ chl@

Revision 1.329 / (download) - annotate - [select for diffs], Tue Aug 21 20:19:46 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.328: +7 -3 lines
Diff to previous 1.328 (colored)

Allow smtpd to work as a backup MX, relaying only to MXs with higher
priority in the DNS record. For example:

   accept for domain "foo.org" relay backup "mx3.foo.org"

will relay mails for "foo.org" using only hosts with higher priority
(i.e. lower value) than "mx3.foo.org", which is supposed to be the
current server.

If the specified backup MX is not found in the DNS record, relaying
works as normal.

ok gilles@

Revision 1.328 / (download) - annotate - [select for diffs], Tue Aug 21 13:13:17 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.327: +2 -1 lines
Diff to previous 1.327 (colored)

Re-enable loop detection, but in mta and mda this time.

ok gilles@

Revision 1.327 / (download) - annotate - [select for diffs], Mon Aug 20 21:14:17 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.326: +3 -6 lines
Diff to previous 1.326 (colored)

MAX_LINE_SIZE is supposed to define the max length of a SMTP line ...
... but SMTP_LINE_MAX *also* defines it ... with a different value ...
and did I mention both were too small anyway ?

quick fix until we kill one or the other: bump MAX_LINE_SIZE and define
SMTP_LINE_MAX to be MAX_LINE_SIZE. this fixes the immediate issue while
we decide which one bites the dust.

fixes the crashes and "line too long" errors spotted by todd@
ok todd@, ok chl@

Revision 1.326 / (download) - annotate - [select for diffs], Sun Aug 19 14:16:58 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.325: +71 -71 lines
Diff to previous 1.325 (colored)

coding style: replace all occurences of u_int* with uint*

ok eric@

Revision 1.325 / (download) - annotate - [select for diffs], Sun Aug 19 10:32:32 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.324: +3 -3 lines
Diff to previous 1.324 (colored)

Kill envelope_{dump,load}_file() and replace them with envelope_{dump,load}_buffer().

with input from eric@

ok eric@

Revision 1.324 / (download) - annotate - [select for diffs], Sun Aug 19 08:45:44 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.323: +3 -3 lines
Diff to previous 1.323 (colored)

fix an issue where too long lines were not spot properly.

issue reported by todd@

ok eric@

Revision 1.323 / (download) - annotate - [select for diffs], Sat Aug 18 20:52:36 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.322: +1 -8 lines
Diff to previous 1.322 (colored)

zap struct mta_batch. Only pass ids where needed.

ok gilles@

Revision 1.322 / (download) - annotate - [select for diffs], Sat Aug 18 18:18:23 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.321: +38 -77 lines
Diff to previous 1.321 (colored)

- introduce stat_backend, an API for pluggable statistic backends
  > statistics are no longer static structures in shared memory
  > statistics are only set, smtpd never uses them in its logic
  > each statistic is a key/value where key can be any (dynamic) string
- convert all uses of the former API to use the new one
- implement stat_ramstat that keeps non-persistent stats in ram structure

ok eric@, ok chl@

Revision 1.321 / (download) - annotate - [select for diffs], Sat Aug 18 15:45:12 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.320: +54 -1 lines
Diff to previous 1.320 (colored)

Major update of the mta internals.

Add a mta_route structure which describes a route through which
outgoing mails are to be sent.  This structure holds connection
parameters and limits.  When an envelope is received in a batch,
the route for it is looked up, and the envelope is added to the
a list of envelope to be sent for this message on that route: a
task.  When the batch is closed, each task is added to the list
of tasks for their respective route.

The routes are drained when new work can happen. The route will
create new mta sessions if necessary.  When a session is up and
ready, it picks the first pending task on the route if any.  In
the other case, it just closes the connection.

Errors on the connection are reported to the route, so that the
route could be flagged as broken.  Currently, three errors on a
an attempt to open a route is reported as a failure for all pen-
ding tasks.

ok gilles@

Revision 1.320 / (download) - annotate - [select for diffs], Fri Aug 10 11:05:55 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.319: +1 -45 lines
Diff to previous 1.319 (colored)

Move mta and smtp specific defines into their own files.
Some formatting cleanups while there.

ok gilles@

Revision 1.319 / (download) - annotate - [select for diffs], Thu Aug 9 16:00:31 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.318: +4 -3 lines
Diff to previous 1.318 (colored)

Allow failure reports for different recipients of the same message
to be grouped into a single bounce message.

The bounce structure keeps a list of envelopes.  For now, the list
is constructed by delaying the re-enqueuing of a bounce envelope a
bit, to wait for other bounces from the same message to be part of
the same report.

Revision 1.318 / (download) - annotate - [select for diffs], Thu Aug 9 11:52:32 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.317: +1 -3 lines
Diff to previous 1.317 (colored)

remove unused function and prototypes

Revision 1.317 / (download) - annotate - [select for diffs], Thu Aug 9 09:48:02 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.316: +2 -6 lines
Diff to previous 1.316 (colored)

Improve the message flows to completely isolate operations on the
queue backend within the queue process.

The scheduler sends envelope ids to the queue process which loads
the envelope and forward the request to the agent responsible for
the delivery.  The result is sent by the agent to the queue which
updates the storage before notifying the scheduler.

Bounces are created and enqueued (from the client side) by the
queue process, rather than the scheduler.

ok gilles@

Revision 1.316 / (download) - annotate - [select for diffs], Wed Aug 8 17:33:55 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.315: +1 -8 lines
Diff to previous 1.315 (colored)

remove useless defines

ok gilles@ chl@

Revision 1.315 / (download) - annotate - [select for diffs], Wed Aug 8 08:50:42 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.314: +33 -22 lines
Diff to previous 1.314 (colored)

Improve the scheduler backend API.

New envelopes are pushed into the scheduler through the insert()
commit() rollback() transactional interface functions.

Worklists are pulled from the scheduler through a single batch()
interface function, which returns a list of envelope ids and the
type of processing. Envelopes returned in this batch are said to
be "in-flight", as opposed to "pending". They are supposed to be
processed in some way, and either updated() or deleted() at some
point.

The schedule()/remove() functions are used to alter the internal
state of "pending" envelopes to make them schedulable.  The enve-
lopes will be part of a worklist on the next call to batch().

Rewrite the scheduler_ramqueue backend.

The initial queue loading in now done by the queue.

ok gilles@

Revision 1.314 / (download) - annotate - [select for diffs], Tue Aug 7 21:47:57 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.313: +21 -1 lines
Diff to previous 1.313 (colored)

Implement a set of tree_* functions for storing arbitrary pointers in splay
trees with uint64_t keys. Also add x{m,c}alloc and xstrdup helpers.

ok gilles@

Revision 1.313 / (download) - annotate - [select for diffs], Sun Jul 29 17:21:43 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.312: +3 -2 lines
Diff to previous 1.312 (colored)

- introduce xlowercase() and allow lowercase() to fail gracefully
- replace all calls to lowercase() with calls to xlowercase()
- in the format string expansion, lowercase() all formats

we will have to reassess all calls to xlowercase() even though it has never
triggered as far as I know, we can probably gracefully fail some of them.
right now we're just keeping former behaviour.

this commit fixes issue reported by Hugo Osvaldo Barrera where a %u format
could lead to a delivery failure (ie: GILLES@openbsd.org should be expanded
to gilles, not GILLES ... only for local deliveries).

ok chl@ on the idea, ok eric@ on the diff

Revision 1.312 / (download) - annotate - [select for diffs], Sun Jul 29 16:33:01 2012 UTC (11 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.311: +1 -3 lines
Diff to previous 1.311 (colored)

get rid of A_INVALID.
little code cleanup while here.

ok gilles@

Revision 1.311 / (download) - annotate - [select for diffs], Sun Jul 29 13:56:24 2012 UTC (11 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.310: +1 -32 lines
Diff to previous 1.310 (colored)

remove the session tree from the global env and move it to mta_session.c,
along with mta_relay and mta_session definition.

ok gilles@

Revision 1.310 / (download) - annotate - [select for diffs], Thu Jul 12 08:51:43 2012 UTC (11 years, 10 months ago) by chl
Branch: MAIN
CVS Tags: OPENBSD_5_2_BASE, OPENBSD_5_2
Changes since 1.309: +2 -3 lines
Diff to previous 1.309 (colored)

add support for maildir tagging/folders.

ok gilles@
ok eric@ on previous versions of this patch

Revision 1.309 / (download) - annotate - [select for diffs], Tue Jul 10 23:21:34 2012 UTC (11 years, 10 months ago) by chl
Branch: MAIN
Changes since 1.308: +2 -1 lines
Diff to previous 1.308 (colored)

backout the:
- remove the /envelopes subdirectory, envelopes are at the same level than
  the message file
- kill PATH_ENVELOPES define

but keep the:
- reduce the number of buckets from 0xfff to 0xff, this avoid performances
  of the queue to decrease when we start having tons of buckets

ok eric@ gilles@

Revision 1.308 / (download) - annotate - [select for diffs], Tue Jul 10 16:11:43 2012 UTC (11 years, 10 months ago) by chl
Branch: MAIN
Changes since 1.307: +3 -3 lines
Diff to previous 1.307 (colored)

accept address literal for the recipient domain.
while there, change valid_{local,domain}part() prototypes to use const char *.

with input from gilles@ and eric@

ok gilles@ eric@

Revision 1.307 / (download) - annotate - [select for diffs], Tue Jul 10 11:13:40 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.306: +2 -2 lines
Diff to previous 1.306 (colored)

- simplify the scheduler loop logic further, it is ridiculously simple now
  and I don't think we can do much better (at that level) :-p
- always break out of the handler after processing an envelope, this will
  avoid a busy scheduler from not getting a chance to handle SIGTERM/SIGINT
  YES we can now ctrl-c a maaaaad scheduler !

ok eric@, ok chl@

Revision 1.306 / (download) - annotate - [select for diffs], Mon Jul 9 17:57:54 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.305: +2 -1 lines
Diff to previous 1.305 (colored)

- introduce log_trace(TRACE_SCHEDULER, ...)
- simplify a tiny tiny bit the scheduler loop
- no functional change (yet)

Revision 1.305 / (download) - annotate - [select for diffs], Mon Jul 9 09:57:53 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.304: +8 -8 lines
Diff to previous 1.304 (colored)

- runner is the terminology we used back when we had runqueues, we no
  longer have them and runner is actually a scheduler so rename.
- introduce scheduler_backend which does the same to scheduler than
  queue_backend does to queue and map_backend does to maps
- remove all occurences of RUNNER and runner, replace them with SCHEDULER
  and scheduler

ok eric@, ok chl@

Revision 1.304 / (download) - annotate - [select for diffs], Mon Jul 9 08:08:29 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.303: +1 -2 lines
Diff to previous 1.303 (colored)

first step of simplifying fsqueue:

- remove the /envelopes subdirectory, envelopes are at the same level than
  the message file
- kill PATH_ENVELOPES define
- reduce the number of buckets from 0xfff to 0xff, this avoid performances
  of the queue to decrease when we start having tons of buckets

this diff introduces a change to the queue layout, you will want to empty
your queue before updating. more cleanup to come

ok eric@, ok chl@

Revision 1.303 / (download) - annotate - [select for diffs], Sun Jul 8 18:13:08 2012 UTC (11 years, 10 months ago) by chl
Branch: MAIN
Changes since 1.302: +7 -2 lines
Diff to previous 1.302 (colored)

remove enum queue_kind from queue_fsqueue.c.
incoming messages are now always stored in /incoming, whatever the queue_backend is.
remove QOP_FD_RW and fsqueue_message_fd_rw().
while there check return value of generated paths before calling rmtree()

with advice from gilles@ and eric@

ok gilles@ eric@

Revision 1.302 / (download) - annotate - [select for diffs], Mon Jul 2 17:00:05 2012 UTC (11 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.301: +3 -15 lines
Diff to previous 1.301 (colored)

Lookup queue and scheduler backends by name, rather than enum.
Add a command-line option to specify the backend to use at runtime.

ok gilles@

Revision 1.301 / (download) - annotate - [select for diffs], Wed Jun 20 20:45:23 2012 UTC (11 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.300: +15 -21 lines
Diff to previous 1.300 (colored)

Finally get rid of the queue_kind enum in the queue API. Keep that
internally in fsqueue backend for now, and let the fsqueue_message()
and fsqueue_envelope() dispatchers do the right thing.

Based on a diff by chl@

ok chl@ gilles@

Revision 1.300 / (download) - annotate - [select for diffs], Sun Jun 17 15:17:08 2012 UTC (11 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.299: +14 -2 lines
Diff to previous 1.299 (colored)

- introduce struct scheduler_info and the scheduler_info() function to fill
  a struct scheduler_info given a struct envelope
- adapt the scheduler API and the scheduler_ramqueue backend to use the new
  struct scheduler_info instead of a struct envelope

idea discussed with eric@ and chl@, mechanical diff, no functional change

Revision 1.299 / (download) - annotate - [select for diffs], Thu Jun 14 21:56:13 2012 UTC (11 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.298: +2 -2 lines
Diff to previous 1.298 (colored)

- rename filter.h -> filter_api.h to be consistent with upcoming changes

Revision 1.298 / (download) - annotate - [select for diffs], Sun Jun 3 19:52:56 2012 UTC (12 years ago) by eric
Branch: MAIN
Changes since 1.297: +2 -1 lines
Diff to previous 1.297 (colored)

Do not store the envelope id within the envelope, only the message id.
Make sure existing envelopes can be properly loaded.

ok chl@ gilles@

Revision 1.297 / (download) - annotate - [select for diffs], Fri Jun 1 10:40:14 2012 UTC (12 years ago) by chl
Branch: MAIN
Changes since 1.296: +1 -3 lines
Diff to previous 1.296 (colored)

remove unused fields

ok eric@

Revision 1.296 / (download) - annotate - [select for diffs], Fri Jun 1 09:24:58 2012 UTC (12 years ago) by eric
Branch: MAIN
Changes since 1.295: +4 -4 lines
Diff to previous 1.295 (colored)

move envelope dump/load functions to envelope.c

ok gilles@

Revision 1.295 / (download) - annotate - [select for diffs], Tue May 29 19:53:10 2012 UTC (12 years ago) by gilles
Branch: MAIN
Changes since 1.294: +2 -2 lines
Diff to previous 1.294 (colored)

- introduce map_static.c as a backend to static maps in parse.y, this has
  the benefit that we no longer have two code paths whenever we deal with
  maps, we can always use the backend mechanism.

I have not plugged this in yet, I'll do it in a later commit, just get it
out of my sandbox

Revision 1.294 / (download) - annotate - [select for diffs], Tue May 29 19:29:44 2012 UTC (12 years ago) by gilles
Branch: MAIN
Changes since 1.293: +2 -1 lines
Diff to previous 1.293 (colored)

- introduce text_to_relayhost() which converts an url into a relayhost.
  urls are of the form: [schema://]host[:ip]

not used, yet other commits are following ;-)

Revision 1.293 / (download) - annotate - [select for diffs], Sun May 13 00:10:49 2012 UTC (12 years ago) by gilles
Branch: MAIN
Changes since 1.292: +9 -3 lines
Diff to previous 1.292 (colored)

- cleanup parse.y by removing lots of code that should not have been there,
  but in ruleset.c and util.c instead.

- introduce the new map_compare() map API call to allow iterating over keys
  and comparing them with provided key using provided function. this allows
  checking a partial key in a key set, very useful for comparing an address
  to a set of netmask.

- introduce new map kind K_NETADDR
- implement K_NETADDR for map_db and map_stdio
- teach ruleset checking how to use the map_compare() with K_NETADDR

we can now do the following:

   map "srcaddr" source plain "/etc/mail/srcaddr.txt"

   accept from map srcaddr for domain "openbsd.org" [...]

Revision 1.292 / (download) - annotate - [select for diffs], Sat May 12 21:49:31 2012 UTC (12 years ago) by gilles
Branch: MAIN
Changes since 1.291: +2 -5 lines
Diff to previous 1.291 (colored)

- remove unused sources S_EXT, S_DYN and S_EXT from enum map_src
- continue simplification of parse.y
- remove "for network", if we ever need it we can reimport, probably no
  one knows of that undocumented strange feature ;-)
- change syntax for virtual domains configuration:

	  accept for virtual vmap	[...] <- wrong
	  accept for virtual map vmap	[...] <- right

  the reason for this change is that we will soon implement relay rules
  through maps and that keeping that syntax would make it inconsistent
  with the other rules.

- update man pages for makemap and smtpd.conf to reflect changes

ok eric@, looks ok chl@

Revision 1.291 / (download) - annotate - [select for diffs], Sat May 12 18:41:10 2012 UTC (12 years ago) by gilles
Branch: MAIN
Changes since 1.290: +1 -11 lines
Diff to previous 1.290 (colored)

- simplify a bit maps by removing fields which are still unused years
  after the initial ambitious implementation: byebye map type & map flags

- simplify a bit parse.y by removing assignations to these otherwise unused
  fields

- remove the DNS map source, it may be a good idea, but we can just add it
  when we plan to implement it (if we do)

- make the { } options in map declaration, it's been annoying me for a long
  time now, this allows the following to work:

       map "foobar" source plain "/etc/mail/foobar"

- update smtpd.conf.5 accordingly ;-)

Revision 1.290 / (download) - annotate - [select for diffs], Sat May 12 17:41:27 2012 UTC (12 years ago) by eric
Branch: MAIN
Changes since 1.289: +4 -2 lines
Diff to previous 1.289 (colored)

Update the internal mta implementation so that a session now has a
list of messages to send to the remote smtp server over the same
connection. It's not currently used as the scheduler/runner is not
aware of this yet, and the imsg protocol would need to be updated.

ok gilles@

Revision 1.289 / (download) - annotate - [select for diffs], Sat May 12 15:29:16 2012 UTC (12 years ago) by gilles
Branch: MAIN
Changes since 1.288: +3 -3 lines
Diff to previous 1.288 (colored)

- rename all occurences of K_SECRET to K_CREDENTIALS
- rename all occurences of struct map_secret to map_credentials
- do not fatal if the credentials map has disappeared, instead make the
  auth fail with a lookup failure. the mail will be temporary failed so
  it stays in queue until admin fixes smtpd.conf, removes mail, or lets
  it expires

Revision 1.288 / (download) - annotate - [select for diffs], Fri May 11 12:12:02 2012 UTC (12 years ago) by eric
Branch: MAIN
Changes since 1.287: +3 -3 lines
Diff to previous 1.287 (colored)

split the session logic off mta.c into mta_session.c

ok gilles@

Revision 1.287 / (download) - annotate - [select for diffs], Wed Mar 7 22:54:49 2012 UTC (12 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.286: +4 -3 lines
Diff to previous 1.286 (colored)

various reliability fixes:

- prevent queue_fsqueue from fatal() when it hits an ENOENT, it can happen
- change a bit the scheduler API to simplify it, fix runner accordingly

- we can't remove msg/batch from ramqueue while envelope is offloaded or
  it will cause a double, instead we add refcnt to both msg/batch and
  only free them when it hits 0

Revision 1.286 / (download) - annotate - [select for diffs], Tue Jan 31 21:05:26 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_1_BASE, OPENBSD_5_1
Changes since 1.285: +5 -2 lines
Diff to previous 1.285 (colored)

fix an issue observed this week-end while flooding ajacoutot@ :

we keep track of available fd's to prevent scheduling of messages if we
know that we are going to fail. however, since the envelope is not
removed from the scheduler, it will be rescheduled right away leading to
a busy loop in the scheduler. we know flag the mda/mta processes as BUSY
and do not schedule envelopes that target a BUSY process.

also, fix a potential bug that could lead to a use after free when doing
a batch/message/host traversal of schedulable envelopes.

while at it fix misuse of env->sc_opts as env->sc_flags, was not really
causing any issue as the misuse was constant ...

Revision 1.285 / (download) - annotate - [select for diffs], Sun Jan 29 11:37:32 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.284: +41 -21 lines
Diff to previous 1.284 (colored)

Rewrite io code in smtp and mta using the iobuf/ioev interface to have
a better separation between io and protocol logic.  As a side-effect,
it fixes a couple of long-standing issues in the io path, and
hopefully add fresh ones instead.  Kill client.c in the process.

ok gilles@

Revision 1.284 / (download) - annotate - [select for diffs], Sat Jan 28 16:50:02 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.283: +3 -1 lines
Diff to previous 1.283 (colored)

add optional display handler to scheduler_backend, if not NULL the handler
will be called for each iteration of the runner

implement a display handler for scheduler_ramqueue to display the entire
ramqueue (hosttree, msgtree and linear queue) in log_debug

Revision 1.283 / (download) - annotate - [select for diffs], Sat Jan 28 15:11:06 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.282: +1 -2 lines
Diff to previous 1.282 (colored)

remove useless state

ok gilles@

Revision 1.282 / (download) - annotate - [select for diffs], Sat Jan 28 11:33:07 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.281: +36 -77 lines
Diff to previous 1.281 (colored)

- introduce the scheduler_backend API
- introduce the scheduler_ramqueue backend
- remove all occurences of ramqueue outside of the ramqueue backend
- teach runner how to use the new API

it is now possible to write custom schedulers !

ok eric@, ok chl@

Revision 1.281 / (download) - annotate - [select for diffs], Tue Jan 24 12:20:18 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.280: +2 -2 lines
Diff to previous 1.280 (colored)

Add a parameter to the queue backend init() call to specify wether the
call is issued by smtpd or smtpctl.  In the latter case, only perform
sanity checks and do not touch directories.  A running server no
longer lose its "incoming/" directory each time smtpctl is called...

ok gilles@

Revision 1.280 / (download) - annotate - [select for diffs], Wed Jan 18 13:41:54 2012 UTC (12 years, 4 months ago) by chl
Branch: MAIN
Changes since 1.279: +15 -8 lines
Diff to previous 1.279 (colored)

Add new filters callbacks for:
- network events (CONNECT/CLOSE)
- commands (QUIT/RSET)

ok gilles@ eric@

Revision 1.279 / (download) - annotate - [select for diffs], Fri Jan 13 21:58:35 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.278: +1 -3 lines
Diff to previous 1.278 (colored)

queue_message_purge() and queue_message_delete() are actually the same
thing. Remove queue_message_purge() in favor of queue_message_delete
and simplify fsqueue_message_delete() implementation to move the
message dir to purge/

ok gilles@

Revision 1.278 / (download) - annotate - [select for diffs], Fri Jan 13 14:27:55 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.277: +4 -7 lines
Diff to previous 1.277 (colored)

remove the status field from struct envelope, move it to the smtp
session, and cleanup the DS_* flags.

ok gilles@ chl@

Revision 1.277 / (download) - annotate - [select for diffs], Fri Jan 13 14:01:58 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.276: +4 -2 lines
Diff to previous 1.276 (colored)

Stop using envelope->status to report delivery outcome to the
runner/queue.  Instead, replace IMSG_QUEUE_MESSAGE_UPDATE with three
messages:

- IMSG_QUEUE_DELIVERY_OK
- IMSG_QUEUE_DELIVERY_TEMPFAIL
- IMSG_QUEUE_DELIVERY_PERMFAIL

1) it's less confusing as status is also used by smtp
2) it's easier to see what happens just looking at imsg traces
3) it makes the code path generally easier to follow
4) it's safer because it enforces clear semantics and intent, whereas
   the status field is loosely defined and could carry bogus values.

ok gilles@ chl@

Revision 1.276 / (download) - annotate - [select for diffs], Thu Jan 12 20:59:07 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.275: +2 -2 lines
Diff to previous 1.275 (colored)

use mbox backend for mbox delivery.

ok gilles@

Revision 1.275 / (download) - annotate - [select for diffs], Thu Jan 12 18:06:18 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.274: +1 -7 lines
Diff to previous 1.274 (colored)

Remove dead code for config reloading for now. It is not functionnal
and confusing.

ok gilles@

Revision 1.274 / (download) - annotate - [select for diffs], Thu Jan 12 15:01:33 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.273: +2 -3 lines
Diff to previous 1.273 (colored)

remove envelope_get_errormsg() and move envelope_set_errormsg()
to envelope.c

ok gilles@

Revision 1.273 / (download) - annotate - [select for diffs], Wed Jan 11 22:55:16 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.272: +1 -2 lines
Diff to previous 1.272 (colored)

remove stateful iteration from ramqueue, if we ever need to reintroduce it
we'll do it, but it isn't used and causes potential bugs

idea by Nathanael Rensel, diff by me, ok eric@

Revision 1.272 / (download) - annotate - [select for diffs], Wed Jan 11 22:24:37 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.271: +34 -1 lines
Diff to previous 1.271 (colored)

implement an envelope_ascii API that's not tied to a specific queue_backend
simplify queue_fsqueue

Revision 1.271 / (download) - annotate - [select for diffs], Wed Jan 11 17:46:36 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.270: +4 -2 lines
Diff to previous 1.270 (colored)

Simplify runner/queue by getting rid of Q_PURGE.  Instead, let smtpd
periodically clear the purge/ directory.  At init time, the fsqueue
backend simply moves the existing incoming/ dir in purge/ to discard
aborted sessions.

ok gilles@ chl@

Revision 1.270 / (download) - annotate - [select for diffs], Wed Jan 11 17:07:18 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.269: +1 -2 lines
Diff to previous 1.269 (colored)

remove dead prototype

from Nathanael Rensen

ok gilles@

Revision 1.269 / (download) - annotate - [select for diffs], Tue Dec 27 17:13:05 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.268: +1 -3 lines
Diff to previous 1.268 (colored)

Q_BOUNCE is not used anymore

ok gilles@

Revision 1.268 / (download) - annotate - [select for diffs], Sun Dec 18 22:55:31 2011 UTC (12 years, 5 months ago) by chl
Branch: MAIN
Changes since 1.267: +4 -4 lines
Diff to previous 1.267 (colored)

Sync comments with latest cleanup changes

ok eric@

Revision 1.267 / (download) - annotate - [select for diffs], Wed Dec 14 23:08:40 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.266: +2 -7 lines
Diff to previous 1.266 (colored)

finally kill queue_shared.c and move what is left to bounce.c
where it belongs.

ok gilles@

Revision 1.266 / (download) - annotate - [select for diffs], Wed Dec 14 22:28:02 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.265: +2 -4 lines
Diff to previous 1.265 (colored)

split auth_backend.c for consistency

ok chl@ gilles@

Revision 1.265 / (download) - annotate - [select for diffs], Wed Dec 14 18:42:27 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.264: +1 -2 lines
Diff to previous 1.264 (colored)

move show_queue() and related functions from queue_shared.c
to smtpctl.c

ok gilles@

Revision 1.264 / (download) - annotate - [select for diffs], Wed Dec 14 17:55:55 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.263: +1 -3 lines
Diff to previous 1.263 (colored)

make queue_fsqueue backend consistent with the backend scheme.

ok gilles@

Revision 1.263 / (download) - annotate - [select for diffs], Tue Dec 13 23:55:00 2011 UTC (12 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.262: +3 -1 lines
Diff to previous 1.262 (colored)

*finally* make use of certificate authority file if available !

bits from relayd, ok chl@, ok eric@

Revision 1.262 / (download) - annotate - [select for diffs], Tue Dec 13 22:04:35 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.261: +2 -4 lines
Diff to previous 1.261 (colored)

split user_backend.c into user.c and user_pwd.c to be consistent with the
backend scheme. Also rename USER_GETPWNAM to USER_PWD.

ok chl@ gilles@

Revision 1.261 / (download) - annotate - [select for diffs], Tue Dec 13 21:44:47 2011 UTC (12 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.260: +14 -3 lines
Diff to previous 1.260 (colored)

- introduce delivery backend API (delivery.c)
- move each delivery method to it's own delivery backend
- simplify smtpd.c accordingly
- rename A_EXT -> A_MDA since that's what we really do

ok eric@

Revision 1.260 / (download) - annotate - [select for diffs], Mon Dec 12 17:20:36 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.259: +7 -3 lines
Diff to previous 1.259 (colored)

add a session_enter_state() function to change the state of an smtp
session and allow those state changes to be traced (add traces flags
for upcoming changes while there).

ok chl@ gilles@

Revision 1.259 / (download) - annotate - [select for diffs], Mon Dec 12 16:45:16 2011 UTC (12 years, 5 months ago) by chl
Branch: MAIN
Changes since 1.258: +1 -3 lines
Diff to previous 1.258 (colored)

remove comments about dead "struct delivery"

"obvious ok" gilles@

Revision 1.258 / (download) - annotate - [select for diffs], Sun Dec 11 19:58:09 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.257: +2 -1 lines
Diff to previous 1.257 (colored)

utility function for parsing and validating SMTP response lines

ok gilles@

Revision 1.257 / (download) - annotate - [select for diffs], Sun Dec 11 17:02:10 2011 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.256: +3 -3 lines
Diff to previous 1.256 (colored)

Make the mta code a bit more straightforward:
- fetch the ssl cert earlier on if needed
- skip mta_pickup() when handling the incoming fd

ok gilles@

Revision 1.256 / (download) - annotate - [select for diffs], Thu Dec 8 17:00:28 2011 UTC (12 years, 5 months ago) by todd
Branch: MAIN
Changes since 1.255: +4 -4 lines
Diff to previous 1.255 (colored)

rename struct user to struct mta_user to avoid namespace conflict elsewhere
ok chl@ & gilles@

Revision 1.255 / (download) - annotate - [select for diffs], Mon Nov 28 20:29:27 2011 UTC (12 years, 6 months ago) by chl
Branch: MAIN
Changes since 1.254: +2 -2 lines
Diff to previous 1.254 (colored)

fix STATE_COUNT count

ok eric@ gilles@

Revision 1.254 / (download) - annotate - [select for diffs], Mon Nov 21 18:57:54 2011 UTC (12 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.253: +1 -2 lines
Diff to previous 1.253 (colored)

get rid of the "enqueue/" queue; use "incoming/" instead.

ok gilles@ chl@

Revision 1.253 / (download) - annotate - [select for diffs], Wed Nov 16 19:38:56 2011 UTC (12 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.252: +1 -5 lines
Diff to previous 1.252 (colored)

remove unused functions

ok gilles@ chl@

Revision 1.252 / (download) - annotate - [select for diffs], Wed Nov 16 11:18:55 2011 UTC (12 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.251: +2 -1 lines
Diff to previous 1.251 (colored)

Do not unlink an offline message until it has been correctly enqueued.
While there, simplify the offline_enqueue() function by doing all the
sanity checks in the forked process,  and remove all fatal(): on error,
the offline message is left untouched in the directory.  Also, get rid
of the path_starts_with() check since all paths to offline messages are
now constructed internally.

ok gilles@ chl@

Revision 1.251 / (download) - annotate - [select for diffs], Tue Nov 15 23:06:39 2011 UTC (12 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.250: +12 -17 lines
Diff to previous 1.250 (colored)

Qwalk, our API to linearly walk over the persistent queue, did not take the
queue_backend into account and assumed a filesystem with a specific layout.

This commit does plenty of things:

- make qwalk an abstraction in the queue_backend API, and impose queue
  drivers to implement qwalk_open(), qwalk() and qwalk_close();

- move previous qwalk_open(), qwalk() and qwalk_close() to the fsqueue
  driver since they were fsqueue specific ...

- make qwalk API work with msgid/evpid instead of pathnames since we're
  going to use the queue_backend API to load envelopes by evpid anyway;

- makes smtpd use *solely* the queue_backend API when manipulating the
  queue. pathnames were removed from smtpd.h and moved into the fsqueue
  which means we can now store a queue anywhere ... as long as we write
  the ten functions or so required for a queue driver ;-)

ok eric@, ok chl@

Revision 1.250 / (download) - annotate - [select for diffs], Mon Nov 14 19:23:41 2011 UTC (12 years, 6 months ago) by chl
Branch: MAIN
Changes since 1.249: +3 -1 lines
Diff to previous 1.249 (colored)

when receiving an unexpected imsg, print its name.

with help and ideas from eric@

ok eric@ gilles@

Revision 1.249 / (download) - annotate - [select for diffs], Mon Nov 14 11:53:10 2011 UTC (12 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.248: +2 -1 lines
Diff to previous 1.248 (colored)

The spool and offline directories are backend-independent, so they
must be created early by smtpd, rather than in fsqueue.

ok gilles@ chl@

Revision 1.248 / (download) - annotate - [select for diffs], Mon Nov 7 11:14:10 2011 UTC (12 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.247: +1 -2 lines
Diff to previous 1.247 (colored)

Let the smtpd process handle the enqueueing of offline messages at
startup, rather than playing tricks with the runner. This will allow
further simplifications and improvements in the runner/queue.

ok gilles@

Revision 1.247 / (download) - annotate - [select for diffs], Wed Oct 26 20:47:31 2011 UTC (12 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.246: +5 -5 lines
Diff to previous 1.246 (colored)

- fix smtpctl pause/resume so the ramqueue scheduling is done correctly
- rename IMSG and smtpctl pause/resume parameters
- update man page

tested by me, ok chl@, eric@

Revision 1.246 / (download) - annotate - [select for diffs], Sun Oct 23 15:36:53 2011 UTC (12 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.245: +5 -5 lines
Diff to previous 1.245 (colored)

a few important fixes:

- use correct endianness when dumping/loading port
- use the right flag set when dumping/loading flags
- keep and use the authmap name when needed, rather than an id that
  might change when smtpd is restarted
- dump/load the authmap name with the envelope
- remove the rule struct from rq_batch as only the relay info is useful

ok gilles@

Revision 1.245 / (download) - annotate - [select for diffs], Sun Oct 23 13:08:18 2011 UTC (12 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.244: +1 -2 lines
Diff to previous 1.244 (colored)

relay_as is not used anuwhere

ok gilles@

Revision 1.244 / (download) - annotate - [select for diffs], Sun Oct 23 13:03:05 2011 UTC (12 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.243: +7 -3 lines
Diff to previous 1.243 (colored)

introduce Q_CORRUPT and queue_backend operation to move a message from
schedule queue to corrupt queue upon envelope loading failure.

tested by me, ok eric@

Revision 1.243 / (download) - annotate - [select for diffs], Sun Oct 23 09:30:07 2011 UTC (12 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.242: +36 -41 lines
Diff to previous 1.242 (colored)

fsqueue no longer stores envelopes by dumping the structure, instead use a
couple of load/dump functions to convert to and from a human readable fmt.
while at it kill struct delivery and merge back its fields to the envelope.

this basically means we shouldn't require users to flush their queues every
time we make a change to struct envelope.

work is not done, but we're at a better state than the binary fsqueue so
we'll improve it in-tree.

has been running on my own box for the last 12 hours or so
ok eric@, chl@

Revision 1.242 / (download) - annotate - [select for diffs], Sat Oct 22 00:16:34 2011 UTC (12 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.241: +4 -1 lines
Diff to previous 1.241 (colored)

Add a log_trace() call to toggle logging of specific debugging info in
verbose mode, and an associated -T command line option.  Use it for
the imsg traces.

Requested by gilles@ who doesn't like verbose to be too verbose.

ok gilles@ chl@

Revision 1.241 / (download) - annotate - [select for diffs], Sun Oct 9 18:39:54 2011 UTC (12 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.240: +2 -1 lines
Diff to previous 1.240 (colored)

show messages sent between processes in debug mode

ok gilles@ chl@

Revision 1.240 / (download) - annotate - [select for diffs], Mon Sep 19 13:10:47 2011 UTC (12 years, 8 months ago) by chl
Branch: MAIN
Changes since 1.239: +6 -6 lines
Diff to previous 1.239 (colored)

Quick fix to avoid fatal() when we receive a line which have a length of exactly 1024.

Better fix comming soon.

Committing on behalf of gilles@

Revision 1.239 / (download) - annotate - [select for diffs], Sun Sep 18 21:37:53 2011 UTC (12 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.238: +2 -2 lines
Diff to previous 1.238 (colored)

a single ramqueue message may be shared by many ramqueue envelopes to be
delivered to many ramqueue hosts, therefore storing the rq_host pointer in
the rq_msg envelope is wrong and causes baaaaad behavior.

this commit fixes reliability issues in runner process, experienced and
reported by many

Revision 1.238 / (download) - annotate - [select for diffs], Mon Sep 12 20:47:15 2011 UTC (12 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.237: +2 -1 lines
Diff to previous 1.237 (colored)

- introduce filtermask in struct smtpd
- do not forward lines to mfa when FILTER_DATALINE is not set in filtermask

prevents smtpd from handling mails slowly while I'm hacking on filters
support

Revision 1.237 / (download) - annotate - [select for diffs], Thu Sep 1 19:56:49 2011 UTC (12 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.236: +50 -66 lines
Diff to previous 1.236 (colored)

Introduce a small set of functions to manage stat counters in a
simpler and hopefully saner way.

ok gilles@ chl@

Revision 1.236 / (download) - annotate - [select for diffs], Wed Aug 31 18:56:30 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.235: +3 -2 lines
Diff to previous 1.235 (colored)

add support for per-line DATA callbacks, this allows filters to take their
decisions *while* the message is being received by the client.

Revision 1.235 / (download) - annotate - [select for diffs], Sat Aug 27 22:37:35 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.234: +2 -2 lines
Diff to previous 1.234 (colored)

typo

Revision 1.234 / (download) - annotate - [select for diffs], Sat Aug 27 22:32:41 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.233: +39 -9 lines
Diff to previous 1.233 (colored)

initial support for a session-time filtering API

currently only HELO/EHLO, MAIL, RCPT are supported, however ... I have
voluntarily disabled filters at smtpd.conf level so people don't play with
it until the API has stabilized a bit

discussed with several people in private, no one opposed the feature

Revision 1.233 / (download) - annotate - [select for diffs], Wed Aug 17 20:35:11 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.232: +8 -5 lines
Diff to previous 1.232 (colored)

move ramqueue_host pointer from ramqueue_envelope to ramqueue_message.
this allows us to save one pointer from each envelope stored in ram while
still allowing O(1) host lookups by ramqueue_envelope.

Revision 1.232 / (download) - annotate - [select for diffs], Wed Aug 17 20:04:43 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.231: +6 -4 lines
Diff to previous 1.231 (colored)

- teach smtpctl remove about the new ramqueue structure
- bonus #1: O(log n) removal of envelopes
- bonus #2: removing all envelopes that have the same msgid works again

Revision 1.231 / (download) - annotate - [select for diffs], Wed Aug 17 19:36:23 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.230: +8 -2 lines
Diff to previous 1.230 (colored)

- introduce ramqueue_lookup_{host,message,envelope} to perform lookups in
  the new ramqueue structure
- introduce ramqueue_reschedule() and ramqueue_reschedule_envelope() which
  to reschedule a message or a specific envelope.

O(n) ->  O(log n)  \o/

Revision 1.230 / (download) - annotate - [select for diffs], Tue Aug 16 19:02:03 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.229: +21 -5 lines
Diff to previous 1.229 (colored)

add a host-tree and an envelope-tree in the ramqueue, they will be used to
improve scheduling and general ramqueue operations. unused yet

Revision 1.229 / (download) - annotate - [select for diffs], Thu Jul 21 23:29:24 2011 UTC (12 years, 10 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_0_BASE, OPENBSD_5_0
Changes since 1.228: +6 -1 lines
Diff to previous 1.228 (colored)

- update smtpctl.8 to reflect reality
- bring back 'smtpctl schedule' and 'smtpctl remove' to life

Things you should know:

The ramqueue data structure is not finished yet and lacks an envelope tree
for evpid lookups. I wanted to wait until I'm done but too many people are
affected by not being able to reschedule envelopes, this is a quick fix.

So right now there's an O(rrible) complexity as both commands will perform
a (possibly aborted) queue scan leading to O(n). I will make that O(log n)
soon.

Also, smtpctl remove no longer supports removing an entire message, I will
fix that very soon too.

Revision 1.228 / (download) - annotate - [select for diffs], Wed Jul 20 10:22:54 2011 UTC (12 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.227: +9 -1 lines
Diff to previous 1.227 (colored)

Fix reporting of permanent/temporary failures for MX lookups.
Simplify code a bit while there.

ok gilles@

Revision 1.227 / (download) - annotate - [select for diffs], Thu Jun 9 17:41:52 2011 UTC (12 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.226: +3 -1 lines
Diff to previous 1.226 (colored)

'relay as' and 'relay via as' rules allow smtpd to rewrite the user part,
the domain part or the entire address of the sender at the SMTP sesssion
level. this is not masquerade but allows smtpd to communicate with hosts
that do a check of SMTP sender fqdn.

sent to tech@, a couple 'no regression' feedbacks

Revision 1.226 / (download) - annotate - [select for diffs], Sat May 21 19:57:34 2011 UTC (13 years ago) by gilles
Branch: MAIN
Changes since 1.225: +1 -13 lines
Diff to previous 1.225 (colored)

remove unused commented structure

Revision 1.225 / (download) - annotate - [select for diffs], Sat May 21 18:43:08 2011 UTC (13 years ago) by gilles
Branch: MAIN
Changes since 1.224: +3 -8 lines
Diff to previous 1.224 (colored)

until now the map_backend API was not really useful for backends that are
not key/val stores. refactored a bit so that smtpd can really take
advantage of backends. preliminary work for ldap support ;-)

no functionnal change

Revision 1.224 / (download) - annotate - [select for diffs], Tue May 17 18:54:32 2011 UTC (13 years ago) by gilles
Branch: MAIN
Changes since 1.223: +29 -3 lines
Diff to previous 1.223 (colored)

introduce new user_backend API for smtpd to lookup the users it wants to
deliver mail to. the only backend supported for now is USER_GETPWNAM and
it is not yet possible to switch to an alternate backend.

yes this means that we're very close from smtpd being able to handle fully
virtual accounts for both incoming and outgoing messages.

Revision 1.223 / (download) - annotate - [select for diffs], Tue May 17 16:42:06 2011 UTC (13 years ago) by gilles
Branch: MAIN
Changes since 1.222: +16 -3 lines
Diff to previous 1.222 (colored)

smtpd now uses an auth_backend API to authenticate users that are allowed
to send mail so they do not necessarily need a local system account.

two backends are provided by default, bsd_auth(3) and getpwnam(3), however
smtpd will only select bsd_auth(3) for the moment and not provide a way to
chose any other backend (that's on purpose ;p).

bye bye authenticate() !

Revision 1.222 / (download) - annotate - [select for diffs], Mon May 16 21:05:52 2011 UTC (13 years ago) by gilles
Branch: MAIN
Changes since 1.221: +117 -101 lines
Diff to previous 1.221 (colored)

murder struct path and make sure smtpd uses simpler structures that do not
bring a shitload of unnecessary information everywhere. this required many
parts of smtpd to be refactored and more specifically envelope expansion.

in the process lots of code got simplified, and the envelope expansion code
has been isolated to lka_session.c with some longstanding bugs fixed.

Diff has been tested by many with no major regression reported.
armani@ spotted a bug in a setup where a domain is listed a both primary
and virtual, I will fix that in-tree as it's becoming painful to maintain
this diff out.

Revision 1.221 / (download) - annotate - [select for diffs], Fri May 6 19:21:43 2011 UTC (13 years ago) by eric
Branch: MAIN
Changes since 1.220: +1 -21 lines
Diff to previous 1.220 (colored)

move dns session specific structs and prototypes out of smtpd.h.

ok gilles@

Revision 1.220 / (download) - annotate - [select for diffs], Sun May 1 12:57:11 2011 UTC (13 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.219: +52 -64 lines
Diff to previous 1.219 (colored)

the smtpd env is meant to be global, so do not pass it all around.

discussed with and ok gilles@

Revision 1.219 / (download) - annotate - [select for diffs], Sun Apr 17 13:36:07 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.218: +111 -114 lines
Diff to previous 1.218 (colored)

cleanups, cosmethic changes, functions that should be static are now static
no functionnal change

Revision 1.218 / (download) - annotate - [select for diffs], Sun Apr 17 12:46:46 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.217: +4 -15 lines
Diff to previous 1.217 (colored)

remove unused IMSG_ defines

Revision 1.217 / (download) - annotate - [select for diffs], Sun Apr 17 11:39:23 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.216: +23 -23 lines
Diff to previous 1.216 (colored)

a structure describing an envelope should be called struct envelope, not
struct message ...

Revision 1.216 / (download) - annotate - [select for diffs], Sun Apr 17 11:16:57 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.215: +7 -34 lines
Diff to previous 1.215 (colored)

no functionnal change, getting rid of deprecated prototypes

Revision 1.215 / (download) - annotate - [select for diffs], Fri Apr 15 19:03:28 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.214: +1 -4 lines
Diff to previous 1.214 (colored)

whenever an envelope is reinserted into the ramqueue after a trip to mda or
mta, call runner_reset_events() so runner starts reprocessing ramqueue

Revision 1.214 / (download) - annotate - [select for diffs], Fri Apr 15 17:01:05 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.213: +19 -15 lines
Diff to previous 1.213 (colored)

kill message_id and message_uid

smtpd now has an evpid associated to each delivery message, the evpid is an
u_int64_t where the upper 32 bits are the msgid, and the 32 bits are the
envelope unique identifier for that message. this results in lots of space
saved in both disk-based and ram-based queues, but also simplifies a lot of
code.

change has been stressed on my desktop, and has ran on my MX for the entire
afternoon without a regression.

Revision 1.213 / (download) - annotate - [select for diffs], Thu Apr 14 23:26:16 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.212: +4 -5 lines
Diff to previous 1.212 (colored)

- implement missing operations for fsqueue:
	fsqueue_envelope_create(), fsqueue_message_purge()
- kill deprecated functions in queue_shared.c

At this point fsqueue backend is almost complete, all that is left to do is
to move the qwalk() API inside the queue_backend API, then make sure smtpd
is no longer calling anything queue related directly.

Revision 1.212 / (download) - annotate - [select for diffs], Thu Apr 14 22:46:38 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.211: +2 -2 lines
Diff to previous 1.211 (colored)

no functionnal change

Revision 1.211 / (download) - annotate - [select for diffs], Thu Apr 14 22:36:09 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.210: +3 -3 lines
Diff to previous 1.210 (colored)

- implement fsqueue_message_create() and fsqueue_message_commit()
- change a few prototypes to allow bounce messages to use the
	queue_backend API until it gets merged in
- kill functions of the queue API that have been deprecated

Revision 1.210 / (download) - annotate - [select for diffs], Thu Apr 14 20:11:08 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.209: +3 -2 lines
Diff to previous 1.209 (colored)

fsqueue queue backend will implement a filesystem queue:
- fsqueue->setup() performs the queue initialization;
- fsqueue->message() controls messages;
- fsqueue->envelope() controls envelopes;

This commit brings the following to fsbackend:
fsqueue_setup(), fsqueue_message_delete(), fsqueue_envelope_load(),
fsqueue_envelope_update(), fsqueue_envelope_delete().

It also makes smtpd use the queue_backend API for these operations.

Revision 1.209 / (download) - annotate - [select for diffs], Thu Apr 14 17:06:43 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.208: +55 -1 lines
Diff to previous 1.208 (colored)

smtpd makes too many assumptions about the structure and layout of its
disk-based queue, it makes it near impossible to make changes to it without
editing twenty files... how am i going to implement mongodb support ? :-)

bring a new queue_backend API which hides the details of the disk-based
queue to smtpd. it is not "plugged in" yet and I'm filling the holes.

Revision 1.208 / (download) - annotate - [select for diffs], Wed Apr 13 20:53:18 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.207: +67 -39 lines
Diff to previous 1.207 (colored)

following an idea from jacekm@, smtpd now uses a ram-queue instead of doing
a continuous walk on the disk-queue. the implementation differs from what
jacekm@ commited (and I backed out) a while ago in that it uses a queue and
a host tree required for upcoming features.

code will be improved in tree, it requires changes to be done in queue and
bounce API, I just wanted to commit a working version first ...

tested by todd@ and I

Revision 1.207 / (download) - annotate - [select for diffs], Sat Apr 2 16:40:19 2011 UTC (13 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.206: +12 -1 lines
Diff to previous 1.206 (colored)

add stat counters for the lookup agent

ok gilles@

Revision 1.206 / (download) - annotate - [select for diffs], Tue Mar 29 20:43:51 2011 UTC (13 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.205: +1 -2 lines
Diff to previous 1.205 (colored)

remove unused code now that reverse lookups are done through asr.

ok gilles@

Revision 1.205 / (download) - annotate - [select for diffs], Sat Mar 26 21:41:31 2011 UTC (13 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.204: +2 -1 lines
Diff to previous 1.204 (colored)

missing file in previous commit

ok gilles@

Revision 1.204 / (download) - annotate - [select for diffs], Sat Mar 26 10:59:59 2011 UTC (13 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.203: +2 -2 lines
Diff to previous 1.203 (colored)

have the client API receive a stdio stream rather than a fd to the message
fd. this shifts responsibility for the fclose to the caller, prevents a
memory leak and makes everyone happy.

diff by Jared Yanovich, thanks !

Revision 1.203 / (download) - annotate - [select for diffs], Sat Mar 26 10:54:22 2011 UTC (13 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.202: +2 -3 lines
Diff to previous 1.202 (colored)

use an index for iterating into the mx list.

ok gilles@

Revision 1.202 / (download) - annotate - [select for diffs], Tue Mar 15 19:24:55 2011 UTC (13 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.201: +3 -1 lines
Diff to previous 1.201 (colored)

let smtpd use user-provided Diffie-Hellman parameters for ephemeral key
exchange. if no DH parameters are found, fallback to builtin parameters
as was done until now.

since we now accept user-provided DH parameters, make smtpd more strict
and fatal() if the parameters are bogus.

bump the key size of the DH parameters from 512bits to 1024bits, it might
be bumped further after some more research.

thanks to mikeb@ for his suggestions

diff ok mikeb@ , man ok jmc@

Revision 1.201 / (download) - annotate - [select for diffs], Wed Mar 9 20:59:22 2011 UTC (13 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.200: +3 -1 lines
Diff to previous 1.200 (colored)

smtpctl show stats displays inet4/inet6 repartition for incoming sessions

Revision 1.200 / (download) - annotate - [select for diffs], Mon Nov 29 15:25:56 2010 UTC (13 years, 6 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_4_9_BASE, OPENBSD_4_9
Changes since 1.199: +32 -8 lines
Diff to previous 1.199 (colored)

replace the fork-based-non-blocking-resolver-hack by shiny async resolver
written by eric@. it is still experimental but still better than what we
had earlier so ... we'll improve in tree :)

diff by me with *lots* of help from eric@, tested by todd and I (and a
few people out there)

Revision 1.199 / (download) - annotate - [select for diffs], Sun Nov 28 13:56:43 2010 UTC (13 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.198: +76 -69 lines
Diff to previous 1.198 (colored)

a bit of .h cleanups, no functionnal change

Revision 1.198 / (download) - annotate - [select for diffs], Wed Nov 24 23:27:04 2010 UTC (13 years, 6 months ago) by todd
Branch: MAIN
Changes since 1.197: +11 -1 lines
Diff to previous 1.197 (colored)

add *maxactive stats
"ok and no need to keep them for yourself" gilles@

Revision 1.197 / (download) - annotate - [select for diffs], Fri Oct 29 09:16:08 2010 UTC (13 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.196: +6 -3 lines
Diff to previous 1.196 (colored)

smtpd no longer knows a map called "secrets" which holds credentials for
authenticated relaying. one can create many maps holding credentials and
name them however he/she wants, just like any other map.

teach smtpd how to select a credentials map at the rule-level allowing a
setup to relay through the same MX with different credentials depending
on the source.

smtpd.conf.5 updated to reflect changes with help from jmc@

Revision 1.196 / (download) - annotate - [select for diffs], Thu Oct 28 21:15:50 2010 UTC (13 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.195: +3 -1 lines
Diff to previous 1.195 (colored)

teach smtpd how to handle per-rule delays for message expiry, this allows
some rules to have a longer expiry delay than the default:

	accept for [...] relay expire 8d  # will stay 8 days in queue

I added the man page bits so I don't forget but I need to reword it a bit

Revision 1.195 / (download) - annotate - [select for diffs], Sat Oct 9 22:05:35 2010 UTC (13 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.194: +196 -124 lines
Diff to previous 1.194 (colored)

backout the "new" queue code commited 4 months ago. it has many good ideas,
is way more optimized than what we had earlier and there's definitely stuff
we want to keep, however it is early optimization that doesn't account for
many features and makes them hard (if not impossible) to write without
ugly workarounds that ruin the purpose of the optimizations.

the backout goes to 30 May's right before the commit and catches up on all
the non-queue related commits that happened since then.

i'll work on reintroducing the ideas from this queue when the basic
features we expect from a MTA are implemented.

suggested on tech@ about a week ago, no objections, several "please make
smtpd move forward" mails from hackers and tech readers.

Revision 1.194 / (download) - annotate - [select for diffs], Mon Sep 20 09:01:09 2010 UTC (13 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.193: +3 -4 lines
Diff to previous 1.193 (colored)

- fix a regression caused by latest commit (long story made short: do not
  attempt to expand the local delivery buffer when relaying mail, it was
  kind of ok before but no longer is)
- use the same buffer for local deliveries to files and commands

tested by jmc@ and I

Revision 1.193 / (download) - annotate - [select for diffs], Thu Jun 10 19:34:51 2010 UTC (13 years, 11 months ago) by chl
Branch: MAIN
CVS Tags: OPENBSD_4_8_BASE, OPENBSD_4_8
Changes since 1.192: +2 -1 lines
Diff to previous 1.192 (colored)

allow configure queue expiry

with help from jacekm@

ok gilles@ jacekm@

Revision 1.192 / (download) - annotate - [select for diffs], Wed Jun 9 20:00:55 2010 UTC (13 years, 11 months ago) by zinovik
Branch: MAIN
Changes since 1.191: +2 -2 lines
Diff to previous 1.191 (colored)

switch `ref' data type to int, because it is tested for negative value in
queue_mem_content_unref() function

ok jacekm@, gilles@

Revision 1.191 / (download) - annotate - [select for diffs], Tue Jun 1 23:06:25 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.190: +122 -194 lines
Diff to previous 1.190 (colored)

new queue, again; gcc2 compile tested by deraadt

Revision 1.190 / (download) - annotate - [select for diffs], Tue Jun 1 19:47:09 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.189: +194 -122 lines
Diff to previous 1.189 (colored)

New queue doesn't compile on gcc2, back out.  Spotted by deraadt@

Revision 1.189 / (download) - annotate - [select for diffs], Tue Jun 1 14:21:52 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.188: +2 -1 lines
Diff to previous 1.188 (colored)

Schedule newly arrived mail immediately, ie. place it at the beginning
of the list of next items to try, or near the beginning if the schedule
contains expired mail, which is of highest priority.

Revision 1.188 / (download) - annotate - [select for diffs], Mon May 31 23:50:28 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.187: +2 -2 lines
Diff to previous 1.187 (colored)

oops

Revision 1.187 / (download) - annotate - [select for diffs], Mon May 31 23:38:56 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.186: +122 -195 lines
Diff to previous 1.186 (colored)

Rewrite entire queue code.

Major goals:

1) Fix bad performance caused by the runner process doing full queue
read in 1s intervals.  My Soekris can now happily accept >50 msg/s
while having multi-thousand queue; before, one hundred queue would
bring the system to its knees.

2) Introduce Qmail-like scheduler that doesn't write as much to the
disk so that it needs less code for servicing error conditions,
which in some places can be tricky to get right.

3) Introduce separation between the scheduler and the backend; these
two queue aspects shouldn't be too tied too each other.  This means
that eg. storing queue in SQL requires rewrite of just queue_backend.c.

4) Make on-disk queue format architecture independent, and more
easily extensible, to reduce number of flag days in the future.

Minor goals:

ENOSPC no longer prevents delivery attempts, fixed session limiting
for relayed mail, improved batching of "relay via" mails, human-readable
mailq output, "show queue raw" command, clearer logging, sending
of single bounce about multiple recipients, exact delay= computation,
zero delay between deliveries while within session limit (currently
1s delay between re-scheduling is enforced), mta no longer requests
content fd, corrected session limit for bounce submissions, tiny
<100B queue files instead of multi-KB, detect loops before accepting
mail, reduce traffic on imsg channels by killing enormous struct
submit_status.

Revision 1.186 / (download) - annotate - [select for diffs], Mon May 31 22:25:26 2010 UTC (14 years ago) by chl
Branch: MAIN
Changes since 1.185: +2 -2 lines
Diff to previous 1.185 (colored)

imsg_compose_event() return value was never checked. Make it fatal() if needed.

ok jacekm@ gilles@

Revision 1.185 / (download) - annotate - [select for diffs], Thu May 27 15:36:04 2010 UTC (14 years ago) by gilles
Branch: MAIN
Changes since 1.184: +2 -3 lines
Diff to previous 1.184 (colored)

when a rule has two conditions (ie: accept for { domain foo, domain bar } )
expand to two rules each having its own condition rather than one rule
with a tail queue of conditions. this simplifies code a bit and removes a
couple hacks.

basic testing by oga and me

Revision 1.184 / (download) - annotate - [select for diffs], Thu May 27 11:18:34 2010 UTC (14 years ago) by gilles
Branch: MAIN
Changes since 1.183: +1 -11 lines
Diff to previous 1.183 (colored)

struct opt as not used anywhere else, kill it, we can reintroduce it later
if we feel a need for it

Revision 1.183 / (download) - annotate - [select for diffs], Thu May 27 11:17:29 2010 UTC (14 years ago) by gilles
Branch: MAIN
Changes since 1.182: +1 -2 lines
Diff to previous 1.182 (colored)

kill struct opt from struct rule, we don't use it, we don't need it

Revision 1.182 / (download) - annotate - [select for diffs], Tue Apr 27 10:17:53 2010 UTC (14 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.181: +2 -2 lines
Diff to previous 1.181 (colored)

this commit enables "plain" as a backend for maps (that means aliases,
virtual AND secrets), adds a description in smtpd.conf.5 and removes a
mention to special map "aliases" which was removed a while ago.

to use plain maps:  map "myaliases" { source plain "/etc/mail/aliases" }

code diff was okayd a while ago by jacekm@

Revision 1.181 / (download) - annotate - [select for diffs], Tue Apr 27 09:49:23 2010 UTC (14 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.180: +34 -24 lines
Diff to previous 1.180 (colored)

initial work at fixing aliases support:
- kill struct alias, struct expandnode is used instead
- introduce map_parse_alias() and map_parse_virtual()
- aliases and virtual code no longer assume db(3) but use the map API which
  lets them become backend agnostic AND value-checked. this actually makes
  the code simpler by removing all values parsing from aliases.c
- rename K_SECRETS -> K_SECRET, K_ALIASES -> K_ALIAS for consistency the
  enum has singular names.
- aliases, virtual and forward now work with an expandtree and deal with
  multiple levels of resolving by merging expandtree's

more coming soon ;)

Revision 1.180 / (download) - annotate - [select for diffs], Thu Apr 22 12:56:33 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.179: +1 -2 lines
Diff to previous 1.179 (colored)

Fix a case of runner trying to send imsg directly to smtp process instead
of forwarding it via queue.

Revision 1.179 / (download) - annotate - [select for diffs], Wed Apr 21 21:47:39 2010 UTC (14 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.178: +7 -2 lines
Diff to previous 1.178 (colored)

introduce first map parser for maps of kind K_SECRETS !

map_parse_secret() converts a map value into a struct map_secret. lka no
longer needs to do any parsing, it simply calls map_lookup() with kind
K_SECRETS, checks if it returned a !NULL value, and call lka_encode_secret
to safely do the base64 encoding.

Revision 1.178 / (download) - annotate - [select for diffs], Wed Apr 21 21:04:29 2010 UTC (14 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.177: +15 -1 lines
Diff to previous 1.177 (colored)

map.c is growing big, split it into three different files:
map.c contains the map API calls as usable by smtpd processes,
map_backend.c contains backend implementations and map_parser.c contains
parser implementations used internally by the map API

Revision 1.177 / (download) - annotate - [select for diffs], Wed Apr 21 19:53:16 2010 UTC (14 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.176: +4 -3 lines
Diff to previous 1.176 (colored)

map_lookup() takes an additionnal parameter of type enum map_kind which
will be used to select the appropriate map parser. make sure every call
to map_lookup() is updated. map_lookup() currently ignores the value.

Revision 1.176 / (download) - annotate - [select for diffs], Wed Apr 21 19:45:07 2010 UTC (14 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.175: +7 -1 lines
Diff to previous 1.175 (colored)

introduce enum map_kind, the map_parsers array and map_parser_lookup()
which will be used to perform stronger checks on map values. they are
unused yet ...

Revision 1.175 / (download) - annotate - [select for diffs], Wed Apr 21 18:54:43 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.174: +9 -14 lines
Diff to previous 1.174 (colored)

Runner process is just a helper for queue, so tear down its imsg
channels to parent, mda, mta, lka, smtp, and control.  This leaves
just the channel to queue, which forwards imsgs on runner's behalf
and redirects any replies back to it.

OK gilles@

Revision 1.174 / (download) - annotate - [select for diffs], Tue Apr 20 15:34:56 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.173: +6 -1 lines
Diff to previous 1.173 (colored)

Kill *2400* lines of code by abstracting common bits of the imsg handlers.

Revision 1.173 / (download) - annotate - [select for diffs], Mon Apr 19 10:12:48 2010 UTC (14 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.172: +2 -1 lines
Diff to previous 1.172 (colored)

basic support for SIZE extension, has been sitting in my tree for a month
or so ...

okayd by jacekm@ a while ago

Revision 1.172 / (download) - annotate - [select for diffs], Mon Apr 19 08:14:07 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.171: +22 -20 lines
Diff to previous 1.171 (colored)

Simplify local delivery codepath:
- replace uses of struct batch in the parent with simpler struct delivery.
- replace IMSG_BATCH_* dance with single IMSG_MDA_SESS_NEW.
- make mda assume it delivers to external program over a pipe.
- fork helper process when delivering to maildir or a file.

New feature: upon external mda failure use last line of its output
as an error message.

With input and tests from nicm@.

OK nicm@ gilles@

Revision 1.171 / (download) - annotate - [select for diffs], Sun Apr 11 22:46:28 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.170: +2 -2 lines
Diff to previous 1.170 (colored)

Increase limit on the length of "user" in user@domain.com to a value
higher than specified in rfc.  Too many mailers don't conform to it,
and it is harmless as far as I can see.

From Tim van der Molen <tbvdm@xs4all.nl>

OK gilles@

Revision 1.170 / (download) - annotate - [select for diffs], Wed Mar 3 11:11:55 2010 UTC (14 years, 3 months ago) by jacekm
Branch: MAIN
CVS Tags: OPENBSD_4_7_BASE, OPENBSD_4_7
Changes since 1.169: +2 -3 lines
Diff to previous 1.169 (colored)

tweak mda.c rev. 1.36: eliminate risk of busy waiting for socket
to become writable, and make code more idiomatic.

tested by nicm@

ok gilles@

Revision 1.169 / (download) - annotate - [select for diffs], Wed Feb 17 13:47:31 2010 UTC (14 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.168: +2 -3 lines
Diff to previous 1.168 (colored)

the map api becomes backend-agnostic with initial support for db(3) and
stdio(3) backends, though for now we only enable db(3). this is the first
commit of a serie to improve maps and everything related.

idea discussed with and diff okay jacekm@

Revision 1.168 / (download) - annotate - [select for diffs], Sun Jan 10 16:42:35 2010 UTC (14 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.167: +8 -1 lines
Diff to previous 1.167 (colored)

- teach runner how to remove a message from queue given a message id/uid
	and assuming message is not in processing/scheduled state
- teach smtpctl how to request message removal from runner

discussed with todd@, idea ok jacekm@

Revision 1.167 / (download) - annotate - [select for diffs], Sun Jan 3 14:37:37 2010 UTC (14 years, 5 months ago) by chl
Branch: MAIN
Changes since 1.166: +3 -1 lines
Diff to previous 1.166 (colored)

Implement "log verbose" and "log brief" to enable or disable verbose debug
logging on runtime.

Based on claudio@'s work on ripd, ospfd, ospf6d, dvmrpd, ldpd, bgpd.

With help/ideas/testing from gilles@ jacekm@ todd@

ok jacekm@

Revision 1.166 / (download) - annotate - [select for diffs], Thu Dec 24 14:19:46 2009 UTC (14 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.165: +1 -3 lines
Diff to previous 1.165 (colored)

kill PATH_RUNQUEUEHIGH and PATH_RUNQUEUELOW, i had great plans for them but
they're still unused months later and there's many ways to achieve having a
set of priorities on envelopes without needing additionnal queues.

Revision 1.165 / (download) - annotate - [select for diffs], Mon Dec 14 19:56:55 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.164: +2 -2 lines
Diff to previous 1.164 (colored)

Tweak the logic behind setting the fd limits so that smtpd is less likely
to get upset by custom soft/hard ulimit settings.

Suggested by todd@

Revision 1.164 / (download) - annotate - [select for diffs], Mon Dec 14 18:21:53 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.163: +3 -1 lines
Diff to previous 1.163 (colored)

Control maximum number of bounce sessions similarly to how the mta and mda
are now controlled.

Revision 1.163 / (download) - annotate - [select for diffs], Mon Dec 14 16:44:14 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.162: +4 -2 lines
Diff to previous 1.162 (colored)

Impose sessions limit on the delivery sessions (mta and mda).

Revision 1.162 / (download) - annotate - [select for diffs], Mon Dec 14 13:17:51 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.161: +4 -1 lines
Diff to previous 1.161 (colored)

Do non-blocking I/O when delivering locally over a pipe.

Revision 1.161 / (download) - annotate - [select for diffs], Sun Dec 13 22:02:55 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.160: +10 -1 lines
Diff to previous 1.160 (colored)

Use safe fd limits in smtp, lka, queue, and control.  Removes a
possibility for fd-starvation fatal when under heavy load.

Revision 1.160 / (download) - annotate - [select for diffs], Sat Dec 12 10:33:11 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.159: +2 -2 lines
Diff to previous 1.159 (colored)

Simplify client_* api, mainly by making fatal conditions result in immediate
fatals instead of passing the error up (kills ~300 lines).

Implement sending of the QUIT command which replaces crude close(2).

tested by gilles@, todd@

Revision 1.159 / (download) - annotate - [select for diffs], Fri Nov 13 11:27:52 2009 UTC (14 years, 6 months ago) by jacekm
Branch: MAIN
Changes since 1.158: +4 -1 lines
Diff to previous 1.158 (colored)

Log FQDN and IP of the server we handed mail to.  As a bonus, don't delay
logging of successful deliveries until all MXs were tried, plus add logging
of 5yz replies.

tested by todd@, "reads ok" gilles@

Revision 1.158 / (download) - annotate - [select for diffs], Mon Nov 9 23:49:34 2009 UTC (14 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.157: +12 -4 lines
Diff to previous 1.157 (colored)

- add a reference count and flags to struct expand_node
- during expansion, no longer create a new node for each result but try to
  lookup for an existing equivalent node and increment its reference count
  so that: a) we save on memory, b) we don't need to expand the same users
  again and again just because they keep appearing in expansion results.
- while expanding, flag nodes as F_EXPAND_DONE so that we know which nodes
  we already processed
- be smarter when expanding, if we have a clue that an iteration has not
  brought any new result (because no new nodes were added and all existing
  nodes have F_EXPAND_DONE), end expansion and proceed to delivery.
- various small cleanups

discussed with jacekm@ yesterday, rebuild aliases db, make clean

Revision 1.157 / (download) - annotate - [select for diffs], Sun Nov 8 23:20:07 2009 UTC (14 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.156: +8 -7 lines
Diff to previous 1.156 (colored)

add an alias_to_expand_node() function and use it in aliases.c

Revision 1.156 / (download) - annotate - [select for diffs], Sun Nov 8 23:08:56 2009 UTC (14 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.155: +38 -29 lines
Diff to previous 1.155 (colored)

rework a bit expansion and data structures involved in the expansion so we
no longer have a direct mapping between structures saved in aliases/virtual
db and structures used at runtime during expansion.

side effects ? struct alias is smaller, databases are smaller and it is no
longer necessary to rebuild aliases/virtual databases each time jacekm@ or
I make changes to some obscure structure used indirectely during expansion

rebuild databases, flush queues, make clean

Revision 1.155 / (download) - annotate - [select for diffs], Sun Nov 8 21:40:05 2009 UTC (14 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.154: +15 -8 lines
Diff to previous 1.154 (colored)

- make aliases expansion use a rb tree instead of a tail queue, the code
  doesn't take advantage of the new structure yet, but this was a needed
  change for upcoming improvements.
- introduce aliasestree_{lookup,insert,remove} to the aliases api
- rename queue_generate_id() to generate_uid() and move it to utils.c as
  it is used all over the place and not only in queue

tree idea discussed with jacekm@, if you update rebuild aliases db, make
clean and flush queue

Revision 1.154 / (download) - annotate - [select for diffs], Sun Nov 8 19:38:26 2009 UTC (14 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.153: +5 -1 lines
Diff to previous 1.153 (colored)

first commit of a serie to cleanup, simplify and improve aliases resolution
which is probably the most complex code in smtpd right now. no longer use a
single list to hold aliases to be resolved and resolved aliases, and do not
use struct alias to hold resolved aliases. instead use a delivery list that
is a list of struct path, and populate it with resolved aliases.

idea discussed with jacekm@, this needs some testing to make sure it does
not introduce a regression with aliases. flush your queue and make clean.

Revision 1.153 / (download) - annotate - [select for diffs], Thu Nov 5 10:27:24 2009 UTC (14 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.152: +3 -1 lines
Diff to previous 1.152 (colored)

- move a couple prototypes in smtpd.h
- remove prototypes from deprecated functions

Revision 1.152 / (download) - annotate - [select for diffs], Tue Nov 3 22:57:41 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.151: +3 -2 lines
Diff to previous 1.151 (colored)

teach makemap how to build a set, which is a map containing only keys.
smtpd is now capable of looking primary domains at runtime in a set, which
means that the following becomes possible:

	map "primary" { source db "/etc/mail/primary.db" }
	accept for domain map "primary" deliver to mbox

while at it fix a couple bugs in the aliases resolution path which caused
recipients to bounce if a ruleset did not have an "accept for local" rule

"diff reads good" jacekm@, flush queue & make clean

Revision 1.151 / (download) - annotate - [select for diffs], Tue Nov 3 20:55:23 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.150: +4 -3 lines
Diff to previous 1.150 (colored)

this commit removes the hardcoded special "aliases" map and brings support
for multiple aliases maps that can be attached at the rule level. with it,
you can for example define different aliases maps for different domains or
different aliases maps for the same domain depending on the client source:

map "localiases" { source db "/etc/mail/localiases.db" }
map "netaliases" { source db "/etc/mail/netaliases.db" }

accept from 192.168.0.0/16 for local alias "localiases" deliver to mbox
accept from all for local alias "netaliases" deliver to mbox

idea discussed with jacekm@ and various other hackers, diff contains some
bug fixes too which were not part of the original diff. man page follows
very shortly ... make clean & flush queue !

Revision 1.150 / (download) - annotate - [select for diffs], Tue Nov 3 19:13:34 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.149: +4 -5 lines
Diff to previous 1.149 (colored)

- remove a useless member of struct cond
- have virtual related functions take a map id instead of a map
- shrink a tiny bit ruleset matching
- add missing lka_resolve_path() call in aliases resolution leading to
  issues spotted by nicm@

Revision 1.149 / (download) - annotate - [select for diffs], Mon Oct 19 20:48:13 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.148: +6 -1 lines
Diff to previous 1.148 (colored)

currently, smtpd is capable of having multiple listeners with different
options but they will all share the same ruleset. this means that there
is no way to have a rule apply to a session established on one listener
but not applied on another.

this commit brings initial support for tagging listeners and having the
rules able to match these specific listeners. The following will define
a rule which will only apply to interfaces tagged as "mynet":

listen on lo0			# implicit lo0 tag
listen on fxp0 tag mynet
listen on fxp1 tag mynet

accept on mynet for domain "example.org" deliver to mbox

Revision 1.148 / (download) - annotate - [select for diffs], Mon Oct 19 20:00:46 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.147: +3 -1 lines
Diff to previous 1.147 (colored)

users within virtual domains do not necessarily map to a real user account,
teach smtpd how to deliver using specific user permissions:

	accept for virtual map "foo" deliver to maildir "/m/%d/%u" user foo

will deliver mail under /m/domain/user as user foo

idea and initial diff discussed with jacekm@

Revision 1.147 / (download) - annotate - [select for diffs], Mon Oct 12 22:34:37 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.146: +5 -4 lines
Diff to previous 1.146 (colored)

- fix a null deref which could happen after a couple iterations of the
  aliases/virtual domains resolution code.

- fix a logic bug which caused virtual domains not to be correctly
  handled after one iteration of the aliases resolution code.

- introduce a few helper functions to help clean up and simplify the
  lka code.

- simplify the IS_EXT/IS_MAILBOX/IS_RELAY macros so they manipulate a
  struct path * instead of the mess of dereferences we were passing them.

Revision 1.146 / (download) - annotate - [select for diffs], Sun Oct 11 17:40:49 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.145: +7 -4 lines
Diff to previous 1.145 (colored)

implement proper virtual domains instead of faking them on top of primary
domains. this means that:

- virtual domains no longer deliver to a local user when not told to
- they no longer attempt to resolve aliases when not told to
- they no longer need an explicit rule in smtpd.conf for EACH domain
- the "virtual" map is no longer hardcoded
- smtpd no longer needs a restart to support a new domain

instead we introduce the: accept for virtual map "mapname" [...] syntax
which refers to a map that can be manipulated at runtime.

idea discussed and okayd with jacekm@

Revision 1.145 / (download) - annotate - [select for diffs], Wed Oct 7 18:09:12 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.144: +7 -1 lines
Diff to previous 1.144 (colored)

currently both mfa and lka perform ruleset matching for their own purposes.
make lka the only caller of ruleset_match(), mfa request match through imsg
which will shrink its code and help me implement virtual domains properly.

idea discussed with jacekm@

Revision 1.144 / (download) - annotate - [select for diffs], Wed Sep 23 09:40:39 2009 UTC (14 years, 8 months ago) by jacekm
Branch: MAIN
Changes since 1.143: +3 -3 lines
Diff to previous 1.143 (colored)

Plug memleak.

Revision 1.143 / (download) - annotate - [select for diffs], Tue Sep 15 16:50:06 2009 UTC (14 years, 8 months ago) by jacekm
Branch: MAIN
Changes since 1.142: +49 -34 lines
Diff to previous 1.142 (colored)

Extend SMTP client_* API to support SSL+AUTH, and use it in the mta
process to relay mails.  ok gilles@

Revision 1.142 / (download) - annotate - [select for diffs], Sat Sep 12 12:24:51 2009 UTC (14 years, 8 months ago) by jacekm
Branch: MAIN
Changes since 1.141: +3 -7 lines
Diff to previous 1.141 (colored)

Simplify line len checking: use one limit for all SMTP exchanges.
This is not as strict as various RFCs want it, but we can make it
more granular later.  For now, let all mails flow by using the
maximum of all limits found in the RFCs.
ok gilles@

Revision 1.141 / (download) - annotate - [select for diffs], Sat Sep 12 09:38:45 2009 UTC (14 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.140: +2 -1 lines
Diff to previous 1.140 (colored)

cmdlinetoolong stat was incremented at the wrong place, fix

Revision 1.140 / (download) - annotate - [select for diffs], Sat Sep 12 09:22:33 2009 UTC (14 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.139: +5 -4 lines
Diff to previous 1.139 (colored)

- fix a compile time warning
- add a new stats counter to distinguish between command lines which
  exceeded the limit and data lines which exceeded the limits

Revision 1.139 / (download) - annotate - [select for diffs], Fri Sep 4 11:49:23 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.138: +24 -26 lines
Diff to previous 1.138 (colored)

Major mda update:
- Fix: check external mda / mail.local exit code.
- Fix: check maildir rename(2) return code.
- Fix: check read(2) and write(2) return codes.
- Fix: in parent, batchp->env was not set to the env of the current process.
- Fix: clean file in tmp if maildir delivery fails.
- Fix: mark message as temporarily failed upon start, unmark upon sucessful
delivery. (safe default)
- Fix: kill all message drops, aka. PERMFAILUREs, with one exception: when the
local user no longer exists.
- Cleanup: store.c is merged with its only user, mda.c
- Feature: in parent, child_add now returns pointer to the new child struct.
This is used to store and later access child->mda_batch member in order to
associate children with their batches.
- Feature: in parent, external mda / mail.local will timeout after 5 minutes.

Revision 1.138 / (download) - annotate - [select for diffs], Wed Sep 2 12:47:06 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.137: +6 -1 lines
Diff to previous 1.137 (colored)

Include mbox write errors in "show stats".  Suggested by gilles.

Revision 1.137 / (download) - annotate - [select for diffs], Thu Aug 27 11:37:30 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.136: +2 -1 lines
Diff to previous 1.136 (colored)

Implement client side of the SMTP protocol in a library-like module.
Make bounce code and /usr/sbin/sendmail interface use this new API.
The mta process continues to use its own implementation, but
eventually will be switched to use this shared module.

Buffer routines are taken from buffer.c rather than from evbuffer.
This is one step forward to using a single buffer API across the
program.

"it looks sexy" gilles@

Revision 1.136 / (download) - annotate - [select for diffs], Thu Aug 27 09:21:28 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.135: +2 -2 lines
Diff to previous 1.135 (colored)

getpwnam failure that results in setting errno could confuse the
check for non-existent ~/.forward, so make the check more robust;
fix a fd leak under rare circumstances; use secure_file for
testing .forward file security.

Revision 1.135 / (download) - annotate - [select for diffs], Fri Aug 7 19:02:55 2009 UTC (14 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.134: +4 -1 lines
Diff to previous 1.134 (colored)

wrap auth_userokay in a new function authenticate_user() and make sure
that only authenticate.c knows about bsd_auth, no functionnal change.

Revision 1.134 / (download) - annotate - [select for diffs], Thu Aug 6 16:46:57 2009 UTC (14 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.133: +5 -4 lines
Diff to previous 1.133 (colored)

- change all occurences of T_DAEMON_BATCH,MESSAGE to T_BOUNCE_BATCH/MESSAGE
- make sure T_BOUNCE_MESSAGE is no longer OR-ed to T_MDA/MTA_MESSAGE
- define F_MESSAGE_BOUNCE flag and make sure bounce sessions set it
- teach smtpctl show queue how to recognize a bounce message

Revision 1.133 / (download) - annotate - [select for diffs], Thu Aug 6 14:27:41 2009 UTC (14 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.132: +2 -3 lines
Diff to previous 1.132 (colored)

factorize file_copy_session() and file_copy() so file_copy() can handle
both deliveries to mailboxes (mbox/maildir) and copying to a session.

Revision 1.132 / (download) - annotate - [select for diffs], Thu Aug 6 14:12:48 2009 UTC (14 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.131: +3 -1 lines
Diff to previous 1.131 (colored)

- introduce message_set_errormsg() to set the error description that will
  appear in a bounce message, and message_get_errormsg() to retrieve that
  message.
- when loop is detected, call message_set_errormsg()
- in mta, call message_set_errormsg() for each recipient failure
- in mta, call message_set_errormsg() to copy batch errors to recipients if
  we failed to deliver for a session related error
- when bouncing, add the recipient and error reason to the bounce message

Revision 1.131 / (download) - annotate - [select for diffs], Thu Aug 6 13:40:45 2009 UTC (14 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.130: +17 -11 lines
Diff to previous 1.130 (colored)

This commit reworks the entire mailer daemon support to actually make it
work for real. As an added bonus, it simplifies it, makes it follow the
same code path as regular messages and kills quite some code from mta,
mda and store. There's still some work needed but the most painful part
is behind us now ;)

ok jacekm@

Revision 1.130 / (download) - annotate - [select for diffs], Tue Jul 28 22:03:55 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.129: +8 -1 lines
Diff to previous 1.129 (colored)

introduce daemon_record_message() and needed glue in queue_shared.c to
atomically create a mailer daemon out of a message. it is unreferenced
for now, this commit makes no functional change.

Revision 1.129 / (download) - annotate - [select for diffs], Fri Jun 26 11:48:00 2009 UTC (14 years, 11 months ago) by okan
Branch: MAIN
CVS Tags: OPENBSD_4_6_BASE, OPENBSD_4_6
Changes since 1.128: +5 -1 lines
Diff to previous 1.128 (colored)

locally define nitems() macro, #ifndef'd.

"please commit" gilles@ jacekm@

Revision 1.128 / (download) - annotate - [select for diffs], Sat Jun 6 18:31:42 2009 UTC (14 years, 11 months ago) by pyr
Branch: MAIN
Changes since 1.127: +2 -2 lines
Diff to previous 1.127 (colored)

Get ready for including imsg.h from a lib, when it comes along.

Revision 1.127 / (download) - annotate - [select for diffs], Sat Jun 6 04:14:21 2009 UTC (14 years, 11 months ago) by pyr
Branch: MAIN
Changes since 1.126: +14 -6 lines
Diff to previous 1.126 (colored)

Sync with relayd:
Stop pushing event handling in the imsg framework.
Instead, provide a small glue layer on top of both imsg and libevent.
This finally clearly separates event handling and imsg construction.

Sidetrack bonus: remove the mega-ugly hack of having a dummy imsg_event_add
stub in smtpctl.
ok jaceckm@

Revision 1.126 / (download) - annotate - [select for diffs], Fri Jun 5 20:43:57 2009 UTC (14 years, 11 months ago) by pyr
Branch: MAIN
Changes since 1.125: +12 -98 lines
Diff to previous 1.125 (colored)

make smtpd's imsg lib ready, just like relayd and ospfd.
ok gilles@, jacekm@

Revision 1.125 / (download) - annotate - [select for diffs], Fri Jun 5 08:50:00 2009 UTC (15 years ago) by pyr
Branch: MAIN
Changes since 1.124: +1 -3 lines
Diff to previous 1.124 (colored)

Destroy gilles' fantasy function.
ok by a desillusionned gilles@

Revision 1.124 / (download) - annotate - [select for diffs], Wed Jun 3 22:04:15 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.123: +12 -30 lines
Diff to previous 1.123 (colored)

simplify struct batch; ok gilles@

Revision 1.123 / (download) - annotate - [select for diffs], Wed Jun 3 18:16:29 2009 UTC (15 years ago) by gilles
Branch: MAIN
Changes since 1.122: +2 -2 lines
Diff to previous 1.122 (colored)

rename sc_config to sc_conffile since that's what this field is about, it
will prevent a confusion with sc_config which will be a pointer to the
actual configuration.

Revision 1.122 / (download) - annotate - [select for diffs], Tue Jun 2 22:23:36 2009 UTC (15 years ago) by gilles
Branch: MAIN
Changes since 1.121: +3 -3 lines
Diff to previous 1.121 (colored)

make env->sc_listeners and env->sc_ssl pointers, one step further toward
configuration reloading without killing active sessions; ok jacekm@

Revision 1.121 / (download) - annotate - [select for diffs], Mon Jun 1 18:24:01 2009 UTC (15 years ago) by deraadt
Branch: MAIN
Changes since 1.120: +1 -3 lines
Diff to previous 1.120 (colored)

No need for save_getpwuid and such functions which call endpwent.
endpwent is not needed unless setpwent (with stayopen = 1) or getpwent
was used.  getpwuid/getpwnam do not use getpwent; i think people must
be assuming this.  we are going to improve the man pages for this.
ok gilles

Revision 1.120 / (download) - annotate - [select for diffs], Sat May 30 23:53:41 2009 UTC (15 years ago) by gilles
Branch: MAIN
Changes since 1.119: +7 -2 lines
Diff to previous 1.119 (colored)

It is now possible to specify a certificate to use when relaying to another
host which requests client certificates:

	accept [...] relay via [...] ssl certificate "mycert"

diff from Josh Elsasser <josh@elsasser.org>, tested and okayed by me with
no change but the addition of status 554 to the state machine to deal with
remote host telling us it doesn't like our certificate.

Revision 1.119 / (download) - annotate - [select for diffs], Thu May 28 08:50:08 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.118: +2 -1 lines
Diff to previous 1.118 (colored)

Log the incoming message size; ok gilles@

Revision 1.118 / (download) - annotate - [select for diffs], Wed May 27 13:11:39 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.117: +3 -2 lines
Diff to previous 1.117 (colored)

After a successful AUTH command completes, reject any further AUTH commands.

ok gilles@

Revision 1.117 / (download) - annotate - [select for diffs], Wed May 27 13:09:07 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.116: +11 -2 lines
Diff to previous 1.116 (colored)

request flood mitigation:
1) each state may have 2 responses sent quickly;
2) more responses are delayed exponentially, up to a defined limit.

Delay count is user visible (smtp.errors.delays).

ok gilles@

Revision 1.116 / (download) - annotate - [select for diffs], Mon May 25 14:00:36 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.115: +1 -2 lines
Diff to previous 1.115 (colored)

merge smtp_listener_setup into its only caller; ok gilles@

Revision 1.115 / (download) - annotate - [select for diffs], Sun May 24 15:47:31 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.114: +7 -10 lines
Diff to previous 1.114 (colored)

Parent process shouldn't be base64-decoding untrusted strings, move
this code to privsep smtp process; ok gilles@

Revision 1.114 / (download) - annotate - [select for diffs], Sun May 24 14:38:56 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.113: +15 -10 lines
Diff to previous 1.113 (colored)

Parent process forks 3 types of processes, track them all in a single tree
using newly introduced child struct.

Manage process titles centrally in struct smtpd.

ok gilles@

Revision 1.113 / (download) - annotate - [select for diffs], Sun May 24 14:22:24 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.112: +8 -9 lines
Diff to previous 1.112 (colored)

Instead of keeping stats private to each process, and querying every
process individually whenever stats need to be fetched, keep stats
in MAP_ANON shared memory allocated by parent.

This means control has direct access to stats and can respond very
quickly without troubling any other daemon processes.

ok gilles@

Revision 1.112 / (download) - annotate - [select for diffs], Wed May 20 14:29:44 2009 UTC (15 years ago) by gilles
Branch: MAIN
Changes since 1.111: +12 -3 lines
Diff to previous 1.111 (colored)

first step towards configuration reload in smtpd, smtpctl reload will parse
the configuration file again and replace current configuration with new one
in all processes. what we don't support yet is graceful restart, clients in
sessions at the moment of the reload will have a temp failure thrown at 'em
which is ok RFC-wise but which we will try to improve anyway.

tested with various setups, "diff reads good" jacekm@

Revision 1.111 / (download) - annotate - [select for diffs], Tue May 19 11:42:52 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.110: +6 -1 lines
Diff to previous 1.110 (colored)

- Don't advertise nor accept STARTTLS command when session is secure.
- Make the condition when STARTTLS and AUTH are advertised & accepted
more readable.

ok gilles@

Revision 1.110 / (download) - annotate - [select for diffs], Tue May 19 11:37:44 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.109: +2 -1 lines
Diff to previous 1.109 (colored)

Do not ever run /usr/sbin/sendmail, it could link to sendmail
interface for a different MTA. Use the binary in /usr/libexec
directly instead.

Based on remark claudio@ made in passing.

ok gilles@

Revision 1.109 / (download) - annotate - [select for diffs], Tue May 19 11:24:24 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.108: +7 -1 lines
Diff to previous 1.108 (colored)

Verify the amount of IMSG payload is exactly as expected; ok gilles@

Revision 1.108 / (download) - annotate - [select for diffs], Mon May 18 20:23:35 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.107: +12 -5 lines
Diff to previous 1.107 (colored)

Complete rework of bufferevent event masking allowing for  more
strictness:
- Drop clients attempting command pipelining; protects the daemon
from all kinds of abuse.
- Replace F_EVLOCKED flag with F_WRITEONLY which has cleaner sematics:
when up, session must not be destroyed nor read from, but may be
written to.
- Write callback becomes a central place for enabling EV_READ.
- Delay bufferevent creation until after ssl handshake is completed.

A bunch of session error stats were added to smtpctl's "show stats".
These could help spotting event masking errors in the future.

ok gilles@

Revision 1.107 / (download) - annotate - [select for diffs], Sat May 9 20:03:07 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.106: +5 -4 lines
Diff to previous 1.106 (colored)

give few states more meaningful names; ok gilles@

Revision 1.106 / (download) - annotate - [select for diffs], Sat May 9 17:04:55 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.105: +28 -17 lines
Diff to previous 1.105 (colored)

- New API to handle all DNS query types (A, MX, PTR) asynchronously.

- Improve RFC compliance: CNAMEs are resolved, equal preference MXs
are randomized, relaying via MX that has equal/lower preference
than local server is prevented, decision on when to treat domain
name as implicit MX is better.

ok gilles@

Revision 1.105 / (download) - annotate - [select for diffs], Tue Apr 28 23:11:25 2009 UTC (15 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.104: +8 -7 lines
Diff to previous 1.104 (colored)

smtp auth happened to work by luck because delivery to mbox action was the
first action of an enum and we allocated a struct using calloc, but we did
not properly initialize the action for authenticated users.

while at it, change the name of path action flags so that we know at first
sight that they are path related.

this diff fixes the immediate issue but still needs work.
ok jacekm@, "we'll work out a long term solution"

Revision 1.104 / (download) - annotate - [select for diffs], Fri Apr 24 10:02:35 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.103: +2 -1 lines
Diff to previous 1.103 (colored)

Make aliases case-insensitive, issue reported by Daniel Ouelett; ok gilles@

Revision 1.103 / (download) - annotate - [select for diffs], Fri Apr 24 08:35:48 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.102: +1 -2 lines
Diff to previous 1.102 (colored)

use same timeout at each session state, and make use of
bufferevent_settimeout instead of rolling our own; ok gilles@

Revision 1.102 / (download) - annotate - [select for diffs], Fri Apr 24 08:32:12 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.101: +1 -9 lines
Diff to previous 1.101 (colored)

kill unnecessary struct message_recipient; ok gilles@

Revision 1.101 / (download) - annotate - [select for diffs], Tue Apr 21 18:12:05 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.100: +15 -1 lines
Diff to previous 1.100 (colored)

Make /usr/sbin/sendmail not fail due to smtpd being down.

The approach is to save cmdline + stdin in a file under a newly
added directory /var/spool/smtpd/offline (uid 0 gid 0 mode 1777).
Next time daemon starts, it uses information in that directory
to replay sendmail on user's behalf.

ok gilles@

Revision 1.100 / (download) - annotate - [select for diffs], Mon Apr 20 17:07:01 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.99: +3 -2 lines
Diff to previous 1.99 (colored)

Rewrite smtp session timeouts; use evtimer_* from event(3) instead
of regularly walking session tree in search of idle clients. Gives
the smtp process a chance to become idle.

As a bonus, add smtp.sessions.timeout counter to "smtpctl show stats".

ok gilles@

Revision 1.99 / (download) - annotate - [select for diffs], Thu Apr 16 15:35:06 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.98: +4 -11 lines
Diff to previous 1.98 (colored)

Total rewrite of the sendmail interface. Adds support for -t, -v,
and -F cmdline args. Also, date and Message-Id headers are added
when missing.

The main trouble with the current enqueue code is that it requires
dealing with problems in the control process that are already solved
in the smtp process, ie. duplicating a lot of code which interacts
with untrusted clients. This diff solves this by making sendmail
obtain a SMTP socket from smtp via smtpd.sock, and using that socket
to deliver the message. For smtpd it looks as if connection was
made from the network, only difference being the F_MESSAGE_ENQUEUED
message flag, handy when differentation between local and remote
deliveries is wanted.

Most of the features come from the femail program, created by henning@.

Additional testing by Nigel J. Taylor.

ok gilles@, henning@ happy with smtpd using femail code

Revision 1.98 / (download) - annotate - [select for diffs], Wed Apr 15 20:34:59 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.97: +3 -3 lines
Diff to previous 1.97 (colored)

s/ssmtp/smtps/; ok gilles@

Revision 1.97 / (download) - annotate - [select for diffs], Thu Apr 9 19:49:34 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.96: +3 -3 lines
Diff to previous 1.96 (colored)

change syntax of the "listen on" and "relay via" directives:
1) kill the ssmtp keyword in "ssmtp listen on ...";
2) kill the use keyword in "... use certificate foo";
3) tls no longer implicit, user must explicitely use the tls or smtps option.
4) for "relay via", move the tls/smtps options to right after the
port specification; makes it similar to "listen on".

These directives:

  ssmtp listen on fxp0 use ceritifate "foo"
  accept for all relay via tls "mx.bar.com"

now become:

  listen on fxp0 smtps certificate "foo"
  accept for all relay via "mx.bar.com" tls

ok gilles@

Revision 1.96 / (download) - annotate - [select for diffs], Sun Mar 22 22:53:47 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.95: +2 -1 lines
Diff to previous 1.95 (colored)

fix a bug in the resolution of forward files which would cause usernames
not to be taken into account if they had no ~/.forward file AND were the
result of an alias expansion that expanded to more than one username.

while at it, I spotted another bug where I would check T_MDA_MESSAGE on
the flags field instead of the type field. the bug could cause two MDA
message to end up in the same batch which is no longer valid.

Revision 1.95 / (download) - annotate - [select for diffs], Thu Mar 19 20:27:49 2009 UTC (15 years, 2 months ago) by jacekm
Branch: MAIN
Changes since 1.94: +2 -1 lines
Diff to previous 1.94 (colored)

make action_type == 0 mean A_INVALID, not A_RELAY; ok gilles@

Revision 1.94 / (download) - annotate - [select for diffs], Sun Mar 15 19:32:11 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.93: +4 -3 lines
Diff to previous 1.93 (colored)

since we are going to share code in smtp sessions and mta sessions, we need
to also share the statistics structure, still being worked on

Revision 1.93 / (download) - annotate - [select for diffs], Sun Mar 15 19:15:25 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.92: +3 -4 lines
Diff to previous 1.92 (colored)

the mda process no longer uses struct batch as its central structure to
deal with deliveries, it now uses struct session just like mta and smtp
processes. we now keep the mbox and message descriptors in the session,
saving space in struct message which is now as small as we can make it.
While at it, plugged a memory leak and did some cosmethic changes

This was the last planned change to our struct message which means that
later changes will no longer require a queue flush before rebuild.

Revision 1.92 / (download) - annotate - [select for diffs], Sun Mar 15 18:12:15 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.91: +3 -2 lines
Diff to previous 1.91 (colored)

save 4 bytes per message by moving the datafp field of struct message to
struct session where it really belongs.

Revision 1.91 / (download) - annotate - [select for diffs], Thu Mar 12 11:08:26 2009 UTC (15 years, 2 months ago) by pea
Branch: MAIN
Changes since 1.90: +2 -1 lines
Diff to previous 1.90 (colored)

Add new function time_to_text to correctly display the date.
Use it to display the date in received from header and when we
store headers.

ok jacekm@

Revision 1.90 / (download) - annotate - [select for diffs], Tue Mar 10 22:33:26 2009 UTC (15 years, 2 months ago) by jacekm
Branch: MAIN
Changes since 1.89: +2 -2 lines
Diff to previous 1.89 (colored)

implement basic logging, needs more work; ok gilles@

Revision 1.89 / (download) - annotate - [select for diffs], Mon Mar 9 23:35:04 2009 UTC (15 years, 2 months ago) by jacekm
Branch: MAIN
Changes since 1.88: +9 -1 lines
Diff to previous 1.88 (colored)

run external mda directly, not via sh -c; this steals addargs() API
from OpenSSH; ok gilles@

Revision 1.88 / (download) - annotate - [select for diffs], Mon Mar 9 16:31:09 2009 UTC (15 years, 2 months ago) by jacekm
Branch: MAIN
Changes since 1.87: +2 -7 lines
Diff to previous 1.87 (colored)

Drop ALIAS_TEXT, plain maps got implemented in a different way; ok gilles@

Revision 1.87 / (download) - annotate - [select for diffs], Mon Mar 9 01:43:19 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.86: +20 -12 lines
Diff to previous 1.86 (colored)

add basic support for outgoing authentication (AUTH PLAIN over ssl) which
can be turned on by adding "enable auth" to a "relay via" rule. this made
me rework the mx resolution so that it is done by the mta process and not
the runner process anymore.

Revision 1.86 / (download) - annotate - [select for diffs], Sun Mar 8 20:39:49 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.85: +10 -1 lines
Diff to previous 1.85 (colored)

when operating in enqueue mode, it was easy to make smtpctl fatal() by
writing a small app that sent out of order imsg's. prevent this by use
of a state machine and read event masking.

issue spotted by jacekm@, temporary fix by me. there are ideas around
this, but we want to experiment them a bit and they are low priority.

Revision 1.85 / (download) - annotate - [select for diffs], Sun Mar 8 19:11:22 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.84: +3 -6 lines
Diff to previous 1.84 (colored)

shrink struct message a bit by removing a couple fields which are no longer
used and by moving the rcpt count in struct session where it really belongs

while at it, remove some unused splay tree generation in mfa

Revision 1.84 / (download) - annotate - [select for diffs], Sun Mar 8 17:54:20 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.83: +22 -23 lines
Diff to previous 1.83 (colored)

~/.forward files handling was fixed recently so that it is the privileged
process that does the opening, this commit does some cleanup, and fixes a
bug I experienced today which was caused by a use-after-free.

I did some testing to make sure a user cannot cause smtpd to deadlock, or
loop, with broken setups (self-referencing forwards/aliases, empty files,
broken files...), but if you are playing with aliases/forwards PLEASE let
me know of any bug you run into.

Revision 1.83 / (download) - annotate - [select for diffs], Fri Mar 6 23:45:00 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.82: +4 -1 lines
Diff to previous 1.82 (colored)

missing prototype

Revision 1.82 / (download) - annotate - [select for diffs], Tue Mar 3 23:33:52 2009 UTC (15 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.81: +2 -1 lines
Diff to previous 1.81 (colored)

when forwards/aliases expansion fails in an lka session, make sure that mfa
is notified so that the session does not hang

Revision 1.81 / (download) - annotate - [select for diffs], Tue Mar 3 23:23:52 2009 UTC (15 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.80: +27 -2 lines
Diff to previous 1.80 (colored)

Fix a long standing issue where ~/.forward files were opened by user _smtpd
causing them not to be handled when a user's homedir is set to mode 0700. I
still need to do some cleanup and make sure it works as it should, but this
diff provides better behavior than what we had.

Revision 1.80 / (download) - annotate - [select for diffs], Tue Mar 3 15:47:27 2009 UTC (15 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.79: +3 -1 lines
Diff to previous 1.79 (colored)

fix mbox locking by having it done by mail.local(8) which in turns uses
lockspool(1). this means our mbox delivery follows a code path that has
become almost identical to external mda deliveries. this is the first
of a serie of diffs actually...

lockspool(1) suggestion by deraadt@, mail.local(8) idea by jacekm@, and
fix and testing by me

Revision 1.79 / (download) - annotate - [select for diffs], Sun Mar 1 21:58:53 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.78: +2 -1 lines
Diff to previous 1.78 (colored)

- Refuse delivery to mbox that is a symlink, pipe, chardev, etc. etc.
- Introduce secure_file, based on secure_filename from OpenSSH, it
checks that mbox has right perms, and that path components are
trustworthy, too.

ok gilles@

Revision 1.78 / (download) - annotate - [select for diffs], Sun Mar 1 12:12:58 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.77: +3 -2 lines
Diff to previous 1.77 (colored)

In "smtpctl show stats", break queue.inserts into queue.inserts.remote
and queue.inserts.local; ok gilles@

Revision 1.77 / (download) - annotate - [select for diffs], Tue Feb 24 12:07:47 2009 UTC (15 years, 3 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_4_5_BASE, OPENBSD_4_5
Changes since 1.76: +9 -1 lines
Diff to previous 1.76 (colored)

teach smtpctl's parser how to deal with parameters that are not necessarily
a token so that it is possible to do: smtpctl schedule <message id/uid>

introduce F_MESSAGE_FORCESCHEDULE which lets the runner schedule a message
even if the retry delay has not been expired.

F_MESSAGE_ENQUEUED is a valid flag for a message and should not cause an\
errx() in smtpctl show queue

Revision 1.76 / (download) - annotate - [select for diffs], Mon Feb 23 22:59:40 2009 UTC (15 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.75: +3 -1 lines
Diff to previous 1.75 (colored)

add valid_message_id() and valid_message_uid() which test that a message id
and uid do not look wrong. this was not needed earlier because we only deal
with message id's coming from trusted sources, but smtpctl will soon have a
new feature which requires us to deal with user provided message id's.

Revision 1.75 / (download) - annotate - [select for diffs], Sun Feb 22 11:59:12 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.74: +2 -1 lines
Diff to previous 1.74 (colored)

put repeated inet_ntop calls into new func ss_to_text, which uses
getnameinfo internally; ok gilles@

Revision 1.74 / (download) - annotate - [select for diffs], Thu Feb 19 11:33:25 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.73: +2 -1 lines
Diff to previous 1.73 (colored)

don't let libevent buffer long lines forever; ok gilles@

Revision 1.73 / (download) - annotate - [select for diffs], Wed Feb 18 22:39:12 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.72: +3 -1 lines
Diff to previous 1.72 (colored)

- add basic syntax checking to mfa
- decline source routing at MAIL FROM, strip at RCPT TO

ok gilles@

Revision 1.72 / (download) - annotate - [select for diffs], Tue Feb 17 23:43:57 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.71: +1 -5 lines
Diff to previous 1.71 (colored)

make newaliases read aliases path from smtpd.conf; idea discussed with
gilles@, pyr@ and henning@, diff ok gilles@.

Revision 1.71 / (download) - annotate - [select for diffs], Sun Feb 15 13:12:19 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.70: +2 -2 lines
Diff to previous 1.70 (colored)

If MX query fails due to DNS error, do not attempt more queries; ok gilles@

Revision 1.70 / (download) - annotate - [select for diffs], Sun Feb 15 10:32:23 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.69: +6 -3 lines
Diff to previous 1.69 (colored)

New config.c that allows for process cloning. Done by pyr@ for
relayd at n2k9, and adapted to smtpd; ok gilles@

Revision 1.69 / (download) - annotate - [select for diffs], Sat Feb 14 18:37:12 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.68: +7 -2 lines
Diff to previous 1.68 (colored)

Implement makemap -t and -o, for output type and dest resp.; ok gilles@

Revision 1.68 / (download) - annotate - [select for diffs], Fri Jan 30 21:52:55 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.67: +3 -1 lines
Diff to previous 1.67 (colored)

when decreasing ssl related counters, make sure the session was flagged as
F_SECURE. while at it, add "smtp.sessions.aborted" which keeps track of
sessions which were interrupted before completion.

Revision 1.67 / (download) - annotate - [select for diffs], Fri Jan 30 21:40:21 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.66: +6 -1 lines
Diff to previous 1.66 (colored)

improve statistics for smtp process. not only collect the current sessions
count, but also the total sessions count, ssmtp sessions (both current and
total) and starttls sessions (both current and total)

# ./smtpctl/smtpctl show stats|grep smtp.sessions
smtp.sessions = 0
smtp.sessions.active = 0
smtp.sessions.ssmtp = 0
smtp.sessions.ssmtp.active = 0
smtp.sessions.starttls = 0
smtp.sessions.starttls.active = 0
#

Revision 1.66 / (download) - annotate - [select for diffs], Fri Jan 30 21:22:33 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.65: +4 -2 lines
Diff to previous 1.65 (colored)

clear the F_EVLOCK flag earlier to prevent the error event handler from
being called again with F_EVLOCK set. this fixes a bug where disconnect
after smtpd sends greeting and before entering any command failed to go
into session_destroy().

while at it, rename the "smtp.clients" statistic to "smtp.sessions" and
add counters to struct s_smtp so that I can add ssmtp and starttls with
my next commit ;)

Revision 1.65 / (download) - annotate - [select for diffs], Fri Jan 30 20:11:13 2009 UTC (15 years, 4 months ago) by form
Branch: MAIN
Changes since 1.64: +2 -2 lines
Diff to previous 1.64 (colored)

Do not break header for messages received via smtp.

look ok gilles@

Revision 1.64 / (download) - annotate - [select for diffs], Fri Jan 30 17:34:58 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.63: +2 -1 lines
Diff to previous 1.63 (colored)

bump descriptors limit to the max and set the maximum connections count to
three quarters of that limit (a session typically has 3 descriptors). when
we hit that limit, we stop accepting connections, and when client closes a
session, we start accepting connections again. this prevents us from going
into a session that is likely to fail because of scarce resources.

idea discussed with jacekm@, code mostly ripped from relayd

Revision 1.63 / (download) - annotate - [select for diffs], Fri Jan 30 16:37:52 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.62: +3 -2 lines
Diff to previous 1.62 (colored)

fix a very annoying events masking issue which would cause a fatal() to be
hit under certain conditions; while tracking the bug I ran into other bugs
which were kind of related and could cause us to hit a fatal() too.

fix by me, but with lots of testing and investigation with jacekm@,
ok jacekm@

Revision 1.62 / (download) - annotate - [select for diffs], Thu Jan 29 21:59:15 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.61: +34 -2 lines
Diff to previous 1.61 (colored)

Implement "smtpctl show stats"; ok gilles@

Revision 1.61 / (download) - annotate - [select for diffs], Thu Jan 29 15:40:35 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.60: +4 -1 lines
Diff to previous 1.60 (colored)

missing prototype for session_respond() in mta, move session_respond()'s
prototype to smtpd.h
spotted and diff from Oleg Safiullin <form@pdp-11.org.ru>

Revision 1.60 / (download) - annotate - [select for diffs], Thu Jan 29 15:20:34 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.59: +5 -1 lines
Diff to previous 1.59 (colored)

Received header line was incomplete for mail submitted through the enqueuer
as well as for some outgoing messages, this is fixed now

Revision 1.59 / (download) - annotate - [select for diffs], Thu Jan 29 14:25:55 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.58: +3 -2 lines
Diff to previous 1.58 (colored)

initial starttls support in mta, this allows:

	accept for domain "openbsd.org" relay via tls "mx.example.org"

to ensure the relaying of mail for whoever@openbsd.org will happen through
a secure tls (STARTTLS) session. failure to establish a tls session will be
considered as a permanent failure. As a side effect:

	accept for domain "openbsd.org" relay via ssl "mx.example.org"

can now work as well and ensure that the relaying happens through ssmtp OR
tls, but never through an unsafe channel. no need to specify a port, they
are automatically detected if not specified.

still a work in progress, don't expect that it will work flawlessly.

Revision 1.58 / (download) - annotate - [select for diffs], Thu Jan 29 13:00:12 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.57: +1 -2 lines
Diff to previous 1.57 (colored)

bring initial support for SSL in the mta part of smtpd, allowing for:

	accept for domain "openbsd.org" relay via ssmtp "mx1.example.org"

to ensure that deliveries for whatever@openbsd.org goes through an SSL session
to mx1.example.org

Revision 1.57 / (download) - annotate - [select for diffs], Thu Jan 29 12:43:25 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.56: +26 -22 lines
Diff to previous 1.56 (colored)

Common queue walking code for smtpd and smtpctl. Kills majority of showqueue.c,
the remaining code was moved to queue_shared.c; ok gilles@

Revision 1.56 / (download) - annotate - [select for diffs], Wed Jan 28 23:13:42 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.55: +1 -2 lines
Diff to previous 1.55 (colored)

mta session state belongs to struct session, not struct batch, remove the
state field from struct batch and propagate the change

Revision 1.55 / (download) - annotate - [select for diffs], Wed Jan 28 22:54:10 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.54: +4 -5 lines
Diff to previous 1.54 (colored)

everything we need for the event handling dance is in struct session, the
write handler has been changed to set the bufferevent that's in there
rather than the one in struct batch. since struct batch is no longer doing
anything useful for events handling, we can remove many fields of it.

Revision 1.54 / (download) - annotate - [select for diffs], Wed Jan 28 21:44:15 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.53: +9 -1 lines
Diff to previous 1.53 (colored)

first steps towards better mta code. currently mta uses struct batch to
store a lot of its session related code, but this is just not right and
this commit starts making mta code aware of struct session. This will
ease the implementation of ssl sessions in mta.

while at it, make mta autodetect port to use if it isn't provided in a
rule but can be derived from a parameter (i.e: "relay via ssmtp ...").

Revision 1.53 / (download) - annotate - [select for diffs], Wed Jan 28 19:38:46 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.52: +2 -1 lines
Diff to previous 1.52 (colored)

when pausing listeners, do not simply disable their events as new
clients would still be able to connect. instead, at pause time we
close and remove the listeners, and at resume time we request the
parent to reconfigure all listeners.

discussed with pyr@

Revision 1.52 / (download) - annotate - [select for diffs], Wed Jan 28 17:29:11 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.51: +3 -1 lines
Diff to previous 1.51 (colored)

Make races between queue and runner impossible by implementing the policy:
1) queue never reads /queue.
2) queue writes to /queue only at message injection time.
3) runner does all reading, and all writing apart from 2).

ok gilles@

Revision 1.51 / (download) - annotate - [select for diffs], Wed Jan 28 13:29:40 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.50: +3 -1 lines
Diff to previous 1.50 (colored)

bring loop detection support. we handle this with a qmail-like approach which
consists of checking headers for a custom header, but we also count how many
hops the mail went through and use a hard limit (currently set to 100 as was
recommanded by RFC) as a safe-guard.

idea discussed with jacekm@, qmail approach suggested by claudio@ a long time
ago

Revision 1.50 / (download) - annotate - [select for diffs], Wed Jan 28 12:58:17 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.49: +6 -2 lines
Diff to previous 1.49 (colored)

move some functions from queue.c to queue_shared.c as they are not only
used by queue process but also by runner, while at it change the prototype
of queue_open_message_file() so it takes the message id and not a batch,
runner process requires the decriptor before it even starts building a
batch.

Revision 1.49 / (download) - annotate - [select for diffs], Wed Jan 28 11:27:57 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.48: +7 -5 lines
Diff to previous 1.48 (colored)

add a struct path to struct message so that we can keep track of the RCPT
provided recipient even after aliases/forwards expansion, we'll need this
for loop detection.

message id and uid being MAXPATHLEN long is a waste, define MAX_ID_SIZE
which is currently set to 64 (but can probably be further reduced) and
make sure that structures and the few strlcpy's use the right define.

original idea by jacekm@ a while ago

Revision 1.48 / (download) - annotate - [select for diffs], Tue Jan 27 22:48:29 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.47: +17 -2 lines
Diff to previous 1.47 (colored)

first bricks of enqueue code which allows smtpctl to submit mail to queue
without "talking" smtp to listeners. currently, a big part of the server
side code is done (and requires a cleanup), next step is to get it usable
properly from a mail user agent.

Revision 1.47 / (download) - annotate - [select for diffs], Mon Jan 26 22:20:31 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.46: +11 -1 lines
Diff to previous 1.46 (colored)

move some queue related functions that are needed outside of smtpd to the
sharedqueue.c file, smtpctl cannot link queue.o without creating a mess
otherwise. while at it, move some prototypes to smtpd.h as they will be
needed by enqueue code

Revision 1.46 / (download) - annotate - [select for diffs], Wed Jan 14 23:48:35 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.45: +2 -2 lines
Diff to previous 1.45 (colored)

live testing shows that some clients will not even send EHLO if banner does
not contain ESMTP. Now that we support some extensions, let's just say that
we are ESMTP ...

Revision 1.45 / (download) - annotate - [select for diffs], Thu Jan 8 19:17:31 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.44: +5 -4 lines
Diff to previous 1.44 (colored)

ensure getpwnam is always followed by endpwent; ok gilles@ henning@

Revision 1.44 / (download) - annotate - [select for diffs], Sun Jan 4 22:35:09 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.43: +9 -5 lines
Diff to previous 1.43 (colored)

- smtp can now pause/resume the accepting of incoming messages
- smtpctl recognizes "pause incoming" and "resume incoming"
- setup imsg communication between control process and smtp process

Revision 1.43 / (download) - annotate - [select for diffs], Sun Jan 4 19:37:41 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.42: +8 -2 lines
Diff to previous 1.42 (colored)

- runner is now capable of pausing/resuming the scheduling of deliveries
for both mda and mta batches.
- smtpctl can be used to disable/enable deliveries at runtime using the
pause/resume commands.

ok jacekm@

Revision 1.42 / (download) - annotate - [select for diffs], Sun Jan 4 19:25:19 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.41: +4 -5 lines
Diff to previous 1.41 (colored)

kill F_MESSAGE_EXPIRED; ok gilles@

Revision 1.41 / (download) - annotate - [select for diffs], Sun Jan 4 17:45:58 2009 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.40: +2 -1 lines
Diff to previous 1.40 (colored)

When matching a recipient domain to a rule, do not use strcasecmp, but use
new hostname_match() function which recognizes * as a wildcard. We can now
do: accept for domain "*.example.org" to match all subdomains.

idea from Nicholas Marriott <nicholas.marriott@gmail.com>, hostname_match()
from me in place of his fnmatch() calls.

ok jacekm@

Revision 1.40 / (download) - annotate - [select for diffs], Sun Jan 4 16:40:58 2009 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.39: +2 -2 lines
Diff to previous 1.39 (colored)

- change name of "masked" member in struct netaddr, it was misleading
- allow "from all" so that the ugly "accept from { 0.0.0.0/0, ::/0 }"
construct becomes a nice looking "accept from all"

ok jacekm@

Revision 1.39 / (download) - annotate - [select for diffs], Sun Jan 4 14:46:14 2009 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.38: +1 -3 lines
Diff to previous 1.38 (colored)

cleanup; ok gilles@

Revision 1.38 / (download) - annotate - [select for diffs], Sun Jan 4 00:58:59 2009 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.37: +8 -2 lines
Diff to previous 1.37 (colored)

aliases/forwards expansion was not done correctly and a race could
cause delivery to happen before expansion is over, causing some of
the recipients to never receive the mail. change how the mfa, lka,
queue and smtp processes communicate to ensure smtp never receives
an acknowledgment before ALL expanded envelopes are on disk. While
at it, lka was doing work which belongs in mfa, fix that also.

this is based on an idea from a talk with jacekm@, change not over
but already better than what we had.

Revision 1.37 / (download) - annotate - [select for diffs], Sat Dec 27 17:36:37 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.36: +6 -2 lines
Diff to previous 1.36 (colored)

cleanup; ok gilles@

Revision 1.36 / (download) - annotate - [select for diffs], Sat Dec 27 17:03:29 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.35: +2 -1 lines
Diff to previous 1.35 (colored)

Merge hash() and queue_message_hash() into one func, queue_hash(). Fix callers
to use this interface consistently; ok chl@ gilles@

Revision 1.35 / (download) - annotate - [select for diffs], Fri Dec 26 10:28:31 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.34: +2 -4 lines
Diff to previous 1.34 (colored)

parse.y doesn't allow listen backlog configuration, so "bzero default" is used.
Hardcode it instead: 5 is a popular choice across the tree; ok gilles@

Revision 1.34 / (download) - annotate - [select for diffs], Sun Dec 21 02:18:46 2008 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.33: +8 -2 lines
Diff to previous 1.33 (colored)

- AUTH PLAIN may receive credentials as a parameter to AUTH or on a
	following line, this commit brings support for the latter which was
	not supported yet.
- AUTH LOGIN is now supported, allowing smtp auth support on clients that
	do not support AUTH PLAIN (ie: my mobile phone for instance ;)

Revision 1.33 / (download) - annotate - [select for diffs], Sat Dec 20 00:18:03 2008 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.32: +24 -19 lines
Diff to previous 1.32 (colored)

- import first bricks of SMTP AUTH support. currently only AUTH PLAIN is
	supported, AUTH LOGIN will follow soon. AUTH will only work if a
	listen directive has "enable auth" keywords, AND session is safe
	(ssmtp or starttls).

Revision 1.32 / (download) - annotate - [select for diffs], Fri Dec 19 00:39:05 2008 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.31: +9 -8 lines
Diff to previous 1.31 (colored)

- smtpd handled mbox locking failures as "regular" temporary failures which
	is not good at all. As a result, under heavy load messages would be
	kept in queue, and delayed for hours just because we failed locking
	a few times. This commit makes smtpd distinguish between lock fails
	and "regular" temporary fails.
- delivery scheduler will reschedule immediately a message that couldn't be
	delivered because of a lock fail. If we fail to lock too many times
	we fallback to previous "delay increase" logic.

	"looks sane" jacekm@

Revision 1.31 / (download) - annotate - [select for diffs], Thu Dec 18 23:57:17 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.30: +2 -1 lines
Diff to previous 1.30 (colored)

Introduce safe_fclose, which tries to push file to the disk as
quickly as possible; it fails under temporary error conditions,
letting caller react appropriately.

ok gilles@

Revision 1.30 / (download) - annotate - [select for diffs], Thu Dec 18 23:49:56 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.29: +1 -5 lines
Diff to previous 1.29 (colored)

Declarations for functions used only in smtp_session.c were moved
to that file from smtpd.h.

ok gilles@

Revision 1.29 / (download) - annotate - [select for diffs], Thu Dec 18 23:38:12 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.28: +4 -1 lines
Diff to previous 1.28 (colored)

Check fwrite return code at DATA stage.

Add basic line length checking, as required by rfc.

It is no longer required to disable EV_READ upon evbuffer_readline failure.

ok gilles@

Revision 1.28 / (download) - annotate - [select for diffs], Wed Dec 17 18:47:37 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.27: +3 -1 lines
Diff to previous 1.27 (colored)

Introduce /purge, where all msgs scheduled for deletion are put by
queue, and removed from disk by runner.

On startup, clean /incoming by moving msgs within it to /purge.

ok gilles@

Revision 1.27 / (download) - annotate - [select for diffs], Sat Dec 13 23:19:34 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.26: +10 -41 lines
Diff to previous 1.26 (colored)

IMSG_* namespace cleanup.

ok gilles@

Revision 1.26 / (download) - annotate - [select for diffs], Sat Dec 13 13:15:38 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.25: +2 -1 lines
Diff to previous 1.25 (colored)

Declare alias_parse in smtpd.h, and fix callers that pass it wrong
number of arguments.

ok gilles@

Revision 1.25 / (download) - annotate - [select for diffs], Fri Dec 12 20:19:56 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.24: +3 -2 lines
Diff to previous 1.24 (colored)

Format string checking for bsnprintf.

ok gilles@

Revision 1.24 / (download) - annotate - [select for diffs], Thu Dec 11 22:18:15 2008 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.23: +4 -1 lines
Diff to previous 1.23 (colored)

- missing prototype

Revision 1.23 / (download) - annotate - [select for diffs], Sun Dec 7 15:38:35 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.22: +2 -2 lines
Diff to previous 1.22 (colored)

Replace evbuffer_add_printf calls with wrapper function, session_respond,
which additionally suffixes <CRLF>, and enables EV_WRITE.

Remove bufferevent_enable(.., EV_WRITE) from session_command and
session_pickup so that EV_WRITE is enabled in exactly one place,
session_respond.

Change some responses slightly to make code fit 80 columns.

ok gilles@

Revision 1.22 / (download) - annotate - [select for diffs], Sun Dec 7 03:14:24 2008 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.21: +2 -2 lines
Diff to previous 1.21 (colored)

- getaddrinfo() uses negative values for its error defines, our use of an
	u_int8_t to hold the value leads to invalid checking in
	runner_batch_resolved(), this lead to a crash in MTA because we
	assumed a batch had its mx resolved when it had not. while at it,
	be more strict about errors we don't know and fatal(), it should
	not happen.

	ok jacekm@, ok chl@

Revision 1.21 / (download) - annotate - [select for diffs], Sat Dec 6 15:18:36 2008 UTC (15 years, 5 months ago) by weerd
Branch: MAIN
Changes since 1.20: +8 -8 lines
Diff to previous 1.20 (colored)

Get rid of anonymous unions. Discussed with and ok gilles@

Revision 1.20 / (download) - annotate - [select for diffs], Fri Dec 5 02:51:32 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.19: +8 -4 lines
Diff to previous 1.19 (colored)

- last part of the new queue code: the runner process (unprivileged and
	chrooted) is now in charge of doing the scheduling of deliveries,
	and the dispatching of messages to MDA and MTA. queue process only
	does inserts/updates/removals from the queue and can no longer be
	so busy that it delays answers to imsg from smtp server.

Revision 1.19 / (download) - annotate - [select for diffs], Thu Dec 4 17:24:13 2008 UTC (15 years, 6 months ago) by cloder
Branch: MAIN
Changes since 1.18: +9 -5 lines
Diff to previous 1.18 (colored)

Declare printf-style functions with __attribute__((format(printf,x,x)))
and fix some of the errors caught by this. Part of a general push to
make yyerror() -Wformat clean throughout the tree.

Revision 1.18 / (download) - annotate - [select for diffs], Thu Dec 4 01:16:14 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.17: +2 -1 lines
Diff to previous 1.17 (colored)

- fix event masking for DATA and make DATA look more like MAIL and RCPT
	with regard to communication with queue process (one state before
	sending imsg, another state when imsg has returned). this fixes an
	issue that I observed when clients send DATA and content without
	even looking at server replies.

Revision 1.17 / (download) - annotate - [select for diffs], Wed Dec 3 17:58:00 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.16: +24 -10 lines
Diff to previous 1.16 (colored)

- fix event masking issues in smtp process which could lead to a fatal() if
	queue process did not answer fast enough to an imsg. spotted by
	Jacek Masiulaniec <jacekm@dobremiasto.net>
- queue layout was mostly to bootstrap the project, it does not behave good
	under load, it does complex things to stay in a recoverable state
	and it probably didnt do it too well. New queue code is simpler,
	smaller and allows for atomic submissions (a mail can never be in a
	state where it needs to be recovered). It still needs some work but
	works better than previous code, no regression.

Revision 1.16 / (download) - annotate - [select for diffs], Tue Nov 25 23:06:15 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.15: +5 -1 lines
Diff to previous 1.15 (colored)

- more prototype moving to smtpd.h

Revision 1.15 / (download) - annotate - [select for diffs], Tue Nov 25 23:03:24 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.14: +6 -1 lines
Diff to previous 1.14 (colored)

- move prototype to smtpd.h

Revision 1.14 / (download) - annotate - [select for diffs], Tue Nov 25 23:01:00 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.13: +6 -2 lines
Diff to previous 1.13 (colored)

- move prototypes to smtpd.h

Revision 1.13 / (download) - annotate - [select for diffs], Tue Nov 25 20:26:40 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.12: +20 -11 lines
Diff to previous 1.12 (colored)

- recent change in parse.y caused htons() to be called twice on the port
	provided to "relay via" rules, once in parse.y once in lka.c, fix.
- rename struct address to struct relayhost, introduce struct mxhost which
	not only holds the sockaddr_storage, but also additionnal flags we
	want forwarded to the mta process.
- propagate the change

Revision 1.12 / (download) - annotate - [select for diffs], Tue Nov 25 15:55:13 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.11: +4 -5 lines
Diff to previous 1.11 (colored)

- F_IMSG_SENT is no longer used, kill
	suggested by Jacek Masiulaniec <jacekm@dobremiasto.net>

Revision 1.11 / (download) - annotate - [select for diffs], Mon Nov 17 21:50:43 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.10: +7 -6 lines
Diff to previous 1.10 (colored)

- until now a client could issue a command from an extension even though it
	greeted with helo and not ehlo. introduce session flag F_EHLO and
	make sure the session_command() dispatch only looks at extensions
	when a session does not have the F_EHLO flag.

Revision 1.10 / (download) - annotate - [select for diffs], Mon Nov 17 21:27:50 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.9: +1 -6 lines
Diff to previous 1.9 (colored)

- remove prototypes for the atomic API, we don't use it anymore

Revision 1.9 / (download) - annotate - [select for diffs], Tue Nov 11 01:08:08 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.8: +5 -2 lines
Diff to previous 1.8 (colored)

- queue process no longer schedules messages which do not have flag
	F_MESSAGE_COMPLETE
- submit recipients to the queue as we read them from RCPT instead of
	submiting them all at once when DATA is over. this prevents us
	from having to keep a potentially large number of recipients in
	memory during the whole session.
- remove all code that dealt with the recipients queue of a message as
	it is no longer used.
- several small changes to make sure the server is always in a recoverable
	state in case of an unexpected shutdown.

Revision 1.8 / (download) - annotate - [select for diffs], Mon Nov 10 23:18:47 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.7: +2 -1 lines
Diff to previous 1.7 (colored)

- open the message file earlier after a successful MAIL command instead of
	waiting for the DATA command. this currently has no impact on the
	session but is needed for another change that will make submission
	of recipients safer with regard to "unexpected shutdowns at bad
	timings"

Revision 1.7 / (download) - annotate - [select for diffs], Mon Nov 10 22:35:23 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.6: +8 -8 lines
Diff to previous 1.6 (colored)

- define MAX_LINE_SIZE which is the maximum length of a line we allow from
	a client. it must be set to the highest value we have from all of
	the extensions which are/will be implemented.
- replace all occurences of STRLEN define with MAX_LINE_SIZE, kill STRLEN

Revision 1.6 / (download) - annotate - [select for diffs], Mon Nov 10 21:29:18 2008 UTC (15 years, 6 months ago) by chl
Branch: MAIN
Changes since 1.5: +2 -2 lines
Diff to previous 1.5 (colored)

rename h_errno field into getaddrinfo_error, to avoid confusion with errno.

h_errno has been obsoleted since the gethostbyname() --> getaddrinfo() replacement.

ok gilles@

Revision 1.5 / (download) - annotate - [select for diffs], Mon Nov 10 20:10:38 2008 UTC (15 years, 6 months ago) by chl
Branch: MAIN
Changes since 1.4: +2 -2 lines
Diff to previous 1.4 (colored)

fix store_write_header() prototype.

ok gilles@

Revision 1.4 / (download) - annotate - [select for diffs], Mon Nov 10 17:24:24 2008 UTC (15 years, 6 months ago) by deraadt
Branch: MAIN
Changes since 1.3: +9 -9 lines
Diff to previous 1.3 (colored)

spaces fixed while reading code

Revision 1.3 / (download) - annotate - [select for diffs], Mon Nov 10 00:22:05 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.2: +4 -2 lines
Diff to previous 1.2 (colored)

- make READ_BUF_SIZE a power of 2

Revision 1.2 / (download) - annotate - [select for diffs], Wed Nov 5 12:14:45 2008 UTC (15 years, 6 months ago) by sobrado
Branch: MAIN
Changes since 1.1: +2 -0 lines
Diff to previous 1.1 (colored)

add a few missing id tags; there are a bunch of files, and developers
will probably miss this change when working on more important matters,
so it is probably better to sort them now.  there is a risk of losing
the tags if a change needs to be reverted too.

written with excellent advice from jmc@

ok gilles@

Revision 1.1 / (download) - annotate - [select for diffs], Sat Nov 1 21:35:28 2008 UTC (15 years, 7 months ago) by gilles
Branch: MAIN

smtpd is a smtp server implementation for OpenBSD. It is a work in progress
which still lacks many features. bringing it in tree will help working on it
more easily.

"at this stage it should go in" henning@, "move ahead" deraadt@

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.