OpenBSD CVS

CVS log for src/usr.sbin/smtpd/scheduler.c


[BACK] Up to [local] / src / usr.sbin / smtpd

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.62 / (download) - annotate - [select for diffs], Mon Jun 14 17:58:16 2021 UTC (2 years, 11 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5, OPENBSD_7_4_BASE, OPENBSD_7_4, OPENBSD_7_3_BASE, OPENBSD_7_3, OPENBSD_7_2_BASE, OPENBSD_7_2, OPENBSD_7_1_BASE, OPENBSD_7_1, OPENBSD_7_0_BASE, OPENBSD_7_0, HEAD
Changes since 1.61: +1 -17 lines
Diff to previous 1.61 (colored)

add required headers for smtpd.h and remove unnecessary ones in other files.

ok jung@

Revision 1.61 / (download) - annotate - [select for diffs], Wed May 26 18:08:55 2021 UTC (3 years ago) by eric
Branch: MAIN
Changes since 1.60: +4 -5 lines
Diff to previous 1.60 (colored)

replaces calls to err(3)/errx(3) with fatal()/fatalx() from log.c
for code that runs in the daemon.

ok florian@ millert@

Revision 1.60 / (download) - annotate - [select for diffs], Sun Dec 30 23:09:58 2018 UTC (5 years, 5 months ago) by guenther
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE, OPENBSD_6_9, OPENBSD_6_8_BASE, OPENBSD_6_8, OPENBSD_6_7_BASE, OPENBSD_6_7, OPENBSD_6_6_BASE, OPENBSD_6_6, OPENBSD_6_5_BASE, OPENBSD_6_5
Changes since 1.59: +1 -2 lines
Diff to previous 1.59 (colored)

Delete unnecessary <libgen.h> #includes

ok deraadt@

Revision 1.59 / (download) - annotate - [select for diffs], Sat Dec 8 08:01:15 2018 UTC (5 years, 5 months ago) by sunil
Branch: MAIN
Changes since 1.58: +2 -2 lines
Diff to previous 1.58 (colored)

Use correct RFC 3464 specified values for Action field in a DSN.
error -> failed
success -> delivered

This fixes DSN parsing for Mailman. Issue reported by Cristiano
Costa on misc@opensmtpd.org.

While here, rename enums to reflect the intent and properly handle
envelope ascii load/dump to understand change in the values.

Suggestions and ok gilles@

Revision 1.58 / (download) - annotate - [select for diffs], Thu May 31 21:06:12 2018 UTC (6 years ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_6_4_BASE, OPENBSD_6_4
Changes since 1.57: +5 -5 lines
Diff to previous 1.57 (colored)

remove 'where' parameter from all x*() functions in utils.c, it doesn't
really help us with anything, propagate the change in codebase

ok millert@

Revision 1.57 / (download) - annotate - [select for diffs], Thu May 24 11:38:24 2018 UTC (6 years ago) by gilles
Branch: MAIN
Changes since 1.56: +3 -3 lines
Diff to previous 1.56 (colored)

switch smtpd to new grammar

ok eric@

Revision 1.56 / (download) - annotate - [select for diffs], Mon Jan 9 14:49:22 2017 UTC (7 years, 4 months ago) by reyk
Branch: MAIN
CVS Tags: OPENBSD_6_3_BASE, OPENBSD_6_3, OPENBSD_6_2_BASE, OPENBSD_6_2, OPENBSD_6_1_BASE, OPENBSD_6_1
Changes since 1.55: +2 -2 lines
Diff to previous 1.55 (colored)

Stop accessing verbose and debug variables from log.c directly.

This replaces log_verbose() and "extern int verbose" with the two functions
log_setverbose() and log_getverbose().

Pointed out by benno@
OK krw@ eric@ gilles@ (OK gilles@ for the snmpd bits as well)

Revision 1.55 / (download) - annotate - [select for diffs], Thu Sep 8 12:06:43 2016 UTC (7 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.54: +7 -22 lines
Diff to previous 1.54 (colored)

Streamline the daemon shutdown sequence.

Only the parent process handles SIGTERM and SIGINT.  Upon receiving one
of those, it closes all imsg sockets and waitpid() for the children. It
fatal()s if one of the sockets is closed unexpectedly.  Other processes
exit() "normally" when one of the imsg sockets is closed.

ok gilles@ sunil@

Revision 1.54 / (download) - annotate - [select for diffs], Sun Sep 4 16:10:31 2016 UTC (7 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.53: +3 -4 lines
Diff to previous 1.53 (colored)

The smtpd processes are not expected to ever leave their event loop.
So stop pretending that the *_shutdown() functions could ever be called
in this context, and just fatal() if event_dispatch() returns.

ok gilles@ sunil@ giovanni@

Revision 1.53 / (download) - annotate - [select for diffs], Thu Sep 1 10:54:25 2016 UTC (7 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.52: +1 -2 lines
Diff to previous 1.52 (colored)

remove noop function

ok sunil@

Revision 1.52 / (download) - annotate - [select for diffs], Sat May 28 21:21:20 2016 UTC (8 years ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_6_0_BASE, OPENBSD_6_0
Changes since 1.51: +2 -13 lines
Diff to previous 1.51 (colored)

Implement the fork+exec pattern in smtpd.

The parent process forks child processes and re-exec each of them with
an additional "-x <proc>" argument.  During the early setup phase, the
parent process sends ipc socket pairs to interconnect the child
processes as needed, and it passes the queue encryption key to the
queue if necessary. When this is done, all processes have their
environment set as in the fork-only case, and they can start doing
their work as before.

ok gilles@ jung@

Revision 1.51 / (download) - annotate - [select for diffs], Mon Dec 28 22:08:30 2015 UTC (8 years, 5 months ago) by jung
Branch: MAIN
CVS Tags: OPENBSD_5_9_BASE, OPENBSD_5_9
Changes since 1.50: +2 -2 lines
Diff to previous 1.50 (colored)

remove spaces after '!'

no binary change

ok millert

Revision 1.50 / (download) - annotate - [select for diffs], Thu Oct 29 10:25:36 2015 UTC (8 years, 7 months ago) by sunil
Branch: MAIN
Changes since 1.49: +36 -1 lines
Diff to previous 1.49 (colored)

Implement smtpctl discover <evpid|msgid>.

discover subcommand schedules envelopes manually moved to the queue.
It triggers a queue walk searching for envelopes with the given id,
schedules them and informs the user number of envelopes scheduled.
Admins no longer would need to restart the daemon to discover
manually moved messages.

Ok gilles@

Revision 1.49 / (download) - annotate - [select for diffs], Tue Oct 13 08:09:25 2015 UTC (8 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.48: +5 -1 lines
Diff to previous 1.48 (colored)

pledge("stdio") the scheduler process

Revision 1.48 / (download) - annotate - [select for diffs], Tue Jan 20 17:37:54 2015 UTC (9 years, 4 months ago) by deraadt
Branch: MAIN
CVS Tags: OPENBSD_5_8_BASE, OPENBSD_5_8, OPENBSD_5_7_BASE, OPENBSD_5_7
Changes since 1.47: +2 -1 lines
Diff to previous 1.47 (colored)

use <limits.h> comprehensively.  For now try to push <> includes to
each .c file, and out of the .h files.  To avoid overinclude.
ok gilles, in principle.  If this has been done right, -portable should
become easier to maintain.

Revision 1.47 / (download) - annotate - [select for diffs], Thu Jul 10 14:45:02 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_6_BASE, OPENBSD_5_6
Changes since 1.46: +125 -177 lines
Diff to previous 1.46 (colored)

Improve the scheduler, better and simpler.

- Get rid of the scheduler_batch structure. The scheduler can now return
  envelopes of different types in a single run, interlacing them to avoid
  batch effects.

- Ask for an acknowledgement from the queue when removing or expiring
  an envelope to benefit from the inflight envelope limitation mechanism.
  This ensures that the scheduler always keeps sending envelopes at a rate
  that the queue can sustain in all cases.

- Limit the number of envelopes in a holdq.  When a holdq is full,
  new envelopes are put back in the pending queue instead, with a
  shorter retry time.

- Plumbing for proc-ified schedulers.

imsg version bump. smtpctl stop before updating.

ok gilles@

Revision 1.46 / (download) - annotate - [select for diffs], Tue Jul 8 07:59:31 2014 UTC (9 years, 10 months ago) by sobrado
Branch: MAIN
Changes since 1.45: +2 -2 lines
Diff to previous 1.45 (colored)

fix typos.

ok jmc@

Revision 1.45 / (download) - annotate - [select for diffs], Tue May 20 18:47:01 2014 UTC (10 years ago) by eric
Branch: MAIN
Changes since 1.44: +2 -2 lines
Diff to previous 1.44 (colored)

format string cleanup: change "%i" to "%d" and fix a few typos

Revision 1.44 / (download) - annotate - [select for diffs], Tue Apr 8 15:25:43 2014 UTC (10 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.43: +2 -2 lines
Diff to previous 1.43 (colored)

use correct imsg

ok gilles@

Revision 1.43 / (download) - annotate - [select for diffs], Fri Apr 4 16:10:42 2014 UTC (10 years, 2 months ago) by eric
Branch: MAIN
Changes since 1.42: +16 -16 lines
Diff to previous 1.42 (colored)

Merge the mda, mta and smtp processes into a single unprivileged
process managing message reception, delivery and transfer.  Mostly
mechanical, but very intrusive as it required to rewamp all IMSG to
fix ambiguities.

with and ok gilles@

Revision 1.42 / (download) - annotate - [select for diffs], Tue Feb 4 14:56:03 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_5_BASE, OPENBSD_5_5
Changes since 1.41: +41 -19 lines
Diff to previous 1.41 (colored)

internal improvements and cleanups

- get rid of the whole penalty thing for failed envelopes in the mta and scheduler.
- do not disable routes on smtp errors
- try to schedule all types of envelopes on each scheduler frame.

Revision 1.41 / (download) - annotate - [select for diffs], Tue Feb 4 09:05:06 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
Changes since 1.40: +1 -3 lines
Diff to previous 1.40 (colored)

get rid of fdlimit()

Revision 1.40 / (download) - annotate - [select for diffs], Thu Dec 26 17:25:32 2013 UTC (10 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.39: +2 -2 lines
Diff to previous 1.39 (colored)

bcopy -> memmove
bzero -> memset

Revision 1.39 / (download) - annotate - [select for diffs], Thu Dec 5 09:26:47 2013 UTC (10 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.38: +21 -1 lines
Diff to previous 1.38 (colored)

When a relay fails, let the scheduler update all envelopes in the
holdq as if they tempfailed.

Revision 1.38 / (download) - annotate - [select for diffs], Sat Nov 30 10:11:57 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.37: +11 -13 lines
Diff to previous 1.37 (colored)

do not hardcode scheduler batch size, and reduce default limit to avoid
hammering effects.

Revision 1.37 / (download) - annotate - [select for diffs], Wed Nov 20 09:22:42 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.36: +5 -3 lines
Diff to previous 1.36 (colored)

Rework the mda and scheduler to use the holdq mechanism instead of
tempfail for limiting the number of pending deliveries to the same
user.  This allows to reach optimal delivery time even in case of
burst, while keeping the number of inflight envelopes low.

Revision 1.36 / (download) - annotate - [select for diffs], Tue Nov 19 10:01:20 2013 UTC (10 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.35: +18 -4 lines
Diff to previous 1.35 (colored)

Add a limit on the number of inflight envelopes.  The scheduler suspends
scheduling of mta/mda envelopes until the number of inflight envelopes
falls below that line.

Revision 1.35 / (download) - annotate - [select for diffs], Sun Oct 27 17:47:53 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.34: +26 -2 lines
Diff to previous 1.34 (colored)

Implement a feedback mechanism which allows the mta to "hold" envelopes
in the scheduler when it has too many tasks for a given relay.  The
envelopes are put on a wait queue, and are not scheduled again until
the mta "releases" some envelopes from that queue.

It prevents from having too many inflight envelopes, which are out of reach
for the admin.

Revision 1.34 / (download) - annotate - [select for diffs], Sun Oct 27 07:56:25 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.33: +2 -1 lines
Diff to previous 1.33 (colored)

Create the control socket in the parent process to abort early if
another smtpd instance is running.  Close the inherited socket in
every forked process but control.

Revision 1.33 / (download) - annotate - [select for diffs], Fri Oct 25 18:58:10 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.32: +14 -6 lines
Diff to previous 1.32 (colored)

Improve reporting in smtpctl and a few fixes.

When sending a request to the scheduler, wait for the success/failure
report from the scheduler.  Simplify the code by introducing generic
functions for interruptible iteration over envelopes.  Report the total
number of affected envelopes for schedule, pause, resume and remove
envelope operations.

Revision 1.32 / (download) - annotate - [select for diffs], Fri Jul 19 21:14:52 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_4_BASE, OPENBSD_5_4
Changes since 1.31: +1 -5 lines
Diff to previous 1.31 (colored)

Many MTA improvements:

- Better transient error handling logic: failing destinations are
  automatically disabled for a while.  When a destination is active
  again, ask the scheduler to retry previous envelopes immediatly.
- More informative error report when all routes fail for a mail.
- Implement a "smtpctl show hoststats" command to get the latest stat
  message per MX domain.
- Implement a "smtpctl show routes" command to show the state the
  currently known routes to remote MXs.
- Implement a "smtpctl resume route" command to re-enable a route that
  has been disabled.
- Do not hardcode limits
- Minor code improvements

Revision 1.31 / (download) - annotate - [select for diffs], Fri Jul 19 15:14:23 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.30: +36 -4 lines
Diff to previous 1.30 (colored)

scheduler improvements:
- implement suspend/resume scheduling for individual envelopes or message,
  with the associated smtpctl commands.
- allow the mta to request immediate scheduling of an envelope.
- on temporary failures a penalty can be given to further delay the next try.

Revision 1.30 / (download) - annotate - [select for diffs], Fri Jul 19 11:14:08 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.29: +5 -3 lines
Diff to previous 1.29 (colored)

Get rid of env->sc_pw and env->sc_pwqueue.  Early queue initialization
now happens in queue_init(), and backends take the queue passwd as
parameter in their init function.

Remove useless SMTPD_FILTER_USER while there.

Revision 1.29 / (download) - annotate - [select for diffs], Fri Jul 19 07:49:08 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.28: +1 -2 lines
Diff to previous 1.28 (colored)

Remove useless sc_pid from struct smtpd.

Revision 1.28 / (download) - annotate - [select for diffs], Fri May 24 17:03:14 2013 UTC (11 years ago) by eric
Branch: MAIN
Changes since 1.27: +41 -24 lines
Diff to previous 1.27 (colored)

sync with OpenSMTPD 5.3.2

ok gilles@

Revision 1.27 / (download) - annotate - [select for diffs], Sun Feb 10 15:01:16 2013 UTC (11 years, 3 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_3_BASE, OPENBSD_5_3
Changes since 1.26: +43 -35 lines
Diff to previous 1.26 (colored)

When getting the next batch of envelope to schedule, use an array to
store envelope ids, rather than a dynamic list.

ok gilles@

Revision 1.26 / (download) - annotate - [select for diffs], Thu Jan 31 18:34:43 2013 UTC (11 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.25: +3 -2 lines
Diff to previous 1.25 (colored)

assorted fixes spotted by Coverity.
some log message updates.

ok gilles@

Revision 1.25 / (download) - annotate - [select for diffs], Sat Jan 26 09:37:23 2013 UTC (11 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.24: +107 -67 lines
Diff to previous 1.24 (colored)

Sync with our smtpd repo:

* first bricks of ldap and sqlite support (not finished but both working)
* new table API to replace map API, all lookups are done through tables
* improved handling of temporary errors throughout the daemon
* improved scheduler and mta logic: connection reuse, optimizes batches
* improved queue: more tolerant to admin errors, new layout, less disk-IO
* improved memory usage under high load
* SSL certs/keys isolated to lookup process to avoid facing network
* VIRTUAL support improved, fully virtual setups possible now
* runtime tracing of processes through smtpctl trace
* ssl_privsep.c sync-ed with relayd
* ssl.c no longer contains smtpd specific interfaces
* smtpd-specific ssl bits moved to ssl_smtpd.c
* update mail address in copyright

FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE.

smtpd.conf(5) simplified, it will require adaptations

ok eric@

Revision 1.24 / (download) - annotate - [select for diffs], Tue Nov 20 09:47:45 2012 UTC (11 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.23: +45 -19 lines
Diff to previous 1.23 (colored)

Allow "smtpctl show queue" to run in "online" mode if the smtpd server
is running.  The scheduler sends the runtime state of each envelope to
the queue process which loads the envelope, fills the runtime bits and
sends the envelope back to the client. Iteration over the envelope set
happens in small chunks to make the request interruptible and to allow
the server to keep doing its job in the meantime.

Adpat "smtpctl schedule-all" to schedule the messages one by one using
the same iteration mechanism.

Document "smtpctl monitor" and "smtpctl show queue".

ok gilles@

Revision 1.23 / (download) - annotate - [select for diffs], Mon Nov 12 14:58:53 2012 UTC (11 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.22: +11 -11 lines
Diff to previous 1.22 (colored)

Cleanups and improvements:

* Log more events (especially client session) and use a better scheme
  for that: each messages is prefixed with a token to easily identify
  its class:
    - info/warn/debug: general server messages
    - smtp-in: smtp client connections
    - relay: status update for relayed messages
    - delivery: status update for local deliveries

* Implement "smtpctl monitor" to display updates of selected internal
  counters.

* When reloading the on-disk queue at startup do not commit a message
  if no envelope was submitted for that message.

* Remove unused stuff in the config parser.

ok gilles@

Revision 1.22 / (download) - annotate - [select for diffs], Fri Nov 2 14:46:43 2012 UTC (11 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.21: +33 -13 lines
Diff to previous 1.21 (colored)

Make counters more informative in the scheduler:

- Change the scheduler backend API a bit: commit() and rollback()
  API calls return the number of envelopes added or canceled; put
  the number of envelopes in the structure returned by batch().

- Properly report the number of incoming, registered, removed and
  expired envelopes, as well as the outcome of deliveries.

ok gilles@

Revision 1.21 / (download) - annotate - [select for diffs], Sun Sep 16 16:43:29 2012 UTC (11 years, 8 months ago) by chl
Branch: MAIN
Changes since 1.20: +1 -3 lines
Diff to previous 1.20 (colored)

Factorize log_imsg() in imsg_dispatch() instead of in each imsg_callback()'s
and put it out of profiling, so it's not accounted.

While there, for PROC_PARENT:
- set smtpd_process for PROC_PARENT
- use setproctitle() like other processes

ok gilles@

Revision 1.20 / (download) - annotate - [select for diffs], Tue Sep 11 08:37:52 2012 UTC (11 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.19: +3 -4 lines
Diff to previous 1.19 (colored)

Rework the scheduler internals.  Fix some scheduling loop issues and
handle envelope scheduling/expiration better.

ok gilles@

Revision 1.19 / (download) - annotate - [select for diffs], Sat Aug 25 22:03:26 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.18: +1 -4 lines
Diff to previous 1.18 (colored)

- add myself to the copyright in control.c, i've done quite a few changes
there in the last few years ;-)
- get rid of availdesc(): getdtablecount() is so much more reliable
- get rid of env->sc_maxconn, we can be much smarter with getdtablecount()
	and getdtablesize()
- disable accept when we hit the control process fd reserve
- disable accept when we fail
- enable accept when we're back below the limit

this is not the full fd exhaustion diff, i'll merge changes from relayd
tomorrow, this was only required to get rid of the env->sc_maxconn and
availdesc() mess

"reads alright" eric@

Revision 1.18 / (download) - annotate - [select for diffs], Sat Aug 25 10:23:12 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.17: +10 -10 lines
Diff to previous 1.17 (colored)

- introduce struct stat_value
- statistics can now have a type (counter, timestamp, timeval, timespec and
  possibly others in the future)
- stat_increment() / stat_decrement() now take an increment/decrement value
  and are at the moment only of type counter
- stat_set() now takes a stat_value
- provide helpers to convert raw values to stat_value

ok eric@, ok chl@

while at it fix a rq_queue_dump() call using a bogus timestamp in scheduler
ramqueue.

Revision 1.17 / (download) - annotate - [select for diffs], Fri Aug 24 18:46:46 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.16: +2 -2 lines
Diff to previous 1.16 (colored)

When an smtp session fails and IMSG_QUEUE_REMOVE_MESSAGE is sent to the
queue, also notify the scheduler so it can rollback the current update.
Send only the msgid while there.

ok gilles@

Revision 1.16 / (download) - annotate - [select for diffs], Fri Aug 24 18:21:06 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.15: +10 -1 lines
Diff to previous 1.15 (colored)

add stat counter for the number of envelopes inflight.

ok gilles@

Revision 1.15 / (download) - annotate - [select for diffs], Tue Aug 21 13:13:17 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.14: +9 -1 lines
Diff to previous 1.14 (colored)

Re-enable loop detection, but in mta and mda this time.

ok gilles@

Revision 1.14 / (download) - annotate - [select for diffs], Mon Aug 20 09:34:53 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.13: +2 -3 lines
Diff to previous 1.13 (colored)

chroot scheduler in /var/empty instead of /var/spool/smtpd now that eric@ fixed the initialization layer violation

committing on behalf of gilles@

ok gilles@ chl@

Revision 1.13 / (download) - annotate - [select for diffs], Sun Aug 19 15:06:36 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.12: +7 -2 lines
Diff to previous 1.12 (colored)

fix smtpctl schedule-all and schedule-id <msgid>

with help from eric@

ok eric@

Revision 1.12 / (download) - annotate - [select for diffs], Sat Aug 18 18:18:23 2012 UTC (11 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.11: +1 -3 lines
Diff to previous 1.11 (colored)

- introduce stat_backend, an API for pluggable statistic backends
  > statistics are no longer static structures in shared memory
  > statistics are only set, smtpd never uses them in its logic
  > each statistic is a key/value where key can be any (dynamic) string
- convert all uses of the former API to use the new one
- implement stat_ramstat that keeps non-persistent stats in ram structure

ok eric@, ok chl@

Revision 1.11 / (download) - annotate - [select for diffs], Thu Aug 9 19:16:26 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.10: +1 -2 lines
Diff to previous 1.10 (colored)

Envelopes scheduled as "removed" are already deleted from the scheduler.
Do not delete them twice.

Revision 1.10 / (download) - annotate - [select for diffs], Thu Aug 9 12:19:33 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.9: +1 -2 lines
Diff to previous 1.9 (colored)

dead prototype

Revision 1.9 / (download) - annotate - [select for diffs], Thu Aug 9 09:48:02 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.8: +64 -155 lines
Diff to previous 1.8 (colored)

Improve the message flows to completely isolate operations on the
queue backend within the queue process.

The scheduler sends envelope ids to the queue process which loads
the envelope and forward the request to the agent responsible for
the delivery.  The result is sent by the agent to the queue which
updates the storage before notifying the scheduler.

Bounces are created and enqueued (from the client side) by the
queue process, rather than the scheduler.

ok gilles@

Revision 1.8 / (download) - annotate - [select for diffs], Wed Aug 8 08:50:42 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.7: +178 -284 lines
Diff to previous 1.7 (colored)

Improve the scheduler backend API.

New envelopes are pushed into the scheduler through the insert()
commit() rollback() transactional interface functions.

Worklists are pulled from the scheduler through a single batch()
interface function, which returns a list of envelope ids and the
type of processing. Envelopes returned in this batch are said to
be "in-flight", as opposed to "pending". They are supposed to be
processed in some way, and either updated() or deleted() at some
point.

The schedule()/remove() functions are used to alter the internal
state of "pending" envelopes to make them schedulable.  The enve-
lopes will be part of a worklist on the next call to batch().

Rewrite the scheduler_ramqueue backend.

The initial queue loading in now done by the queue.

ok gilles@

Revision 1.7 / (download) - annotate - [select for diffs], Wed Jul 18 22:04:49 2012 UTC (11 years, 10 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_2_BASE, OPENBSD_5_2
Changes since 1.6: +4 -3 lines
Diff to previous 1.6 (colored)

make the scheduler less verbose by default.

suggested by Percy Piper.

ok gilles@

Revision 1.6 / (download) - annotate - [select for diffs], Tue Jul 10 11:13:40 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.5: +20 -30 lines
Diff to previous 1.5 (colored)

- simplify the scheduler loop logic further, it is ridiculously simple now
  and I don't think we can do much better (at that level) :-p
- always break out of the handler after processing an envelope, this will
  avoid a busy scheduler from not getting a chance to handle SIGTERM/SIGINT
  YES we can now ctrl-c a maaaaad scheduler !

ok eric@, ok chl@

Revision 1.5 / (download) - annotate - [select for diffs], Mon Jul 9 17:57:54 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.4: +60 -51 lines
Diff to previous 1.4 (colored)

- introduce log_trace(TRACE_SCHEDULER, ...)
- simplify a tiny tiny bit the scheduler loop
- no functional change (yet)

Revision 1.4 / (download) - annotate - [select for diffs], Mon Jul 9 09:57:53 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.3: +570 -16 lines
Diff to previous 1.3 (colored)

- runner is the terminology we used back when we had runqueues, we no
  longer have them and runner is actually a scheduler so rename.
- introduce scheduler_backend which does the same to scheduler than
  queue_backend does to queue and map_backend does to maps
- remove all occurences of RUNNER and runner, replace them with SCHEDULER
  and scheduler

ok eric@, ok chl@

Revision 1.3 / (download) - annotate - [select for diffs], Mon Jul 2 17:00:05 2012 UTC (11 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.2: +3 -7 lines
Diff to previous 1.2 (colored)

Lookup queue and scheduler backends by name, rather than enum.
Add a command-line option to specify the backend to use at runtime.

ok gilles@

Revision 1.2 / (download) - annotate - [select for diffs], Sun Jun 17 15:17:08 2012 UTC (11 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.1: +13 -1 lines
Diff to previous 1.1 (colored)

- introduce struct scheduler_info and the scheduler_info() function to fill
  a struct scheduler_info given a struct envelope
- adapt the scheduler API and the scheduler_ramqueue backend to use the new
  struct scheduler_info instead of a struct envelope

idea discussed with eric@ and chl@, mechanical diff, no functional change

Revision 1.1 / (download) - annotate - [select for diffs], Sat Jan 28 11:33:07 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_1_BASE, OPENBSD_5_1

- introduce the scheduler_backend API
- introduce the scheduler_ramqueue backend
- remove all occurences of ramqueue outside of the ramqueue backend
- teach runner how to use the new API

it is now possible to write custom schedulers !

ok eric@, ok chl@

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.