OpenBSD CVS

CVS log for src/usr.sbin/smtpd/scheduler_backend.c


[BACK] Up to [local] / src / usr.sbin / smtpd

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.18 / (download) - annotate - [select for diffs], Mon Jun 14 17:58:16 2021 UTC (2 years, 11 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5, OPENBSD_7_4_BASE, OPENBSD_7_4, OPENBSD_7_3_BASE, OPENBSD_7_3, OPENBSD_7_2_BASE, OPENBSD_7_2, OPENBSD_7_1_BASE, OPENBSD_7_1, OPENBSD_7_0_BASE, OPENBSD_7_0, HEAD
Changes since 1.17: +1 -14 lines
Diff to previous 1.17 (colored)

add required headers for smtpd.h and remove unnecessary ones in other files.

ok jung@

Revision 1.17 / (download) - annotate - [select for diffs], Wed May 26 18:08:55 2021 UTC (3 years ago) by eric
Branch: MAIN
Changes since 1.16: +1 -2 lines
Diff to previous 1.16 (colored)

replaces calls to err(3)/errx(3) with fatal()/fatalx() from log.c
for code that runs in the daemon.

ok florian@ millert@

Revision 1.16 / (download) - annotate - [select for diffs], Thu May 24 11:38:24 2018 UTC (6 years ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE, OPENBSD_6_9, OPENBSD_6_8_BASE, OPENBSD_6_8, OPENBSD_6_7_BASE, OPENBSD_6_7, OPENBSD_6_6_BASE, OPENBSD_6_6, OPENBSD_6_5_BASE, OPENBSD_6_5, OPENBSD_6_4_BASE, OPENBSD_6_4
Changes since 1.15: +20 -3 lines
Diff to previous 1.15 (colored)

switch smtpd to new grammar

ok eric@

Revision 1.15 / (download) - annotate - [select for diffs], Tue Jan 20 17:37:54 2015 UTC (9 years, 4 months ago) by deraadt
Branch: MAIN
CVS Tags: OPENBSD_6_3_BASE, OPENBSD_6_3, OPENBSD_6_2_BASE, OPENBSD_6_2, OPENBSD_6_1_BASE, OPENBSD_6_1, OPENBSD_6_0_BASE, OPENBSD_6_0, OPENBSD_5_9_BASE, OPENBSD_5_9, OPENBSD_5_8_BASE, OPENBSD_5_8, OPENBSD_5_7_BASE, OPENBSD_5_7
Changes since 1.14: +2 -1 lines
Diff to previous 1.14 (colored)

use <limits.h> comprehensively.  For now try to push <> includes to
each .c file, and out of the .h files.  To avoid overinclude.
ok gilles, in principle.  If this has been done right, -portable should
become easier to maintain.

Revision 1.14 / (download) - annotate - [select for diffs], Thu Jul 10 14:45:02 2014 UTC (9 years, 10 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_6_BASE, OPENBSD_5_6
Changes since 1.13: +1 -20 lines
Diff to previous 1.13 (colored)

Improve the scheduler, better and simpler.

- Get rid of the scheduler_batch structure. The scheduler can now return
  envelopes of different types in a single run, interlacing them to avoid
  batch effects.

- Ask for an acknowledgement from the queue when removing or expiring
  an envelope to benefit from the inflight envelope limitation mechanism.
  This ensures that the scheduler always keeps sending envelopes at a rate
  that the queue can sustain in all cases.

- Limit the number of envelopes in a holdq.  When a holdq is full,
  new envelopes are put back in the pending queue instead, with a
  shorter retry time.

- Plumbing for proc-ified schedulers.

imsg version bump. smtpctl stop before updating.

ok gilles@

Revision 1.13 / (download) - annotate - [select for diffs], Tue Feb 4 14:56:03 2014 UTC (10 years, 3 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_5_BASE, OPENBSD_5_5
Changes since 1.12: +2 -3 lines
Diff to previous 1.12 (colored)

internal improvements and cleanups

- get rid of the whole penalty thing for failed envelopes in the mta and scheduler.
- do not disable routes on smtp errors
- try to schedule all types of envelopes on each scheduler frame.

Revision 1.12 / (download) - annotate - [select for diffs], Sun Oct 27 17:47:53 2013 UTC (10 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.11: +0 -0 lines
Diff to previous 1.11 (colored)

Implement a feedback mechanism which allows the mta to "hold" envelopes
in the scheduler when it has too many tasks for a given relay.  The
envelopes are put on a wait queue, and are not scheduled again until
the mta "releases" some envelopes from that queue.

It prevents from having too many inflight envelopes, which are out of reach
for the admin.

Revision 1.11 / (download) - annotate - [select for diffs], Fri Jul 19 21:34:31 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
CVS Tags: OPENBSD_5_4_BASE, OPENBSD_5_4
Changes since 1.10: +3 -0 lines
Diff to previous 1.10 (colored)

Implement a scheduler_proc backend

Revision 1.10 / (download) - annotate - [select for diffs], Fri Jul 19 15:14:23 2013 UTC (10 years, 10 months ago) by eric
Branch: MAIN
Changes since 1.9: +8 -5 lines
Diff to previous 1.9 (colored)

scheduler improvements:
- implement suspend/resume scheduling for individual envelopes or message,
  with the associated smtpctl commands.
- allow the mta to request immediate scheduling of an envelope.
- on temporary failures a penalty can be given to further delay the next try.

Revision 1.9 / (download) - annotate - [select for diffs], Fri May 24 17:03:14 2013 UTC (11 years ago) by eric
Branch: MAIN
Changes since 1.8: +1 -2 lines
Diff to previous 1.8 (colored)

sync with OpenSMTPD 5.3.2

ok gilles@

Revision 1.8 / (download) - annotate - [select for diffs], Sat Jan 26 09:37:23 2013 UTC (11 years, 4 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_3_BASE, OPENBSD_5_3
Changes since 1.7: +11 -6 lines
Diff to previous 1.7 (colored)

Sync with our smtpd repo:

* first bricks of ldap and sqlite support (not finished but both working)
* new table API to replace map API, all lookups are done through tables
* improved handling of temporary errors throughout the daemon
* improved scheduler and mta logic: connection reuse, optimizes batches
* improved queue: more tolerant to admin errors, new layout, less disk-IO
* improved memory usage under high load
* SSL certs/keys isolated to lookup process to avoid facing network
* VIRTUAL support improved, fully virtual setups possible now
* runtime tracing of processes through smtpctl trace
* ssl_privsep.c sync-ed with relayd
* ssl.c no longer contains smtpd specific interfaces
* smtpd-specific ssl bits moved to ssl_smtpd.c
* update mail address in copyright

FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE.

smtpd.conf(5) simplified, it will require adaptations

ok eric@

Revision 1.7 / (download) - annotate - [select for diffs], Fri Nov 23 10:55:25 2012 UTC (11 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.6: +2 -2 lines
Diff to previous 1.6 (colored)

knf

ok gilles@

Revision 1.6 / (download) - annotate - [select for diffs], Tue Sep 11 19:19:13 2012 UTC (11 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.5: +7 -13 lines
Diff to previous 1.5 (colored)

- simplify scheduler_compute_schedule by making it a quadratic delaying.
  no more if/else, we derive the delay from the retry count. works for
  qmail, should work for us too.

ok eric@

Revision 1.5 / (download) - annotate - [select for diffs], Fri Aug 24 12:29:50 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.4: +1 -3 lines
Diff to previous 1.4 (colored)

Remove the rq_host and rq_batch structures from the ramqueue scheduler.
The scheduler should only allow admin to schedule specific envelopes by
id, or msgid. More advanced scheduling (per host/route/whatever) should
be achieved using smtpctl schedule-id and proper filtering on the queue,
or using ad-hoc scheduler backend and tools.

ok gilles@ chl@

Revision 1.4 / (download) - annotate - [select for diffs], Fri Aug 24 07:32:12 2012 UTC (11 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.3: +2 -2 lines
Diff to previous 1.3 (colored)

fix assignment typo

ok eric@ gilles@

Revision 1.3 / (download) - annotate - [select for diffs], Wed Aug 8 08:50:42 2012 UTC (11 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.2: +22 -1 lines
Diff to previous 1.2 (colored)

Improve the scheduler backend API.

New envelopes are pushed into the scheduler through the insert()
commit() rollback() transactional interface functions.

Worklists are pulled from the scheduler through a single batch()
interface function, which returns a list of envelope ids and the
type of processing. Envelopes returned in this batch are said to
be "in-flight", as opposed to "pending". They are supposed to be
processed in some way, and either updated() or deleted() at some
point.

The schedule()/remove() functions are used to alter the internal
state of "pending" envelopes to make them schedulable.  The enve-
lopes will be part of a worklist on the next call to batch().

Rewrite the scheduler_ramqueue backend.

The initial queue loading in now done by the queue.

ok gilles@

Revision 1.2 / (download) - annotate - [select for diffs], Sat Jul 28 12:06:41 2012 UTC (11 years, 10 months ago) by chl
Branch: MAIN
Changes since 1.1: +3 -2 lines
Diff to previous 1.1 (colored)

fix uninitialized field type in scheduler_info struct.

found with valgrind on -portable.

ok gilles@

Revision 1.1 / (download) - annotate - [select for diffs], Mon Jul 9 09:57:53 2012 UTC (11 years, 10 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_2_BASE, OPENBSD_5_2

- runner is the terminology we used back when we had runqueues, we no
  longer have them and runner is actually a scheduler so rename.
- introduce scheduler_backend which does the same to scheduler than
  queue_backend does to queue and map_backend does to maps
- remove all occurences of RUNNER and runner, replace them with SCHEDULER
  and scheduler

ok eric@, ok chl@

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.