![[BACK]](/icons/back.gif){kind=icon}
Up to [local] / src / usr.sbin / smtpd
Request diff between arbitrary revisions
Default branch: MAIN
Revision 1.145, Mon Jul 9 09:57:53 2012 UTC (11 years, 11 months ago) by gilles
Branch: MAIN
CVS Tags: HEAD
Changes since 1.144: +1 -1 lines
FILE REMOVED
- runner is the terminology we used back when we had runqueues, we no longer have them and runner is actually a scheduler so rename. - introduce scheduler_backend which does the same to scheduler than queue_backend does to queue and map_backend does to maps - remove all occurences of RUNNER and runner, replace them with SCHEDULER and scheduler ok eric@, ok chl@
Revision 1.144 / (download) - annotate - [select for diffs], Mon Jul 9 08:08:29 2012 UTC (11 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.143: +3 -1 lines
Diff to previous 1.143 (colored)
first step of simplifying fsqueue: - remove the /envelopes subdirectory, envelopes are at the same level than the message file - kill PATH_ENVELOPES define - reduce the number of buckets from 0xfff to 0xff, this avoid performances of the queue to decrease when we start having tons of buckets this diff introduces a change to the queue layout, you will want to empty your queue before updating. more cleanup to come ok eric@, ok chl@
Revision 1.143 / (download) - annotate - [select for diffs], Sun Jul 8 18:13:08 2012 UTC (11 years, 11 months ago) by chl
Branch: MAIN
Changes since 1.142: +3 -5 lines
Diff to previous 1.142 (colored)
remove enum queue_kind from queue_fsqueue.c. incoming messages are now always stored in /incoming, whatever the queue_backend is. remove QOP_FD_RW and fsqueue_message_fd_rw(). while there check return value of generated paths before calling rmtree() with advice from gilles@ and eric@ ok gilles@ eric@
Revision 1.142 / (download) - annotate - [select for diffs], Mon Jul 2 17:00:05 2012 UTC (11 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.141: +6 -2 lines
Diff to previous 1.141 (colored)
Lookup queue and scheduler backends by name, rather than enum. Add a command-line option to specify the backend to use at runtime. ok gilles@
Revision 1.141 / (download) - annotate - [select for diffs], Wed Jun 20 20:45:23 2012 UTC (11 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.140: +17 -18 lines
Diff to previous 1.140 (colored)
Finally get rid of the queue_kind enum in the queue API. Keep that internally in fsqueue backend for now, and let the fsqueue_message() and fsqueue_envelope() dispatchers do the right thing. Based on a diff by chl@ ok chl@ gilles@
Revision 1.140 / (download) - annotate - [select for diffs], Wed Jun 20 20:27:41 2012 UTC (11 years, 11 months ago) by eric
Branch: MAIN
Changes since 1.139: +2 -2 lines
Diff to previous 1.139 (colored)
fix runner spinning on message loop. ok gilles@
Revision 1.139 / (download) - annotate - [select for diffs], Sun Jun 17 15:17:08 2012 UTC (11 years, 11 months ago) by gilles
Branch: MAIN
Changes since 1.138: +16 -7 lines
Diff to previous 1.138 (colored)
- introduce struct scheduler_info and the scheduler_info() function to fill a struct scheduler_info given a struct envelope - adapt the scheduler API and the scheduler_ramqueue backend to use the new struct scheduler_info instead of a struct envelope idea discussed with eric@ and chl@, mechanical diff, no functional change
Revision 1.138 / (download) - annotate - [select for diffs], Sun Apr 15 12:12:35 2012 UTC (12 years, 1 month ago) by chl
Branch: MAIN
Changes since 1.137: +1 -3 lines
Diff to previous 1.137 (colored)
Remove dead assignments and newly created unused variables. Found by LLVM/Clang Static Analyzer. ok gilles@
Revision 1.137 / (download) - annotate - [select for diffs], Mon Mar 19 20:38:49 2012 UTC (12 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.136: +3 -3 lines
Diff to previous 1.136 (colored)
when a mailer daemon loop is detected, do not insert new bounce in ramqueue fixes an issue experienced by myself and Roman Kravchuck ok eric@
Revision 1.136 / (download) - annotate - [select for diffs], Wed Mar 7 22:54:49 2012 UTC (12 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.135: +14 -12 lines
Diff to previous 1.135 (colored)
various reliability fixes: - prevent queue_fsqueue from fatal() when it hits an ENOENT, it can happen - change a bit the scheduler API to simplify it, fix runner accordingly - we can't remove msg/batch from ramqueue while envelope is offloaded or it will cause a double, instead we add refcnt to both msg/batch and only free them when it hits 0
Revision 1.135 / (download) - annotate - [select for diffs], Tue Jan 31 21:05:26 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_1_BASE,
OPENBSD_5_1
Changes since 1.134: +32 -15 lines
Diff to previous 1.134 (colored)
fix an issue observed this week-end while flooding ajacoutot@ : we keep track of available fd's to prevent scheduling of messages if we know that we are going to fail. however, since the envelope is not removed from the scheduler, it will be rescheduled right away leading to a busy loop in the scheduler. we know flag the mda/mta processes as BUSY and do not schedule envelopes that target a BUSY process. also, fix a potential bug that could lead to a use after free when doing a batch/message/host traversal of schedulable envelopes. while at it fix misuse of env->sc_opts as env->sc_flags, was not really causing any issue as the misuse was constant ...
Revision 1.134 / (download) - annotate - [select for diffs], Sat Jan 28 16:50:02 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.133: +4 -1 lines
Diff to previous 1.133 (colored)
add optional display handler to scheduler_backend, if not NULL the handler will be called for each iteration of the runner implement a display handler for scheduler_ramqueue to display the entire ramqueue (hosttree, msgtree and linear queue) in log_debug
Revision 1.133 / (download) - annotate - [select for diffs], Sat Jan 28 11:33:07 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.132: +130 -143 lines
Diff to previous 1.132 (colored)
- introduce the scheduler_backend API - introduce the scheduler_ramqueue backend - remove all occurences of ramqueue outside of the ramqueue backend - teach runner how to use the new API it is now possible to write custom schedulers ! ok eric@, ok chl@
Revision 1.132 / (download) - annotate - [select for diffs], Fri Jan 13 14:01:58 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.131: +21 -33 lines
Diff to previous 1.131 (colored)
Stop using envelope->status to report delivery outcome to the runner/queue. Instead, replace IMSG_QUEUE_MESSAGE_UPDATE with three messages: - IMSG_QUEUE_DELIVERY_OK - IMSG_QUEUE_DELIVERY_TEMPFAIL - IMSG_QUEUE_DELIVERY_PERMFAIL 1) it's less confusing as status is also used by smtp 2) it's easier to see what happens just looking at imsg traces 3) it makes the code path generally easier to follow 4) it's safer because it enforces clear semantics and intent, whereas the status field is loosely defined and could carry bogus values. ok gilles@ chl@
Revision 1.131 / (download) - annotate - [select for diffs], Thu Jan 12 23:17:02 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.130: +2 -2 lines
Diff to previous 1.130 (colored)
we should never use ramqueue_first_envelope() as a mean to determine the first schedulable envelope otherwise we end up with nsched wrong ... heck, let's kill ramqueue_first_envelope() and be done with it ;)
Revision 1.130 / (download) - annotate - [select for diffs], Thu Jan 12 22:59:55 2012 UTC (12 years, 4 months ago) by eric
Branch: MAIN
Changes since 1.129: +1 -3 lines
Diff to previous 1.129 (colored)
The status field in the envelope is confusing. Its only purpose is to notify the runner of what happened with an envelope that has been scheduled. It is not part of the state of the envelope, and it is not even dumped. So it should only be set by mta/mda, checked by runner to decide what to do with the envelope, and ignored everywhere else. ok gilles@
Revision 1.129 / (download) - annotate - [select for diffs], Thu Jan 12 22:40:16 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.128: +7 -4 lines
Diff to previous 1.128 (colored)
the ramqueue filling at startup was busted, it would load envelopes one by one instead of doing it efficiently. fix runner_timeout() and ramqueue_load() so that at startup smtpd fills the ramqueue as long as there's no schedulable envelope in it, interrupts filling if there is and resume once it's scheduled. bug spotted by Nathanael Rensel, bug fix by me w/ help from eric@ tested by eric@ and I
Revision 1.128 / (download) - annotate - [select for diffs], Thu Jan 12 22:00:21 2012 UTC (12 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.127: +4 -13 lines
Diff to previous 1.127 (colored)
now that we no longer do a stateful iteration on schedule queue, we need to make ramqueue_next_envelope() aware of pauses in mda/mta. while at it kill the pausing of bounces since they are reinjected in smtp and end up paused by the mda/mta cases. fixes an infinite loop observed by eric@ when pausing deliveries and trying to interrupt smtpd while it attempts to fetch next envelope :) tested by eric@ and I, ok eric@
Revision 1.127 / (download) - annotate - [select for diffs], Wed Jan 11 17:46:36 2012 UTC (12 years, 5 months ago) by eric
Branch: MAIN
Changes since 1.126: +1 -19 lines
Diff to previous 1.126 (colored)
Simplify runner/queue by getting rid of Q_PURGE. Instead, let smtpd periodically clear the purge/ directory. At init time, the fsqueue backend simply moves the existing incoming/ dir in purge/ to discard aborted sessions. ok gilles@ chl@
Revision 1.126 / (download) - annotate - [select for diffs], Wed Nov 16 19:38:56 2011 UTC (12 years, 6 months ago) by eric
Branch: MAIN
Changes since 1.125: +1 -6 lines
Diff to previous 1.125 (colored)
remove unused functions ok gilles@ chl@
Revision 1.125 / (download) - annotate - [select for diffs], Tue Nov 15 23:06:39 2011 UTC (12 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.124: +14 -102 lines
Diff to previous 1.124 (colored)
Qwalk, our API to linearly walk over the persistent queue, did not take the queue_backend into account and assumed a filesystem with a specific layout. This commit does plenty of things: - make qwalk an abstraction in the queue_backend API, and impose queue drivers to implement qwalk_open(), qwalk() and qwalk_close(); - move previous qwalk_open(), qwalk() and qwalk_close() to the fsqueue driver since they were fsqueue specific ... - make qwalk API work with msgid/evpid instead of pathnames since we're going to use the queue_backend API to load envelopes by evpid anyway; - makes smtpd use *solely* the queue_backend API when manipulating the queue. pathnames were removed from smtpd.h and moved into the fsqueue which means we can now store a queue anywhere ... as long as we write the ten functions or so required for a queue driver ;-) ok eric@, ok chl@
Revision 1.124 / (download) - annotate - [select for diffs], Mon Nov 14 19:23:41 2011 UTC (12 years, 6 months ago) by chl
Branch: MAIN
Changes since 1.123: +2 -2 lines
Diff to previous 1.123 (colored)
when receiving an unexpected imsg, print its name. with help and ideas from eric@ ok eric@ gilles@
Revision 1.123 / (download) - annotate - [select for diffs], Mon Nov 7 11:14:10 2011 UTC (12 years, 7 months ago) by eric
Branch: MAIN
Changes since 1.122: +3 -17 lines
Diff to previous 1.122 (colored)
Let the smtpd process handle the enqueueing of offline messages at startup, rather than playing tricks with the runner. This will allow further simplifications and improvements in the runner/queue. ok gilles@
Revision 1.122 / (download) - annotate - [select for diffs], Thu Oct 27 14:32:57 2011 UTC (12 years, 7 months ago) by chl
Branch: MAIN
Changes since 1.121: +10 -8 lines
Diff to previous 1.121 (colored)
Use PRI{x,d}64 in format strings instead of %llx, %lld or %qd to print {u_,}int64_t or time_t
While there, cast some time_t to int64_t
These will fix build warnings for portable smptd
ok gilles@ eric@
Revision 1.121 / (download) - annotate - [select for diffs], Wed Oct 26 20:47:31 2011 UTC (12 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.120: +19 -14 lines
Diff to previous 1.120 (colored)
- fix smtpctl pause/resume so the ramqueue scheduling is done correctly - rename IMSG and smtpctl pause/resume parameters - update man page tested by me, ok chl@, eric@
Revision 1.120 / (download) - annotate - [select for diffs], Sun Oct 23 09:30:07 2011 UTC (12 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.119: +40 -29 lines
Diff to previous 1.119 (colored)
fsqueue no longer stores envelopes by dumping the structure, instead use a couple of load/dump functions to convert to and from a human readable fmt. while at it kill struct delivery and merge back its fields to the envelope. this basically means we shouldn't require users to flush their queues every time we make a change to struct envelope. work is not done, but we're at a better state than the binary fsqueue so we'll improve it in-tree. has been running on my own box for the last 12 hours or so ok eric@, chl@
Revision 1.119 / (download) - annotate - [select for diffs], Sun Oct 9 18:39:53 2011 UTC (12 years, 8 months ago) by eric
Branch: MAIN
Changes since 1.118: +3 -1 lines
Diff to previous 1.118 (colored)
show messages sent between processes in debug mode ok gilles@ chl@
Revision 1.118 / (download) - annotate - [select for diffs], Sun Sep 18 21:37:53 2011 UTC (12 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.117: +2 -2 lines
Diff to previous 1.117 (colored)
a single ramqueue message may be shared by many ramqueue envelopes to be delivered to many ramqueue hosts, therefore storing the rq_host pointer in the rq_msg envelope is wrong and causes baaaaad behavior. this commit fixes reliability issues in runner process, experienced and reported by many
Revision 1.117 / (download) - annotate - [select for diffs], Thu Sep 1 19:56:49 2011 UTC (12 years, 9 months ago) by eric
Branch: MAIN
Changes since 1.116: +17 -28 lines
Diff to previous 1.116 (colored)
Introduce a small set of functions to manage stat counters in a simpler and hopefully saner way. ok gilles@ chl@
Revision 1.116 / (download) - annotate - [select for diffs], Tue Aug 30 17:06:01 2011 UTC (12 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.115: +4 -19 lines
Diff to previous 1.115 (colored)
Call free() into ramqueue_remove_{batch,host,message,envelope}() instead of just after the function call.
While there, also include stats counters into their respective function.
ok gilles@
Revision 1.115 / (download) - annotate - [select for diffs], Mon Aug 29 21:43:09 2011 UTC (12 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.114: +3 -3 lines
Diff to previous 1.114 (colored)
cast all printed time_t to long long int, and change format string accordingly. ok gilles@
Revision 1.114 / (download) - annotate - [select for diffs], Mon Aug 29 18:49:29 2011 UTC (12 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.113: +2 -1 lines
Diff to previous 1.113 (colored)
add missing header needed by signal() ok gilles@
Revision 1.113 / (download) - annotate - [select for diffs], Fri Aug 26 14:39:47 2011 UTC (12 years, 9 months ago) by chl
Branch: MAIN
Changes since 1.112: +2 -1 lines
Diff to previous 1.112 (colored)
add missing header needed by time() ok gilles@
Revision 1.112 / (download) - annotate - [select for diffs], Wed Aug 17 20:54:16 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.111: +16 -15 lines
Diff to previous 1.111 (colored)
- plug a memory leak in runner_remove_envelope() - cosmetic change
Revision 1.111 / (download) - annotate - [select for diffs], Wed Aug 17 20:35:11 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.110: +26 -23 lines
Diff to previous 1.110 (colored)
move ramqueue_host pointer from ramqueue_envelope to ramqueue_message. this allows us to save one pointer from each envelope stored in ram while still allowing O(1) host lookups by ramqueue_envelope.
Revision 1.110 / (download) - annotate - [select for diffs], Wed Aug 17 20:04:43 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.109: +50 -20 lines
Diff to previous 1.109 (colored)
- teach smtpctl remove about the new ramqueue structure - bonus #1: O(log n) removal of envelopes - bonus #2: removing all envelopes that have the same msgid works again
Revision 1.109 / (download) - annotate - [select for diffs], Tue Aug 16 19:02:03 2011 UTC (12 years, 9 months ago) by gilles
Branch: MAIN
Changes since 1.108: +9 -3 lines
Diff to previous 1.108 (colored)
add a host-tree and an envelope-tree in the ramqueue, they will be used to improve scheduling and general ramqueue operations. unused yet
Revision 1.108 / (download) - annotate - [select for diffs], Thu Jul 21 23:29:24 2011 UTC (12 years, 10 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_5_0_BASE,
OPENBSD_5_0
Changes since 1.107: +28 -1 lines
Diff to previous 1.107 (colored)
- update smtpctl.8 to reflect reality - bring back 'smtpctl schedule' and 'smtpctl remove' to life Things you should know: The ramqueue data structure is not finished yet and lacks an envelope tree for evpid lookups. I wanted to wait until I'm done but too many people are affected by not being able to reschedule envelopes, this is a quick fix. So right now there's an O(rrible) complexity as both commands will perform a (possibly aborted) queue scan leading to O(n). I will make that O(log n) soon. Also, smtpctl remove no longer supports removing an entire message, I will fix that very soon too.
Revision 1.107 / (download) - annotate - [select for diffs], Mon May 16 21:05:52 2011 UTC (13 years ago) by gilles
Branch: MAIN
Changes since 1.106: +41 -45 lines
Diff to previous 1.106 (colored)
murder struct path and make sure smtpd uses simpler structures that do not bring a shitload of unnecessary information everywhere. this required many parts of smtpd to be refactored and more specifically envelope expansion. in the process lots of code got simplified, and the envelope expansion code has been isolated to lka_session.c with some longstanding bugs fixed. Diff has been tested by many with no major regression reported. armani@ spotted a bug in a setup where a domain is listed a both primary and virtual, I will fix that in-tree as it's becoming painful to maintain this diff out.
Revision 1.106 / (download) - annotate - [select for diffs], Sun May 1 12:57:11 2011 UTC (13 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.105: +48 -50 lines
Diff to previous 1.105 (colored)
the smtpd env is meant to be global, so do not pass it all around. discussed with and ok gilles@
Revision 1.105 / (download) - annotate - [select for diffs], Sun Apr 17 13:36:07 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.104: +17 -19 lines
Diff to previous 1.104 (colored)
cleanups, cosmethic changes, functions that should be static are now static no functionnal change
Revision 1.104 / (download) - annotate - [select for diffs], Sun Apr 17 11:39:23 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.103: +10 -10 lines
Diff to previous 1.103 (colored)
a structure describing an envelope should be called struct envelope, not struct message ...
Revision 1.103 / (download) - annotate - [select for diffs], Fri Apr 15 19:03:28 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.102: +16 -5 lines
Diff to previous 1.102 (colored)
whenever an envelope is reinserted into the ramqueue after a trip to mda or mta, call runner_reset_events() so runner starts reprocessing ramqueue
Revision 1.102 / (download) - annotate - [select for diffs], Fri Apr 15 17:01:05 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.101: +49 -25 lines
Diff to previous 1.101 (colored)
kill message_id and message_uid smtpd now has an evpid associated to each delivery message, the evpid is an u_int64_t where the upper 32 bits are the msgid, and the 32 bits are the envelope unique identifier for that message. this results in lots of space saved in both disk-based and ram-based queues, but also simplifies a lot of code. change has been stressed on my desktop, and has ran on my MX for the entire afternoon without a regression.
Revision 1.101 / (download) - annotate - [select for diffs], Fri Apr 15 11:39:57 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.100: +2 -1 lines
Diff to previous 1.100 (colored)
temporarily add fsqueue_hash() prototype until runner is fully converted to queue_backend API
Revision 1.100 / (download) - annotate - [select for diffs], Thu Apr 14 23:29:56 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.99: +2 -3 lines
Diff to previous 1.99 (colored)
bye bye queue_hash() you can now rest in peace.
Revision 1.99 / (download) - annotate - [select for diffs], Thu Apr 14 22:36:09 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.98: +3 -3 lines
Diff to previous 1.98 (colored)
- implement fsqueue_message_create() and fsqueue_message_commit() - change a few prototypes to allow bounce messages to use the queue_backend API until it gets merged in - kill functions of the queue API that have been deprecated
Revision 1.98 / (download) - annotate - [select for diffs], Thu Apr 14 21:53:46 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.97: +6 -6 lines
Diff to previous 1.97 (colored)
fsqueue now provides fsqueue_message_fd_r() and fsqueue_message_fd_rw() to
obtain a read{-only,/write} descriptor to the message file.
make sure smtpd uses the new API everywhere it needs a fd, and kill the
many functions that were used until now.
Revision 1.97 / (download) - annotate - [select for diffs], Thu Apr 14 20:11:08 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.96: +14 -10 lines
Diff to previous 1.96 (colored)
fsqueue queue backend will implement a filesystem queue: - fsqueue->setup() performs the queue initialization; - fsqueue->message() controls messages; - fsqueue->envelope() controls envelopes; This commit brings the following to fsbackend: fsqueue_setup(), fsqueue_message_delete(), fsqueue_envelope_load(), fsqueue_envelope_update(), fsqueue_envelope_delete(). It also makes smtpd use the queue_backend API for these operations.
Revision 1.96 / (download) - annotate - [select for diffs], Wed Apr 13 20:53:18 2011 UTC (13 years, 1 month ago) by gilles
Branch: MAIN
Changes since 1.95: +231 -480 lines
Diff to previous 1.95 (colored)
following an idea from jacekm@, smtpd now uses a ram-queue instead of doing a continuous walk on the disk-queue. the implementation differs from what jacekm@ commited (and I backed out) a while ago in that it uses a queue and a host tree required for upcoming features. code will be improved in tree, it requires changes to be done in queue and bounce API, I just wanted to commit a working version first ... tested by todd@ and I
Revision 1.95 / (download) - annotate - [select for diffs], Sun Nov 28 14:35:58 2010 UTC (13 years, 6 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_4_9_BASE,
OPENBSD_4_9
Changes since 1.94: +1 -8 lines
Diff to previous 1.94 (colored)
remove all unused headers
Revision 1.94 / (download) - annotate - [select for diffs], Sun Nov 28 13:56:43 2010 UTC (13 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.93: +3 -1 lines
Diff to previous 1.93 (colored)
a bit of .h cleanups, no functionnal change
Revision 1.93 / (download) - annotate - [select for diffs], Wed Nov 24 23:27:04 2010 UTC (13 years, 6 months ago) by todd
Branch: MAIN
Changes since 1.92: +9 -1 lines
Diff to previous 1.92 (colored)
add *maxactive stats "ok and no need to keep them for yourself" gilles@
Revision 1.92 / (download) - annotate - [select for diffs], Thu Oct 28 21:15:50 2010 UTC (13 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.91: +5 -4 lines
Diff to previous 1.91 (colored)
teach smtpd how to handle per-rule delays for message expiry, this allows some rules to have a longer expiry delay than the default: accept for [...] relay expire 8d # will stay 8 days in queue I added the man page bits so I don't forget but I need to reword it a bit
Revision 1.91 / (download) - annotate - [select for diffs], Sat Oct 9 22:12:26 2010 UTC (13 years, 8 months ago) by gilles
Branch: MAIN
Changes since 1.90: +1 -1 lines
Diff to previous 1.90 (colored)
these need to be re-added too
Revision 1.90, Tue Jun 1 23:06:23 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.89: +1 -1 lines
FILE REMOVED
new queue, again; gcc2 compile tested by deraadt
Revision 1.89 / (download) - annotate - [select for diffs], Tue Jun 1 19:47:09 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.88: +0 -0 lines
Diff to previous 1.88 (colored)
New queue doesn't compile on gcc2, back out. Spotted by deraadt@
Revision 1.88, Mon May 31 23:38:56 2010 UTC (14 years ago) by jacekm
Branch: MAIN
Changes since 1.87: +1 -1 lines
FILE REMOVED
Rewrite entire queue code. Major goals: 1) Fix bad performance caused by the runner process doing full queue read in 1s intervals. My Soekris can now happily accept >50 msg/s while having multi-thousand queue; before, one hundred queue would bring the system to its knees. 2) Introduce Qmail-like scheduler that doesn't write as much to the disk so that it needs less code for servicing error conditions, which in some places can be tricky to get right. 3) Introduce separation between the scheduler and the backend; these two queue aspects shouldn't be too tied too each other. This means that eg. storing queue in SQL requires rewrite of just queue_backend.c. 4) Make on-disk queue format architecture independent, and more easily extensible, to reduce number of flag days in the future. Minor goals: ENOSPC no longer prevents delivery attempts, fixed session limiting for relayed mail, improved batching of "relay via" mails, human-readable mailq output, "show queue raw" command, clearer logging, sending of single bounce about multiple recipients, exact delay= computation, zero delay between deliveries while within session limit (currently 1s delay between re-scheduling is enforced), mta no longer requests content fd, corrected session limit for bounce submissions, tiny <100B queue files instead of multi-KB, detect loops before accepting mail, reduce traffic on imsg channels by killing enormous struct submit_status.
Revision 1.87 / (download) - annotate - [select for diffs], Wed May 19 20:57:10 2010 UTC (14 years ago) by gilles
Branch: MAIN
Changes since 1.86: +1 -2 lines
Diff to previous 1.86 (colored)
cleanup-only commit, removes unrequired includes, no functionnal change
Revision 1.86 / (download) - annotate - [select for diffs], Thu Apr 22 12:56:33 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.85: +4 -2 lines
Diff to previous 1.85 (colored)
Fix a case of runner trying to send imsg directly to smtp process instead of forwarding it via queue.
Revision 1.85 / (download) - annotate - [select for diffs], Thu Apr 22 12:13:33 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.84: +19 -32 lines
Diff to previous 1.84 (colored)
- kill the runner_imsg_compose wrapper to reduce indirection - kill noisy log_debug
Revision 1.84 / (download) - annotate - [select for diffs], Wed Apr 21 18:54:43 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.83: +93 -105 lines
Diff to previous 1.83 (colored)
Runner process is just a helper for queue, so tear down its imsg channels to parent, mda, mta, lka, smtp, and control. This leaves just the channel to queue, which forwards imsgs on runner's behalf and redirects any replies back to it. OK gilles@
Revision 1.83 / (download) - annotate - [select for diffs], Wed Apr 21 08:29:01 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.82: +1 -7 lines
Diff to previous 1.82 (colored)
Remove unusable ifdef DEBUG code.
Revision 1.82 / (download) - annotate - [select for diffs], Wed Apr 21 00:12:49 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.81: +2 -2 lines
Diff to previous 1.81 (colored)
Fix logic error just like the one in previous revision.
Revision 1.81 / (download) - annotate - [select for diffs], Tue Apr 20 20:55:04 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.80: +2 -2 lines
Diff to previous 1.80 (colored)
Fix crash that could happen when attempting ``smtpctl remove'' while the message is being delivered.
Revision 1.80 / (download) - annotate - [select for diffs], Tue Apr 20 15:34:56 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.79: +82 -381 lines
Diff to previous 1.79 (colored)
Kill *2400* lines of code by abstracting common bits of the imsg handlers.
Revision 1.79 / (download) - annotate - [select for diffs], Mon Apr 19 08:14:07 2010 UTC (14 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.78: +51 -72 lines
Diff to previous 1.78 (colored)
Simplify local delivery codepath: - replace uses of struct batch in the parent with simpler struct delivery. - replace IMSG_BATCH_* dance with single IMSG_MDA_SESS_NEW. - make mda assume it delivers to external program over a pipe. - fork helper process when delivering to maildir or a file. New feature: upon external mda failure use last line of its output as an error message. With input and tests from nicm@. OK nicm@ gilles@
Revision 1.78 / (download) - annotate - [select for diffs], Sun Jan 10 16:42:35 2010 UTC (14 years, 5 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_4_7_BASE,
OPENBSD_4_7
Changes since 1.77: +60 -1 lines
Diff to previous 1.77 (colored)
- teach runner how to remove a message from queue given a message id/uid and assuming message is not in processing/scheduled state - teach smtpctl how to request message removal from runner discussed with todd@, idea ok jacekm@
Revision 1.77 / (download) - annotate - [select for diffs], Sun Jan 3 14:37:37 2010 UTC (14 years, 5 months ago) by chl
Branch: MAIN
Changes since 1.76: +10 -1 lines
Diff to previous 1.76 (colored)
Implement "log verbose" and "log brief" to enable or disable verbose debug logging on runtime. Based on claudio@'s work on ripd, ospfd, ospf6d, dvmrpd, ldpd, bgpd. With help/ideas/testing from gilles@ jacekm@ todd@ ok jacekm@
Revision 1.76 / (download) - annotate - [select for diffs], Mon Dec 14 19:56:55 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.75: +2 -2 lines
Diff to previous 1.75 (colored)
Tweak the logic behind setting the fd limits so that smtpd is less likely to get upset by custom soft/hard ulimit settings. Suggested by todd@
Revision 1.75 / (download) - annotate - [select for diffs], Mon Dec 14 18:21:53 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.74: +30 -14 lines
Diff to previous 1.74 (colored)
Control maximum number of bounce sessions similarly to how the mta and mda are now controlled.
Revision 1.74 / (download) - annotate - [select for diffs], Mon Dec 14 16:44:14 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.73: +28 -3 lines
Diff to previous 1.73 (colored)
Impose sessions limit on the delivery sessions (mta and mda).
Revision 1.73 / (download) - annotate - [select for diffs], Sun Dec 13 22:02:55 2009 UTC (14 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.72: +4 -1 lines
Diff to previous 1.72 (colored)
Use safe fd limits in smtp, lka, queue, and control. Removes a possibility for fd-starvation fatal when under heavy load.
Revision 1.72 / (download) - annotate - [select for diffs], Sun Nov 8 23:08:56 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.71: +3 -1 lines
Diff to previous 1.71 (colored)
rework a bit expansion and data structures involved in the expansion so we no longer have a direct mapping between structures saved in aliases/virtual db and structures used at runtime during expansion. side effects ? struct alias is smaller, databases are smaller and it is no longer necessary to rebuild aliases/virtual databases each time jacekm@ or I make changes to some obscure structure used indirectely during expansion rebuild databases, flush queues, make clean
Revision 1.71 / (download) - annotate - [select for diffs], Sun Nov 8 21:40:05 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.70: +2 -2 lines
Diff to previous 1.70 (colored)
- make aliases expansion use a rb tree instead of a tail queue, the code
doesn't take advantage of the new structure yet, but this was a needed
change for upcoming improvements.
- introduce aliasestree_{lookup,insert,remove} to the aliases api
- rename queue_generate_id() to generate_uid() and move it to utils.c as
it is used all over the place and not only in queue
tree idea discussed with jacekm@, if you update rebuild aliases db, make
clean and flush queue
Revision 1.70 / (download) - annotate - [select for diffs], Tue Nov 3 11:10:43 2009 UTC (14 years, 7 months ago) by jacekm
Branch: MAIN
Changes since 1.69: +1 -3 lines
Diff to previous 1.69 (colored)
Improve error logging.
Revision 1.69 / (download) - annotate - [select for diffs], Sun Nov 1 22:15:27 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.68: +2 -2 lines
Diff to previous 1.68 (colored)
fix a couple log_info()s, from Tim van der Molen <tbvdm@xs4all.nl>
Revision 1.68 / (download) - annotate - [select for diffs], Mon Oct 12 22:34:37 2009 UTC (14 years, 7 months ago) by gilles
Branch: MAIN
Changes since 1.67: +2 -3 lines
Diff to previous 1.67 (colored)
- fix a null deref which could happen after a couple iterations of the aliases/virtual domains resolution code. - fix a logic bug which caused virtual domains not to be correctly handled after one iteration of the aliases resolution code. - introduce a few helper functions to help clean up and simplify the lka code. - simplify the IS_EXT/IS_MAILBOX/IS_RELAY macros so they manipulate a struct path * instead of the mess of dereferences we were passing them.
Revision 1.67 / (download) - annotate - [select for diffs], Fri Sep 4 19:11:32 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.66: +12 -5 lines
Diff to previous 1.66 (colored)
Fix scheduling of bounces that could not be delivered. ok gilles@
Revision 1.66 / (download) - annotate - [select for diffs], Fri Sep 4 13:33:00 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.65: +4 -6 lines
Diff to previous 1.65 (colored)
Fix scheduling of bounce messages. From gilles@
Revision 1.65 / (download) - annotate - [select for diffs], Fri Sep 4 11:44:23 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.64: +3 -2 lines
Diff to previous 1.64 (colored)
Fix a crash when messagep->batch_id is read from disk and used long after the batch has ceased to exist. From gilles@
Revision 1.64 / (download) - annotate - [select for diffs], Thu Sep 3 08:19:13 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.63: +8 -8 lines
Diff to previous 1.63 (colored)
imsg_get sets errno so use fatal instead of fatalx.
Revision 1.63 / (download) - annotate - [select for diffs], Thu Aug 27 11:37:30 2009 UTC (14 years, 9 months ago) by jacekm
Branch: MAIN
Changes since 1.62: +7 -18 lines
Diff to previous 1.62 (colored)
Implement client side of the SMTP protocol in a library-like module. Make bounce code and /usr/sbin/sendmail interface use this new API. The mta process continues to use its own implementation, but eventually will be switched to use this shared module. Buffer routines are taken from buffer.c rather than from evbuffer. This is one step forward to using a single buffer API across the program. "it looks sexy" gilles@
Revision 1.62 / (download) - annotate - [select for diffs], Tue Aug 11 14:46:58 2009 UTC (14 years, 10 months ago) by jacekm
Branch: MAIN
Changes since 1.61: +3 -1 lines
Diff to previous 1.61 (colored)
call reset_flags in bounce_session failure code path.
Revision 1.61 / (download) - annotate - [select for diffs], Tue Aug 11 14:45:19 2009 UTC (14 years, 10 months ago) by jacekm
Branch: MAIN
Changes since 1.60: +15 -12 lines
Diff to previous 1.60 (colored)
do not repeat flag resetting code, introduce reset_flags(). discussed with gilles
Revision 1.60 / (download) - annotate - [select for diffs], Sat Aug 8 00:02:22 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.59: +2 -2 lines
Diff to previous 1.59 (colored)
import some changes from portable smtpd to reduce the delta between both. this commit contains mostly missing casts and cosmethic changes, do not expect to build this anywhere but on OpenBSD, it does not contain any of the portable glue.
Revision 1.59 / (download) - annotate - [select for diffs], Thu Aug 6 16:46:57 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.58: +9 -12 lines
Diff to previous 1.58 (colored)
- change all occurences of T_DAEMON_BATCH,MESSAGE to T_BOUNCE_BATCH/MESSAGE - make sure T_BOUNCE_MESSAGE is no longer OR-ed to T_MDA/MTA_MESSAGE - define F_MESSAGE_BOUNCE flag and make sure bounce sessions set it - teach smtpctl show queue how to recognize a bounce message
Revision 1.58 / (download) - annotate - [select for diffs], Thu Aug 6 16:29:35 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.57: +3 -3 lines
Diff to previous 1.57 (colored)
replace X-OpenSMPTD-Loop with Delivered-To idea by jacekm@ a few weeks ago, discussed with pyr
Revision 1.57 / (download) - annotate - [select for diffs], Thu Aug 6 14:32:34 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.56: +4 -1 lines
Diff to previous 1.56 (colored)
when a message in queue expires, generate a bounce before we remove it
Revision 1.56 / (download) - annotate - [select for diffs], Thu Aug 6 14:12:48 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.55: +2 -1 lines
Diff to previous 1.55 (colored)
- introduce message_set_errormsg() to set the error description that will appear in a bounce message, and message_get_errormsg() to retrieve that message. - when loop is detected, call message_set_errormsg() - in mta, call message_set_errormsg() for each recipient failure - in mta, call message_set_errormsg() to copy batch errors to recipients if we failed to deliver for a session related error - when bouncing, add the recipient and error reason to the bounce message
Revision 1.55 / (download) - annotate - [select for diffs], Thu Aug 6 13:40:45 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.54: +102 -13 lines
Diff to previous 1.54 (colored)
This commit reworks the entire mailer daemon support to actually make it work for real. As an added bonus, it simplifies it, makes it follow the same code path as regular messages and kills quite some code from mta, mda and store. There's still some work needed but the most painful part is behind us now ;) ok jacekm@
Revision 1.54 / (download) - annotate - [select for diffs], Tue Jul 28 22:03:55 2009 UTC (14 years, 10 months ago) by gilles
Branch: MAIN
Changes since 1.53: +2 -2 lines
Diff to previous 1.53 (colored)
introduce daemon_record_message() and needed glue in queue_shared.c to atomically create a mailer daemon out of a message. it is unreferenced for now, this commit makes no functional change.
Revision 1.53 / (download) - annotate - [select for diffs], Mon Jun 29 10:11:07 2009 UTC (14 years, 11 months ago) by martynas
Branch: MAIN
CVS Tags: OPENBSD_4_6_BASE,
OPENBSD_4_6
Changes since 1.52: +3 -3 lines
Diff to previous 1.52 (colored)
since upper-bound is not a power of two here, use arc4random_uniform instead, to avoid modulo bias. ok jacekm@, gilles@
Revision 1.52 / (download) - annotate - [select for diffs], Sat Jun 6 04:14:21 2009 UTC (15 years ago) by pyr
Branch: MAIN
Changes since 1.51: +36 -24 lines
Diff to previous 1.51 (colored)
Sync with relayd: Stop pushing event handling in the imsg framework. Instead, provide a small glue layer on top of both imsg and libevent. This finally clearly separates event handling and imsg construction. Sidetrack bonus: remove the mega-ugly hack of having a dummy imsg_event_add stub in smtpctl. ok jaceckm@
Revision 1.51 / (download) - annotate - [select for diffs], Fri Jun 5 20:43:57 2009 UTC (15 years ago) by pyr
Branch: MAIN
Changes since 1.50: +6 -6 lines
Diff to previous 1.50 (colored)
make smtpd's imsg lib ready, just like relayd and ospfd. ok gilles@, jacekm@
Revision 1.50 / (download) - annotate - [select for diffs], Wed Jun 3 22:04:15 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.49: +1 -2 lines
Diff to previous 1.49 (colored)
simplify struct batch; ok gilles@
Revision 1.49 / (download) - annotate - [select for diffs], Mon Jun 1 13:20:56 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.48: +31 -49 lines
Diff to previous 1.48 (colored)
Fix EV_READ/EV_WRITE testing inside IMSG handlers. Based on similar change to the routing daemons by claudio@; ok gilles@
Revision 1.48 / (download) - annotate - [select for diffs], Sun May 24 14:38:56 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.47: +2 -2 lines
Diff to previous 1.47 (colored)
Parent process forks 3 types of processes, track them all in a single tree using newly introduced child struct. Manage process titles centrally in struct smtpd. ok gilles@
Revision 1.47 / (download) - annotate - [select for diffs], Sun May 24 14:22:23 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.46: +3 -14 lines
Diff to previous 1.46 (colored)
Instead of keeping stats private to each process, and querying every process individually whenever stats need to be fetched, keep stats in MAP_ANON shared memory allocated by parent. This means control has direct access to stats and can respond very quickly without troubling any other daemon processes. ok gilles@
Revision 1.46 / (download) - annotate - [select for diffs], Tue May 19 11:24:24 2009 UTC (15 years ago) by jacekm
Branch: MAIN
Changes since 1.45: +11 -6 lines
Diff to previous 1.45 (colored)
Verify the amount of IMSG payload is exactly as expected; ok gilles@
Revision 1.45 / (download) - annotate - [select for diffs], Thu May 14 15:05:12 2009 UTC (15 years ago) by eric
Branch: MAIN
Changes since 1.44: +3 -3 lines
Diff to previous 1.44 (colored)
use the nitems() macro where appropriate ok gilles@, jacekm@
Revision 1.44 / (download) - annotate - [select for diffs], Sun May 10 11:29:40 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.43: +2 -1 lines
Diff to previous 1.43 (colored)
assert copyright; ok gilles@
Revision 1.43 / (download) - annotate - [select for diffs], Tue Apr 21 18:39:00 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.42: +2 -2 lines
Diff to previous 1.42 (colored)
small corrections to log_warns and fatals
Revision 1.42 / (download) - annotate - [select for diffs], Tue Apr 21 18:12:05 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.41: +72 -3 lines
Diff to previous 1.41 (colored)
Make /usr/sbin/sendmail not fail due to smtpd being down. The approach is to save cmdline + stdin in a file under a newly added directory /var/spool/smtpd/offline (uid 0 gid 0 mode 1777). Next time daemon starts, it uses information in that directory to replay sendmail on user's behalf. ok gilles@
Revision 1.41 / (download) - annotate - [select for diffs], Tue Apr 21 14:37:32 2009 UTC (15 years, 1 month ago) by eric
Branch: MAIN
Changes since 1.40: +6 -6 lines
Diff to previous 1.40 (colored)
correct some fatal(x) calls and error strings ok gilles@ jacekm@
Revision 1.40 / (download) - annotate - [select for diffs], Sat Apr 18 21:13:56 2009 UTC (15 years, 1 month ago) by jacekm
Branch: MAIN
Changes since 1.39: +10 -10 lines
Diff to previous 1.39 (colored)
correct function name in fatal
Revision 1.39 / (download) - annotate - [select for diffs], Sun Apr 12 15:42:13 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.38: +6 -2 lines
Diff to previous 1.38 (colored)
When runner_check_loop() detects a mail loop, remove the envelope. This is only half of the problem solved, we need to generate a mailer daemon too. This fixes a bug in my smtpd where it would try to schedule a message, detect that it is a loop and not schedule it, then trying again at next queue walk ...
Revision 1.38 / (download) - annotate - [select for diffs], Sun Mar 29 14:18:20 2009 UTC (15 years, 2 months ago) by jacekm
Branch: MAIN
Changes since 1.37: +11 -11 lines
Diff to previous 1.37 (colored)
turn some log_debugs into log_warns or even fatals; "looks ok" gilles@
Revision 1.37 / (download) - annotate - [select for diffs], Sun Mar 22 22:53:47 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.36: +2 -2 lines
Diff to previous 1.36 (colored)
fix a bug in the resolution of forward files which would cause usernames not to be taken into account if they had no ~/.forward file AND were the result of an alias expansion that expanded to more than one username. while at it, I spotted another bug where I would check T_MDA_MESSAGE on the flags field instead of the type field. the bug could cause two MDA message to end up in the same batch which is no longer valid.
Revision 1.36 / (download) - annotate - [select for diffs], Wed Mar 18 10:29:27 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.35: +5 -1 lines
Diff to previous 1.35 (colored)
we can no longer create batches of MDA deliveries, so make sure runner does not do a batch lookup when dealing with a MDA message
Revision 1.35 / (download) - annotate - [select for diffs], Sun Mar 15 19:15:25 2009 UTC (15 years, 2 months ago) by gilles
Branch: MAIN
Changes since 1.34: +1 -7 lines
Diff to previous 1.34 (colored)
the mda process no longer uses struct batch as its central structure to deal with deliveries, it now uses struct session just like mta and smtp processes. we now keep the mbox and message descriptors in the session, saving space in struct message which is now as small as we can make it. While at it, plugged a memory leak and did some cosmethic changes This was the last planned change to our struct message which means that later changes will no longer require a queue flush before rebuild.
Revision 1.34 / (download) - annotate - [select for diffs], Mon Mar 9 01:43:19 2009 UTC (15 years, 3 months ago) by gilles
Branch: MAIN
Changes since 1.33: +7 -77 lines
Diff to previous 1.33 (colored)
add basic support for outgoing authentication (AUTH PLAIN over ssl) which can be turned on by adding "enable auth" to a "relay via" rule. this made me rework the mx resolution so that it is done by the mta process and not the runner process anymore.
Revision 1.33 / (download) - annotate - [select for diffs], Tue Feb 24 12:07:47 2009 UTC (15 years, 3 months ago) by gilles
Branch: MAIN
CVS Tags: OPENBSD_4_5_BASE,
OPENBSD_4_5
Changes since 1.32: +65 -1 lines
Diff to previous 1.32 (colored)
teach smtpctl's parser how to deal with parameters that are not necessarily a token so that it is possible to do: smtpctl schedule <message id/uid> introduce F_MESSAGE_FORCESCHEDULE which lets the runner schedule a message even if the retry delay has not been expired. F_MESSAGE_ENQUEUED is a valid flag for a message and should not cause an\ errx() in smtpctl show queue
Revision 1.32 / (download) - annotate - [select for diffs], Sun Feb 22 23:29:54 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.31: +3 -8 lines
Diff to previous 1.31 (colored)
Fix race possible between runner and queue when starting up. runner_reset_flags would attempt to update envelope, but its envelope.tmp could be renamed by queue_purge. The fix is to store envelope.tmp in a place no other process writes to, and /queue/envelope.tmp is perfect for this. ok gilles@
Revision 1.31 / (download) - annotate - [select for diffs], Sun Feb 22 11:44:29 2009 UTC (15 years, 3 months ago) by form
Branch: MAIN
Changes since 1.30: +6 -5 lines
Diff to previous 1.30 (colored)
replace MAX* constants by sizeof where possible ok jacekm@
Revision 1.30 / (download) - annotate - [select for diffs], Sun Feb 15 10:32:23 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.29: +2 -1 lines
Diff to previous 1.29 (colored)
New config.c that allows for process cloning. Done by pyr@ for relayd at n2k9, and adapted to smtpd; ok gilles@
Revision 1.29 / (download) - annotate - [select for diffs], Sat Feb 14 17:49:09 2009 UTC (15 years, 3 months ago) by jacekm
Branch: MAIN
Changes since 1.28: +2 -1 lines
Diff to previous 1.28 (colored)
Initialize time_t now as necessary, fixes delivery rescheduling; ok gilles@
Revision 1.28 / (download) - annotate - [select for diffs], Thu Jan 29 21:59:15 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.27: +13 -1 lines
Diff to previous 1.27 (colored)
Implement "smtpctl show stats"; ok gilles@
Revision 1.27 / (download) - annotate - [select for diffs], Thu Jan 29 12:43:25 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.26: +75 -170 lines
Diff to previous 1.26 (colored)
Common queue walking code for smtpd and smtpctl. Kills majority of showqueue.c, the remaining code was moved to queue_shared.c; ok gilles@
Revision 1.26 / (download) - annotate - [select for diffs], Wed Jan 28 17:43:45 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.25: +5 -2 lines
Diff to previous 1.25 (colored)
relayhost flags were not properly copied to the relayhost array of the batch we're handing over to mta. this prevented mta from knowing if a session has to be established over ssl or not.
Revision 1.25 / (download) - annotate - [select for diffs], Wed Jan 28 17:29:11 2009 UTC (15 years, 4 months ago) by jacekm
Branch: MAIN
Changes since 1.24: +10 -7 lines
Diff to previous 1.24 (colored)
Make races between queue and runner impossible by implementing the policy: 1) queue never reads /queue. 2) queue writes to /queue only at message injection time. 3) runner does all reading, and all writing apart from 2). ok gilles@
Revision 1.24 / (download) - annotate - [select for diffs], Wed Jan 28 13:29:40 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.23: +67 -1 lines
Diff to previous 1.23 (colored)
bring loop detection support. we handle this with a qmail-like approach which consists of checking headers for a custom header, but we also count how many hops the mail went through and use a hard limit (currently set to 100 as was recommanded by RFC) as a safe-guard. idea discussed with jacekm@, qmail approach suggested by claudio@ a long time ago
Revision 1.23 / (download) - annotate - [select for diffs], Tue Jan 27 22:48:29 2009 UTC (15 years, 4 months ago) by gilles
Branch: MAIN
Changes since 1.22: +1 -2 lines
Diff to previous 1.22 (colored)
first bricks of enqueue code which allows smtpctl to submit mail to queue without "talking" smtp to listeners. currently, a big part of the server side code is done (and requires a cleanup), next step is to get it usable properly from a mail user agent.
Revision 1.22 / (download) - annotate - [select for diffs], Thu Jan 8 19:15:23 2009 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.21: +25 -9 lines
Diff to previous 1.21 (colored)
clear processing flags early so that there's never doubt whether they belong to previous or current smtpd instance; ok gilles@
Revision 1.21 / (download) - annotate - [select for diffs], Tue Jan 6 23:02:07 2009 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.20: +110 -101 lines
Diff to previous 1.20 (colored)
rework /queue traversal; ok gilles@
Revision 1.20 / (download) - annotate - [select for diffs], Tue Jan 6 20:17:23 2009 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.19: +11 -2 lines
Diff to previous 1.19 (colored)
make file update in queue_update_envelope atomic; ok gilles@
Revision 1.19 / (download) - annotate - [select for diffs], Sun Jan 4 22:35:09 2009 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.18: +5 -5 lines
Diff to previous 1.18 (colored)
- smtp can now pause/resume the accepting of incoming messages - smtpctl recognizes "pause incoming" and "resume incoming" - setup imsg communication between control process and smtp process
Revision 1.18 / (download) - annotate - [select for diffs], Sun Jan 4 22:18:23 2009 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.17: +11 -12 lines
Diff to previous 1.17 (colored)
- remove runstates global, we don't need it actually. - while at it, move the runner states check a bit earlier.
Revision 1.17 / (download) - annotate - [select for diffs], Sun Jan 4 20:52:06 2009 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.16: +9 -2 lines
Diff to previous 1.16 (colored)
make sure runner resets scheduling related flags on envelopes when the queue is processed for the first time since a (re)start. this ensures that deliveries scheduling can recover properly in case of a badly timed shutdown. ok jacekm@
Revision 1.16 / (download) - annotate - [select for diffs], Sun Jan 4 19:37:41 2009 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.15: +21 -1 lines
Diff to previous 1.15 (colored)
- runner is now capable of pausing/resuming the scheduling of deliveries for both mda and mta batches. - smtpctl can be used to disable/enable deliveries at runtime using the pause/resume commands. ok jacekm@
Revision 1.15 / (download) - annotate - [select for diffs], Sun Jan 4 19:26:30 2009 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.14: +1 -5 lines
Diff to previous 1.14 (colored)
remove unnecessary "messagep->retry == 255" expiry condition; ok gilles@
Revision 1.14 / (download) - annotate - [select for diffs], Sun Jan 4 19:25:19 2009 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.13: +3 -9 lines
Diff to previous 1.13 (colored)
kill F_MESSAGE_EXPIRED; ok gilles@
Revision 1.13 / (download) - annotate - [select for diffs], Sun Jan 4 19:23:06 2009 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.12: +4 -7 lines
Diff to previous 1.12 (colored)
fix bug where runner would expire message that is being delivered, leading to double unlink on envelope file; ok gilles@
Revision 1.12 / (download) - annotate - [select for diffs], Sat Dec 27 17:45:54 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.11: +2 -2 lines
Diff to previous 1.11 (colored)
log_warn -> log_warnx
Revision 1.11 / (download) - annotate - [select for diffs], Sat Dec 27 17:36:37 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.10: +13 -32 lines
Diff to previous 1.10 (colored)
cleanup; ok gilles@
Revision 1.10 / (download) - annotate - [select for diffs], Sat Dec 27 17:03:29 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.9: +3 -5 lines
Diff to previous 1.9 (colored)
Merge hash() and queue_message_hash() into one func, queue_hash(). Fix callers to use this interface consistently; ok chl@ gilles@
Revision 1.9 / (download) - annotate - [select for diffs], Fri Dec 19 00:39:05 2008 UTC (15 years, 5 months ago) by gilles
Branch: MAIN
Changes since 1.8: +12 -5 lines
Diff to previous 1.8 (colored)
- smtpd handled mbox locking failures as "regular" temporary failures which is not good at all. As a result, under heavy load messages would be kept in queue, and delayed for hours just because we failed locking a few times. This commit makes smtpd distinguish between lock fails and "regular" temporary fails. - delivery scheduler will reschedule immediately a message that couldn't be delivered because of a lock fail. If we fail to lock too many times we fallback to previous "delay increase" logic. "looks sane" jacekm@
Revision 1.8 / (download) - annotate - [select for diffs], Wed Dec 17 18:47:37 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.7: +79 -1 lines
Diff to previous 1.7 (colored)
Introduce /purge, where all msgs scheduled for deletion are put by queue, and removed from disk by runner. On startup, clean /incoming by moving msgs within it to /purge. ok gilles@
Revision 1.7 / (download) - annotate - [select for diffs], Sat Dec 13 23:19:34 2008 UTC (15 years, 5 months ago) by jacekm
Branch: MAIN
Changes since 1.6: +5 -5 lines
Diff to previous 1.6 (colored)
IMSG_* namespace cleanup. ok gilles@
Revision 1.6 / (download) - annotate - [select for diffs], Thu Dec 11 22:59:14 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.5: +7 -14 lines
Diff to previous 1.5 (colored)
- snprintf -> bsnprintf
Revision 1.5 / (download) - annotate - [select for diffs], Thu Dec 11 22:32:27 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.4: +4 -7 lines
Diff to previous 1.4 (colored)
- fix a bug that would cause the runner to hit a fatal() when running out of luck under load. Long story made short: the runner process opens the queue and sequentially opens each bucket to process messages. If a message is delivered by MDA/MTA after the opendir(), then the queue process will garbage collect the message from the queue and the runner will attempt to opendir() a path that no longer exists. Reported by Daniel Lidberg <daniel.lidberg@gmail.com>, observed by jacekm@ and debugged by me, that's collaborative work ;-)
Revision 1.4 / (download) - annotate - [select for diffs], Sun Dec 7 03:14:24 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.3: +7 -4 lines
Diff to previous 1.3 (colored)
- getaddrinfo() uses negative values for its error defines, our use of an u_int8_t to hold the value leads to invalid checking in runner_batch_resolved(), this lead to a crash in MTA because we assumed a batch had its mx resolved when it had not. while at it, be more strict about errors we don't know and fatal(), it should not happen. ok jacekm@, ok chl@
Revision 1.3 / (download) - annotate - [select for diffs], Sun Dec 7 01:55:41 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.2: +2 -2 lines
Diff to previous 1.2 (colored)
- fix function name in fatal()
Revision 1.2 / (download) - annotate - [select for diffs], Fri Dec 5 19:09:59 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
Changes since 1.1: +4 -5 lines
Diff to previous 1.1 (colored)
- more err/errx -> fatal/fatalx, warn/warnx -> log_warn/log_warnx contains bits based on an old diff from Jacek Masiulaniec and other bits from me.
Revision 1.1 / (download) - annotate - [select for diffs], Fri Dec 5 02:51:32 2008 UTC (15 years, 6 months ago) by gilles
Branch: MAIN
- last part of the new queue code: the runner process (unprivileged and chrooted) is now in charge of doing the scheduling of deliveries, and the dispatching of messages to MDA and MTA. queue process only does inserts/updates/removals from the queue and can no longer be so busy that it delays answers to imsg from smtp server.