OpenBSD CVS

CVS log for src/usr.bin/mandoc/mandocdb.c


[BACK] Up to [local] / src / usr.bin / mandoc

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.221 / (download) - annotate - [select for diffs], Tue May 14 21:12:44 2024 UTC (4 days, 18 hours ago) by schwarze
Branch: MAIN
CVS Tags: HEAD
Changes since 1.220: +12 -26 lines
Diff to previous 1.220 (colored)

Garbage collect dead code intended to write five- and six-byte UTF-8
sequences since the Unicode standard has been explicitly prohibiting
the use of such sequences when encoding Unicode characters for more
than 20 years now.

While here, also weed out UTF-16 surrogates and codepoints in the
invalid range 110000 to 1FFFFF if any are encountered.  I hoped to
write "no functional change", but to my shame it turns out there
are unrelated bugs with \[uXXXX] parsing in roff_escape.c, so this
new anti-surrogate check is actually reachable until those other
bugs get fixed, and even after fixing those other bugs, it will
remain useful as a defense in depth.

Revision 1.220 / (download) - annotate - [select for diffs], Tue May 14 18:38:13 2024 UTC (4 days, 20 hours ago) by schwarze
Branch: MAIN
Changes since 1.219: +20 -10 lines
Diff to previous 1.219 (colored)

The makewhatis(8) program already provided a "-T utf8" option
to put UTF-8 strings into the database, but that only worked
for input files containing the manually written, mnemonic roff(7)
character escape sequences documented in mandoc_char(7).
Even though mandoc(1), man(1), and man.cgi(8) have been able to
properly handle UTF-8 and ISO-Latin-1 encoded input files for many
years, makewhatis(8) unconditionally replaced all non-ASCII bytes
in all input files with ASCII question marks ("?").

Improve this by changing two aspects of non-ASCII character handling
in makewhatis(8) at the same time.

1. In the makewhatis(8) main program, when configuring the roff(7) parser,
enable UTF-8 and ISO-Latin-1 autorecognition and translation
to \[uXXXX] roff(7) Unicode character escape sequences.
The man(1) and man.cgi(8) programs prove that this option has
been working very reliably for many years, so there is no risk.

2. In the makewhatis(8) string rendering code, if "-T utf8" was
requested, translate these escape sequences to UTF-8 strings,
just like makewhatis(8) already did it for ESCAPE_SPECIAL sequences.
Otherwise, i.e. if an ASCII-only database is desired, replace
all character escape sequences by ASCII transliterations, again
like it was already done for ESCAPE_SPECIAL sequences.

With this change, giving UTF-8 command line arguments to apropos(1)
allows searching in UTF-8 and ISO-Latin-1 encoded manual pages if the
respective mandoc.db(5) has been built with makewhatis(8) -T utf8.

Issue found while investigating a question from
Valid-Amirali-Averiva at rambler dot ru, who is using mandoc
on FreeBSD to process documents containing cyrillic letters.

Revision 1.219 / (download) - annotate - [select for diffs], Mon Dec 26 19:16:02 2022 UTC (16 months, 3 weeks ago) by jmc
Branch: MAIN
CVS Tags: OPENBSD_7_5_BASE, OPENBSD_7_5, OPENBSD_7_4_BASE, OPENBSD_7_4, OPENBSD_7_3_BASE, OPENBSD_7_3
Changes since 1.218: +2 -2 lines
Diff to previous 1.218 (colored)

spelling fixes; from paul tagliamonte
amendments to his diff are noted on tech

Revision 1.218 / (download) - annotate - [select for diffs], Sun Oct 24 21:24:16 2021 UTC (2 years, 6 months ago) by deraadt
Branch: MAIN
CVS Tags: OPENBSD_7_2_BASE, OPENBSD_7_2, OPENBSD_7_1_BASE, OPENBSD_7_1
Changes since 1.217: +3 -3 lines
Diff to previous 1.217 (colored)

For open/openat, if the flags parameter does not contain O_CREAT, the
3rd (variadic) mode_t parameter is irrelevant.  Many developers in the past
have passed mode_t (0, 044, 0644, or such), which might lead future people
to copy this broken idiom, and perhaps even believe this parameter has some
meaning or implication or application. Delete them all.
This comes out of a conversation where tb@ noticed that a strange (but
intentional) pledge behaviour is to always knock-out high-bits from
mode_t on a number of system calls as a safety factor, and his bewilderment
that this appeared to be happening against valid modes (at least visually),
but no sorry, they are all irrelevant junk.  They could all be 0xdeafbeef.
ok millert

Revision 1.217 / (download) - annotate - [select for diffs], Thu Aug 19 16:55:24 2021 UTC (2 years, 9 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_7_0_BASE, OPENBSD_7_0
Changes since 1.216: +3 -1 lines
Diff to previous 1.216 (colored)

do not crash when a manpath directory contains a symbolic link
that points to a directory rather than to a regular file;
bug reported by Lukas Epple <sternenseemann at systemli dot org>,
and my patch also tested by him on NixOS

Revision 1.216 / (download) - annotate - [select for diffs], Fri Apr 3 11:34:19 2020 UTC (4 years, 1 month ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_9_BASE, OPENBSD_6_9, OPENBSD_6_8_BASE, OPENBSD_6_8, OPENBSD_6_7_BASE, OPENBSD_6_7
Changes since 1.215: +5 -3 lines
Diff to previous 1.215 (colored)

Remove some stray argument names from function prototypes,
for consistency with the dominant style used in mandoc.
No functional change.
Patch from Martin Vahlensieck <academicsolutions dot ch>.

Revision 1.215 / (download) - annotate - [select for diffs], Sun Jan 26 21:24:58 2020 UTC (4 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.214: +84 -34 lines
Diff to previous 1.214 (colored)

Repair more of the issues that i found in filescan() while investigating
the report from <Andreas dot Kahari at abc dot se> on ports@:

For a symlink, use the first of the following names that is available:
1. In -t mode, the symlink itself (unchanged).
2. When the (unresolved) symlink already resides inside the manpath,
just strip the manpath and use the rest (unchanged).
3. When prefix(es) of the unresolved symlink point to the manpath,
strip the longest such prefix and use the rest (new); this fixes
situations where the manpath or one of its parent directories is a
symlink and at the same time contains symlinks to manual pages.
4. Fall back to the fully resolved symlink, with the manpath stripped
(new); this may for example happen when the command line passes
symlinks from outside the manpath that point to manual pages inside
the manpath, or if manual page trees contain symlinks to symlinks and
not all of them are given on the command line.

The fallback (4) isn't perfect.  You can construct symlink spaghetti
in such a way that this algorithm will not enter all manual page
names into the database that a human would be able to deduce.  But
i do not expect such spaghetti to actually occur in practice (not
even in ports), and a full fix would require re-implementing
realpath(3) in terms of step-by-step readlink(2) calls, repeating
the complicated algorithm (3) after each step.

While here, also stop using PATH_MAX as the size of a static buffer
in filescan(); on some systems, it can be unreasonably large.
Instead, allocate path strings dynamically.

Revision 1.214 / (download) - annotate - [select for diffs], Sun Jan 26 11:15:49 2020 UTC (4 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.213: +3 -3 lines
Diff to previous 1.213 (colored)

Fix incorrect file type tests.
This bug caused sockets and character special devices to be accepted
as manual pages if they appeared inside manpaths, and
it caused incorrect file names to be entered into the database when
the manpath or one of its parent directories was a symbolic link.
This fixes the issues reported by <Andreas dot Kahari at abc dot se>
on ports@, but additional issues remain when symbolic links are
contained in a manpath that involves another symbolic link.

Revision 1.213 / (download) - annotate - [select for diffs], Sat Jan 25 22:59:14 2020 UTC (4 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.212: +58 -53 lines
Diff to previous 1.212 (colored)

Minor cleanup, no functional change:
Do not abuse strstr(3) to check whether one long string starts with
another long string.  Instead, use strncmp(3) with the proper length.
In set_basedir(), also reset *basedir in the error brances for extra safety.
While here, invert some more Yoda conditions in the neighbourhood.

Revision 1.212 / (download) - annotate - [select for diffs], Fri May 3 18:16:57 2019 UTC (5 years ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_6_BASE, OPENBSD_6_6
Changes since 1.211: +12 -7 lines
Diff to previous 1.211 (colored)

Enter dangling .so links into the database, to avoid harassing
users of man(1) about running makewhatis(8), which won't help.
Seeing the content of the broken .so request might even help
users to figure out how to access the manual page they want.

Fixing the last issue reported by Lorenzo Beretta <loreb at github>
as part of https://github.com/void-linux/void-packages/issues/9868 .

Revision 1.211 / (download) - annotate - [select for diffs], Sun Dec 30 00:48:47 2018 UTC (5 years, 4 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_5_BASE, OPENBSD_6_5
Changes since 1.210: +21 -25 lines
Diff to previous 1.210 (colored)

Cleanup, no functional change:

The struct roff_man used to be a bad mixture of internal parser
state and public parsing results.  Move the public results to the
parsing result struct roff_meta, which is already public.  Move the
rest of struct roff_man to the parser-internal header roff_int.h.

Since the validators need access to the parser state, call them
from the top level parser during mparse_result() rather than from
the main programs, also reducing code duplication.

This keeps parser internal state out of thee main programs (five
in mandoc portable) and out of eight formatters.

Revision 1.210 / (download) - annotate - [select for diffs], Fri Dec 14 01:17:46 2018 UTC (5 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.209: +2 -3 lines
Diff to previous 1.209 (colored)

Major cleanup; may imply minor changes in edge cases of error reporting.

Finally, drop support for the run-time configurable mandocmsg()
callback.  It was over-engineered from the start, never used for
anything in a decade, and repeatedly caused maintenance headaches.

Consolidate reporting infrastructure into two files, mandoc.h and
mandoc_msg.c, mopping up the bits and pieces that were scattered
around main.c, read.c, mandoc_parse.h, libmandoc.h, the prototypes
of four parsing-related functions, and both parser structs.

Revision 1.209 / (download) - annotate - [select for diffs], Thu Dec 13 11:55:14 2018 UTC (5 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.208: +2 -1 lines
Diff to previous 1.208 (colored)

Cleanup, no functional change:
Split the top level parser interface out of the utility header
mandoc.h, into a new header mandoc_parse.h, for use in the main
program and in the main parser only.
Move enum mandoc_os into roff.h because struct roff_man is the
place where it is stored.
This allows removal of mandoc.h from seven files in low-level
parsers and in formatters.

Revision 1.208 / (download) - annotate - [select for diffs], Fri Aug 17 20:31:52 2018 UTC (5 years, 9 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_4_BASE, OPENBSD_6_4
Changes since 1.207: +13 -11 lines
Diff to previous 1.207 (colored)

Remove more pointer arithmetic passing via regions outside the array
that is undefined according to the C standard.  Robert Elz <kre at
munnari dot oz dot au> pointed out i wasn't quite done yet.

Revision 1.207 / (download) - annotate - [select for diffs], Fri Feb 23 18:24:41 2018 UTC (6 years, 2 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_3_BASE, OPENBSD_6_3
Changes since 1.206: +27 -6 lines
Diff to previous 1.206 (colored)

Logically, the following are are type names - just like .Vt,
some of them with an optional variable name following:
- .Ft
- .Fa in the SYNOPSIS
- .Fn second and later arguments in the SYNOPSIS
So add these to the .Vt macro table in the mandoc.db(5) database.
During my LibreSSL work, i'm getting really tired of typing
$ man -k Vt,Ft,Fa=some_type_name
over and over again; now, this becomes just:
$ man -k Vt=some_type_name

Revision 1.206 / (download) - annotate - [select for diffs], Wed Feb 7 20:31:32 2018 UTC (6 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.205: +13 -3 lines
Diff to previous 1.205 (colored)

In man(7) and cat pages, cut off excessive one line descriptions.
An extreme example of how rogue files could mess up apropos(1) output
was reported by bentley@: qwtlicense(3) in the x11/qwt port.

Revision 1.205 / (download) - annotate - [select for diffs], Sat Aug 26 20:38:09 2017 UTC (6 years, 8 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_2_BASE, OPENBSD_6_2
Changes since 1.204: +6 -6 lines
Diff to previous 1.204 (colored)

Fix previous: mmap(2) returns MAP_FAILED on failure, not NULL.
Bug pointed out by tedu@.

Revision 1.204 / (download) - annotate - [select for diffs], Sat Aug 26 15:55:41 2017 UTC (6 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.203: +54 -40 lines
Diff to previous 1.203 (colored)

Do not fork and exec cmp(1); instead, simply fstat(2), mmap(2), and
compare the files directly, allowing a much stricter pledge(2), at
very little cost: merely 15 additional lines of very simple code.
Suggested by George Brown <321 dot george at gmail dot com> on misc@.

Revision 1.203 / (download) - annotate - [select for diffs], Sat Aug 26 12:59:13 2017 UTC (6 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.202: +4 -21 lines
Diff to previous 1.202 (colored)

No need to fork and exec rm(1) -rf, we know that we have exactly
one file and exactly one directory to remove.  While here, increase
the size of the buffer such that the file name actually fits.
Minus 17 lines of code, no functional change.

Opportunity for simplification reported by George Brown <321 dot
george at gmail dot com> on misc@.

Revision 1.202 / (download) - annotate - [select for diffs], Fri Jul 28 14:46:46 2017 UTC (6 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.201: +2 -2 lines
Diff to previous 1.201 (colored)

When the mparse_alloc() prototype changed in read.c 1.151, updating
the enum constant was forgotten in this call.  No functional change
since the mmsg argument is NULL anyway.
Found by florian@ with clang.

Revision 1.201 / (download) - annotate - [select for diffs], Sat Jul 15 13:17:07 2017 UTC (6 years, 10 months ago) by schwarze
Branch: MAIN
Changes since 1.200: +2 -2 lines
Diff to previous 1.200 (colored)

Improve rev. 1.199 further:
If a database is empty, we already don't write a database file and
delete an existing one.  If none exists, that only means the database
was empty and remains so, which is not an error.  So don't nag about
it, in particular because that would be a weekly(8) annoyance.
Issue reported by jmc@.

Revision 1.200 / (download) - annotate - [select for diffs], Sat Jun 24 14:38:27 2017 UTC (6 years, 10 months ago) by schwarze
Branch: MAIN
Changes since 1.199: +3 -2 lines
Diff to previous 1.199 (colored)

Split -Wstyle into -Wstyle and the even lower -Wbase, and add
-Wopenbsd and -Wnetbsd to check conventions for the base system of
a specific operating system.  Mark operating system specific messages
with "(OpenBSD)" at the end.

Please use just "-Tlint" to check base system manuals (defaulting
to -Wall, which is now -Wbase), but prefer "-Tlint -Wstyle" for the
manuals of portable software projects you maintain that are not
part of OpenBSD base, to avoid bogus recommendations about base
system conventions that do not apply.

Issue originally reported by semarie@, solution using
an idea from tedu@, discussed with jmc@ and jca@.

Revision 1.199 / (download) - annotate - [select for diffs], Wed May 17 22:26:52 2017 UTC (7 years ago) by schwarze
Branch: MAIN
Changes since 1.198: +23 -1 lines
Diff to previous 1.198 (colored)

Never create empty databases.

When pkg_add(1)ing packages installing manual pages into some directory,
the database in that directory automatically gets created or updated,
no change so far.  This patch causes the database file to be
automatically unlinked when pkg_delete(1)ing the last package having
manual pages in that directory, to leave less cruft behind.

Suggested by ajacoutot@.

Revision 1.198 / (download) - annotate - [select for diffs], Fri May 5 15:16:25 2017 UTC (7 years ago) by schwarze
Branch: MAIN
Changes since 1.197: +1 -2 lines
Diff to previous 1.197 (colored)

Move .sp to the roff modules.  Enough infrastructure is in place
now that this actually saves code: -70 LOC.

Revision 1.197 / (download) - annotate - [select for diffs], Fri May 5 13:17:04 2017 UTC (7 years ago) by schwarze
Branch: MAIN
Changes since 1.196: +1 -2 lines
Diff to previous 1.196 (colored)

move .ll to the roff modules

Revision 1.196 / (download) - annotate - [select for diffs], Thu May 4 17:48:24 2017 UTC (7 years ago) by schwarze
Branch: MAIN
Changes since 1.195: +5 -3 lines
Diff to previous 1.195 (colored)

Parser reorg:
Generate the first node on the roff level: .br
Fix some column numbers in diagnostic messages while here.

Revision 1.195 / (download) - annotate - [select for diffs], Mon Apr 24 23:06:09 2017 UTC (7 years ago) by schwarze
Branch: MAIN
Changes since 1.194: +9 -10 lines
Diff to previous 1.194 (colored)

Continue parser unification:
* Make enum rofft an internal interface as enum roff_tok in "roff.h".
* Represent mdoc and man macros in enum roff_tok.
* Make TOKEN_NONE a proper enum value and use it throughout.
* Put the prologue macros first in the macro tables.
* Unify mdoc_macroname[] and man_macroname[] into roff_name[].

Revision 1.194 / (download) - annotate - [select for diffs], Fri Mar 3 13:41:28 2017 UTC (7 years, 2 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_1_BASE, OPENBSD_6_1
Changes since 1.193: +2 -2 lines
Diff to previous 1.193 (colored)

Fix a copy-and-paste error that caused man(7) manuals without
a section number in .TH to be misinterpreted as preformatted.
Found by jsg@ with cppcheck.

Revision 1.193 / (download) - annotate - [select for diffs], Thu Feb 9 19:50:47 2017 UTC (7 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.192: +2 -2 lines
Diff to previous 1.192 (colored)

No need to cast NULL when assigning it to a variable;
from Christos Zoulas <christos at NetBSD>.

Revision 1.192 / (download) - annotate - [select for diffs], Thu Feb 9 17:19:07 2017 UTC (7 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.191: +2 -2 lines
Diff to previous 1.191 (colored)

Be consistent in protecting __attribute__ attributes with __;
from Christos Zoulas <christos @ NetBSD>.

Revision 1.191 / (download) - annotate - [select for diffs], Fri Jan 27 11:33:08 2017 UTC (7 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.190: +9 -4 lines
Diff to previous 1.190 (colored)

warn about missing one-line description;
missing feature found in the TODO file

Revision 1.190 / (download) - annotate - [select for diffs], Fri Jan 27 01:14:34 2017 UTC (7 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.189: +31 -14 lines
Diff to previous 1.189 (colored)

Parse the section number from the content of preformatted pages
and warn if it doesn't match the directory where the file was found.

Revision 1.189 / (download) - annotate - [select for diffs], Fri Jan 27 01:09:02 2017 UTC (7 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.188: +17 -13 lines
Diff to previous 1.188 (colored)

If parsing a page reveals that it is neither mdoc(7) nor man(7),
fall back to treating it as preformatted rather than treating
it as man(7) anyway.

Revision 1.188 / (download) - annotate - [select for diffs], Fri Jan 27 01:04:13 2017 UTC (7 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.187: +15 -1 lines
Diff to previous 1.187 (colored)

fix base directory detection for makewhatis -t

Revision 1.187 / (download) - annotate - [select for diffs], Fri Jan 27 00:55:49 2017 UTC (7 years, 3 months ago) by schwarze
Branch: MAIN
Changes since 1.186: +7 -5 lines
Diff to previous 1.186 (colored)

Always register names taken from file names in the names ohash.
Bug found by makewhatis -p crashing in mlink_check().

Revision 1.186 / (download) - annotate - [select for diffs], Wed Jan 11 17:39:45 2017 UTC (7 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.185: +3 -3 lines
Diff to previous 1.185 (colored)

Do text production for .Bt, .Ex, .Rv, .Ud at the validation stage
rather than in the formatters.  Use NODE_NOSRC flag for .Lb and
NODE_NOSRC and NODE_NOPRT for .St.  Results in a more rigorous
syntax tree and in 135 lines less code.

This work was triggered by a question from Abhinav Upadhyay <er dot
abhinav dot upadhyay at gmail dot com> (NetBSD) on discuss@.

Revision 1.185 / (download) - annotate - [select for diffs], Tue Jan 10 23:36:24 2017 UTC (7 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.184: +3 -3 lines
Diff to previous 1.184 (colored)

Use new NODE_NOSRC and NODE_NOPRT flags for .Bx and .At.
More rigorous AST and 40 lines less code.

Revision 1.184 / (download) - annotate - [select for diffs], Tue Jan 10 21:54:34 2017 UTC (7 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.183: +136 -131 lines
Diff to previous 1.183 (colored)

For the .Ux/.Ox family of macros, do text production at the validation
stage rather than in each and every individual formatter, using the
new NODE_NOSRC flag.  More rigorous and also ten lines less code.

Revision 1.183 / (download) - annotate - [select for diffs], Sat Oct 22 10:08:31 2016 UTC (7 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.182: +3 -2 lines
Diff to previous 1.182 (colored)

cast off_t and uint64_t for printf(3)
to make sure that widths match on all platforms;
from Ed Maste <emaste at freebsd dot org>;
OK guenther@

Revision 1.182 / (download) - annotate - [select for diffs], Thu Oct 20 16:29:13 2016 UTC (7 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.181: +2 -2 lines
Diff to previous 1.181 (colored)

0x%llu is a bad idea, make that 0x%llx;
noticed while investigating a report from Ed Maste

Revision 1.181 / (download) - annotate - [select for diffs], Tue Oct 18 14:20:07 2016 UTC (7 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.180: +2 -3 lines
Diff to previous 1.180 (colored)

Fix a comment: We don't have manpath(1).  No code change.

Revision 1.180 / (download) - annotate - [select for diffs], Tue Oct 18 14:13:46 2016 UTC (7 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.179: +26 -36 lines
Diff to previous 1.179 (colored)

Simplify and correct support for reproducible builds, such that database
entries come in a well-defined order even in the presence of MLINKS.
Do this by using the compar() argument of fts_open(3) rather than
trying to sort later, which missed some cases.
This also shortens the code by a few lines.

Diff from Ed Maste <emaste @ FreeBSD>, adapted to our tree
and tweaked a bit by me, final version confirmed by Ed.

Revision 1.179 / (download) - annotate - [select for diffs], Fri Sep 2 14:03:24 2016 UTC (7 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.178: +4 -4 lines
Diff to previous 1.178 (colored)

When "makewhatis -d" tries to add to a database that doesn't (yet) exist,
silently create it from scratch instead of printing a warning.
The annoying warning message was reported by ajacoutot@, and espie@
convincingly argues that a non-existing database can be considered
equivalent to an empty one.

Revision 1.178 / (download) - annotate - [select for diffs], Thu Sep 1 15:07:46 2016 UTC (7 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.177: +2 -2 lines
Diff to previous 1.177 (colored)

remove "sparc" from a comment

Revision 1.177 / (download) - annotate - [select for diffs], Mon Aug 22 16:12:52 2016 UTC (7 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.176: +5 -5 lines
Diff to previous 1.176 (colored)

When trying to edit an existing database with makewhatis(8) -d or -u
but reading the database fails, report the full path to the database
on standard error, and mention that the database is automatically
recreated from scratch.
Suggested by espie@.

Revision 1.176 / (download) - annotate - [select for diffs], Wed Aug 17 20:46:06 2016 UTC (7 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.175: +3 -2 lines
Diff to previous 1.175 (colored)

When the content of a manual page does not specify a section, the
empty string got added to the list of sections, breaking the database
format slightly and causing the page to not be considered part of
any section, not even if a section could be deduced from the directory
or from the file name.
Bug found due to the bogus pcredemo(3) "manual" in the pcre-8.38p0 package.

Revision 1.175 / (download) - annotate - [select for diffs], Fri Aug 5 17:20:30 2016 UTC (7 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.174: +2 -2 lines
Diff to previous 1.174 (colored)

fix a typo that prevented names from .Dt from getting priority over
names from .Sh NAME; no dire consequences on OpenBSD since we no
longer have MLINKS for mdoc(7) pages

Revision 1.174 / (download) - annotate - [select for diffs], Mon Aug 1 20:46:33 2016 UTC (7 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.173: +4 -3 lines
Diff to previous 1.173 (colored)

Repair makewhatis -t, sorry for breaking it in the previous commit.
Committing this quickly because it hurts package builders.
Regression reported by naddy@.

Revision 1.173 / (download) - annotate - [select for diffs], Mon Aug 1 10:32:39 2016 UTC (7 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.172: +115 -388 lines
Diff to previous 1.172 (colored)

Remove the dependency on SQLite without loss of functionality.
Drop the obsolete names_check() now that we deleted MLINKS.
Run "doas makewhatis" after compiling and installing this.

Earlier version tested by jmc@ and jturner@;
"commit it all" deraadt@   "commit and dodge" krw@

Revision 1.172 / (download) - annotate - [select for diffs], Tue Jul 19 13:30:16 2016 UTC (7 years, 10 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_6_0_BASE, OPENBSD_6_0
Changes since 1.171: +5 -4 lines
Diff to previous 1.171 (colored)

Use __attribute__((__format__ throughout.
Triggered by a smaller patch from Christos Zoulas.

Revision 1.171 / (download) - annotate - [select for diffs], Fri Jul 15 18:02:32 2016 UTC (7 years, 10 months ago) by schwarze
Branch: MAIN
Changes since 1.170: +4 -1 lines
Diff to previous 1.170 (colored)

add missing prototypes, no code change;
noticed by Christos Zoulas with -Wmissing-prototypes

Revision 1.170 / (download) - annotate - [select for diffs], Sat Jul 9 15:23:36 2016 UTC (7 years, 10 months ago) by schwarze
Branch: MAIN
Changes since 1.169: +1 -2 lines
Diff to previous 1.169 (colored)

getopt(3) is declared in <unistd.h>, and <getopt.h> is not needed;
from Joerg Sonnenberger via Thomas Klausner, NetBSD.

Revision 1.169 / (download) - annotate - [select for diffs], Tue Mar 15 20:50:23 2016 UTC (8 years, 2 months ago) by krw
Branch: MAIN
Changes since 1.168: +2 -2 lines
Diff to previous 1.168 (colored)

'accomodate' -> 'accommodate' in comments.

Started by diff from Mical Mazurek.

Revision 1.168 / (download) - annotate - [select for diffs], Fri Jan 8 17:48:04 2016 UTC (8 years, 4 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_9_BASE, OPENBSD_5_9
Changes since 1.167: +10 -9 lines
Diff to previous 1.167 (colored)

Delete the redundant "nchild" member of struct roff_node, replacing
most uses by one, a few by two pointer checks, and only one by a
tiny loop - not only making data smaller, but code shorter as well.

This gets rid of an implicit invariant that confused both static
analysis tools and human auditors.  No functional change.

Revision 1.167 / (download) - annotate - [select for diffs], Fri Jan 8 15:01:58 2016 UTC (8 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.166: +4 -4 lines
Diff to previous 1.166 (colored)

Prefer warn(3) over perror(3) at the few places where it was used.
It is useful to see the program name.
Suggested by Christos Zoulas (NetBSD).

Revision 1.166 / (download) - annotate - [select for diffs], Fri Jan 8 02:53:09 2016 UTC (8 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.165: +2 -3 lines
Diff to previous 1.165 (colored)

Simplify the mparse_open() interface.
Just return the file descriptor or -1 on error;
there is just one kind of error anyway.
Suggested by Christos Zoulas (NetBSD).

Revision 1.165 / (download) - annotate - [select for diffs], Fri Jan 8 02:13:35 2016 UTC (8 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.164: +2 -1 lines
Diff to previous 1.164 (colored)

It was very surprising that a function called mparse_readfd()
closed the file descriptor passed to it after completing its work,
in particular considering the fact that it required its callers
to call open(2) or mparse_open() beforehand.

Change mparse_readfd() to not call close(2) and change the callers
to call close(2) afterwards, more or less bringing open and close
to the same level of the code and making review easier.  Note that
man.cgi(8) already did that, even though it was wrong in the past.

Small restructuring suggested by Christos Zoulas (NetBSD).

Revision 1.164 / (download) - annotate - [select for diffs], Mon Jan 4 14:44:02 2016 UTC (8 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.163: +29 -14 lines
Diff to previous 1.163 (colored)

Improve handling of .Va and .Vt macros.
tedu@ noticed that no Vt= database entries were generated.
Serguey Parkhomovsky suggested the deletion of parse_mdoc_body().
tb@ noticed that the fix requires more than just adding TYPE_Vt
to the MDOC_Vt mask in the mdoc_handler array.

Revision 1.163 / (download) - annotate - [select for diffs], Tue Dec 15 17:36:19 2015 UTC (8 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.162: +12 -9 lines
Diff to previous 1.162 (colored)

pledge(2) style:
Make sure to always use the idiom 'if (pledge("'
such that it can easily be searched for.
No functional change.
Requested by deraadt@ some time ago.

Revision 1.162 / (download) - annotate - [select for diffs], Sat Nov 7 17:58:52 2015 UTC (8 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.161: +22 -18 lines
Diff to previous 1.161 (colored)

Modernization, no functional change intended:
Use the POSIX function getline(3) rather than the slightly
dangerous BSD function fgetln(3).

Revision 1.161 / (download) - annotate - [select for diffs], Fri Nov 6 16:27:13 2015 UTC (8 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.160: +4 -6 lines
Diff to previous 1.160 (colored)

Use getprogname(3) rather than __progname.
Suggested by Joerg@ Sonnenberger (NetBSD).
Last year, deraadt@ confirmed on tech@ that this "has the potential
to be more portable", and micro-optimizing for speed is not relevant
here.  Also gets rid of one global variable.

Revision 1.160 / (download) - annotate - [select for diffs], Thu Oct 22 22:05:42 2015 UTC (8 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.159: +2 -1 lines
Diff to previous 1.159 (colored)

use the new function man_validate() here, too

Revision 1.159 / (download) - annotate - [select for diffs], Tue Oct 20 02:00:49 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.158: +2 -1 lines
Diff to previous 1.158 (colored)

In order to become able to generate syntax tree nodes on the roff(7)
level, validation must be separated from parsing and rewinding.
This first big step moves calling of the mdoc(7) post_*() functions
out of the parser loop into their own mdoc_validate() pass, while
using a new mdoc_state() module to make syntax tree state handling
available to both the parser loop and the validation pass.

Revision 1.158 / (download) - annotate - [select for diffs], Fri Oct 16 13:37:44 2015 UTC (8 years, 7 months ago) by millert
Branch: MAIN
Changes since 1.157: +3 -3 lines
Diff to previous 1.157 (colored)

Implement real "flock" request and add it to userland programs that
use pledge and file locking.  OK deraadt@

Revision 1.157 / (download) - annotate - [select for diffs], Tue Oct 13 22:57:49 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.156: +6 -8 lines
Diff to previous 1.156 (colored)

Major character table cleanup:
* Use ohash(3) rather than a hand-rolled hash table.
* Make the character table static in the chars.c module:
There is no need to pass a pointer around, we most certainly
never want to use two different character tables concurrently.
* No need to keep the characters in a separate file chars.in;
that merely encourages downstream porters to mess with them.
* Sort the characters to agree with the mandoc_chars(7) manual page.
* Specify Unicode codepoints in hex, not decimal (that's the detail
that originally triggered this patch).
No functional change, minus 100 LOC, and i don't see a performance change.

Revision 1.156 / (download) - annotate - [select for diffs], Tue Oct 13 15:50:15 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.155: +9 -47 lines
Diff to previous 1.155 (colored)

Reduce the amount of code by moving the three copies of the ohash
callback functions into one common place, preparing for the use of
ohash for some additional purposes.  No functional change.

Revision 1.155 / (download) - annotate - [select for diffs], Mon Oct 12 22:30:27 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.154: +17 -1 lines
Diff to previous 1.154 (colored)

pledge(2) for makewhatis(8):
Always needed: stdio rpath.
In addition when writing to an existing db: wpath cpath fattr.
In addition when creating a new db: proc exec.
Based on work by and OK bentley@, "get moving" deraadt@.

Revision 1.154 / (download) - annotate - [select for diffs], Mon Oct 12 21:16:32 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.153: +3 -4 lines
Diff to previous 1.153 (colored)

Garbage collect an unused variable, no functional change;
found by Svyatoslav Mishyn <juef at openmailbox dot org> with cppcheck.

Revision 1.153 / (download) - annotate - [select for diffs], Mon Oct 12 00:32:37 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.152: +23 -20 lines
Diff to previous 1.152 (colored)

Clear dform and dsec when exiting a first-level directory in treescan().
Fixes a segfault reported by bentley@.
While here, do some style cleanup in the same function.

Revision 1.152 / (download) - annotate - [select for diffs], Mon Oct 12 00:07:27 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.151: +1 -7 lines
Diff to previous 1.151 (colored)

To make the code more readable, delete 283 /* FALLTHROUGH */ comments
that were right between two adjacent case statement.  Keep only
those 24 where the first case actually executes some code before
falling through to the next case.

Revision 1.151 / (download) - annotate - [select for diffs], Sun Oct 11 21:06:59 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.150: +10 -17 lines
Diff to previous 1.150 (colored)

Finally use __progname, err(3) and warn(3).
That's more readable and less error-prone than fumbling around
with argv[0], fprintf(3), strerror(3), perror(3), and exit(3).
It also shortens the code by 50 lines.

It's a bad idea to boycott good interfaces merely because standards
committees ignore them.  Instead, it's the job of the portable
distribution to provide compatibility modules for archaic systems
(like commercial Solaris) that still don't have them.  Actually,
the compat code for the portable distribution already exists and
will be committed right after this.

Revision 1.150 / (download) - annotate - [select for diffs], Tue Oct 6 18:30:43 2015 UTC (8 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.149: +43 -43 lines
Diff to previous 1.149 (colored)

modernize style: "return" is not a function; ok cmp(1)

Revision 1.149 / (download) - annotate - [select for diffs], Wed Sep 16 09:49:48 2015 UTC (8 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.148: +3 -3 lines
Diff to previous 1.148 (colored)

for portability, use (char *)NULL in execlp(3) as discussed on tech@
(didn't blow up anywhere yet, but better safe than sorry)

Revision 1.148 / (download) - annotate - [select for diffs], Sun Jul 19 05:59:07 2015 UTC (8 years, 10 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_8_BASE, OPENBSD_5_8
Changes since 1.147: +1 -5 lines
Diff to previous 1.147 (colored)

Do not fork and exec gunzip(1), just link with libz instead.
As discussed with deraadt@, that's cleaner and will help tame(2).
Something like this was also suggested earlier by bapt at FreeBSD.
Minus 50 lines of code, deleting one interface function (mparse_wait),
no functional change intended.

Revision 1.147 / (download) - annotate - [select for diffs], Thu Apr 23 16:17:04 2015 UTC (9 years ago) by schwarze
Branch: MAIN
Changes since 1.146: +3 -3 lines
Diff to previous 1.146 (colored)

Unify mdoc_deroff() and man_deroff() into a common function deroff().
No functional change except that for mdoc(7), it now skips leading
escape sequences just like it already did for man(7).
Escape sequences rarely occur in mdoc(7) code and if they do,
skipping them is an improvement in this context.
Minus 30 lines of code.

Revision 1.146 / (download) - annotate - [select for diffs], Sat Apr 18 17:50:02 2015 UTC (9 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.145: +8 -9 lines
Diff to previous 1.145 (colored)

Delete the wrapper functions mdoc_meta(), man_meta(), mdoc_node(),
man_node() from the mandoc(3) semi-public interface and the internal
wrapper functions print_mdoc() and print_man() from the HTML formatters.
Minus 60 lines of code, no functional change.

Revision 1.145 / (download) - annotate - [select for diffs], Sat Apr 18 16:34:03 2015 UTC (9 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.144: +9 -11 lines
Diff to previous 1.144 (colored)

Profit from the unified struct roff_man and reduce the number of
arguments of mparse_result() by one.  No functional change.
Written on the ICE Bruxelles-Koeln on the way back from p2k15.

Revision 1.144 / (download) - annotate - [select for diffs], Sat Apr 18 16:04:40 2015 UTC (9 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.143: +3 -3 lines
Diff to previous 1.143 (colored)

Replace the structs mdoc and man by a unified struct roff_man.
Almost completely mechanical, no functional change.
Written on the train from Exeter to London returning from p2k15.

Revision 1.143 / (download) - annotate - [select for diffs], Thu Apr 2 23:47:43 2015 UTC (9 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.142: +24 -24 lines
Diff to previous 1.142 (colored)

Third step towards parser unification:
Replace struct mdoc_meta and struct man_meta by a unified struct roff_meta.
Written of the train from London to Exeter on the way to p2k15.

Revision 1.142 / (download) - annotate - [select for diffs], Thu Apr 2 22:06:17 2015 UTC (9 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.141: +29 -29 lines
Diff to previous 1.141 (colored)

Second step towards parser unification:
Replace struct mdoc_node and struct man_node by a unified struct roff_node.
To be able to use the tok member for both mdoc(7) and man(7) without
defining all the macros in roff.h, sacrifice a tiny bit of type safety
and make tok an int rather than an enum.
Almost mechanical, no functional change.
Written on the Eurostar from Bruxelles to London on the way to p2k15.

Revision 1.141 / (download) - annotate - [select for diffs], Thu Apr 2 21:03:18 2015 UTC (9 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.140: +24 -23 lines
Diff to previous 1.140 (colored)

First step towards parser unification:
Replace enum mdoc_type and enum man_type by a unified enum roff_type.
Almost mechanical, no functional change.
Written on the ICE train from Frankfurt to Bruxelles on the way to p2k15.

Revision 1.140 / (download) - annotate - [select for diffs], Fri Mar 27 17:36:56 2015 UTC (9 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.139: +23 -23 lines
Diff to previous 1.139 (colored)

Parse the new man.conf(5) "output" directive.
The next step will be to actually use the parsed data.

Revision 1.139 / (download) - annotate - [select for diffs], Fri Mar 13 00:18:45 2015 UTC (9 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.138: +2 -1 lines
Diff to previous 1.138 (colored)

Fix hardlink detection on platforms having padding in struct inodev,
typically 64bit platforms.  This was basically broken since forever.
Not only is the padding used, but it was used uninitialized.
Problem reported by jmc@.

Revision 1.138 / (download) - annotate - [select for diffs], Fri Feb 27 16:21:42 2015 UTC (9 years, 2 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_7_BASE, OPENBSD_5_7
Changes since 1.137: +3 -2 lines
Diff to previous 1.137 (colored)

When makewhatis(8) scans a tree, ignore trailing garbage on filenames.
This is relevant because some ports install files like man1/xsel.1x,
as reported by patrick keshishian <pkeshish at gmail dot com> on misc@.

We can probably improve functionality and simplify the code by ignoring
file name extensions altogether; we already know the section number from
the name of the directory.  But so close to lock, i'm keeping the fix
minimal.

Revision 1.137 / (download) - annotate - [select for diffs], Thu Jan 15 04:26:06 2015 UTC (9 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.136: +6 -9 lines
Diff to previous 1.136 (colored)

Fatal errors no longer exist.
If a file can be opened, mandoc will produce some output;
at worst, the output may be almost empty.
Simplifies error handling and frees a message type for future use.

Revision 1.136 / (download) - annotate - [select for diffs], Sat Jan 3 12:54:49 2015 UTC (9 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.135: +16 -23 lines
Diff to previous 1.135 (colored)

Fix a potential NULL pointer access in an error message after waitpid()
failure; found using detailed information provided by Ulrich Spoerlein
<uqs at FreeBSD> about FreeBSD Coverity CID 1261304.

Revision 1.135 / (download) - annotate - [select for diffs], Fri Jan 2 17:01:31 2015 UTC (9 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.134: +4 -2 lines
Diff to previous 1.134 (colored)

Explicitly set the *data member of struct ohash_info to NULL.
It is never dereferenced, but it gets copied around, which worries
static analysis tools and might also confuse human auditors.
FreeBSD Coverity CID 1261298, 1261299, 1261300, reported by
Pedro Giffuni and Ulrich Spörlein <pfg@ and uqs@ at FreeBSD>.

Revision 1.134 / (download) - annotate - [select for diffs], Tue Dec 30 20:40:05 2014 UTC (9 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.133: +2 -1 lines
Diff to previous 1.133 (colored)

When a file is given on the command line, actually exists, and its name
relative to the respective manual tree is longer than PATH_MAX, do not
leak the memory allocated to hold the name.  Not sure that can actually
happen, but better safe than sorry.
FreeBSD Coverity Scan CID 1261303, reported by Pedro Giffuni <pfg@>.

Revision 1.133 / (download) - annotate - [select for diffs], Fri Dec 5 16:17:56 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.132: +28 -25 lines
Diff to previous 1.132 (colored)

Improve parsing of function names.
This gets rid of the last bogus entries in base and Xenocara.

Revision 1.132 / (download) - annotate - [select for diffs], Fri Dec 5 15:16:25 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.131: +4 -1 lines
Diff to previous 1.131 (colored)

Do not misinterpret function arguments as function names;
improves semantic analysis of more than 300 manuals.

Revision 1.131 / (download) - annotate - [select for diffs], Fri Dec 5 14:26:23 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.130: +113 -122 lines
Diff to previous 1.130 (colored)

Render text before, not after accumulating flag bits, such that flags
for different representations of the same string end up in the same
database entry.  Improves name classification for 500 manuals.

Revision 1.130 / (download) - annotate - [select for diffs], Thu Dec 4 21:48:26 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.129: +17 -4 lines
Diff to previous 1.129 (colored)

When finding a .so link after the page was already processed,
do not clobber the existing names flags;
instead, OR the additional flags into them.

Revision 1.129 / (download) - annotate - [select for diffs], Thu Dec 4 20:13:13 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.128: +28 -11 lines
Diff to previous 1.128 (colored)

correctly store .Dt and .TH information in the names table

Revision 1.128 / (download) - annotate - [select for diffs], Thu Dec 4 18:31:04 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.127: +17 -2 lines
Diff to previous 1.127 (colored)

in the SYNOPSIS, add .Fo and first .Fn arguments to the names table

Revision 1.127 / (download) - annotate - [select for diffs], Thu Dec 4 17:35:42 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.126: +48 -25 lines
Diff to previous 1.126 (colored)

In the SYNOPSIS, if .Nm occurs without argument, give the first .Nm
that occurred in the document a NAME_SYN entry in the names table.

Revision 1.126 / (download) - annotate - [select for diffs], Thu Nov 27 01:57:42 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.125: +19 -15 lines
Diff to previous 1.125 (colored)

Make makewhatis(8) understand .so links to .gz pages.
Drop the FORM_GZ annotation in the mpages table; it is conceptually wrong
because it ought to be in the mlinks table: An uncompressed .so link file
can point to a compressed manual page file and vice versa.
Besides, it is no longer needed because mparse_open() handles it all.
Sprinkle some KNF while here.

Revision 1.125 / (download) - annotate - [select for diffs], Wed Nov 26 21:40:11 2014 UTC (9 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.124: +3 -6 lines
Diff to previous 1.124 (colored)

Simplify the mparse_open()/mparse_wait() interface.
Don't bother the user with the PID of the child process,
store it inside the opaque mparse handle.

Revision 1.124 / (download) - annotate - [select for diffs], Wed Nov 19 20:40:28 2014 UTC (9 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.123: +14 -4 lines
Diff to previous 1.123 (colored)

Three fixes with respect to the names table:
1. Do not mask out NAME_FIRST before its first use.
2. Avoid duplicate NAME_FILE entries.
3. Correctly mask NAME_FILE for .so links.

Revision 1.123 / (download) - annotate - [select for diffs], Tue Oct 28 17:35:42 2014 UTC (9 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.122: +4 -4 lines
Diff to previous 1.122 (colored)

Make the character table available to libroff so it can check the
validity of character escape names and warn about unknown ones.
This requires mchars_spec2cp() to report unknown names again.
Fortunately, that doesn't require changing the calling code because
according to groff, invalid character escapes should not produce
output anyway, and now that we warn about them, that's fine.

Revision 1.122 / (download) - annotate - [select for diffs], Mon Oct 27 13:29:30 2014 UTC (9 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.121: +2 -2 lines
Diff to previous 1.121 (colored)

Fix a regression in term.c rev. 1.89 reported by bentley@:
In UTF-8 output, do not print anything if mchars_spec2cp() returns 0.
In particular, this repairs handling of zero-width spaces (\&).

While here, let mchars_spec2cp() return 0xFFFD instead of -1
if the character is not found, simplifying the using code.
In HTML output, do not print obfuscated ASCII characters and
do not test for one-char escapes, mchars_spec2cp() already does that.

Revision 1.121 / (download) - annotate - [select for diffs], Sun Oct 12 20:32:16 2014 UTC (9 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.120: +2 -1 lines
Diff to previous 1.120 (colored)

add missing entry for .ll to the dispatch table;
patch from Martin <Natano at natano dot net>, thanks

Revision 1.120 / (download) - annotate - [select for diffs], Tue Sep 9 19:33:55 2014 UTC (9 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.119: +16 -12 lines
Diff to previous 1.119 (colored)

If a manpath directory (for example, a _whatdb entry from man.conf(5)
or an entry in the MANPATH environment variable) does not exist,
silently skip it.  This brings makewhatis(8) back closer to the
behaviour of espie@'s version and ought to shut up the weekly(8)
whining observed by henning@ on machines not having xbase installed.
Also, don't error out after the first unusable manpath entry, still
try the others.

Of course, still complain about non-existent directories specified
on the command line and about any directories failing for other
reasons than ENOENT.

Revision 1.119 / (download) - annotate - [select for diffs], Sun Sep 7 03:08:42 2014 UTC (9 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.118: +3 -3 lines
Diff to previous 1.118 (colored)

Do not report a page as arch=any merely because .Dt lacks the third argument.
Pages found outside arch-specific dirs still get arch=any, of course.
Issue reported by justinhenryhaynes at gmail dot com on misc@, thanks!

Revision 1.118 / (download) - annotate - [select for diffs], Sun Sep 7 03:00:03 2014 UTC (9 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.117: +2 -2 lines
Diff to previous 1.117 (colored)

fix debugging output

Revision 1.117 / (download) - annotate - [select for diffs], Wed Sep 3 23:20:33 2014 UTC (9 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.116: +13 -51 lines
Diff to previous 1.116 (colored)

Add *.gz support to apropos(1) -a, manm(1), and even mandoc(1).
Implemented by moving the zip code from makewhatis(8) to the parser lib.

Revision 1.116 / (download) - annotate - [select for diffs], Wed Sep 3 18:08:26 2014 UTC (9 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.115: +8 -11 lines
Diff to previous 1.115 (colored)

If a manual page is installed gzip(1)ed, let makewhatis(8) take
note in mandoc.db(5), such that man(1) -w and apropos(1) -w can
report the correct filename.
This is a prerequisite for letting apropos -a and man support
gzip'ed manuals in the future, which doesn't work yet.

Revision 1.115 / (download) - annotate - [select for diffs], Mon Sep 1 23:47:56 2014 UTC (9 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.114: +8 -1 lines
Diff to previous 1.114 (colored)

When makewhatis(8) finds an .so link after the manual being pointed to
has already been processed, add the file names to the names table, too,
not just to the mlinks table.
This fixes a bug where apropos(1) and the new man(1) wouldn't find some
of the Xenocara manuals via some of their .so links.  After rebuilding,
run "makewhatis /usr/X11R6/man" or just wait for weekly(8).

Revision 1.114 / (download) - annotate - [select for diffs], Sat Aug 23 01:00:20 2014 UTC (9 years, 8 months ago) by doug
Branch: MAIN
Changes since 1.113: +3 -1 lines
Diff to previous 1.113 (colored)

Fix fd leak when fdopen fails.

ok schwarze@

Revision 1.113 / (download) - annotate - [select for diffs], Fri Aug 8 17:36:21 2014 UTC (9 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.112: +5 -5 lines
Diff to previous 1.112 (colored)

Absurdly, the return value of sqlite3_column_text()
is "const unsigned char *", which causes warnings with GCC on Linux.
Explicitly cast to "const char *" to avoid this.
Issue noticed by kristaps@.

Revision 1.112 / (download) - annotate - [select for diffs], Fri Aug 8 16:17:09 2014 UTC (9 years, 9 months ago) by schwarze
Branch: MAIN
Changes since 1.111: +4 -3 lines
Diff to previous 1.111 (colored)

Bring the handling of defective prologues even closer to groff,
in particular relaxing the distinction between prologue and body
and further improving messages.
* The last .Dd wins and the last .Os wins, even in the body.
* The last .Dt before the first body macro wins.
* Missing title in .Dt defaults to UNTITLED.  Warn about it.
* Missing section in .Dt does not default to 1.  But warn about it.
* Do not warn multiple times about the same mdoc(7) prologue macro.
* Warn about missing .Os.
* Incomplete .TH defaults to empty strings.  Warn about it.

Revision 1.111 / (download) - annotate - [select for diffs], Sat Jun 21 16:17:56 2014 UTC (9 years, 11 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_6_BASE, OPENBSD_5_6
Changes since 1.110: +8 -5 lines
Diff to previous 1.110 (colored)

Prefix messages about bad command line options and arguments
with "mandoc: " or "makewhatis: ", respectively,
similar to what we already do for other messages.

Revision 1.110 / (download) - annotate - [select for diffs], Fri Jun 20 01:20:55 2014 UTC (9 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.109: +40 -24 lines
Diff to previous 1.109 (colored)

More tweaking of set_basedir().

1) Do not error out when getcwd(3) fails, only fail when inaccessibility
of the cwd prevents processing of relative paths given on the command line.

2) Do not uselessly call set_basedir() twice in a row.
While fts_read(3) in treescan() does cause the cwd to jump around,
fts_close(3) is always called at the end, putting us back
where we came from.  The -d/-u fallback code already relied on this.

3) Fix the man-root-dir indicator in say().

Revision 1.109 / (download) - annotate - [select for diffs], Thu Jun 19 00:44:59 2014 UTC (9 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.108: +21 -18 lines
Diff to previous 1.108 (colored)

Some simple set_basedir() cleanup; more to come.

1) Refrain from calling set_basedir() in the -t case,
and do not attempt to strip anything from the file names in that case.
Testing individual files cannot reasonably have any notion of a base dir.

2) Remove the possibility of passing NULL to set_basedir().
It was dangerous because it was not idempotent, and it served no purpose
except closing a file descriptor right before exit(), which is pointless.
Besides, the file descriptor is likely to be removed completely, soon.

3) Make sure that /foobar isn't treated as a subdirectory of /foo;
this fixes a bug reported by espie@.

Revision 1.108 / (download) - annotate - [select for diffs], Wed Jun 18 19:17:37 2014 UTC (9 years, 11 months ago) by sthen
Branch: MAIN
Changes since 1.107: +3 -2 lines
Diff to previous 1.107 (colored)

Don't display "unable to open mandoc.db" error messages (SQLITE_CANTOPEN)
in the code which opens mandocdb's sqlite database when updating/deleting
individual files (as used and only really useful for pkg_add/pkg_delete).
ok schwarze@

Revision 1.107 / (download) - annotate - [select for diffs], Thu Jun 5 07:13:53 2014 UTC (9 years, 11 months ago) by espie
Branch: MAIN
Changes since 1.106: +4 -3 lines
Diff to previous 1.106 (colored)

work-around issue in makewhatis, that expects its current dir to NOT be /.
okay aja@, miod@ until Ingo figures things out.

(This fixes the
///sr/ports/pobj/p5-YAML-0.84/fake-amd64/usr/local/man/man3p/YAML::Loader::Base.3p: fopen: No such file or directory
style errors)

Revision 1.106 / (download) - annotate - [select for diffs], Mon May 12 19:11:20 2014 UTC (10 years ago) by espie
Branch: MAIN
Changes since 1.105: +10 -10 lines
Diff to previous 1.105 (colored)

adjust to ohash being in libutil now, and to the interface changes.
fix potential integer overflows in memory allocation (mostly for pedagogical
purposes, these are unlikely to overflow in practice)
move the rest of lst.lib stuff into its own directory.

Revision 1.105 / (download) - annotate - [select for diffs], Wed May 7 16:18:57 2014 UTC (10 years ago) by schwarze
Branch: MAIN
Changes since 1.104: +11 -2 lines
Diff to previous 1.104 (colored)

Render roff escape sequences contained in manual page descriptions
before putting them into the mpages table.
Issue found by bentley@ in OpenBSD::Getopt(3p).

Revision 1.104 / (download) - annotate - [select for diffs], Wed May 7 15:10:33 2014 UTC (10 years ago) by schwarze
Branch: MAIN
Changes since 1.103: +14 -2 lines
Diff to previous 1.103 (colored)

Fix two memory leaks in makewhatis -n:
1. As found by nigel@, names_check() requires database access.
2. Do not leak names and strings in -n mode.

Revision 1.103 / (download) - annotate - [select for diffs], Sun Apr 27 23:03:52 2014 UTC (10 years ago) by schwarze
Branch: MAIN
Changes since 1.102: +22 -7 lines
Diff to previous 1.102 (colored)

Improve error handling in dbopen().  If PRAGMA SQL statements fail,
report the error, close the database, and return failure from dbopen(),
such that the main program can recover and rebuild the database.

As noticed by stsp@, this can happen when database files are
accessible, but corrupt or in the wrong format, which will now
automatically be repaired.

Besides, use a safer idiom after sqlite3_open*() failure that also
handles out-of-memory situations correctly, and do not forget to
close the database after CREATE TABLE failure.

Revision 1.102 / (download) - annotate - [select for diffs], Fri Apr 25 12:12:35 2014 UTC (10 years ago) by schwarze
Branch: MAIN
Changes since 1.101: +3 -1 lines
Diff to previous 1.101 (colored)

Reduce the verbosity of makewhatis -t:
In the past, it always showed the title lines of the files processed.
Now, it only shows them when called with -D.
That is better because pkg_create calls makewhatis -t.
It is also more consistent with -D behaviour in non- -t modes.
Issue reported by ajacoutot@; ok espie@ ajacoutot@ jasper@.

Revision 1.101 / (download) - annotate - [select for diffs], Wed Apr 23 21:06:33 2014 UTC (10 years ago) by schwarze
Branch: MAIN
Changes since 1.100: +5 -5 lines
Diff to previous 1.100 (colored)

Audit malloc(3)/calloc(3)/realloc(3) usage.
* Change eight reallocs to reallocarray to be safe from overflows.
* Change one malloc to reallocarray to be safe from overflows.
* Change one calloc to reallocarray, no zeroing needed.
* Change the order of arguments of three callocs (aesthetical).

Revision 1.100 / (download) - annotate - [select for diffs], Wed Apr 23 19:08:52 2014 UTC (10 years ago) by schwarze
Branch: MAIN
Changes since 1.99: +23 -23 lines
Diff to previous 1.99 (colored)

Audit strlcpy(3)/strlcat(3) usage:
* Add missing truncation checks to three calls.
* In four cases where we know that the distination buffer is large enough,
cast the return vailue to (void).

Revision 1.99 / (download) - annotate - [select for diffs], Wed Apr 23 16:33:37 2014 UTC (10 years ago) by schwarze
Branch: MAIN
Changes since 1.98: +3 -2 lines
Diff to previous 1.98 (colored)

improve SQL style: avoid "SELECT *", be explicit in what columns we want;
suggested by espie@.

Revision 1.98 / (download) - annotate - [select for diffs], Sun Apr 20 16:44:44 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.97: +67 -66 lines
Diff to previous 1.97 (colored)

KNF: case (FOO):  ->  case FOO, remove /* LINTED */ and /* ARGSUSED */,
remove trailing whitespace and blanks before tabs, improve some indenting;
no functional change

Revision 1.97 / (download) - annotate - [select for diffs], Sat Apr 19 02:55:44 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.96: +9 -7 lines
Diff to previous 1.96 (colored)

Two minor tweaks regarding the fallback from -u/-d to default mode:
(1) Use all files found on the command line, but do *not* use all stray
files found during fallback tree recursion.
(2) If the fallback works, call that success, i.e. exit(0).
As pointed out by naddy@, the latter is required for ports' happiness.

Revision 1.96 / (download) - annotate - [select for diffs], Sat Apr 19 02:29:12 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.95: +76 -17 lines
Diff to previous 1.95 (colored)

Properly handle symlinks (hardlinks and .so only files were already ok):
Use the file name of the symlink but the inode number of the file pointed to,
such that we get multiple mlinks records but not multiple mpages records.
Also make sure they do not point outside the tree we are processing.

Issue found by kili@ in desktop-file-edit(1), thanks!

Revision 1.95 / (download) - annotate - [select for diffs], Fri Apr 18 21:54:48 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.94: +18 -8 lines
Diff to previous 1.94 (colored)

In update mode, when opening the database fails, probably because it is
missing or corrupt, just rebuild it from scratch.  This also helps when
installing the very first port on a freshly installed machine
and is similar to what espie@'s classical makewhatis(8) did.

Issue reported by naddy@ via kili@.

Revision 1.94 / (download) - annotate - [select for diffs], Wed Apr 16 21:35:48 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.93: +12 -12 lines
Diff to previous 1.93 (colored)

Rename the mpages.id column to mpages.pageid.  There is no good reason
to call this kid by a different name here than in all other tables.
Easier to polish this now than after enabling.

Revision 1.93 / (download) - annotate - [select for diffs], Wed Apr 16 18:59:38 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.92: +4 -2 lines
Diff to previous 1.92 (colored)

Give the mlinks and keys tables a pageid index,
as suggested by jeremy@ and espie@.

The mlinks index speeds up basic apropos(1) searches by around 30%
because it speeds up the final SELECT FROM mlinks query by about 95%.
For large result sets, the overall speedup gets even larger, in the
extreme case of "apropos Nd~." by more than 90%.
The keys index finally makes the apropos(1) -O option usable: It no longer
incurs relevant extra cost, while in the past it was embarrassingly slow.

This comes at a cost:  Total database build times grow by about 5%,
and each index adds about 10% database size with -Q.  I consider that
acceptable in view of the huge apropos(1) performance gains.
The -Q database for /usr/share/man still remains below 1 MB.

Revision 1.92 / (download) - annotate - [select for diffs], Sun Apr 13 22:02:54 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.91: +4 -9 lines
Diff to previous 1.91 (colored)

Unify description handling across all document types (mdoc, man, cat).
Assert that the description is unset right before calling the parse_*
handler, and assign a default if it's still unset right afterwards.
Remove all stray asserts and default assignments found elsewhere.

This fixes SQL_STEP failures for man(7) pages lacking descriptions.

Revision 1.91 / (download) - annotate - [select for diffs], Sun Apr 13 21:21:27 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.90: +9 -9 lines
Diff to previous 1.90 (colored)

better error reporting in case of SQL errors: mention dir and file

Revision 1.90 / (download) - annotate - [select for diffs], Thu Apr 10 02:45:03 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.89: +65 -27 lines
Diff to previous 1.89 (colored)

Next speed optimization step for the new apropos(1).
Split manual names out of the common "keys" table into their
own "names" table.  This reduces standard apropos(1) search
times (i.e. searching for names and descriptions only) by
typically about 70% for the full /usr/share/man database.
(Yes, that multiplies with the previous optimization step,
so both together have reduced search times by a factor of
more than six.  I'm not done yet, expect more to come.)

Even with the minimal databases built with makewhatis(8) -Q,
this step still reduces search times by 15-20%.  For both cases,
database sizes and build times hardly change (+/-2%).

Revision 1.89 / (download) - annotate - [select for diffs], Wed Apr 9 21:49:31 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.88: +9 -18 lines
Diff to previous 1.88 (colored)

After careful gprof(1)ing of the new apropos(1), move the descriptions
back from the keys table to the mpages table:  I found a good way
to still use them in searches, without complication of the code.

On my notebook, this reduces typical apropos(1) search times by about 40%,
it reduces /usr/share/man database size by 6% in makewhatis(8) -Q mode
and by 2% in standard mode (less overhead storing pointers to mpages),
and it doesn't measurably change database build times (may even be
going down by a percent or so because less data is being copied
around in ohashes).

Revision 1.88 / (download) - annotate - [select for diffs], Fri Apr 4 18:23:07 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.87: +30 -25 lines
Diff to previous 1.87 (colored)

In -p (picky) mode, warn unless each filename (aka mlink)
appears as a name in the NAME section.
While here, garbage collect two unused variables, both called "match".

Revision 1.87 / (download) - annotate - [select for diffs], Fri Apr 4 16:43:08 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.86: +41 -1 lines
Diff to previous 1.86 (colored)

Warn about missing mlinks.
This is really expensive, more than tripling database build times,
so only do it when the -p (picky) option was given, but none of the
following options were given: -Q (quick), -d, -u, or -t.

Revision 1.86 / (download) - annotate - [select for diffs], Fri Apr 4 15:55:17 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.85: +9 -6 lines
Diff to previous 1.85 (colored)

Remember which names are in the NAME section.
This helps to find missing MLINKS.
Database build times do not change and database growth is minimal
(1.2% with -Q, 0.7% without -Q in /usr/share/man),
so making this optional would be pointless.

Revision 1.85 / (download) - annotate - [select for diffs], Fri Apr 4 02:31:01 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.84: +39 -5 lines
Diff to previous 1.84 (colored)

When the -n or -t flag is given to makewhatis(8),
write names and decriptions to stdout,
in a format similar to apropos(1) output.
Inspired by espie@'s makewhatis.

Revision 1.84 / (download) - annotate - [select for diffs], Thu Apr 3 21:45:27 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.83: +6 -1 lines
Diff to previous 1.83 (colored)

Instead of silently doing nothing at all,
warn and return non-zero when the manpath is empty, that is,
when /etc/man.conf is non-existent or unreadable
AND the environment variable MANPATH is unset or empty
AND no directories were given on the command line.

Inspired by the error handling in espie@'s makewhatis(8),
except that one doesn't know about MANPATH.

Revision 1.83 / (download) - annotate - [select for diffs], Thu Apr 3 16:26:53 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.82: +11 -8 lines
Diff to previous 1.82 (colored)

Rename the -W option to -p (mnemonics: picky, print to stderr):
That letter was already chosen by espie@ for OpenBSD 2.7,
so avoid being gratuitiously different more than a decade later.
Accept -v for backward compatibility with espie@'s makewhatis,
even though it does nothing right now.

Revision 1.82 / (download) - annotate - [select for diffs], Thu Apr 3 15:37:14 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.81: +13 -13 lines
Diff to previous 1.81 (colored)

The -v option of mandocdb(8) clashes with the -v option of espie@'s
makewhatis(8), which traditionally does something different,
so rename it to -D (mnemonics: Debug, Dump, Display).

Revision 1.81 / (download) - annotate - [select for diffs], Wed Mar 26 21:39:33 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.80: +81 -17 lines
Diff to previous 1.80 (colored)

Without bloating mandoc(1) itself, let mandocdb(8) support files
called manN/X.N.gz and catN/X.0.gz, reading them through a pipe(2)
from gunzip(1) -c.  Asked for by various people in the past.

Revision 1.80 / (download) - annotate - [select for diffs], Wed Mar 26 20:53:31 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.79: +52 -34 lines
Diff to previous 1.79 (colored)

Improve error reporting.
Simplify combining a custom format string with perror(),
avoiding many manual calls to strerror(errno).
For low-level failures, report attempted function calls.
Do not abuse the say() filename argument for files outside the basedir,
and even less for other text.

Revision 1.79 / (download) - annotate - [select for diffs], Sun Mar 23 12:44:18 2014 UTC (10 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.78: +4 -26 lines
Diff to previous 1.78 (colored)

If an .Nd block contains macros, avoid fragmented entries in mandocdb(8),
instead use the .Nd content recursively.
Improves a couple of index entries in base.

Revision 1.78 / (download) - annotate - [select for diffs], Sat Mar 22 00:56:07 2014 UTC (10 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.77: +8 -47 lines
Diff to previous 1.77 (colored)

If a man(7) NAME section contains macros, avoid truncated or empty
entries for .Nd in mandocdb(8), instead use the macro content
recursively.  This improves indexing of more than 200 manuals
in Xenocara, i.e. more than 15%, in particular GL and some Xkb.

Revision 1.77 / (download) - annotate - [select for diffs], Fri Mar 21 22:52:21 2014 UTC (10 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.76: +2 -5 lines
Diff to previous 1.76 (colored)

avoid repetitive code for asprintf error handling

Revision 1.76 / (download) - annotate - [select for diffs], Fri Mar 21 22:17:01 2014 UTC (10 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.75: +2 -1 lines
Diff to previous 1.75 (colored)

The files mandoc.c and mandoc.h contained both specialised low-level
functions used for multiple languages (mdoc, man, roff), for example
mandoc_escape(), mandoc_getarg(), mandoc_eos(), and generic auxiliary
functions.  Split the auxiliaries out into their own file and header.
While here, do some #include cleanup.

Revision 1.75 / (download) - annotate - [select for diffs], Wed Mar 19 22:33:04 2014 UTC (10 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.74: +68 -20 lines
Diff to previous 1.74 (colored)

Register pure .so pages as mlinks, not as mpages.
This doesn't affect /usr/share/man, but improves /usr/X11R6/man:
* Eliminates multiple apropos(1) output for such pages.
* Reduces X11R6 database size from 450 kB to 240 kB (-47%).
* Reduces X11R6 database build time from 1.68s to 1.00s (-40%).

Revision 1.74 / (download) - annotate - [select for diffs], Wed Mar 19 22:20:36 2014 UTC (10 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.73: +2 -2 lines
Diff to previous 1.73 (colored)

Without the MPARSE_SO option, if the file contains nothing but a
single .so request, do not read the file pointed to, but instead
let mparse_result() provide the file name pointed to as a return
value.  To be used by makewhatis(8) in the future.

Revision 1.73 / (download) - annotate - [select for diffs], Wed Mar 19 21:50:59 2014 UTC (10 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.72: +6 -6 lines
Diff to previous 1.72 (colored)

Generalize the mparse_alloc() and roff_alloc() functions by giving
them an "options" argument, replacing the existing "inttype" and
"quick" arguments, preparing for a future MPARSE_SO option.
Store this argument in struct mparse and struct roff, replacing the
existing "inttype", "parsetype", and "quick" members.
No functional change except one tiny cosmetic fix in roff_TH().

Revision 1.72 / (download) - annotate - [select for diffs], Tue Mar 18 16:56:06 2014 UTC (10 years, 2 months ago) by schwarze
Branch: MAIN
Changes since 1.71: +95 -21 lines
Diff to previous 1.71 (colored)

Allow checking that databases are up to date even when you have no write
permission on the databases, as requested by espie@ quite some time ago.
But make sure to not slow database generation down when you do have write
permission, and to not delay error reporting in -Q mode.

Revision 1.71 / (download) - annotate - [select for diffs], Wed Jan 22 20:58:35 2014 UTC (10 years, 3 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_5_BASE, OPENBSD_5_5
Changes since 1.70: +15 -6 lines
Diff to previous 1.70 (colored)

Implement the \: (optional line break) escape sequence,
documented in the Ossanna-Kernighan-Ritter troff manual
and also supported by groff.

Missing feature reported by Steffen Nurpmeso <sdaoden at gmail dot com>.

Revision 1.70 / (download) - annotate - [select for diffs], Sun Jan 19 22:48:00 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.69: +4 -3 lines
Diff to previous 1.69 (colored)

Avoid the risk of trying to modify a literal string.

Revision 1.69 / (download) - annotate - [select for diffs], Sun Jan 19 22:40:42 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.68: +10 -6 lines
Diff to previous 1.68 (colored)

Always store the arch in lower-case only.
Reduces database size by ~0.5%, and by ~1.5% with -Q.

Revision 1.68 / (download) - annotate - [select for diffs], Sun Jan 19 00:09:33 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.67: +17 -2 lines
Diff to previous 1.67 (colored)

Support a second -v on mandocdb(8) to show keys while they are being added;
i need that for debugging, in particular to be used with -t.
To be able to do so, provide a global table of key names, for reuse.

Revision 1.67 / (download) - annotate - [select for diffs], Sat Jan 18 08:54:20 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.66: +10 -1 lines
Diff to previous 1.66 (colored)

Cope with slightly broken NAME sections in man(7) pages
having a trailing comma after the last name, like this:

ASN1_OBJECT_new, ASN1_OBJECT_free, - object allocation functions

Nit found somewhere above the Tasman Sea,
committing from Dunedin, Otago.

Revision 1.66 / (download) - annotate - [select for diffs], Fri Jan 17 19:16:42 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.65: +2 -4 lines
Diff to previous 1.65 (colored)

Drop the AUTOINCREMENT PRIMARY KEYs from the mlinks and keys tables.
They are completely unused, and i cannot imagine what they *could*
ever be used for; but apparently, they are expensive to generate.

Standard DB build time goes down by 10%, now at 1.9x of makewhatis.
Standard DB size goes down by 4%, now at 11x of makewhatis.
DB build time with -Q goes down by 15%, now at 0.28x of makewhatis.
DB size with -Q goes down by 3%, now at 3.35x of makewhatis.

Optimization found somewhere above the Southern Balochistan desert,
near the Iran-Pakistan border.  Committing from Melbourne, Victoria.

Revision 1.65 / (download) - annotate - [select for diffs], Fri Jan 17 19:07:12 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.64: +2 -4 lines
Diff to previous 1.64 (colored)

Despite some experimenting, i'm unable to find any relevant effect of
creating an index for the keys table on apropos(1) search times;
apparently, adding that index was premature optimization in the first
place; so, stop adding that index.

Its root gone, the following evil is reduced (/usr/share/man on my notebook)
 - DB build time with -Q goes down by 15%, now at 1/3 of makewhatis
 - DB size with -Q goes down by 35%, now at 3.5x of makewhatis
 - full DB build time goes down by 12%, now at 2.1x of makewhatis
 - full DB size goes down by 42%, now at 11.5x of makewhatis

Optimization found somewhere above the Dasht-e Kavir, the Great Salt
Desert, between Tehran (Iran) and Herat (Afghanistan).  Committing
from Melbourne, Australia.

Revision 1.64 / (download) - annotate - [select for diffs], Mon Jan 6 20:53:36 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.63: +7 -5 lines
Diff to previous 1.63 (colored)

Do not sync to disk after each individual manual page (duh!),
only sync to disk one single time when all data is ready.

Rebuild times for /usr/share/man/mandoc.db shrink on my notebook:
In standard mode from 45 seconds to 11 seconds   (75% reduction)
In -Q mode       from 25 seconds to 3.1 seconds  (87% reduction)
For comparison: makewhatis(8):      4.2 seconds

That is, in -Q mode, we are now *faster* than the existing makewhatis(8),
and careful profiling shows there is still a lot of room for improval.

Revision 1.63 / (download) - annotate - [select for diffs], Mon Jan 6 15:32:44 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.62: +36 -15 lines
Diff to previous 1.62 (colored)

Fix mandocdb(8) -d and -u.
It was broken by recent optimizations.

Revision 1.62 / (download) - annotate - [select for diffs], Mon Jan 6 13:54:11 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.61: +6 -6 lines
Diff to previous 1.61 (colored)

Rename dbindex() to dbadd() to be less confusing.
The concept of an index file is gone since the switch to SQLite.
No functional change.

Revision 1.61 / (download) - annotate - [select for diffs], Mon Jan 6 03:52:05 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.60: +2 -4 lines
Diff to previous 1.60 (colored)

Remove the redundant "file" column from the "mlinks" table.
The contents can easily be reconstructed from sec, arch, name, form.
Shrinks the database by another 3% in standard mode and 9% in -Q mode.

Revision 1.60 / (download) - annotate - [select for diffs], Mon Jan 6 03:02:40 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.59: +2 -15 lines
Diff to previous 1.59 (colored)

Drop Nd from the mpages table, it is still in the keys table.
This shrinks the database in standard mode by 3%, in -Q mode by 9%,
without loss of functionality.

Revision 1.59 / (download) - annotate - [select for diffs], Sun Jan 5 20:26:27 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.58: +12 -8 lines
Diff to previous 1.58 (colored)

Add an option -Q (quick) to mandocdb(8)
for accelerated generation of reduced-size databases.

Implement this by allowing the parsers to optionally
abort the parse sequence after the NAME section.

While here, garbage collect the unused void *arg attribute
of struct mparse and mparse_alloc().

This reduces the processing time of mandocdb(8) on /usr/share/man
by a factor of 2 and the database size by a factor of 4.
However, it still takes 5 times the time and 6 times the space
of makewhatis(8), so more work is clearly needed.

Revision 1.58 / (download) - annotate - [select for diffs], Sun Jan 5 04:48:35 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.57: +7 -69 lines
Diff to previous 1.57 (colored)

Rip out the complete "reachable" checks, without replacement.
It's a pity i spent time during t2k13 writing this; however,
when an entire concept is busted, let us not look back.

There is no such thing as an unreachable page.  Even if you are crazy
enough to put a page starting with ".Dt NAMEI 9" into a file man1/cat.1,
we now make sure that it can be found by all of the following:
Nm=namei Nm=cat sec=1 sec=9
It will always be displayed as:
cat(1) - pathname lookup
So you know that you have to type `man cat` to get at it.
That obsoletes the concept of "unreachable manuals" for good.

Revision 1.57 / (download) - annotate - [select for diffs], Sun Jan 5 04:13:46 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.56: +2 -14 lines
Diff to previous 1.56 (colored)

Remove the obsolete file name column from the mpages table.
This column wasn't helpful because one manpage can have multiple MLINKS.
Use the file name column in the mlinks table, instead.

Revision 1.56 / (download) - annotate - [select for diffs], Sun Jan 5 03:25:51 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.55: +4 -8 lines
Diff to previous 1.55 (colored)

Remove the obsolete sec and arch columns from the mpages table.
They were confusing because a manpage can have MLINKS in different
sections and architectures.

Revision 1.55 / (download) - annotate - [select for diffs], Sun Jan 5 03:06:36 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.54: +5 -3 lines
Diff to previous 1.54 (colored)

Reimplement apropos -s NUM -S ARCH EXPR by internally converting it to
apropos \( EXPR \) -a 'sec~^NUM$' -a 'arch~^(ARCH|any)$'
in preparation for removal of sec and arch from the mpage table.

Almost no functional change except for the following bonus:
This also makes sure that for cross-section and cross-arch MLINKs,
all of the following work:
apropos -s 1 encrypt
apropos -s 8 encrypt
apropos -s 1 makekey
apropos -s 8 makekey
Before this commit, they don't, neither for the
old makewhatis(8)/apropos(1) combo nor for the new one.

While here, print error messages about invalid regexps to stderr.

Revision 1.54 / (download) - annotate - [select for diffs], Sun Jan 5 00:29:49 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.53: +10 -2 lines
Diff to previous 1.53 (colored)

Put section and architecture info into the keys table,
in preparation for removing them from the mpages table,
aiming for cleaner and more uniform interfaces.
Database growth is below 4%, part of which will be reclaimed.

As a bonus, this allows searches like:
./obj/apropos An=kettenis -a arch=ppc
./obj/apropos An=kettenis -a sec~[^4]

Revision 1.53 / (download) - annotate - [select for diffs], Thu Jan 2 22:44:07 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.52: +19 -19 lines
Diff to previous 1.52 (colored)

Avoid "utf8" in the names of a function and a struct member
that don't necessarily have anything to do with UTF-8.
Just renaming, no functional change.

Revision 1.52 / (download) - annotate - [select for diffs], Thu Jan 2 22:19:38 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.51: +35 -17 lines
Diff to previous 1.51 (colored)

Do not put UTF-8-encoded strings into the database by default, use ASCII.
Just like for mandoc(1), provide a -Tutf8 option for people who want that.

Revision 1.51 / (download) - annotate - [select for diffs], Thu Jan 2 20:24:35 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.50: +21 -27 lines
Diff to previous 1.50 (colored)

Polish the mlink_add() interface:
Allocate memory inside, not in the callers.
No functional change.

Revision 1.50 / (download) - annotate - [select for diffs], Thu Jan 2 18:51:51 2014 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.49: +60 -43 lines
Diff to previous 1.49 (colored)

Check all MLINKS for consistency with the content of the manual page,
not just the first one.  This doesn't change how the check is done,
but just which MLINKS are checked.

Revision 1.49 / (download) - annotate - [select for diffs], Tue Dec 31 19:39:09 2013 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.48: +6 -34 lines
Diff to previous 1.48 (colored)

Yet another regression introduced by Kristaps when he switched from
Berkeley DB to SQLite3: In the .In parser, the logic got inverted.
The resulting NULL pointer access was found by clang;
scan log provided by Ulrich Spoerlein <uqs at FreeBSD>.

The best fix is to simply remove the whole, pointless custom
handler function for .In and let the framework do its work.
Now searching for included header files actually works.

While here, remove the similarly pointless custom .St handler,
fix the return value of the .Fd handler and disentangle the
spaghetti in the .Nm handler.

Revision 1.48 / (download) - annotate - [select for diffs], Tue Dec 31 18:07:06 2013 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.47: +1 -2 lines
Diff to previous 1.47 (colored)

remove assignments that will be overwritten right afterwards,
and remove pointless local variables;
found in a clang output from Ulrich Spoerlein <uqs at FreeBSD>

Revision 1.47 / (download) - annotate - [select for diffs], Tue Dec 31 00:40:19 2013 UTC (10 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.46: +1652 -1532 lines
Diff to previous 1.46 (colored)

Replace the Berkeley-DB based mandocdb(8) by an SQLite3-based version,
aiming for more flexible development and optimization options.
Kristaps started this during the summer 2012, i did some very heavy
bugfixing during t2k13 and finally, during the last few days,
got it to a state where it is ripe for in-tree development.
Beware, neither the user interfaces nor the database formats
are expected to be stable just yet.

Will not be installed or activated until further discussion.
No functional change to mandoc(1).

"As long as it remains off until we decide the cost, fine." deraadt@

Revision 1.46 / (download) - annotate - [select for diffs], Thu Nov 21 01:47:10 2013 UTC (10 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.45: +2 -2 lines
Diff to previous 1.45 (colored)

My audit of mandoc revealed two missing (unsigned char) casts
in isspace() and tolower() arguments containing arbitrary char data.
Thanks to deraadt@ for triggering the audit.

Revision 1.45 / (download) - annotate - [select for diffs], Sat Oct 5 21:17:29 2013 UTC (10 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.44: +4 -2 lines
Diff to previous 1.44 (colored)

Cleanup suggested by gcc-4.8.1, following hints by Christos Zoulas:
- avoid bad qualifier casting in roff.c, roff_parsetext()
  by changing the mandoc_escape arguments to "const char const **"
- avoid bad qualifier casting in mandocdb.c, index_merge()
- garbage collect a few unused variables elsewhere

Revision 1.44 / (download) - annotate - [select for diffs], Wed Jun 5 02:02:53 2013 UTC (10 years, 11 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_4_BASE, OPENBSD_5_4
Changes since 1.43: +29 -29 lines
Diff to previous 1.43 (colored)

Some places used PATH_MAX from <limits.h>, some MAXPATHLEN from <sys/param.h>.
Consistently use the PATH_MAX since it is specified by POSIX,
while MAXPATHLEN is not.

Revision 1.43 / (download) - annotate - [select for diffs], Wed May 29 23:15:11 2013 UTC (10 years, 11 months ago) by schwarze
Branch: MAIN
Changes since 1.42: +5 -5 lines
Diff to previous 1.42 (colored)

Trivial sync to bsd.lv:
Kristaps changed the size member of struct manpaths from int to size_t.
No functional change.

Revision 1.42 / (download) - annotate - [select for diffs], Thu May 24 23:33:23 2012 UTC (11 years, 11 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_3_BASE, OPENBSD_5_3, OPENBSD_5_2_BASE, OPENBSD_5_2
Changes since 1.41: +2 -2 lines
Diff to previous 1.41 (colored)

Support -Ios='OpenBSD 5.1' to override uname(3) as the source of the
default value for the mdoc(7) .Os macro.
Needed for man.cgi on the OpenBSD website.

Problem with man.cgi first noticed by deraadt@;
beck@ and deraadt@ agree with the way to solve the issue.

Revision 1.41 / (download) - annotate - [select for diffs], Wed Apr 18 01:41:41 2012 UTC (12 years, 1 month ago) by deraadt
Branch: MAIN
Changes since 1.40: +137 -93 lines
Diff to previous 1.40 (colored)

backout non-portable code

Revision 1.40 / (download) - annotate - [select for diffs], Sun Apr 15 13:26:14 2012 UTC (12 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.39: +94 -138 lines
Diff to previous 1.39 (colored)

Clean up code a bit by pushing warnings into a WARNING macro; from kristaps@.

This also backs out the realpath(3) changes for now; according to
Kristaps, they caused regressions.

Revision 1.39 / (download) - annotate - [select for diffs], Sun Apr 15 11:14:38 2012 UTC (12 years, 1 month ago) by schwarze
Branch: MAIN
Changes since 1.38: +54 -6 lines
Diff to previous 1.38 (colored)

Handle multi-line NAME sections in man(7); from kristaps@.

Revision 1.38 / (download) - annotate - [select for diffs], Sun Jan 15 16:31:05 2012 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_1_BASE, OPENBSD_5_1
Changes since 1.37: +28 -22 lines
Diff to previous 1.37 (colored)

Two details improving the quality of the Xenocara mandoc database,
also helping a bit in ports land:
1) Do not skip manuals for section mismatches;
use the section where the file is actually located.
2) Always use the file name as an .Nm search key.

Revision 1.37 / (download) - annotate - [select for diffs], Mon Jan 9 23:21:47 2012 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.36: +70 -8 lines
Diff to previous 1.36 (colored)

Stop -W from complaining about MLINKs; only warn when a file
seems inaccessible by its main title.

This reduces the number of warnings in base to about 140,
which is already low enough to find a couple of actual problems
among the noise.

Revision 1.36 / (download) - annotate - [select for diffs], Mon Jan 9 01:59:08 2012 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.35: +2 -2 lines
Diff to previous 1.35 (colored)

Rename whatis.{db,index} back to mandoc.{db,index}
such that makewhatis(8) called from pkg_delete(1)
does not fail trying to open old-style whatis.db files as btree(3).
Suggested by espie@ deraadt@ millert@

Revision 1.35 / (download) - annotate - [select for diffs], Sat Jan 7 15:32:24 2012 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.34: +39 -16 lines
Diff to previous 1.34 (colored)

Improve handling of relative paths, letting invocations like the
following one reindex certain files in a local tree:
makewhatis -d mytree mytree/*/foo*.*

Revision 1.34 / (download) - annotate - [select for diffs], Thu Jan 5 22:48:52 2012 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.33: +72 -45 lines
Diff to previous 1.33 (colored)

Do not truncate the production database when starting to build a new one,
because that would break apropos(1) during the build, and if the build
fails, you would be left without any database at all.
Instead, build the database in temporary files in the same directory
and rename(2) them into place when they are ready.
Suggested by deraadt@.

There is still a tiny race between the two rename(2)s;
if that's a problem, we can solve it later using locking.
Put this in now such that we can move on.

Revision 1.33 / (download) - annotate - [select for diffs], Wed Dec 28 01:17:01 2011 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.32: +44 -16 lines
Diff to previous 1.32 (colored)

Some parsing improvements from kristaps@:
* Accept multiple lines in cat page NAME parsing.
* Do not trim the description to 70 characters in the database.
* Instead, limit the length of the description during output.
* In man(7), accept \-\- as a Nm-Nd separator; occurs in 3p.

Revision 1.32 / (download) - annotate - [select for diffs], Mon Dec 26 11:44:31 2011 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.31: +19 -4 lines
Diff to previous 1.31 (colored)

Do not skip manuals shared across architectures when building databases.

Revision 1.31 / (download) - annotate - [select for diffs], Sun Dec 25 19:57:20 2011 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.30: +8 -3 lines
Diff to previous 1.30 (colored)

Plug a memleak and constify some strings; from kristaps@.

Revision 1.30 / (download) - annotate - [select for diffs], Sun Dec 25 16:52:55 2011 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.29: +2 -2 lines
Diff to previous 1.29 (colored)

Use the traditional name "whatis.db" for the mandocdb(8) databases.
Requested by deraadt@, ok kristaps@.

CAUTION:
If you run "sudo mandocdb" after this, the old apropos(1) and whatis(1)
will not work any longer.  To get temporary copies of the new ones, run:
cd /usr/bin; rm -f apropos whatis; ln -s mandoc apropos; ln -s mandoc whatis

Revision 1.29 / (download) - annotate - [select for diffs], Sun Dec 25 14:51:33 2011 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.28: +9 -9 lines
Diff to previous 1.28 (colored)

For binary compatability of the databases across architectures,
use pointers to arrays, not pointers to structs.  It is now possible
to create databases on sparc64 and use them on i386 and vice versa.
Kristaps@ can't think of anything else that might be required, either.
Put this in now such that we can move on.

Revision 1.28 / (download) - annotate - [select for diffs], Sun Dec 25 13:05:29 2011 UTC (12 years, 4 months ago) by schwarze
Branch: MAIN
Changes since 1.27: +331 -227 lines
Diff to previous 1.27 (colored)

Implement test mode (makewhatis -t), required for pkg_create(8).
Always do all consistency checks; when any one fails, decide whether
to print a message, or skip the file, or both, or none.

While here, do some cleanup as well:
* Bail out on conflicting options.
* Do not crash with -a if there are plain files in the root dir.
* Collect some related variables into structs.

Feedback and OK kristaps@.

Revision 1.27 / (download) - annotate - [select for diffs], Tue Dec 20 00:41:24 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.26: +13 -5 lines
Diff to previous 1.26 (colored)

Do not cast void pointers to pointers requiring alignment.
This makes mandocdb(8)/apropos(1) work on strict alignment architectures.
Basic way to fix this confirmed by deraadt@ and kettenis@, thanks.

This now works on both sparc64 and i386, but note that the binary
database format is still machine-dependent.

Revision 1.26 / (download) - annotate - [select for diffs], Mon Dec 19 02:26:33 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.25: +39 -37 lines
Diff to previous 1.25 (colored)

Improvements from kristaps@:
(1) Make the database format simpler and smaller by
- storing the file type as a single character and
- storing paths relative to the dir containing the database.
The latter allows to move trees around.
Both together typically save 15-25% of the index size.
(2) Make sure apropos(1) "any" really covers all search keys.
(3) Make manpath_parseline() static, drop manpath_parseconf() completely.

Revision 1.25 / (download) - annotate - [select for diffs], Mon Dec 12 01:59:13 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.24: +12 -6 lines
Diff to previous 1.24 (colored)

implement -C (alternative config file) for apropos(1) and mandocdb(8);
ok kristaps@

Revision 1.24 / (download) - annotate - [select for diffs], Sat Dec 10 22:01:03 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.23: +25 -21 lines
Diff to previous 1.23 (colored)

Avoid leaking index records:
Before allocating a record for a file,
first make sure we actually want to use the file.

Revision 1.23 / (download) - annotate - [select for diffs], Sat Dec 10 16:53:38 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.22: +7 -3 lines
Diff to previous 1.22 (colored)

Fix selection of arch-specific manuals:
(1) Correctly compare cat vs. man paths.
(2) Compare arch (and section) names case-insensitively.
Problem noticed by kristaps@.

Revision 1.22 / (download) - annotate - [select for diffs], Fri Dec 9 01:47:11 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.21: +47 -27 lines
Diff to previous 1.21 (colored)

Improve parsing of preformatted manuals:
* If the first section is empty, reuse the file name as the description.
* Strip backspace encoding from the description.
* Make the loops more readable using string(3) functions.
* Put fclose() at the end, as line isn't valid afterwards.
From kristaps@, tweaked a bit by me.

Revision 1.21 / (download) - annotate - [select for diffs], Fri Dec 9 00:53:10 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.20: +12 -5 lines
Diff to previous 1.20 (colored)

In default mode, use realpath(3) on the "dir" arguments
and put canonicalized absolute pathnames into the database;
from kristaps@.

Revision 1.20 / (download) - annotate - [select for diffs], Thu Dec 8 02:14:00 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.19: +10 -10 lines
Diff to previous 1.19 (colored)

bugfix: make reclaiming of index slots actually work;
index_prune always counted the free slots,
but didn't tell anybody about them, so they weren't reused

Revision 1.19 / (download) - annotate - [select for diffs], Wed Dec 7 01:57:18 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.18: +252 -299 lines
Diff to previous 1.18 (colored)

Implement search support for 24 additional macros, extract more information
from Fn, and lift section restrictions from An Cd Er Ev Fn Fo In Pa St Va Vt
by removing 4 handler functions and 50 lines of code.
ok kristaps@

Revision 1.18 / (download) - annotate - [select for diffs], Sun Dec 4 14:11:55 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.17: +4 -5 lines
Diff to previous 1.17 (colored)

Fix parsing of file names given on the command line; i broke it
when adding support for formatted manual pages.

Revision 1.17 / (download) - annotate - [select for diffs], Sat Dec 3 14:53:12 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.16: +5 -6 lines
Diff to previous 1.16 (colored)

Make the mandocdb(8) format endian-neutral by storing integer data
in network byte order; from kristaps@.

Hopefully, this finishes the database format.
This commit requires another rebuild of your mandoc databases
by running "sudo mandocdb".

Revision 1.16 / (download) - annotate - [select for diffs], Thu Dec 1 23:22:09 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.15: +54 -26 lines
Diff to previous 1.15 (colored)

Fix mandocdb(8) to pass over the type when pruning the database.
This fixed `-d' perpetually adding the same files.
While here, remove -vv, clean up the code and document it.
From kristaps@.

Revision 1.15 / (download) - annotate - [select for diffs], Tue Nov 29 22:30:56 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.14: +13 -1 lines
Diff to previous 1.14 (colored)

Sync to bsd.lv, mostly from kristaps@.
Make `-i' only apply to regular expressions.
For substring matches, always use strcasestr().
Also, improve some manuals and comments.

Revision 1.14 / (download) - annotate - [select for diffs], Mon Nov 28 00:57:28 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.13: +6 -2 lines
Diff to previous 1.13 (colored)

Put back the chdir(2) to the right man page tree before parsing manuals;
this got lost during the bsd.lv sync in rev. 1.10; oops.
This is required for processing .so links: otherwise, several files
in Xenocara fail to parse and cause ugly complaints instead.

Revision 1.13 / (download) - annotate - [select for diffs], Sun Nov 27 23:27:16 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.12: +21 -27 lines
Diff to previous 1.12 (colored)

Reimplement the global command line options -a and -v
as static global variables, reducing the maze of arguments
passed around among various static functions.
Suggested by kristaps@.

Revision 1.12 / (download) - annotate - [select for diffs], Sun Nov 27 23:11:32 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.11: +2 -1 lines
Diff to previous 1.11 (colored)

Save the manual type (mdoc, man, or cat) in the index file
of the mandoc databases, as suggested by kristaps@.
Given the well-structured code, this is surprisingly simple.

This changes the mandoc.index database format.
Run "sudo mandocdb" to regenerate your databases.

Revision 1.11 / (download) - annotate - [select for diffs], Sun Nov 27 22:57:28 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.10: +206 -31 lines
Diff to previous 1.10 (colored)

Rudimentary handling of formatted manuals ("cat pages").
Coded on the train back from p2k11 in Budapest.
Kristaps has seen the patch and agreed with the direction.

Revision 1.10 / (download) - annotate - [select for diffs], Sat Nov 26 16:41:35 2011 UTC (12 years, 5 months ago) by schwarze
Branch: MAIN
Changes since 1.9: +31 -25 lines
Diff to previous 1.9 (colored)

Resync to bsd.lv, most code from kristaps@.
In particular, support the MANPATH environment variable
and don't confuse index record numbers when working with
multiple mandoc databases at once.
Besides, remove lots of gratuitious code and whitespace differences.

Revision 1.9 / (download) - annotate - [select for diffs], Thu Nov 17 15:38:27 2011 UTC (12 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.8: +14 -14 lines
Diff to previous 1.8 (colored)

Change the search type mask from 32 to 64 bit such that a full set
of search types can be supported in the future.  While here, replace
a couple of memcpy(3)s by proper, statically type-checked assignments,
and de-obfuscate some magical numbers using sizeof().

Revision 1.8 / (download) - annotate - [select for diffs], Thu Nov 17 15:02:29 2011 UTC (12 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.7: +28 -17 lines
Diff to previous 1.7 (colored)

Fix a pasto that prevented detection of mandoc.index dbopen(3) failure.
And as suggested by kristaps@, improve a lot of comments.

Revision 1.7 / (download) - annotate - [select for diffs], Thu Nov 17 14:52:32 2011 UTC (12 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.6: +22 -2 lines
Diff to previous 1.6 (colored)

Minimal man.conf(5) _whatdb support for mandocdb(8) and apropos(1),
and add the standard options -M and -m to apropos(1).
Including a bugfix to prevent mandocdb(8) from leaking
database file descriptors.

Revision 1.6 / (download) - annotate - [select for diffs], Mon Nov 14 18:52:05 2011 UTC (12 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.5: +173 -33 lines
Diff to previous 1.5 (colored)

Store page titles in the correct case, and by default, only
put stuff into the database that man(1) will be able to retrieve.
However, support an option to use all directories and files.

Kristaps@ agreed with the general direction and provided some feedback.

Revision 1.5 / (download) - annotate - [select for diffs], Sun Nov 13 10:40:52 2011 UTC (12 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.4: +19 -36 lines
Diff to previous 1.4 (colored)

Inventing new keywords for mostly the same thing when a well-established
set of keywords already exists is a bad idea, so reuse the mdoc(7)
macro names as apropos(1) search types.  This is a gain in brevity
as well.  Some time ago, kristaps@ agreed in principle.

The search type bit field constants are used by both mandocdb(8) and
apropos(1) and should better stay in sync, so give them their own
header file.

Revision 1.4 / (download) - annotate - [select for diffs], Sun Nov 13 00:53:07 2011 UTC (12 years, 6 months ago) by schwarze
Branch: MAIN
Changes since 1.3: +5 -5 lines
Diff to previous 1.3 (colored)

Fix two crashes that occur when walking very large (i.e. real-world) trees:
1) Avoid excessive, needless recursion lest you overflow the stack;
2) Close dir file descriptors, lest you run out of descriptors.
ok kristaps@

Revision 1.3 / (download) - annotate - [select for diffs], Thu Oct 6 23:04:16 2011 UTC (12 years, 7 months ago) by schwarze
Branch: MAIN
Changes since 1.2: +2 -2 lines
Diff to previous 1.2 (colored)

1) Import the future apropos(1) replacement written by kristaps@.
2) Link both that one and mandocdb(8) into the mandoc(1) binary.
3) Install a /usr/bin/mandocdb hardlink and the mandocdb(8) manual.

Do not replace /usr/bin/apropos by a hardlink yet because it is
not ready for production, and ports integration is still missing.
Development will be done in the tree, even the user interfaces
are still subject to change at this point.

Both jmc@ and deraadt@ agree with getting this in.

Revision 1.2 / (download) - annotate - [select for diffs], Sat Sep 17 13:45:28 2011 UTC (12 years, 8 months ago) by schwarze
Branch: MAIN
Changes since 1.1: +388 -190 lines
Diff to previous 1.1 (colored)

Change the mandocdb(8) interface to better agree with makewhatis(8);
in particular, allow recursing multiple directories and create
multiple databases in one call.
This commit includes some reorganization, and exposing mandoc_strdup
as a utility function in mandoc.h.
written by kristaps@

Revision 1.1 / (download) - annotate - [select for diffs], Thu Jul 14 15:10:54 2011 UTC (12 years, 10 months ago) by schwarze
Branch: MAIN
CVS Tags: OPENBSD_5_0_BASE, OPENBSD_5_0

Import mandocdb(8) from bsd.lv, coded by kristaps@, for in-tree development.
Will need adjustments to user interface and functionality, and will be
linked into the mandoc(1) binary and hardlinked from /usr/libexec/mandocdb
to /usr/bin/mandoc when ready for general consumption.
The ultimate goal is to call it from OpenBSD::Makewhatis to replace
OpenBSD::Makewhatis::Unformated with enhanced functionality.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.