Annotation of src/usr.bin/lex/ONEWS, Revision 1.1
1.1 ! tedu 1: Changes between release 2.5.4 (11Sep96) and release 2.5.3:
! 2:
! 3: - Fixed a bug introduced in 2.5.3 that blew it when a call
! 4: to input() occurred at the end of an input file.
! 5:
! 6: - Fixed scanner skeleton so the example in the man page of
! 7: scanning strings using exclusive start conditions works.
! 8:
! 9: - Minor Makefile tweaks.
! 10:
! 11:
! 12: Changes between release 2.5.3 (29May96) and release 2.5.2:
! 13:
! 14: - Some serious bugs in yymore() have been fixed. In particular,
! 15: when using AT&T-lex-compatibility or %array, you can intermix
! 16: calls to input(), unput(), and yymore(). (This still doesn't
! 17: work for %pointer, and isn't likely to in the future.)
! 18:
! 19: - A bug in handling NUL's in the input stream of scanners using
! 20: REJECT has been fixed.
! 21:
! 22: - The default main() in libfl.a now repeatedly calls yylex() until
! 23: it returns 0, rather than just calling it once.
! 24:
! 25: - Minor tweak for Windows NT Makefile, MISC/NT/Makefile.
! 26:
! 27:
! 28: Changes between release 2.5.2 (25Apr95) and release 2.5.1:
! 29:
! 30: - The --prefix configuration option now works.
! 31:
! 32: - A bug that completely broke the "-Cf" table compression
! 33: option has been fixed.
! 34:
! 35: - A major headache involving "const" declarators and Solaris
! 36: systems has been fixed.
! 37:
! 38: - An octal escape sequence in a flex regular expression must
! 39: now contain only the digits 0-7.
! 40:
! 41: - You can now use "--" on the flex command line to mark the
! 42: end of flex options.
! 43:
! 44: - You can now specify the filename '-' as a synonym for stdin.
! 45:
! 46: - By default, the scanners generated by flex no longer
! 47: statically initialize yyin and yyout to stdin and stdout.
! 48: This change is necessary because in some ANSI environments,
! 49: stdin and stdout are not compile-time constant. You can
! 50: force the initialization using "%option stdinit" in the first
! 51: section of your flex input.
! 52:
! 53: - "%option nounput" now correctly omits the unput() routine
! 54: from the output.
! 55:
! 56: - "make clean" now removes config.log, config.cache, and the
! 57: flex binary. The fact that it removes the flex binary means
! 58: you should take care if making changes to scan.l, to make
! 59: sure you don't wind up in a bootstrap problem.
! 60:
! 61: - In general, the Makefile has been reworked somewhat (thanks
! 62: to Francois Pinard) for added flexibility - more changes will
! 63: follow in subsequent releases.
! 64:
! 65: - The .texi and .info files in MISC/texinfo/ have been updated,
! 66: thanks also to Francois Pinard.
! 67:
! 68: - The FlexLexer::yylex(istream* new_in, ostream* new_out) method
! 69: now does not have a default for the first argument, to disambiguate
! 70: it from FlexLexer::yylex().
! 71:
! 72: - A bug in destructing a FlexLexer object before doing any scanning
! 73: with it has been fixed.
! 74:
! 75: - A problem with including FlexLexer.h multiple times has been fixed.
! 76:
! 77: - The alloca() chud necessary to accommodate bison has grown
! 78: even uglier, but hopefully more correct.
! 79:
! 80: - A portability tweak has been added to accommodate compilers that
! 81: use char* generic pointers.
! 82:
! 83: - EBCDIC contact information in the file MISC/EBCDIC has been updated.
! 84:
! 85: - An OS/2 Makefile and config.h for flex 2.5 is now available in
! 86: MISC/OS2/, contributed by Kai Uwe Rommel.
! 87:
! 88: - The descrip.mms file for building flex under VMS has been updated,
! 89: thanks to Pat Rankin.
! 90:
! 91: - The notes on building flex for the Amiga have been updated for
! 92: flex 2.5, contributed by Andreas Scherer.
! 93:
! 94:
! 95: Changes between release 2.5.1 (28Mar95) and release 2.4.7:
! 96:
! 97: - A new concept of "start condition" scope has been introduced.
! 98: A start condition scope is begun with:
! 99:
! 100: <SCs>{
! 101:
! 102: where SCs is a list of one or more start conditions. Inside
! 103: the start condition scope, every rule automatically has the
! 104: prefix <SCs> applied to it, until a '}' which matches the
! 105: initial '{'. So, for example:
! 106:
! 107: <ESC>{
! 108: "\\n" return '\n';
! 109: "\\r" return '\r';
! 110: "\\f" return '\f';
! 111: "\\0" return '\0';
! 112: }
! 113:
! 114: is equivalent to:
! 115:
! 116: <ESC>"\\n" return '\n';
! 117: <ESC>"\\r" return '\r';
! 118: <ESC>"\\f" return '\f';
! 119: <ESC>"\\0" return '\0';
! 120:
! 121: As indicated in this example, rules inside start condition scopes
! 122: (and any rule, actually, other than the first) can be indented,
! 123: to better show the extent of the scope.
! 124:
! 125: Start condition scopes may be nested.
! 126:
! 127: - The new %option directive can be used in the first section of
! 128: a flex scanner to control scanner-generation options. Most
! 129: options are given simply as names, optionally preceded by the
! 130: word "no" (with no intervening whitespace) to negate their
! 131: meaning. Some are equivalent to flex flags, so putting them
! 132: in your scanner source is equivalent to always specifying
! 133: the flag (%option's take precedence over flags):
! 134:
! 135: 7bit -7 option
! 136: 8bit -8 option
! 137: align -Ca option
! 138: backup -b option
! 139: batch -B option
! 140: c++ -+ option
! 141: caseful opposite of -i option (caseful is the default);
! 142: case-sensitive same as above
! 143: caseless -i option;
! 144: case-insensitive same as above
! 145: debug -d option
! 146: default opposite of -s option
! 147: ecs -Ce option
! 148: fast -F option
! 149: full -f option
! 150: interactive -I option
! 151: lex-compat -l option
! 152: meta-ecs -Cm option
! 153: perf-report -p option
! 154: read -Cr option
! 155: stdout -t option
! 156: verbose -v option
! 157: warn opposite of -w option (so use "%option nowarn" for -w)
! 158:
! 159: array equivalent to "%array"
! 160: pointer equivalent to "%pointer" (default)
! 161:
! 162: Some provide new features:
! 163:
! 164: always-interactive generate a scanner which always
! 165: considers its input "interactive" (no call to isatty()
! 166: will be made when the scanner runs)
! 167: main supply a main program for the scanner, which
! 168: simply calls yylex(). Implies %option noyywrap.
! 169: never-interactive generate a scanner which never
! 170: considers its input "interactive" (no call to isatty()
! 171: will be made when the scanner runs)
! 172: stack if set, enable start condition stacks (see below)
! 173: stdinit if unset ("%option nostdinit"), initialize yyin
! 174: and yyout statically to nil FILE* pointers, instead
! 175: of stdin and stdout
! 176: yylineno if set, keep track of the current line
! 177: number in global yylineno (this option is expensive
! 178: in terms of performance). The line number is available
! 179: to C++ scanning objects via the new member function
! 180: lineno().
! 181: yywrap if unset ("%option noyywrap"), scanner does not
! 182: call yywrap() upon EOF but simply assumes there
! 183: are no more files to scan
! 184:
! 185: Flex scans your rule actions to determine whether you use the
! 186: REJECT or yymore features (this is not new). Two %options can be
! 187: used to override its decision, either by setting them to indicate
! 188: the feature is indeed used, or unsetting them to indicate it
! 189: actually is not used:
! 190:
! 191: reject
! 192: yymore
! 193:
! 194: Three %option's take string-delimited values, offset with '=':
! 195:
! 196: outfile="<name>" equivalent to -o<name>
! 197: prefix="<name>" equivalent to -P<name>
! 198: yyclass="<name>" set the name of the C++ scanning class
! 199: (see below)
! 200:
! 201: A number of %option's are available for lint purists who
! 202: want to suppress the appearance of unneeded routines in
! 203: the generated scanner. Each of the following, if unset,
! 204: results in the corresponding routine not appearing in the
! 205: generated scanner:
! 206:
! 207: input, unput
! 208: yy_push_state, yy_pop_state, yy_top_state
! 209: yy_scan_buffer, yy_scan_bytes, yy_scan_string
! 210:
! 211: You can specify multiple options with a single %option directive,
! 212: and multiple directives in the first section of your flex input file.
! 213:
! 214: - The new function:
! 215:
! 216: YY_BUFFER_STATE yy_scan_string( const char *str )
! 217:
! 218: returns a YY_BUFFER_STATE (which also becomes the current input
! 219: buffer) for scanning the given string, which occurs starting
! 220: with the next call to yylex(). The string must be NUL-terminated.
! 221: A related function:
! 222:
! 223: YY_BUFFER_STATE yy_scan_bytes( const char *bytes, int len )
! 224:
! 225: creates a buffer for scanning "len" bytes (including possibly NUL's)
! 226: starting at location "bytes".
! 227:
! 228: Note that both of these functions create and scan a *copy* of
! 229: the string/bytes. (This may be desirable, since yylex() modifies
! 230: the contents of the buffer it is scanning.) You can avoid the
! 231: copy by using:
! 232:
! 233: YY_BUFFER_STATE yy_scan_buffer( char *base, yy_size_t size )
! 234:
! 235: which scans in place the buffer starting at "base", consisting
! 236: of "size" bytes, the last two bytes of which *must* be
! 237: YY_END_OF_BUFFER_CHAR (these bytes are not scanned; thus, scanning
! 238: consists of base[0] through base[size-2], inclusive). If you
! 239: fail to set up "base" in this manner, yy_scan_buffer returns a
! 240: nil pointer instead of creating a new input buffer.
! 241:
! 242: The type yy_size_t is an integral type to which you can cast
! 243: an integer expression reflecting the size of the buffer.
! 244:
! 245: - Three new routines are available for manipulating stacks of
! 246: start conditions:
! 247:
! 248: void yy_push_state( int new_state )
! 249:
! 250: pushes the current start condition onto the top of the stack
! 251: and BEGIN's "new_state" (recall that start condition names are
! 252: also integers).
! 253:
! 254: void yy_pop_state()
! 255:
! 256: pops the top of the stack and BEGIN's to it, and
! 257:
! 258: int yy_top_state()
! 259:
! 260: returns the top of the stack without altering the stack's
! 261: contents.
! 262:
! 263: The start condition stack grows dynamically and so has no built-in
! 264: size limitation. If memory is exhausted, program execution
! 265: is aborted.
! 266:
! 267: To use start condition stacks, your scanner must include
! 268: a "%option stack" directive.
! 269:
! 270: - flex now supports POSIX character class expressions. These
! 271: are expressions enclosed inside "[:" and ":]" delimiters (which
! 272: themselves must appear between the '[' and ']' of a character
! 273: class; other elements may occur inside the character class, too).
! 274: The expressions flex recognizes are:
! 275:
! 276: [:alnum:] [:alpha:] [:blank:] [:cntrl:] [:digit:] [:graph:]
! 277: [:lower:] [:print:] [:punct:] [:space:] [:upper:] [:xdigit:]
! 278:
! 279: These expressions all designate a set of characters equivalent to
! 280: the corresponding isXXX function (for example, [:alnum:] designates
! 281: those characters for which isalnum() returns true - i.e., any
! 282: alphabetic or numeric). Some systems don't provide isblank(),
! 283: so flex defines [:blank:] as a blank or a tab.
! 284:
! 285: For example, the following character classes are all equivalent:
! 286:
! 287: [[:alnum:]]
! 288: [[:alpha:][:digit:]
! 289: [[:alpha:]0-9]
! 290: [a-zA-Z0-9]
! 291:
! 292: If your scanner is case-insensitive (-i flag), then [:upper:]
! 293: and [:lower:] are equivalent to [:alpha:].
! 294:
! 295: - The promised rewrite of the C++ FlexLexer class has not yet
! 296: been done. Support for FlexLexer is limited at the moment to
! 297: fixing show-stopper bugs, so, for example, the new functions
! 298: yy_scan_string() & friends are not available to FlexLexer
! 299: objects.
! 300:
! 301: - The new macro
! 302:
! 303: yy_set_interactive(is_interactive)
! 304:
! 305: can be used to control whether the current buffer is considered
! 306: "interactive". An interactive buffer is processed more slowly,
! 307: but must be used when the scanner's input source is indeed
! 308: interactive to avoid problems due to waiting to fill buffers
! 309: (see the discussion of the -I flag in flex.1). A non-zero value
! 310: in the macro invocation marks the buffer as interactive, a zero
! 311: value as non-interactive. Note that use of this macro overrides
! 312: "%option always-interactive" or "%option never-interactive".
! 313:
! 314: yy_set_interactive() must be invoked prior to beginning to
! 315: scan the buffer.
! 316:
! 317: - The new macro
! 318:
! 319: yy_set_bol(at_bol)
! 320:
! 321: can be used to control whether the current buffer's scanning
! 322: context for the next token match is done as though at the
! 323: beginning of a line (non-zero macro argument; makes '^' anchored
! 324: rules active) or not at the beginning of a line (zero argument,
! 325: '^' rules inactive).
! 326:
! 327: - Related to this change, the mechanism for determining when a scan is
! 328: starting at the beginning of a line has changed. It used to be
! 329: that '^' was active iff the character prior to that at which the
! 330: scan started was a newline. The mechanism now is that '^' is
! 331: active iff the last token ended in a newline (or the last call to
! 332: input() returned a newline). For most users, the difference in
! 333: mechanisms is negligible. Where it will make a difference,
! 334: however, is if unput() or yyless() is used to alter the input
! 335: stream. When in doubt, use yy_set_bol().
! 336:
! 337: - The new beginning-of-line mechanism involved changing some fairly
! 338: twisted code, so it may have introduced bugs - beware ...
! 339:
! 340: - The macro YY_AT_BOL() returns true if the next token scanned from
! 341: the current buffer will have '^' rules active, false otherwise.
! 342:
! 343: - The new function
! 344:
! 345: void yy_flush_buffer( struct yy_buffer_state* b )
! 346:
! 347: flushes the contents of the current buffer (i.e., next time
! 348: the scanner attempts to match a token using b as the current
! 349: buffer, it will begin by invoking YY_INPUT to fill the buffer).
! 350: This routine is also available to C++ scanners (unlike some
! 351: of the other new routines).
! 352:
! 353: The related macro
! 354:
! 355: YY_FLUSH_BUFFER
! 356:
! 357: flushes the contents of the current buffer.
! 358:
! 359: - A new "-ooutput" option writes the generated scanner to "output".
! 360: If used with -t, the scanner is still written to stdout, but
! 361: its internal #line directives (see previous item) use "output".
! 362:
! 363: - Flex now generates #line directives relating the code it
! 364: produces to the output file; this means that error messages
! 365: in the flex-generated code should be correctly pinpointed.
! 366:
! 367: - When generating #line directives, filenames with embedded '\'s
! 368: have those characters escaped (i.e., turned into '\\'). This
! 369: feature helps with reporting filenames for some MS-DOS and OS/2
! 370: systems.
! 371:
! 372: - The FlexLexer class includes two new public member functions:
! 373:
! 374: virtual void switch_streams( istream* new_in = 0,
! 375: ostream* new_out = 0 )
! 376:
! 377: reassigns yyin to new_in (if non-nil) and yyout to new_out
! 378: (ditto), deleting the previous input buffer if yyin is
! 379: reassigned. It is used by:
! 380:
! 381: int yylex( istream* new_in = 0, ostream* new_out = 0 )
! 382:
! 383: which first calls switch_streams() and then returns the value
! 384: of calling yylex().
! 385:
! 386: - C++ scanners now have yy_flex_debug as a member variable of
! 387: FlexLexer rather than a global, and member functions for testing
! 388: and setting it.
! 389:
! 390: - When generating a C++ scanning class, you can now use
! 391:
! 392: %option yyclass="foo"
! 393:
! 394: to inform flex that you have derived "foo" as a subclass of
! 395: yyFlexLexer, so flex will place your actions in the member
! 396: function foo::yylex() instead of yyFlexLexer::yylex(). It also
! 397: generates a yyFlexLexer::yylex() member function that generates a
! 398: run-time error if called (by invoking yyFlexLexer::LexerError()).
! 399: This feature is necessary if your subclass "foo" introduces some
! 400: additional member functions or variables that you need to access
! 401: from yylex().
! 402:
! 403: - Current texinfo files in MISC/texinfo, contributed by Francois
! 404: Pinard.
! 405:
! 406: - You can now change the name "flex" to something else (e.g., "lex")
! 407: by redefining $(FLEX) in the Makefile.
! 408:
! 409: - Two bugs (one serious) that could cause "bigcheck" to fail have
! 410: been fixed.
! 411:
! 412: - A number of portability/configuration changes have been made
! 413: for easier portability.
! 414:
! 415: - You can use "YYSTATE" in your scanner as an alias for YY_START
! 416: (for AT&T lex compatibility).
! 417:
! 418: - input() now maintains yylineno.
! 419:
! 420: - input() no longer trashes yytext.
! 421:
! 422: - interactive scanners now read characters in YY_INPUT up to a
! 423: newline, a large performance gain.
! 424:
! 425: - C++ scanner objects now work with the -P option. You include
! 426: <FlexLexer.h> once per scanner - see comments in <FlexLexer.h>
! 427: (or flex.1) for details.
! 428:
! 429: - C++ FlexLexer objects now use the "cerr" stream to report -d output
! 430: instead of stdio.
! 431:
! 432: - The -c flag now has its full glorious POSIX interpretation (do
! 433: nothing), rather than being interpreted as an old-style -C flag.
! 434:
! 435: - Scanners generated by flex now include two #define's giving
! 436: the major and minor version numbers (YY_FLEX_MAJOR_VERSION,
! 437: YY_FLEX_MINOR_VERSION). These can then be tested to see
! 438: whether certain flex features are available.
! 439:
! 440: - Scanners generated using -l lex compatibility now have the symbol
! 441: YY_FLEX_LEX_COMPAT #define'd.
! 442:
! 443: - When initializing (i.e., yy_init is non-zero on entry to yylex()),
! 444: generated scanners now set yy_init to zero before executing
! 445: YY_USER_INIT. This means that you can set yy_init back to a
! 446: non-zero value in YY_USER_INIT if you need the scanner to be
! 447: reinitialized on the next call.
! 448:
! 449: - You can now use "#line" directives in the first section of your
! 450: scanner specification.
! 451:
! 452: - When generating full-table scanners (-Cf), flex now puts braces
! 453: around each row of the 2-d array initialization, to silence warnings
! 454: on over-zealous compilers.
! 455:
! 456: - Improved support for MS-DOS. The flex sources have been successfully
! 457: built, unmodified, for Borland 4.02 (all that's required is a
! 458: Borland Makefile and config.h file, which are supplied in
! 459: MISC/Borland - contributed by Terrence O Kane).
! 460:
! 461: - Improved support for Macintosh using Think C - the sources should
! 462: build for this platform "out of the box". Contributed by Scott
! 463: Hofmann.
! 464:
! 465: - Improved support for VMS, in MISC/VMS/, contributed by Pat Rankin.
! 466:
! 467: - Support for the Amiga, in MISC/Amiga/, contributed by Andreas
! 468: Scherer. Note that the contributed files were developed for
! 469: flex 2.4 and have not been tested with flex 2.5.
! 470:
! 471: - Some notes on support for the NeXT, in MISC/NeXT, contributed
! 472: by Raf Schietekat.
! 473:
! 474: - The MISC/ directory now includes a preformatted version of flex.1
! 475: in flex.man, and pre-yacc'd versions of parse.y in parse.{c,h}.
! 476:
! 477: - The flex.1 and flexdoc.1 manual pages have been merged. There
! 478: is now just one document, flex.1, which includes an overview
! 479: at the beginning to help you find the section you need.
! 480:
! 481: - Documentation now clarifies that start conditions persist across
! 482: switches to new input files or different input buffers. If you
! 483: want to e.g., return to INITIAL, you must explicitly do so.
! 484:
! 485: - The "Performance Considerations" section of the manual has been
! 486: updated.
! 487:
! 488: - Documented the "yy_act" variable, which when YY_USER_ACTION is
! 489: invoked holds the number of the matched rule, and added an
! 490: example of using yy_act to profile how often each rule is matched.
! 491:
! 492: - Added YY_NUM_RULES, a definition that gives the total number
! 493: of rules in the file, including the default rule (even if you
! 494: use -s).
! 495:
! 496: - Documentation now clarifies that you can pass a nil FILE* pointer
! 497: to yy_create_buffer() or yyrestart() if you've arrange YY_INPUT
! 498: to not need yyin.
! 499:
! 500: - Documentation now clarifies that YY_BUFFER_STATE is a pointer to
! 501: an opaque "struct yy_buffer_state".
! 502:
! 503: - Documentation now stresses that you gain the benefits of removing
! 504: backing-up states only if you remove *all* of them.
! 505:
! 506: - Documentation now points out that traditional lex allows you
! 507: to put the action on a separate line from the rule pattern if
! 508: the pattern has trailing whitespace (ugh!), but flex doesn't
! 509: support this.
! 510:
! 511: - A broken example in documentation of the difference between
! 512: inclusive and exclusive start conditions is now fixed.
! 513:
! 514: - Usage (-h) report now goes to stdout.
! 515:
! 516: - Version (-V) info now goes to stdout.
! 517:
! 518: - More #ifdef chud has been added to the parser in attempt to
! 519: deal with bison's use of alloca().
! 520:
! 521: - "make clean" no longer deletes emacs backup files (*~).
! 522:
! 523: - Some memory leaks have been fixed.
! 524:
! 525: - A bug was fixed in which dynamically-expanded buffers were
! 526: reallocated a couple of bytes too small.
! 527:
! 528: - A bug was fixed which could cause flex to read and write beyond
! 529: the end of the input buffer.
! 530:
! 531: - -S will not be going away.
! 532:
! 533:
! 534: Changes between release 2.4.7 (03Aug94) and release 2.4.6:
! 535:
! 536: - Fixed serious bug in reading multiple files.
! 537:
! 538: - Fixed bug in scanning NUL's.
! 539:
! 540: - Fixed bug in input() returning 8-bit characters.
! 541:
! 542: - Fixed bug in matching text with embedded NUL's when
! 543: using %array or lex compatibility.
! 544:
! 545: - Fixed multiple invocations of YY_USER_ACTION when using '|'
! 546: continuation action.
! 547:
! 548: - Minor prototyping fixes.
! 549:
! 550: Changes between release 2.4.6 (04Jan94) and release 2.4.5:
! 551:
! 552: - Linking with -lfl no longer required if your program includes
! 553: its own yywrap() and main() functions. (This change will cause
! 554: problems if you have a non-ANSI compiler on a system for which
! 555: sizeof(int) != sizeof(void*) or sizeof(int) != sizeof(size_t).)
! 556:
! 557: - The use of 'extern "C++"' in FlexLexer.h has been modified to
! 558: get around an incompatibility with g++'s header files.
! 559:
! 560: Changes between release 2.4.5 (11Dec93) and release 2.4.4:
! 561:
! 562: - Fixed bug breaking C++ scanners that use REJECT or variable
! 563: trailing context.
! 564:
! 565: - Fixed serious input problem for interactive scanners on
! 566: systems for which char is unsigned.
! 567:
! 568: - Fixed bug in incorrectly treating '$' operator as variable
! 569: trailing context.
! 570:
! 571: - Fixed bug in -CF table representation that could lead to
! 572: corrupt tables.
! 573:
! 574: - Fixed fairly benign memory leak.
! 575:
! 576: - Added `extern "C++"' wrapper to FlexLexer.h header. This
! 577: should overcome the g++ 2.5.X problems mentioned in the
! 578: NEWS for release 2.4.3.
! 579:
! 580: - Changed #include of FlexLexer.h to use <> instead of "".
! 581:
! 582: - Added feature to control whether the scanner attempts to
! 583: refill the input buffer once it's exhausted. This feature
! 584: will be documented in the 2.5 release.
! 585:
! 586:
! 587: Changes between release 2.4.4 (07Dec93) and release 2.4.3:
! 588:
! 589: - Fixed two serious bugs in scanning 8-bit characters.
! 590:
! 591: - Fixed bug in YY_USER_ACTION that caused it to be executed
! 592: inappropriately (on the scanner's own internal actions, and
! 593: with incorrect yytext/yyleng values).
! 594:
! 595: - Fixed bug in pointing yyin at a new file and resuming scanning.
! 596:
! 597: - Portability fix regarding min/max/abs macros conflicting with
! 598: function definitions in standard header files.
! 599:
! 600: - Added a virtual LexerError() method to the C++ yyFlexLexer class
! 601: for reporting error messages instead of always using cerr.
! 602:
! 603: - Added warning in flexdoc that the C++ scanning class is presently
! 604: experimental and subject to considerable change between major
! 605: releases.
! 606:
! 607:
! 608: Changes between release 2.4.3 (03Dec93) and release 2.4.2:
! 609:
! 610: - Fixed bug causing fatal scanner messages to fail to print.
! 611:
! 612: - Fixed things so FlexLexer.h can be included in other C++
! 613: sources. One side-effect of this change is that -+ and -CF
! 614: are now incompatible.
! 615:
! 616: - libfl.a now supplies private versions of the the <string.h>/
! 617: <strings.h> string routines needed by flex and the scanners
! 618: it generates, to enhance portability to some BSD systems.
! 619:
! 620: - More robust solution to 2.4.2's flexfatal() bug fix.
! 621:
! 622: - Added ranlib of installed libfl.a.
! 623:
! 624: - Some lint tweaks.
! 625:
! 626: - NOTE: problems have been encountered attempting to build flex
! 627: C++ scanners using g++ version 2.5.X. The problem is due to an
! 628: unfortunate heuristic in g++ 2.5.X that attempts to discern between
! 629: C and C++ headers. Because FlexLexer.h is installed (by default)
! 630: in /usr/local/include and not /usr/local/lib/g++-include, g++ 2.5.X
! 631: decides that it's a C header :-(. So if you have problems, install
! 632: the header in /usr/local/lib/g++-include instead.
! 633:
! 634:
! 635: Changes between release 2.4.2 (01Dec93) and release 2.4.1:
! 636:
! 637: - Fixed bug in libfl.a referring to non-existent "flexfatal" function.
! 638:
! 639: - Modified to produce both compress'd and gzip'd tar files for
! 640: distributions (you probably don't care about this change!).
! 641:
! 642:
! 643: Changes between release 2.4.1 (30Nov93) and release 2.3.8:
! 644:
! 645: - The new '-+' flag instructs flex to generate a C++ scanner class
! 646: (thanks to Kent Williams). flex writes an implementation of the
! 647: class defined in FlexLexer.h to lex.yy.cc. You may include
! 648: multiple scanner classes in your program using the -P flag. Note
! 649: that the scanner class also provides a mechanism for creating
! 650: reentrant scanners. The scanner class uses C++ streams for I/O
! 651: instead of FILE*'s (thanks to Tom Epperly). If the flex executable's
! 652: name ends in '+' then the '-+' flag is automatically on, so creating
! 653: a symlink or copy of "flex" to "flex++" results in a version of
! 654: flex that can be used exclusively for C++ scanners.
! 655:
! 656: Note that without the '-+' flag, flex-generated scanners can still
! 657: be compiled using C++ compilers, though they use FILE*'s for I/O
! 658: instead of streams.
! 659:
! 660: See the "GENERATING C++ SCANNERS" section of flexdoc for details.
! 661:
! 662: - The new '-l' flag turns on maximum AT&T lex compatibility. In
! 663: particular, -l includes support for "yylineno" and makes yytext
! 664: be an array instead of a pointer. It does not, however, do away
! 665: with all incompatibilities. See the "INCOMPATIBILITIES WITH LEX
! 666: AND POSIX" section of flexdoc for details.
! 667:
! 668: - The new '-P' option specifies a prefix to use other than "yy"
! 669: for the scanner's globally-visible variables, and for the
! 670: "lex.yy.c" filename. Using -P you can link together multiple
! 671: flex scanners in the same executable.
! 672:
! 673: - The distribution includes a "texinfo" version of flexdoc.1,
! 674: contributed by Roland Pesch (thanks also to Marq Kole, who
! 675: contributed another version). It has not been brought up to
! 676: date, but reflects version 2.3. See MISC/flex.texinfo.
! 677:
! 678: The flex distribution will soon include G.T. Nicol's flex
! 679: manual; he is presently bringing it up-to-date for version 2.4.
! 680:
! 681: - yywrap() is now a function, and you now *must* link flex scanners
! 682: with libfl.a.
! 683:
! 684: - Site-configuration is now done via an autoconf-generated
! 685: "configure" script contributed by Francois Pinard.
! 686:
! 687: - Scanners now use fread() (or getc(), if interactive) and not
! 688: read() for input. A new "table compression" option, -Cr,
! 689: overrides this change and causes the scanner to use read()
! 690: (because read() is a bit faster than fread()). -f and -F
! 691: are now equivalent to -Cfr and -CFr; i.e., they imply the
! 692: -Cr option.
! 693:
! 694: - In the blessed name of POSIX compliance, flex supports "%array"
! 695: and "%pointer" directives in the definitions (first) section of
! 696: the scanner specification. The former specifies that yytext
! 697: should be an array (of size YYLMAX), the latter, that it should
! 698: be a pointer. The array version of yytext is universally slower
! 699: than the pointer version, but has the advantage that its contents
! 700: remain unmodified across calls to input() and unput() (the pointer
! 701: version of yytext is, still, trashed by such calls).
! 702:
! 703: "%array" cannot be used with the '-+' C++ scanner class option.
! 704:
! 705: - The new '-Ca' option directs flex to trade off memory for
! 706: natural alignment when generating a scanner's tables. In
! 707: particular, table entries that would otherwise be "short"
! 708: become "long".
! 709:
! 710: - The new '-h' option produces a summary of the flex flags.
! 711:
! 712: - The new '-V' option reports the flex version number and exits.
! 713:
! 714: - The new scanner macro YY_START returns an integer value
! 715: corresponding to the current start condition. You can return
! 716: to that start condition by passing the value to a subsequent
! 717: "BEGIN" action. You also can implement "start condition stacks"
! 718: by storing the values in an integer stack.
! 719:
! 720: - You can now redefine macros such as YY_INPUT by just #define'ing
! 721: them to some other value in the first section of the flex input;
! 722: no need to first #undef them.
! 723:
! 724: - flex now generates warnings for rules that can't be matched.
! 725: These warnings can be turned off using the new '-w' flag. If
! 726: your scanner uses REJECT then you will not get these warnings.
! 727:
! 728: - If you specify the '-s' flag but the default rule can be matched,
! 729: flex now generates a warning.
! 730:
! 731: - "yyleng" is now a global, and may be modified by the user (though
! 732: doing so and then using yymore() will yield weird results).
! 733:
! 734: - Name definitions in the first section of a scanner specification
! 735: can now include a leading '^' or trailing '$' operator. In this
! 736: case, the definition is *not* pushed back inside of parentheses.
! 737:
! 738: - Scanners with compressed tables are now "interactive" (-I option)
! 739: by default. You can suppress this attribute (which makes them
! 740: run slightly slower) using the new '-B' flag.
! 741:
! 742: - Flex now generates 8-bit scanners by default, unless you use the
! 743: -Cf or -CF compression options (-Cfe and -CFe result in 8-bit
! 744: scanners). You can force it to generate a 7-bit scanner using
! 745: the new '-7' flag. You can build flex to generate 8-bit scanners
! 746: for -Cf and -CF, too, by adding -DDEFAULT_CSIZE=256 to CFLAGS
! 747: in the Makefile.
! 748:
! 749: - You no longer need to call the scanner routine yyrestart() to
! 750: inform the scanner that you have switched to a new file after
! 751: having seen an EOF on the current input file. Instead, just
! 752: point yyin at the new file and continue scanning.
! 753:
! 754: - You no longer need to invoke YY_NEW_FILE in an <<EOF>> action
! 755: to indicate you wish to continue scanning. Simply point yyin
! 756: at a new file.
! 757:
! 758: - A leading '#' no longer introduces a comment in a flex input.
! 759:
! 760: - flex no longer considers formfeed ('\f') a whitespace character.
! 761:
! 762: - %t, I'm happy to report, has been nuked.
! 763:
! 764: - The '-p' option may be given twice ('-pp') to instruct flex to
! 765: report minor performance problems as well as major ones.
! 766:
! 767: - The '-v' verbose output no longer includes start/finish time
! 768: information.
! 769:
! 770: - Newlines in flex inputs can optionally include leading or
! 771: trailing carriage-returns ('\r'), in support of several PC/Mac
! 772: run-time libraries that automatically include these.
! 773:
! 774: - A start condition of the form "<*>" makes the following rule
! 775: active in every start condition, whether exclusive or inclusive.
! 776:
! 777: - The following items have been corrected in the flex documentation:
! 778:
! 779: - '-C' table compression options *are* cumulative.
! 780:
! 781: - You may modify yytext but not lengthen it by appending
! 782: characters to the end. Modifying its final character
! 783: will affect '^' anchoring for the next rule matched
! 784: if the character is changed to or from a newline.
! 785:
! 786: - The term "backtracking" has been renamed "backing up",
! 787: since it is a one-time repositioning and not a repeated
! 788: search. What used to be the "lex.backtrack" file is now
! 789: "lex.backup".
! 790:
! 791: - Unindented "/* ... */" comments are allowed in the first
! 792: flex input section, but not in the second.
! 793:
! 794: - yyless() can only be used in the flex input source, not
! 795: externally.
! 796:
! 797: - You can use "yyrestart(yyin)" to throw away the
! 798: current contents of the input buffer.
! 799:
! 800: - To write high-speed scanners, attempt to match as much
! 801: text as possible with each rule. See MISC/fastwc/README
! 802: for more information.
! 803:
! 804: - Using the beginning-of-line operator ('^') is fairly
! 805: cheap. Using unput() is expensive. Using yyless() is
! 806: cheap.
! 807:
! 808: - An example of scanning strings with embedded escape
! 809: sequences has been added.
! 810:
! 811: - The example of backing-up in flexdoc was erroneous; it
! 812: has been corrected.
! 813:
! 814: - A flex scanner's internal buffer now dynamically grows if needed
! 815: to match large tokens. Note that growing the buffer presently
! 816: requires rescanning the (large) token, so consuming a lot of
! 817: text this way is a slow process. Also note that presently the
! 818: buffer does *not* grow if you unput() more text than can fit
! 819: into the buffer.
! 820:
! 821: - The MISC/ directory has been reorganized; see MISC/README for
! 822: details.
! 823:
! 824: - yyless() can now be used in the third (user action) section
! 825: of a scanner specification, thanks to Ceriel Jacobs. yyless()
! 826: remains a macro and cannot be used outside of the scanner source.
! 827:
! 828: - The skeleton file is no longer opened at run-time, but instead
! 829: compiled into a large string array (thanks to John Gilmore and
! 830: friends at Cygnus). You can still use the -S flag to point flex
! 831: at a different skeleton file.
! 832:
! 833: - flex no longer uses a temporary file to store the scanner's
! 834: actions.
! 835:
! 836: - A number of changes have been made to decrease porting headaches.
! 837: In particular, flex no longer uses memset() or ctime(), and
! 838: provides a single simple mechanism for dealing with C compilers
! 839: that still define malloc() as returning char* instead of void*.
! 840:
! 841: - Flex now detects if the scanner specification requires the -8 flag
! 842: but the flag was not given or on by default.
! 843:
! 844: - A number of table-expansion fencepost bugs have been fixed,
! 845: making flex more robust for generating large scanners.
! 846:
! 847: - flex more consistently identifies the location of errors in
! 848: its input.
! 849:
! 850: - YY_USER_ACTION is now invoked only for "real" actions, not for
! 851: internal actions used by the scanner for things like filling
! 852: the buffer or handling EOF.
! 853:
! 854: - The rule "[^]]" now matches any character other than a ']';
! 855: formerly it matched any character at all followed by a ']'.
! 856: This change was made for compatibility with AT&T lex.
! 857:
! 858: - A large number of miscellaneous bugs have been found and fixed
! 859: thanks to Gerhard Wilhelms.
! 860:
! 861: - The source code has been heavily reformatted, making patches
! 862: relative to previous flex releases no longer accurate.
! 863:
! 864:
! 865: Changes between 2.3 Patch #8 (21Feb93) and 2.3 Patch #7:
! 866:
! 867: - Fixed bugs in dynamic memory allocation leading to grievous
! 868: fencepost problems when generating large scanners.
! 869: - Fixed bug causing infinite loops on character classes with 8-bit
! 870: characters in them.
! 871: - Fixed bug in matching repetitions with a lower bound of 0.
! 872: - Fixed bug in scanning NUL characters using an "interactive" scanner.
! 873: - Fixed bug in using yymore() at the end of a file.
! 874: - Fixed bug in misrecognizing rules with variable trailing context.
! 875: - Fixed bug compiling flex on Suns using gcc 2.
! 876: - Fixed bug in not recognizing that input files with the character
! 877: ASCII 128 in them require the -8 flag.
! 878: - Fixed bug that could cause an infinite loop writing out
! 879: error messages.
! 880: - Fixed bug in not recognizing old-style lex % declarations if
! 881: followed by a tab instead of a space.
! 882: - Fixed potential crash when flex terminated early (usually due
! 883: to a bad flag) and the -v flag had been given.
! 884: - Added some missing declarations of void functions.
! 885: - Changed to only use '\a' for __STDC__ compilers.
! 886: - Updated mailing addresses.
! 887:
! 888:
! 889: Changes between 2.3 Patch #7 (28Mar91) and 2.3 Patch #6:
! 890:
! 891: - Fixed out-of-bounds array access that caused bad tables
! 892: to be produced on machines where the bad reference happened
! 893: to yield a 1. This caused problems installing or running
! 894: flex on some Suns, in particular.
! 895:
! 896:
! 897: Changes between 2.3 Patch #6 (29Aug90) and 2.3 Patch #5:
! 898:
! 899: - Fixed a serious bug in yymore() which basically made it
! 900: completely broken. Thanks goes to Jean Christophe of
! 901: the Nethack development team for finding the problem
! 902: and passing along the fix.
! 903:
! 904:
! 905: Changes between 2.3 Patch #5 (16Aug90) and 2.3 Patch #4:
! 906:
! 907: - An up-to-date version of initscan.c so "make test" will
! 908: work after applying the previous patches
! 909:
! 910:
! 911: Changes between 2.3 Patch #4 (14Aug90) and 2.3 Patch #3:
! 912:
! 913: - Fixed bug in hexadecimal escapes which allowed only digits,
! 914: not letters, in escapes
! 915: - Fixed bug in previous "Changes" file!
! 916:
! 917:
! 918: Changes between 2.3 Patch #3 (03Aug90) and 2.3 Patch #2:
! 919:
! 920: - Correction to patch #2 for gcc compilation; thanks goes to
! 921: Paul Eggert for catching this.
! 922:
! 923:
! 924: Changes between 2.3 Patch #2 (02Aug90) and original 2.3 release:
! 925:
! 926: - Fixed (hopefully) headaches involving declaring malloc()
! 927: and free() for gcc, which defines __STDC__ but (often) doesn't
! 928: come with the standard include files such as <stdlib.h>.
! 929: Reordered #ifdef maze in the scanner skeleton in the hope of
! 930: getting the declarations right for cfront and g++, too.
! 931:
! 932: - Note that this patch supercedes patch #1 for release 2.3,
! 933: which was never announced but was available briefly for
! 934: anonymous ftp.
! 935:
! 936:
! 937: Changes between 2.3 (full) release of 28Jun90 and 2.2 (alpha) release:
! 938:
! 939: User-visible:
! 940:
! 941: - A lone <<EOF>> rule (that is, one which is not qualified with
! 942: a list of start conditions) now specifies the EOF action for
! 943: *all* start conditions which haven't already had <<EOF>> actions
! 944: given. To specify an end-of-file action for just the initial
! 945: state, use <INITIAL><<EOF>>.
! 946:
! 947: - -d debug output is now contigent on the global yy_flex_debug
! 948: being set to a non-zero value, which it is by default.
! 949:
! 950: - A new macro, YY_USER_INIT, is provided for the user to specify
! 951: initialization action to be taken on the first call to the
! 952: scanner. This action is done before the scanner does its
! 953: own initialization.
! 954:
! 955: - yy_new_buffer() has been added as an alias for yy_create_buffer()
! 956:
! 957: - Comments beginning with '#' and extending to the end of the line
! 958: now work, but have been deprecated (in anticipation of making
! 959: flex recognize #line directives).
! 960:
! 961: - The funky restrictions on when semi-colons could follow the
! 962: YY_NEW_FILE and yyless macros have been removed. They now
! 963: behave identically to functions.
! 964:
! 965: - A bug in the sample redefinition of YY_INPUT in the documentation
! 966: has been corrected.
! 967:
! 968: - A bug in the sample simple tokener in the documentation has
! 969: been corrected.
! 970:
! 971: - The documentation on the incompatibilities between flex and
! 972: lex has been reordered so that the discussion of yylineno
! 973: and input() come first, as it's anticipated that these will
! 974: be the most common source of headaches.
! 975:
! 976:
! 977: Things which didn't used to be documented but now are:
! 978:
! 979: - flex interprets "^foo|bar" differently from lex. flex interprets
! 980: it as "match either a 'foo' or a 'bar', providing it comes at the
! 981: beginning of a line", whereas lex interprets it as "match either
! 982: a 'foo' at the beginning of a line, or a 'bar' anywhere".
! 983:
! 984: - flex initializes the global "yyin" on the first call to the
! 985: scanner, while lex initializes it at compile-time.
! 986:
! 987: - yy_switch_to_buffer() can be used in the yywrap() macro/routine.
! 988:
! 989: - flex scanners do not use stdio for their input, and hence when
! 990: writing an interactive scanner one must explictly call fflush()
! 991: after writing out a prompt.
! 992:
! 993: - flex scanner can be made reentrant (after a fashion) by using
! 994: "yyrestart( yyin );". This is useful for interactive scanners
! 995: which have interrupt handlers that long-jump out of the scanner.
! 996:
! 997: - a defense of why yylineno is not supported is included, along
! 998: with a suggestion on how to convert scanners which rely on it.
! 999:
! 1000:
! 1001: Other changes:
! 1002:
! 1003: - Prototypes and proper declarations of void routines have
! 1004: been added to the flex source code, courtesy of Kevin B. Kenny.
! 1005:
! 1006: - Routines dealing with memory allocation now use void* pointers
! 1007: instead of char* - see Makefile for porting implications.
! 1008:
! 1009: - Error-checking is now done when flex closes a file.
! 1010:
! 1011: - Various lint tweaks were added to reduce the number of gripes.
! 1012:
! 1013: - Makefile has been further parameterized to aid in porting.
! 1014:
! 1015: - Support for SCO Unix added.
! 1016:
! 1017: - Flex now sports the latest & greatest UC copyright notice
! 1018: (which is only slightly different from the previous one).
! 1019:
! 1020: - A note has been added to flexdoc.1 mentioning work in progress
! 1021: on modifying flex to generate straight C code rather than a
! 1022: table-driven automaton, with an email address of whom to contact
! 1023: if you are working along similar lines.
! 1024:
! 1025:
! 1026: Changes between 2.2 Patch #3 (30Mar90) and 2.2 Patch #2:
! 1027:
! 1028: - fixed bug which caused -I scanners to bomb
! 1029:
! 1030:
! 1031: Changes between 2.2 Patch #2 (27Mar90) and 2.2 Patch #1:
! 1032:
! 1033: - fixed bug writing past end of input buffer in yyunput()
! 1034: - fixed bug detecting NUL's at the end of a buffer
! 1035:
! 1036:
! 1037: Changes between 2.2 Patch #1 (23Mar90) and 2.2 (alpha) release:
! 1038:
! 1039: - Makefile fixes: definition of MAKE variable for systems
! 1040: which don't have it; installation of flexdoc.1 along with
! 1041: flex.1; fixed two bugs which could cause "bigtest" to fail.
! 1042:
! 1043: - flex.skel fix for compiling with g++.
! 1044:
! 1045: - README and flexdoc.1 no longer list an out-of-date BITNET address
! 1046: for contacting me.
! 1047:
! 1048: - minor typos and formatting changes to flex.1 and flexdoc.1.
! 1049:
! 1050:
! 1051: Changes between 2.2 (alpha) release of March '90 and previous release:
! 1052:
! 1053: User-visible:
! 1054:
! 1055: - Full user documentation now available.
! 1056:
! 1057: - Support for 8-bit scanners.
! 1058:
! 1059: - Scanners now accept NUL's.
! 1060:
! 1061: - A facility has been added for dealing with multiple
! 1062: input buffers.
! 1063:
! 1064: - Two manual entries now. One which fully describes flex
! 1065: (rather than just its differences from lex), and the
! 1066: other for quick(er) reference.
! 1067:
! 1068: - A number of changes to bring flex closer into compliance
! 1069: with the latest POSIX lex draft:
! 1070:
! 1071: %t support
! 1072: flex now accepts multiple input files and concatenates
! 1073: them together to form its input
! 1074: previous -c (compress) flag renamed -C
! 1075: do-nothing -c and -n flags added
! 1076: Any indented code or code within %{}'s in section 2 is
! 1077: now copied to the output
! 1078:
! 1079: - yyleng is now a bona fide global integer.
! 1080:
! 1081: - -d debug information now gives the line number of the
! 1082: matched rule instead of which number rule it was from
! 1083: the beginning of the file.
! 1084:
! 1085: - -v output now includes a summary of the flags used to generate
! 1086: the scanner.
! 1087:
! 1088: - unput() and yyrestart() are now globally callable.
! 1089:
! 1090: - yyrestart() no longer closes the previous value of yyin.
! 1091:
! 1092: - C++ support; generated scanners can be compiled with C++ compiler.
! 1093:
! 1094: - Primitive -lfl library added, containing default main()
! 1095: which calls yylex(). A number of routines currently living
! 1096: in the scanner skeleton will probably migrate to here
! 1097: in the future (in particular, yywrap() will probably cease
! 1098: to be a macro and instead be a function in the -lfl library).
! 1099:
! 1100: - Hexadecimal (\x) escape sequences added.
! 1101:
! 1102: - Support for MS-DOS, VMS, and Turbo-C integrated.
! 1103:
! 1104: - The %used/%unused operators have been deprecated. They
! 1105: may go away soon.
! 1106:
! 1107:
! 1108: Other changes:
! 1109:
! 1110: - Makefile enhanced for easier testing and installation.
! 1111: - The parser has been tweaked to detect some erroneous
! 1112: constructions which previously were missed.
! 1113: - Scanner input buffer overflow is now detected.
! 1114: - Bugs with missing "const" declarations fixed.
! 1115: - Out-of-date Minix/Atari patches provided.
! 1116: - Scanners no longer require printf() unless FLEX_DEBUG is being used.
! 1117: - A subtle input() bug has been fixed.
! 1118: - Line numbers for "continued action" rules (those following
! 1119: the special '|' action) are now correct.
! 1120: - unput() bug fixed; had been causing problems porting flex to VMS.
! 1121: - yymore() handling rewritten to fix bug with interaction
! 1122: between yymore() and trailing context.
! 1123: - EOF in actions now generates an error message.
! 1124: - Bug involving -CFe and generating equivalence classes fixed.
! 1125: - Bug which made -CF be treated as -Cf fixed.
! 1126: - Support for SysV tmpnam() added.
! 1127: - Unused #define's for scanner no longer generated.
! 1128: - Error messages which are associated with a particular input
! 1129: line are now all identified with their input line in standard
! 1130: format.
! 1131: - % directives which are valid to lex but not to flex are
! 1132: now ignored instead of generating warnings.
! 1133: - -DSYS_V flag can now also be specified -DUSG for System V
! 1134: compilation.
! 1135:
! 1136:
! 1137: Changes between 2.1 beta-test release of June '89 and previous release:
! 1138:
! 1139: User-visible:
! 1140:
! 1141: - -p flag generates a performance report to stderr. The report
! 1142: consists of comments regarding features of the scanner rules
! 1143: which result in slower scanners.
! 1144:
! 1145: - -b flag generates backtracking information to lex.backtrack.
! 1146: This is a list of scanner states which require backtracking
! 1147: and the characters on which they do so. By adding rules
! 1148: one can remove backtracking states. If all backtracking states
! 1149: are eliminated, the generated scanner will run faster.
! 1150: Backtracking is not yet documented in the manual entry.
! 1151:
! 1152: - Variable trailing context now works, i.e., one can have
! 1153: rules like "(foo)*/[ \t]*bletch". Some trailing context
! 1154: patterns still cannot be properly matched and generate
! 1155: error messages. These are patterns where the ending of the
! 1156: first part of the rule matches the beginning of the second
! 1157: part, such as "zx*/xy*", where the 'x*' matches the 'x' at
! 1158: the beginning of the trailing context. Lex won't get these
! 1159: patterns right either.
! 1160:
! 1161: - Faster scanners.
! 1162:
! 1163: - End-of-file rules. The special rule "<<EOF>>" indicates
! 1164: actions which are to be taken when an end-of-file is
! 1165: encountered and yywrap() returns non-zero (i.e., indicates
! 1166: no further files to process). See manual entry for example.
! 1167:
! 1168: - The -r (reject used) flag is gone. flex now scans the input
! 1169: for occurrences of the string "REJECT" to determine if the
! 1170: action is needed. It tries to be intelligent about this but
! 1171: can be fooled. One can force the presence or absence of
! 1172: REJECT by adding a line in the first section of the form
! 1173: "%used REJECT" or "%unused REJECT".
! 1174:
! 1175: - yymore() has been implemented. Similarly to REJECT, flex
! 1176: detects the use of yymore(), which can be overridden using
! 1177: "%used" or "%unused".
! 1178:
! 1179: - Patterns like "x{0,3}" now work (i.e., with lower-limit == 0).
! 1180:
! 1181: - Removed '\^x' for ctrl-x misfeature.
! 1182:
! 1183: - Added '\a' and '\v' escape sequences.
! 1184:
! 1185: - \<digits> now works for octal escape sequences; previously
! 1186: \0<digits> was required.
! 1187:
! 1188: - Better error reporting; line numbers are associated with rules.
! 1189:
! 1190: - yyleng is a macro; it cannot be accessed outside of the
! 1191: scanner source file.
! 1192:
! 1193: - yytext and yyleng should not be modified within a flex action.
! 1194:
! 1195: - Generated scanners #define the name FLEX_SCANNER.
! 1196:
! 1197: - Rules are internally separated by YY_BREAK in lex.yy.c rather
! 1198: than break, to allow redefinition.
! 1199:
! 1200: - The macro YY_USER_ACTION can be redefined to provide an action
! 1201: which is always executed prior to the matched rule's action.
! 1202:
! 1203: - yyrestart() is a new action which can be used to restart
! 1204: the scanner after it has seen an end-of-file (a "real" one,
! 1205: that is, one for which yywrap() returned non-zero). It takes
! 1206: a FILE* argument indicating a new file to scan and sets
! 1207: things up so that a subsequent call to yylex() will start
! 1208: scanning that file.
! 1209:
! 1210: - Internal scanner names all preceded by "yy_"
! 1211:
! 1212: - lex.yy.c is deleted if errors are encountered during processing.
! 1213:
! 1214: - Comments may be put in the first section of the input by preceding
! 1215: them with '#'.
! 1216:
! 1217:
! 1218:
! 1219: Other changes:
! 1220:
! 1221: - Some portability-related bugs fixed, in particular for machines
! 1222: with unsigned characters or sizeof( int* ) != sizeof( int ).
! 1223: Also, tweaks for VMS and Microsoft C (MS-DOS), and identifiers all
! 1224: trimmed to be 31 or fewer characters. Shortened file names
! 1225: for dinosaur OS's. Checks for allocating > 64K memory
! 1226: on 16 bit'ers. Amiga tweaks. Compiles using gcc on a Sun-3.
! 1227: - Compressed and fast scanner skeletons merged.
! 1228: - Skeleton header files done away with.
! 1229: - Generated scanner uses prototypes and "const" for __STDC__.
! 1230: - -DSV flag is now -DSYS_V for System V compilation.
! 1231: - Removed all references to FTL language.
! 1232: - Software now covered by BSD Copyright.
! 1233: - flex will replace lex in subsequent BSD releases.
![[BACK]](/icons/back.gif){kind=icon}
Return to ONEWS CVS log ![[TXT]](/icons/text.gif){kind=icon}
![[DIR]](/icons/dir.gif){kind=icon}
Up to [local] / src / usr.bin / lex