(flex.info.gz) Cxx
Info Catalog
(flex.info.gz) Performance
(flex.info.gz) Top
(flex.info.gz) Reentrant
18 Generating C++ Scanners
**************************
*IMPORTANT*: the present form of the scanning class is _experimental_
and may change considerably between major releases.
`flex' provides two different ways to generate scanners for use with
C++. The first way is to simply compile a scanner generated by `flex'
using a C++ compiler instead of a C compiler. You should not encounter
any compilation errors ( Reporting Bugs). You can then use C++
code in your rule actions instead of C code. Note that the default
input source for your scanner remains `yyin', and default echoing is
still done to `yyout'. Both of these remain `FILE *' variables and not
C++ _streams_.
You can also use `flex' to generate a C++ scanner class, using the
`-+' option (or, equivalently, `%option c++)', which is automatically
specified if the name of the `flex' executable ends in a '+', such as
`flex++'. When using this option, `flex' defaults to generating the
scanner to the file `lex.yy.cc' instead of `lex.yy.c'. The generated
scanner includes the header file `FlexLexer.h', which defines the
interface to two C++ classes.
The first class, `FlexLexer', provides an abstract base class
defining the general scanner class interface. It provides the
following member functions:
`const char* YYText()'
returns the text of the most recently matched token, the
equivalent of `yytext'.
`int YYLeng()'
returns the length of the most recently matched token, the
equivalent of `yyleng'.
`int lineno() const'
returns the current input line number (see `%option yylineno)', or
`1' if `%option yylineno' was not used.
`void set_debug( int flag )'
sets the debugging flag for the scanner, equivalent to assigning to
`yy_flex_debug' ( Scanner Options). Note that you must
build the scanner using `%option debug' to include debugging
information in it.
`int debug() const'
returns the current setting of the debugging flag.
Also provided are member functions equivalent to
`yy_switch_to_buffer()', `yy_create_buffer()' (though the first
argument is an `istream*' object pointer and not a `FILE*)',
`yy_flush_buffer()', `yy_delete_buffer()', and `yyrestart()' (again,
the first argument is a `istream*' object pointer).
The second class defined in `FlexLexer.h' is `yyFlexLexer', which is
derived from `FlexLexer'. It defines the following additional member
functions:
`yyFlexLexer( istream* arg_yyin = 0, ostream* arg_yyout = 0 )'
constructs a `yyFlexLexer' object using the given streams for input
and output. If not specified, the streams default to `cin' and
`cout', respectively.
`virtual int yylex()'
performs the same role is `yylex()' does for ordinary `flex'
scanners: it scans the input stream, consuming tokens, until a
rule's action returns a value. If you derive a subclass `S' from
`yyFlexLexer' and want to access the member functions and variables
of `S' inside `yylex()', then you need to use `%option
yyclass="S"' to inform `flex' that you will be using that subclass
instead of `yyFlexLexer'. In this case, rather than generating
`yyFlexLexer::yylex()', `flex' generates `S::yylex()' (and also
generates a dummy `yyFlexLexer::yylex()' that calls
`yyFlexLexer::LexerError()' if called).
`virtual void switch_streams(istream* new_in = 0, ostream* new_out = 0)'
reassigns `yyin' to `new_in' (if non-null) and `yyout' to
`new_out' (if non-null), deleting the previous input buffer if
`yyin' is reassigned.
`int yylex( istream* new_in, ostream* new_out = 0 )'
first switches the input streams via `switch_streams( new_in,
new_out )' and then returns the value of `yylex()'.
In addition, `yyFlexLexer' defines the following protected virtual
functions which you can redefine in derived classes to tailor the
scanner:
`virtual int LexerInput( char* buf, int max_size )'
reads up to `max_size' characters into `buf' and returns the
number of characters read. To indicate end-of-input, return 0
characters. Note that `interactive' scanners (see the `-B' and
`-I' flags in Scanner Options) define the macro
`YY_INTERACTIVE'. If you redefine `LexerInput()' and need to take
different actions depending on whether or not the scanner might be
scanning an interactive input source, you can test for the
presence of this name via `#ifdef' statements.
`virtual void LexerOutput( const char* buf, int size )'
writes out `size' characters from the buffer `buf', which, while
`NUL'-terminated, may also contain internal `NUL's if the
scanner's rules can match text with `NUL's in them.
`virtual void LexerError( const char* msg )'
reports a fatal error message. The default version of this
function writes the message to the stream `cerr' and exits.
Note that a `yyFlexLexer' object contains its _entire_ scanning
state. Thus you can use such objects to create reentrant scanners, but
see also Reentrant. You can instantiate multiple instances of
the same `yyFlexLexer' class, and you can also combine multiple C++
scanner classes together in the same program using the `-P' option
discussed above.
Finally, note that the `%array' feature is not available to C++
scanner classes; you must use `%pointer' (the default).
Here is an example of a simple C++ scanner:
// An example of using the flex C++ scanner class.
%{
int mylineno = 0;
%}
string \"[^\n"]+\"
ws [ \t]+
alpha [A-Za-z]
dig [0-9]
name ({alpha}|{dig}|\$)({alpha}|{dig}|[_.\-/$])*
num1 [-+]?{dig}+\.?([eE][-+]?{dig}+)?
num2 [-+]?{dig}*\.{dig}+([eE][-+]?{dig}+)?
number {num1}|{num2}
%%
{ws} /* skip blanks and tabs */
"/*" {
int c;
while((c = yyinput()) != 0)
{
if(c == '\n')
++mylineno;
else if(c == @samp{*})
{
if((c = yyinput()) == '/')
break;
else
unput(c);
}
}
}
{number} cout "number " YYText() '\n';
\n mylineno++;
{name} cout "name " YYText() '\n';
{string} cout "string " YYText() '\n';
%%
int main( int /* argc */, char** /* argv */ )
{
@code{flex}Lexer* lexer = new yyFlexLexer;
while(lexer->yylex() != 0)
;
return 0;
}
If you want to create multiple (different) lexer classes, you use the
`-P' flag (or the `prefix=' option) to rename each `yyFlexLexer' to
some other `xxFlexLexer'. You then can include `<FlexLexer.h>' in your
other sources once per lexer class, first renaming `yyFlexLexer' as
follows:
#undef yyFlexLexer
#define yyFlexLexer xxFlexLexer
#include <FlexLexer.h>
#undef yyFlexLexer
#define yyFlexLexer zzFlexLexer
#include <FlexLexer.h>
if, for example, you used `%option prefix="xx"' for one of your
scanners and `%option prefix="zz"' for the other.
Info Catalog
(flex.info.gz) Performance
(flex.info.gz) Top
(flex.info.gz) Reentrant
automatically generated byinfo2html