[lug] parsing tool for linux

[Keith Herold]

Well, yes, I realize it is a tokenizer; but, I want what a *C* compiler
thinks is a token, not what I think a C compiler thinks is a token.  If I
try to replicate the grammar of C, I *will* screw it up.  I would really
like to know what C is treating as names of variables and functions for a
given source file, and that means that C has to know what is and is not a
function, variable, operator, etc.

What I am trying to avoid is me writing incomplete/incorrect grammars, and
thus building a language model that isn't correct, because I did something
wrong.  If there is a parser out there that will output the symbol table,
plus the operators/characters, than I can pipe that output and the original
source file to another tool (that I already have) to count up the
coocurrences.

--Keith

> -----Original Message-----
> From: lug-admin at lug.boulder.co.us [mailto:lug-admin at lug.boulder.co.us]On
> Behalf Of Bear Giles
> Sent: Sunday, April 07, 2002 4:15 PM
> To: lug at lug.boulder.co.us
> Subject: Re: [lug] parsing tool for linux
>
>
> > Hi folks!  I am looking for a tool that already knows C syntax, and can
> > output the results of a parsed C source file, one token per line.
>
> This is a tokenizer, easily written with a tool such as lex/flex.  Almost
> as easily coded by hand.
>
> > Separated by spaces is fine, too, so long as the function names are
> > preserved as functions ( so that func(x) comes out func(, or func (x)
> > comes out func( )
>
> That's a problem.  func(x) is actually four tokens - 'func', '(', 'x'
> and ')'.  While it's possible to write the tokenizer rules so that
> 'func(' is a single token, you'll rarely see this done in practice.
> It's also not really correct to call 'func' the function - what's the
> "function" in this C snippet:
>
>    ((*((struct foobar *) x)).y(2))->z()
>
> Bear
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>