TZTokenizer fails to detect -- comments

[aehimself]

Hello,

In my application I needed to distinguish if a query returns a resultset or not so I wrote a string parser to look for the first valid keyword of a text. For a while I migrated this logic to use TZTokenizer instead and now I found out that the following query:

Code: Select all

-- This is a comment
SELECT * FROM MyTabe

is actually considered one WITHOUT a resultset, as TZTokenizer finds "This" as the first word token.

Code: Select all

/* This is a comment */
SELECT * FROM MyTable

works fine. Which is actually a bug as "-" is declared as a possible comment token, see ZGenericSQLToken.pas : 531 (TZGenericSQLTokenizer.CreateTokenStates):

Code: Select all

  SetCharacterState('-', '-', CommentState);

I sent a pull request on GitHub with the possible fix, however I'm unsure if this is the correct way to do this. Take it with a grain of salt, please :)

[marsupilami]

Hello aehimself,

I finally got around to applying your patch. Currently I am waiting for the results of the test suite.

Best regards,

Jan

[marsupilami]

Your patch triggers two test that fail now:

basic.core.TZTestTokenizer.TZTestTokenizer.TestNumber
basic.parsesql.TZTestGenericStatementAnalyser.TZTestGenericStatementAnalyser.TestSelectParser

Do you have a chance to look into this?

[aehimself]

Jan,

Done, second commit is in the same pull request.
Unfortunately I was unable to execute any tests in the suite (option was greyed out) but I replicated the failing tests in an application and they executed correctly.

Thanks!