Full Unicode/Ansi-Support in /testing branch
Moderators: gto, EgonHugeist, olehs
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
miab3,
And only for D2006? Makes me wondering. I removed the TZPostgreSQLTokenizer interface. It should work now i hope. R.1624
Michael
And only for D2006? Makes me wondering. I removed the TZPostgreSQLTokenizer interface. It should work now i hope. R.1624
Michael
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
@EgonHugeist
You have to move for protected or public not private then Delphi2006 compile(I think is right):
procedure SetEscapeMarkSequence(const Value: String); <-------------------------
protected
protected
procedure SetEscapeMarkSequence(const Value: String);
I suspect that the D2006 and earlier was that this problem or feature.
Michal
You have to move for protected or public not private then Delphi2006 compile(I think is right):
procedure SetEscapeMarkSequence(const Value: String); <-------------------------
protected
protected
procedure SetEscapeMarkSequence(const Value: String);
I suspect that the D2006 and earlier was that this problem or feature.
Michal
Last edited by miab3 on 30.07.2012, 10:04, edited 1 time in total.
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
olehs, miab3,
If nobody did recognize it: These both functions had been in the private section since egonhugeist did exists. So i dont understand it. Anyway i did commit that proposal now. On my side i could run everything with FPC, D7, D2007... Rev 1627.
@olehs,
saw your commit. A good first job. Will test it right now.
Michael
If nobody did recognize it: These both functions had been in the private section since egonhugeist did exists. So i dont understand it. Anyway i did commit that proposal now. On my side i could run everything with FPC, D7, D2007... Rev 1627.
@olehs,
saw your commit. A good first job. Will test it right now.
Michael
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
EgonHugeist,
I'm proceeding with modifying QuoteState tokenizer and I'm gonna need TZPostgreSQLTokenizer back (with a little change).
This is a test I made that uses it.
This test is more important, because depending on StandardConformingStrings queries must be tokenized in different ways.
So, what do you think, can I return that TZPostgreSQLTokenizer back?
I'm proceeding with modifying QuoteState tokenizer and I'm gonna need TZPostgreSQLTokenizer back (with a little change).
This is a test I made that uses it.
Code: Select all
procedure TZTestPostgreSQLTokenizer.TestQuoteState;
const
TokenString1: string = '"a\""\''aa" ''c\'' ''c''''c''';
TokenTypes1Off: array[0..1] of TZTokenType = (ttWord, ttQuoted);
TokenValues1Off: array[0..1] of string = ('"a\""\''aa"', '''c\'' ''c''''c''');
TokenTypes1On: array[0..2] of TZTokenType = (ttWord, ttQuoted, ttQuoted);
TokenValues1On: array[0..2] of string = ('"a\""\''aa"', '''c\''', '''c''''c''');
begin
(Tokenizer as TZPostgreSQLTokenizer).SetStandardConformingStrings(False);
CheckTokens(Tokenizer.TokenizeBuffer(TokenString1,
[toSkipEOF, toSkipWhitespaces]), TokenTypes1Off, TokenValues1Off);
(Tokenizer as TZPostgreSQLTokenizer).SetStandardConformingStrings(True);
CheckTokens(Tokenizer.TokenizeBuffer(TokenString1,
[toSkipEOF, toSkipWhitespaces]), TokenTypes1On, TokenValues1On);
end;
So, what do you think, can I return that TZPostgreSQLTokenizer back?
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
olehs,
The TZPostgreSQLTokenizer does still exist, or not? (:
I don't understand the question right now. What you did was overriding some protected procedures on CppCommentState. No problem. Do you mean you need me interface back i did before?
Generally i so no issue to have it. I'm only wondering why now Michal has compiling issues. But i trust him. He is allways there and does a great job to confirm the running latest commits. But i must admit my packages are not the standarts(i use all available files for all packages), so i testet them with the "normal" packeges again and everything works. Curious...
Accordingly the spezial Tokenizer: That's what i mentioned before. IF he must have different behavior, then there must be a way to inform him about to get him running like expected. In my mind a spezial descendant interface of IZTokenizer is the only way here...
Michael
The TZPostgreSQLTokenizer does still exist, or not? (:
I don't understand the question right now. What you did was overriding some protected procedures on CppCommentState. No problem. Do you mean you need me interface back i did before?
Generally i so no issue to have it. I'm only wondering why now Michal has compiling issues. But i trust him. He is allways there and does a great job to confirm the running latest commits. But i must admit my packages are not the standarts(i use all available files for all packages), so i testet them with the "normal" packeges again and everything works. Curious...
Accordingly the spezial Tokenizer: That's what i mentioned before. IF he must have different behavior, then there must be a way to inform him about to get him running like expected. In my mind a spezial descendant interface of IZTokenizer is the only way here...
Michael
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
olehs,
then add it. Let's see how we get this running later on. Wondering that you do not need the backslash_quote option here. If i understand the documentations right is a \ equal to ' IF backslash_quote=ON. Or am i totaly wrong? So if backslash_quote is ON then the tokenizer should detect \sasf\ as ttQuoted=sasf or not?
Michael
then add it. Let's see how we get this running later on. Wondering that you do not need the backslash_quote option here. If i understand the documentations right is a \ equal to ' IF backslash_quote=ON. Or am i totaly wrong? So if backslash_quote is ON then the tokenizer should detect \sasf\ as ttQuoted=sasf or not?
Michael
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
EgonHugeist,
No, actually.
When standard_conforming_strings=off or E'syntax is used, then \' is parsed as '
But when standard_conforming_strings=on, \' is parsed as \'
backslash_quote controls, whether postgres should allow usage of \'
in SCS=off & E'syntax modes or it should raise exception.
Here are examples
BUT
and the last one
No, actually.
When standard_conforming_strings=off or E'syntax is used, then \' is parsed as '
But when standard_conforming_strings=on, \' is parsed as \'
backslash_quote controls, whether postgres should allow usage of \'
in SCS=off & E'syntax modes or it should raise exception.
Here are examples
Code: Select all
zeoslib=# set standard_conforming_strings=ON;
SET
zeoslib=# select '''' as a, '\'' as b';
a | ?column?
---+----------
' | \' as b
(1 row)
Code: Select all
zeoslib=# set standard_conforming_strings=off;
SET
zeoslib=# select '''' as a, '\'' as b;
a | b
---+---
' | '
(1 row)
zeoslib=# set standard_conforming_strings=on;
SET
zeoslib=# select '''' as a, E'\'' as b;
a | b
---+---
' | '
(1 row)
Code: Select all
zeoslib=# set standard_conforming_strings=off;
SET
zeoslib=# set backslash_quote=off;
SET
zeoslib=# select '''' as a, '\'' as b;
ERROR: unsafe use of \' in a string literal
LINE 1:select '''' as a, '\'' as b;
^
HINT: Use '' to write quotes in strings. \' is insecure in client-only enc
odings.
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
olehs,
Ok i understand....
Hmm what do you think about this? Let's make postgre the stupid escaping work. What i don't know: Is the backward encoding for D12_UP valid, after escaping???? IF not then we should handle these issues one level deeper on the Plain-Layer.
This i allready implemented for MySQL...
Michael
Ok i understand....
Code: Select all
function TZPostgreSQLBaseDriver.EscapeString(Handle: PZPostgreSQLConnect;
Encoding: TZCharEncoding; const Source: String; Dest: String): Integer;
var
ResLen: NativeUInt;
Temp: PAnsiChar;
{$IFDEF DELPHI12_UP}
SourceTemp, DestTemp: RawByteString;
{$ENDIF}
begin
if Assigned(POSTGRESQL_API.PQescapeStringConn) then
begin
{$IFDEF DELPHI12_UP}
if encoding = ceUTF8 then
SourceTemp := UTF8Encode(Source)
else
SourceTemp := AnsiString(Source);
GetMem(Temp, Length(SourceTemp)*2);
ResLen := POSTGRESQL_API.PQescapeStringConn(Handle, Temp,
PAnsiChar(SourceTemp), StrLen(PAnsiChar(SourceTemp)), @Result);
SetLength(DestTemp, ResLen);
Move(Temp^, PAnsiChar(DestTemp)^, ResLen);
if encoding = ceUTF8 then
Dest := UTF8ToString(DestTemp)
else
Dest := String(DestTemp);
{$ELSE}
GetMem(Temp, Length(Source)*2);
ResLen := POSTGRESQL_API.PQescapeStringConn(Handle, Temp,
PAnsiChar(Source), StrLen(PAnsiChar(Source)), @Result);
SetLength(Dest, ResLen);
Move(Temp^, PAnsiChar(Dest)^, ResLen);
{$ENDIF}
end
else
begin
Dest := Source;
Result := 0;
end;
end;
This i allready implemented for MySQL...
Michael
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
@olehs,
solved your tests.. Oh didn't saw you previous post...
Rev. 1629
But like i wrote i'm not 100% sure what happens to more complicated Unicode Strings, than these of our tests..
And i don't know if the Tokenizer behaves correctly for the comming test cases..
@miab3, Michal is everything ok now?
Michael
solved your tests.. Oh didn't saw you previous post...
Rev. 1629
But like i wrote i'm not 100% sure what happens to more complicated Unicode Strings, than these of our tests..
And i don't know if the Tokenizer behaves correctly for the comming test cases..
@miab3, Michal is everything ok now?
Michael
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
olehs,
next question: Do we need now GetServerSetting(), FStandartConfirmingStrings, function EncodeString and so on?? What do you think?
and could you please insert some much more complicated tests, with accedend chars like 'üöäü'? that would be great!
AND Couldn't we say: IF PreprepareSQL -> use AnsiQuoteStr instead of EscapeString and then we do this stuff automatically? Works for MySQL perfect.
Michael
next question: Do we need now GetServerSetting(), FStandartConfirmingStrings, function EncodeString and so on?? What do you think?
and could you please insert some much more complicated tests, with accedend chars like 'üöäü'? that would be great!
AND Couldn't we say: IF PreprepareSQL -> use AnsiQuoteStr instead of EscapeString and then we do this stuff automatically? Works for MySQL perfect.
Michael
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/