Zeos 7.1.0 problem with hungarian characters

[lbalu]

Hi!

Win7 X64 Hungarian
Delphi2010
Zeos 7.1.0 beta

Using Dos852 codepage in varchar field.
With zeos, there are strange characters.



with ibtable, got the correct characters.


I tried utf8, ISOxxxx clientcodepage, but it's not working.

How can I set zconnection to work with this codepage?

Thanks
Balázs

[EgonHugeist]

Not sure what wrong. I need some more Information.

Which default characterset is used for the DB 'em selves? Is it UTf8,NONE... ? This makes significant differences.

Maybe my Codepage setting is wrong? ZPlainFirebirdDriver.pas line 604:

Self.AddCodePage('DOS850', CS_DOS850, ceAnsi, zCP_DOS850); {Latin I (no Euro symbol)} check the constant. is 852 the right CP for windows? I thought so: http://msdn.microsoft.com/en-us/library/cc195066.aspx

[lbalu]

Created the DB with these properties.



Sample database
http://www.proforcom.hu/zeos/proba.fdb

The difference between dos850 and dos852 for example:
ô and ő
Using the ő character.

On delphi7 with zeos 7.0.3 everything is OK.

Regards
Balázs

[miab3]

@lbalu

And as you leave ClientCodepage empty?

Michal

[lbalu]

Same thing with empty clientcodepage.
Displaying wrong characters.

[miab3]

@lbalu

Could you attach a SQL-script or database with this table?

Michal

[lbalu]

Sample database
http://www.proforcom.hu/zeos/proba.fdb

Contain a table, called 'partner', created with ibexpert.

[EgonHugeist]

@Ilbalu

issue confirmed. Did check your example. I can't say why but it seems the Windows-internal ansi to wide mapping doesn't work properly for this codepage. OR???

I could resolve this issue by appling this patch: ZPlainFirebirdSriver.pas, line 589.

Resplace this line with
Self.AddCodePage('DOS852', CS_DOS852, ceAnsi, {$IFDEF MSWINDOWS}zCP_L2_ISO_8859_2{$ELSE}zCP_DOS852{$ENDIF}); {Latin II}

Than it should work. I'm a bit surpriced since this CP should be supported like you could see in my prevous post.

Patch done R2805 \testing-7.1

Also is firebird not able to convert cp 852 corectly to UTF8 (which i can't believe because it works for all CP's except charset 'NONE')OR IBTable did write wrong values(my imression). I can't say whats wrong now. If another user has newly issues with this codepage than IBTable is the trouble maker.

Did you try to write strings and read them back with Zeos only? Can you check this please before you replace my proposed patch?

[lbalu]

First.
tried without the patch.
Now, this is good with zeos, but wrong characters if I look in ibexpert.
Example:
In zeos: Árvíztűrő
In ibexpert: *rv*zt*r*

Second.
Applied the patch, and it is working perfectly.

Thank you.

Problem solved.

[EgonHugeist]

Good news.

The issue isn't resolved 100% for ME.

I internaly do only deal with codepages. Before the patch i used Word(852) as the codepage for CharacterSet DOS852.

Now you write you can successfully read and write the non ASCII chars with Zeos. For me it is not interesting what IBExpert is doing here. I internaly call directly the mapping functions of windows. IBExpert is using a class AFAIK.
And AFAIK does DOS852 not support the € symbol which should be possible now, but this is wrong. See: http://msdn.microsoft.com/en-us/library/cc195066.aspx
Also have i headage with the failing FB string conversion to UTF8. Uge surprice for me.

So can you use another external viewer to compare the strings without the patch? Just to confirm the patch is right or IBTable is wrong. That would be nice to KNOW this patch is right before finaly release the 7.1.2-stable candidate.

[miab3]

Michael,

Sample database:
http://www.proforcom.hu/zeos/proba.fdb
have set Character Set DOS852 (database not table or fields).
It looks like the ZEOS did not provide this setting to table char fields.

Michal

[EgonHugeist]

Michal

nope it did check this and Zeos does it axactly. I had problems to test the changes by my selves, since i can't type in hungarian chars with my german Keybord. Also copying from the forum did fail because of a different encoding.

As i wrote i can't say if this patch is right or wrong. So i simply wrote a comment and if someone else has problems than we'll restart the discussion again. Hope user Ibalu does a reply and support me a bit like i did it for him...

[lbalu]

I'm back.
Sorry this late, but working hard.

Removed the patch from Zplainfirebirddriver, the problem is returned in zeos.
1,Downloaded EMS sql manager, opened the sample database.


2,Append new record in ems


3, Back to delphi, and in zeos:


4,Saved the patch again, reopened the delphi project:

and everything is good.

Regards
Balázs

[marsupilami]

Hello Michael,

I am not sure, wether this will help you, but I did some tests with the example database. I used Flamerobin and retrieved the data. Once using UTF8 as the connection character set (Conversion ANSI->Unicode done by Firebird) and once using DOS852 as the connection character set (Conversion ANSI->Unicode done by Flame Robin). The SQL in both cases was:

SELECT cast(NEV as varchar(50) character set octets), nev FROM PARTNER

which in Flame Robin displays the cast as a string of hex characters. So - these are the results:

Code: Select all

        String in Hex                String
DOS852: 4bf576e172692052656ee17461   Kõvári Renáta
  UTF8: 4bf576e172692052656ee17461   K§vßri Renßta

In both cases the Character at the second position is F5. It seems that Flamerobin and firebird do different conversions when they get data declared to be in DOS852:
Firebird treats the data as MS-DOS CP 852, which has the § sign at position F5. (http://en.wikipedia.org/wiki/Code_page_852)
Flame Robin treats the data as ISO 8859-2 or WIN1250, which does have the õ at position F5. (http://en.wikipedia.org/wiki/Windows-1250)
Given this, I think Zeos works correctly without the patch, because in firebird terms the correct character set for F5 to become an õ is ISO8859_2 or WIN1250.

@Ibalu: Your data seems to be encoded in WIN250 (which is used in hungarian Windows for ANSI strings) and not in DOS852 - which was used by MS DOS and only gets used in the windows CMD by now. Could you try to create your database using WIN1252 as the default character set?

[lbalu]

A new database with cp win1250 is ok (I think this codepage contains ő and ű chars).
Good with zeos and with Ibexpert.

But I've got some dos852 DBs, and on delphi 7 with zeos 7.0.3 there aren't weird characters, so I will use the patch for Delphi 2010.