[patch_done] ZeosLib 7+TZQuery+TStringField error size load

[luizcesarap]

Good evening,

I'm using Zeos 7.0 synchronized with tortoise, DelphiXE2 and firebird 2.5 - DEFAULT CHARACTER SET ISO8859_1 COLLATION PT_BR;

I noticed that the fields when loading a selection TZQuery are coming with wrong size.

Field varchar (30) is coming with (60)

What can it be?

[EgonHugeist]

luizcesarap,

Well that's a patch of mine. I'm still not 100% sure about. I know CHARACTER SET ISO8859_1 is theoretical 1 Byte encoded.

Why i did it: The DataSet FieldBuffers do store everything in Bytes NOT in count of Chars. For all, except ASCII or binary or accendetual chars valid. Each such a char has minimun two bytes(a question of which copdepage/encoding). So the FieldBuffer must be big enough to contain 30 * 'Ä' or 'ö' which means we need min. 60Bytes.

Field.Size is allways the size in Bytes not in chars. ColumnDisplaySize is the size in chars but this i can't change if you as user has assigned the fields to you formular.(I'm not willing to play with the defaults).

So tell me am i wrong? Fixing this is done in 30secs.
But again we have to be sure the FieldBuffer is big enough for all chars of the current charset.

[luizcesarap]

Good morning, Michael

I think you are completely right, but even with twice the field size for this type of case is the way that can cause problems, because the two properties "displayWidth" and "Size" are unrealistic, so a field varchar (100) is doubled varchar (200). In my tests resized the "displayWidth" and "Size" to 100 and the accent was correct without any problem.

I thank you for your attention.

[EgonHugeist]

luizcesarap,

Well i agree partialy. Actually i don't know about a case where we've reports about such a possible case. On the other hand, if the buffer is to small, we can corrupt the stack. Since XE3 we've a TFieldBuffer which is a fixed ByteArray not only a memory address. Much more for this case i've done this patch.

Like i wrote i'm not 100% sure about this issue.

properties "displayWidth" and "Size" are unrealistic

Well Field.Size must be allways big enough to store the string. ColumnDisplaySize is completely your choise.

So what is correct now? Anybody else?

[marsupilami]

Hello EgonHugeist and luizcesarap,

I really would like to have a good opinion about this problem. But may I first ask some questions, where I can only make guesses right now?

For the FieldSize:
@luizcesarap: Your database has a default encoding of ISO8859_1. Is that also the character set, that you set in the Zeos Connection or do you use UTF-8 there?

@EgonHugeist: I don't know much about the FieldBuffer. Does it save the data in the database donnection encoding (UTF-8/ISO8859-1) or does it save the data in the way a normal delphi String is encoded (UTF-16)?

As for the DisplaySize: Normally I would expect Zeos to know the character to bytes ratio and use it for calculating the DisplaySize accordingly. So in this case I would expect Zeos to create new fields with a DisplaySize of 30 but not overwrite user settings if the user has set anything else.

Best regards,

Jan

[EgonHugeist]

Hello Jan,

He can use ISO8859_1 as characterset for delphi. If he uses UTF8 then Field.Size should be Varchar(30)*4. The reason are the EUC encoded chinese chars which are 4 Byte encoded.

I don't know much about the FieldBuffer.

open you Delphi help and have a look at TField.Size.

Does it save the data in the database donnection encoding (UTF-8/ISO8859-1) or does it save the data in the way a normal delphi String is encoded (UTF-16)?

Delphi expects an ansi encoded AnsiString for FieldType ftString.

As for the DisplaySize: Normally I would expect Zeos to know the character to bytes ratio and use it for calculating the DisplaySize accordingly. So in this case I would expect Zeos to create new fields with a DisplaySize of 30 but not overwrite user settings if the user has set anything else.

Well this i've done. Simply link a TZDataSet component with a DBGrid Open it. Then you'll see the corect DisplaySize for the Fields. But if the user assigns the Fields to the formular then i've to start from the premisse he want to change the defaults. So i broke with realigning the DisplaySizes because i've no idea how to handle this behavior. One user wants to have oversized column the other wants to decrease the DisplaySize.

Again the patch is small: ZAbtractRODataset.pas and remove the * 2 on CreateFieldDefs.

Again what's right or wrong?

[marsupilami]

Hello EgonHugeist,

I am not sure, what to think because I simply don't have enough Knowledge...
You pointed me to TField.Size. The delphi help says the same for Fields of ftString in Delphi 6 and Delphi XE2:

ftString: Size is the maximum number of characters in the string.

(see http://docwiki.embarcadero.com/Libraries/XE2/en/Data.DB.TField.Size)

So - TField.Size should be 30 for every varchar(30) column - wether ist is ISO8859-1, UTF-8, UTF-16 or BIG5. I would expect TStringField to know which encoding it uses for storing the String.

Well - this is my interpretation of the documentation.

Maybe you are referring to TField.DataSize and to TField.GetData? For now I cannot find a reference to TFieldBuffer in the Delphi online help.

As for the DisplaySize: I don't know how creation of a TField works. But I assume that Delphi doesn't create new TFields if there already are some TFields? Because of this I don't understand why they are touched at all? Normally I would expect some point where TFieldDefs are created, if there are no TFieldDefs present and that Zeos only interferes at this point?
If it is like this, the TFields are created by the deserialization process and the changed values are written to the fields by it too if the user has his TFields persistent?
Maybe I should take a look in the code of TZAbstarctDataSet here - unfortunately I cannot do so in the next three days...

Best regards,

Jan

[EgonHugeist]

Seem like i loose this battle (:

Well Jan you're right TStringField seems to expect count of chars instead of bytes. The reason why i was missinformed was: http://docwiki.embarcadero.com/Librarie ... ldDef.Size . And we use TFieldRef to create the fields from TFieldRefs.

Patch done R1920