Full Unicode/Ansi-Support in /testing branch
Moderators: gto, EgonHugeist, olehs
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
miab3,
today i made a testcase: New FB-DB, default Charset non.
to test it. First impression with FPC2.7.1 you are right but only @designtime. @runtime everything was ok.
Also did i notice that sometime properties like ClientCodePage, Protocol & Database are missing. LCL Rev:39124 V1.1.
Then i opened the same app with FPC2.6 and Layarus 1.02RC2.
NO ISSUES to see>
today i made a testcase: New FB-DB, default Charset non.
Code: Select all
create table people (
p_id SMALLINT not null,
p_dep_id SMALLINT,
p_name VARCHAR(40) character set UTF8 collate UTF8,
p_begin_work TIME,
p_end_work TIME,
p_picture BLOB,
p_resume BLOB SUB_TYPE TEXT,
p_redundant SMALLINT,
primary key (p_id)
);
Also did i notice that sometime properties like ClientCodePage, Protocol & Database are missing. LCL Rev:39124 V1.1.
Then i opened the same app with FPC2.6 and Layarus 1.02RC2.
NO ISSUES to see>
You do not have the required permissions to view the files attached to this post.
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
Furthermore the columns and editable values are oversized!
A little pic of Lazarus:
I personally can't reproduce this issue on Zeos yet. The reason why i could not fix something, Michal. But what i can do is to modify the column displaysize in dependencies to the column charset....
Is that a unix or an common issue?
A little pic of Lazarus:
I personally can't reproduce this issue on Zeos yet. The reason why i could not fix something, Michal. But what i can do is to modify the column displaysize in dependencies to the column charset....
Is that a unix or an common issue?
You do not have the required permissions to view the files attached to this post.
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
miab3,
me too. That's my problem. The pic above shows 40 chars for a field defined as
VARCHAR(40) character set UTF8 collate UTF8. Not 20 chars. I HAD the issue you are talking about @designtime but not @runtime. Crazy is after recompiling Lazarus the issue was gone. Also was the case solved after assigning the fiels to the formular. Now tell me how i can fix an issue which i can no longer see?
me too. That's my problem. The pic above shows 40 chars for a field defined as
VARCHAR(40) character set UTF8 collate UTF8. Not 20 chars. I HAD the issue you are talking about @designtime but not @runtime. Crazy is after recompiling Lazarus the issue was gone. Also was the case solved after assigning the fiels to the formular. Now tell me how i can fix an issue which i can no longer see?
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
EgonHugeist,
I do not understand.
I say that Lazarus 2.7.1 Win and Lin can not write and read correctly 10 national characters in the field:
VARCHAR (10) CHARACTER SET WIN1250 COLLATE PXW_PLK (Firebird 2.5 NONE).
It can write and read(max):
0123456789 or
ą12345678
ąż234567
ążź3456
ążźć45
ążźćń
Delphi does not have any problem with that.
Michal
I do not understand.
I say that Lazarus 2.7.1 Win and Lin can not write and read correctly 10 national characters in the field:
VARCHAR (10) CHARACTER SET WIN1250 COLLATE PXW_PLK (Firebird 2.5 NONE).
It can write and read(max):
0123456789 or
ą12345678
ąż234567
ążź3456
ążźć45
ążźćń
Delphi does not have any problem with that.
Michal
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
miab3,
that's because delphi writes only ansi for charset none. Lazarus (not FPC) assumes all strings are utf8 encoded. That's why they'll not fir into the column.
Solution:
TZConnection.ClientCodePage = WIN1250
TZConnection.AutoEncode := True;
IF ALL columns are defined as WIN1251.
On the other hand writes the data "as is" und it's possible national chars won't fit...
Another way is what firebird proposes for people with charset none: Write the cast for the charset around your statement values and it should work too.
Example for my 40-Chars testring:
Charset none, column collate utf8. Data written and displayed as: üüüüüüüüüüääääääääää
Each of this chars will be exceptet by your collation but if the encopding is wrong then each à or ¼ eats two bytes again.
I was starting from the premise Zeos does something wrong... Further more its a question how you use it. I checket the IBX components and -> the same behavior.
that's because delphi writes only ansi for charset none. Lazarus (not FPC) assumes all strings are utf8 encoded. That's why they'll not fir into the column.
Solution:
TZConnection.ClientCodePage = WIN1250
TZConnection.AutoEncode := True;
IF ALL columns are defined as WIN1251.
On the other hand writes the data "as is" und it's possible national chars won't fit...
Another way is what firebird proposes for people with charset none: Write the cast for the charset around your statement values and it should work too.
Example for my 40-Chars testring:
Charset none, column collate utf8. Data written and displayed as: üüüüüüüüüüääääääääää
Each of this chars will be exceptet by your collation but if the encopding is wrong then each à or ¼ eats two bytes again.
I was starting from the premise Zeos does something wrong... Further more its a question how you use it. I checket the IBX components and -> the same behavior.
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
miab3,
If that charset is used than no conversations where done from FireBird. But with my proposal you'll run into the same issue if other collations are defined for other columns.
I had a look to the column subtypes where firebird wrote in the article:
But that's not right at all! They determine only these:
CS_NONE = 0;
CS_BINARY = 1;
CS_ASCII = 2;
CS_UNICODE_FSS = 3;
CS_METADATA = CS_UNICODE_FSS;
which means Zeos would never be able to do right char conversations for all columns. That leads to pain since charset 'NONE' does exists.
I propose you recreate you database with either UTF8 or WIN1250 as default charset. Than firbird can do all conversations for the column collations.
i guess you're right for charset none, Michal. But Mssql is a weak database according functionality. Each free RDMS does support a lot of more functionality than Mssql. And Mssql doesn't support a neutral charset like utf8. That's one of the reasons for my implementations...
If that charset is used than no conversations where done from FireBird. But with my proposal you'll run into the same issue if other collations are defined for other columns.
I had a look to the column subtypes where firebird wrote in the article:
Code: Select all
Abstraction layers that have to manage this can read the low byte of the sqlsubtype field in the XSQLVAR structure, which contains the character set identifier.
CS_NONE = 0;
CS_BINARY = 1;
CS_ASCII = 2;
CS_UNICODE_FSS = 3;
CS_METADATA = CS_UNICODE_FSS;
which means Zeos would never be able to do right char conversations for all columns. That leads to pain since charset 'NONE' does exists.
I propose you recreate you database with either UTF8 or WIN1250 as default charset. Than firbird can do all conversations for the column collations.
Yes and no Michal.Is not that very similar to FreeTDS in Lazarus?
i guess you're right for charset none, Michal. But Mssql is a weak database according functionality. Each free RDMS does support a lot of more functionality than Mssql. And Mssql doesn't support a neutral charset like utf8. That's one of the reasons for my implementations...
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
miab3,
well i've debuggedd me down now.
But the ibx components do not support such an implementation!
which controlsCodepage is set for Lazarus? cCP_UTF or cCP_UTF16?
well i've debuggedd me down now.
that was terribly wrong! It seems firebird does it right. On our implementation is a lot missing. Starting right now, Michal...... ):But that's not right at all! They determine only these:
CS_NONE = 0;
CS_BINARY = 1;
CS_ASCII = 2;
CS_UNICODE_FSS = 3;
CS_METADATA = CS_UNICODE_FSS;
But the ibx components do not support such an implementation!
which controlsCodepage is set for Lazarus? cCP_UTF or cCP_UTF16?
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
miab3,
made a patch spezially for CharSet 'NONE'. I do now determine each column collation. To get it working right: TZConnection.ClientcodePage := 'NONE'.
Rev 2002 \testing
You see i use two different collations.
Both can be written and reconverted correctly:
made a patch spezially for CharSet 'NONE'. I do now determine each column collation. To get it working right: TZConnection.ClientcodePage := 'NONE'.
Rev 2002 \testing
Code: Select all
CREATE TABLE PEOPLE
(
P_ID Smallint NOT NULL,
P_DEP_ID Smallint,
P_NAME Varchar(40) CHARACTER SET UTF8,
P_BEGIN_WORK Time,
P_END_WORK Time,
P_PICTURE Blob sub_type 0,
P_RESUME Blob sub_type 1,
P_REDUNDANT Smallint,
BLABLA Varchar(10) CHARACTER SET WIN1252,
PRIMARY KEY (P_ID)
);
GRANT DELETE, INSERT, REFERENCES, SELECT, UPDATE
ON PEOPLE TO SYSDBA WITH GRANT OPTION;
Both can be written and reconverted correctly:
You do not have the required permissions to view the files attached to this post.
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
- EgonHugeist
- Zeos Project Manager
- Posts: 1936
- Joined: 31.03.2011, 22:38
To show you everything is rightly encoded, that's the way it looks with flame robin:
You do not have the required permissions to view the files attached to this post.
Best regards, Michael
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/
You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/