Page 1 of 1
[patch_done] About Unicode and TStringField
Posted: 26.11.2008, 11:00
by mariuszekpl
When I look in delphi 2009 in db.pas i see:
TStringField = class(TField)
protected
function GetAsString: string; override;
public
property Value: AnsiString read GetAsAnsiString write SetAsAnsiString;
And function GetAsString return GetAsAnsiString
In default TStringField i can't see full unicode
when i use .value or .asString
May by we should replace TStringField with TZStringField and add in TZStringField support for unicode ?
STEP 1
Posted: 28.11.2008, 00:04
by mariuszekpl
1) Add new class for string field
Code: Select all
TZStringField = class (TWideStringField)
private
buf : PByte;
public
constructor Create(AOwner: TComponent); override;
destructor Destroy; override;
class procedure CheckTypeSize(Value: Integer); override;
function GetAsString: string; override;
function GetAsVariant: Variant; override;
function GetValue(var Value: string): Boolean;
procedure SetAsString(const Value: string); override;
property Value: UnicodeString read GetAsWideString write SetAsWideString;
end;
{ TZStringField }
class procedure TZStringField.CheckTypeSize(Value: Integer);
begin
if (Value < 0) then
DatabaseError(SInvalidFieldSize);
end;
constructor TZStringField.Create(AOwner: TComponent);
begin
inherited Create(AOwner);
end;
destructor TZStringField.Destroy;
begin
if Assigned(Buf) then
FreeMem(Buf);
inherited;
end;
function TZStringField.GetAsString: string;
begin
if not GetValue(Result) then
Result := '';
end;
function TZStringField.GetAsVariant: Variant;
var
S: string;
begin
if GetValue(S) then
Result := S
else
Result := Null;
end;
function TZStringField.GetValue(var Value: string): Boolean;
begin
if not Assigned(Buf) then
GetMem(Buf, (Size + 1) * SizeOf(PChar));
Result := GetData(Buf);
if Result then
Value := String(PChar(Buf))
end;
procedure TZStringField.SetAsString(const Value: string);
begin
SetData(Pointer(Value), False);
end;
2) override function TZAbstractRODataset.GetFieldClass
Code: Select all
{$IFDEF ZEOS_FULL_UNICODE}
function TZAbstractRODataset.GetFieldClass(FieldType: TFieldType): TFieldClass;
begin
if (FieldType in [ftString,ftFixedChar,ftWideString]) then
result := TZStringField
else
result := inherited ;
end;
{$ENDIF}
Step 2
Posted: 28.11.2008, 00:10
by mariuszekpl
3) Add support unicode for TZSQLiteResultSet
Code: Select all
TZSQLiteResultSet = class(TZAbstractResultSet)
...
...
public
{$IFDEF ZEOS_FULL_UNICODE}
function GetUnicodeString(ColumnIndex: Integer): WideString; override;
{$ENDIF}
{$IFDEF ZEOS_FULL_UNICODE}
function TZSQLiteResultSet.GetUnicodeString(ColumnIndex: Integer): WideString;
var
Buffer: PAnsiChar;
begin
Buffer := GetPChar(ColumnIndex);
if Buffer <> nil then
Result := UTF8ToUnicodeString(StrPas(Buffer))
else
Result := '';
end;
{$ENDIF}
Step 3
Posted: 28.11.2008, 00:15
by mariuszekpl
4) function ConvertSQLiteTypeToSQLType return stUnicodeString for stString field
Code: Select all
if Result = stString then
result := stUnicodeString;
My test code:
Posted: 28.11.2008, 00:24
by mariuszekpl
Code: Select all
procedure TForm2.Button4Click(Sender: TObject);
begin
ZConnection1.Database := 'c:\dane-ŁŁŁĘĘĘĘ-ЫФЫФДЛДЛДЛДкенке.db';
ZConnection1.Connected := true;
ZQuery1.SQL.Text:='drop table прапрпра3;' ;
ZQuery1.ExecSQL;
ZQuery1.SQL.Text:='create table прапрпра3(id integer NOT NULL PRIMARY KEY,daneЙЦЦУ varchar(11) );' ;
ZQuery1.ExecSQL;
ZQuery1.SQL.Text:='INSERT into прапрпра3(id ,daneЙЦЦУ) values ( 101,''dfg1 вапвап'') ';
ZQuery1.ExecSQL;
ZQuery1.SQL.Text:='INSERT into прапрпра3(id ,daneЙЦЦУ) values ( 102,''dfg2 вапвап'') ';
ZQuery1.ExecSQL;
ZQuery1.SQL.Text:='INSERT into прапрпра3(id ,daneЙЦЦУ) values ( 103,''dfg3 вапвап'') ';
ZQuery1.ExecSQL;
ZQuery1.SQL.Text:='select * from прапрпра3;' ;
zquery1.Open;
caption := ZQuery1.FieldByName('daneЙЦЦУ').AsString;
DataSource1.DataSet := zquery1;
end;
Posted: 03.12.2008, 12:00
by mdaems
Do I understand well this is a D2009 shortcoming?
Or does D2009 provide a new field_type for unicode-strings?
In that case we shouldn't add our own field type but make sure zeoslib decides to use the D2009 replacement for the (Anis) TStringfield. Isn't it better to use TWideStringField? That one uses a Unicodestring as Value?
Code: Select all
TWideStringField = class(TStringField)
protected
class procedure CheckTypeSize(Value: Integer); override;
procedure CopyData(Source, Dest: Pointer); override;
function GetAsAnsiString: AnsiString; override;
function GetAsString: string; override;
function GetAsVariant: Variant; override;
function GetAsWideString: UnicodeString; override;
function GetDataSize: Integer; override;
procedure GetText(var Text: string; DisplayText: Boolean); override;
function GetValue(var Value: UnicodeString): Boolean;
procedure SetAsAnsiString(const Value: AnsiString); override;
procedure SetAsString(const Value: string); override;
procedure SetVarValue(const Value: Variant); override;
procedure SetAsWideString(const Value: UnicodeString); override;
public
constructor Create(AOwner: TComponent); override;
property Value: UnicodeString read GetAsWideString write SetAsWideString;
end;
Posted: 05.12.2008, 00:27
by mariuszekpl
I can't understund why TstringField.GetAsString return ansistring
but we can't use TstringField if we want add unicode suport.
In new delphi string == widestring we don't need two types: TstringField and TWideStringField
We can use common type for all string field.
Posted: 05.12.2008, 12:10
by mariuszekpl
We have two roads:
1) Use one common TZStringField for Unicode and Ansi strings (like IBX)
or
2) User can select what he want , if he has data with unicode he should inform ZEOS about it , and now zeos use internal UNICODE , if we want ANSI zeos internal use ANSI
In my opinion first way is better, we stop thinking about difference STRING / UNICODE STRING, but i can make patch with both
In my opinion internal ZEOS should use Unocode to storage internal data
(in d2009) because it is default string format in delphi 2009
Posted: 06.12.2008, 15:56
by mdaems
mariuszekpl,
In a first approach (get D2009 basically supported quickly with almost no impact on the other compilers) I would like a patch like this:
Code: Select all
Index: ZDatasetUtils.pas
===================================================================
--- ZDatasetUtils.pas (revision 535)
+++ ZDatasetUtils.pas (working copy)
@@ -301,7 +301,7 @@
stFloat, stDouble, stBigDecimal:
Result := ftFloat;
stString:
- Result := ftString;
+ Result := {$IFDEF ZEOS_FULL_UNICODE}ftWideString{$ELSE}ftString{$ENDIF};
stBytes:
Result := ftBytes;
stDate:
But I need you to check what extra changes are needed to get this working. (Test suite shows me only changing this causes more errors on string fields)
Could you please have a look at it?
In a later stadium we can talk about adding TZXXXFieldTypes. For the moment Zeoslib only contained TDataset descendants. I'm not sure how adding TField descendants would affect the use of zeoslib datasets in general.
Mark
Posted: 07.12.2008, 02:24
by mariuszekpl
Your patch work
BUT
this change is not enough.
We can use TWideString field to display data and it works with unicode and ansiString but this is not enough.
In TZSQLType we have stString, stUnicodeString
If we want use unicode we shoud use stUnicodeString in all place where is stString
Posted: 13.12.2008, 11:24
by mdaems
Applied your (a little modified) patch. At least for mysql this passes the test suite without problems, even with Delphi 2009.
SVN rev. 543
Posted: 02.03.2009, 23:20
by bravecobra
This patch hasn't been applied to the testing branch as such? I see a
Code: Select all
Result := {$IFDEF DELPHI12_UP}ftWideString{$ELSE}ftString{$ENDIF};
, which no longer offers the choice to the user.
I have code with persistent fields, which now gives an error (ftWideString <> ftString). Shouldn't we be building something that can make the user choose which one to default to.
Does this currently mean I have to replace all my TStringField declarations as TWideStringField??
Posted: 05.03.2009, 12:50
by mdaems
Brave,
I don't think so. But I have no real Delphi 2009 experience. So just test and prove I'm right or wrong. As far as I understand the new situations is comparable to the default D2009 field type choice. But if you have an existing application with TStringFields it seems to me like the behaviour would still be the same as before. I think the code you mention above only applies when defining new fields using the IDE or at runtime when opening the query.
Mark