Page 1 of 2

Bytea/blob - PostgreSQL - UNICODE/UTF8

Posted: 26.07.2006, 01:46
by aperger
Hi Developers,

I can find a solution to solve my problem

From the http://hup.hu pages I got a mail with 2 links:
http://sourceforge.net/tracker/index.ph ... tid=415824
http://sourceforge.net/tracker/index.ph ... tid=415826

These links provide me the solution. The patch was based on 6.1.5 stable release. I made a patch for revision 86 which based on this patch. Attached

Posted: 26.07.2006, 08:16
by Terence
sry, the download links don't work..

Posted: 26.07.2006, 09:31
by aperger
Hi

The link is working for me...
PATCH:

Code: Select all

Index: src/dbc/ZDbcPostgreSqlStatement.pas

===================================================================

--- src/dbc/ZDbcPostgreSqlStatement.pas	(revision 86)

+++ src/dbc/ZDbcPostgreSqlStatement.pas	(working copy)

@@ -377,9 +377,14 @@

             end

             else

             begin

+              (* removed by Perger -> based on SourceForge:

+              [ 1520587 ] Fix for 1484704: bytea corrupted on post when not using utf8,

+              file: 1484704.patch

               Result := EncodeString(TempBlob.GetString);

               Result := Copy(Result, 2, Length(Result) - 2);

               Result := EncodeString(Result);

+              *)

+              Result := EncodeBinaryString(TempBlob.GetString);

             end;

           end else

             Result := 'NULL';

Index: src/dbc/ZDbcPostgreSqlUtils.pas

===================================================================

--- src/dbc/ZDbcPostgreSqlUtils.pas	(revision 86)

+++ src/dbc/ZDbcPostgreSqlUtils.pas	(working copy)

@@ -87,6 +87,17 @@

 function EncodeString(Value: string): string; overload;

 

 {**

+  add by Perger -> based on SourceForge:

+  [ 1520587 ] Fix for 1484704: bytea corrupted on post when not using utf8,

+  file: 1484704.patch

+

+  Converts a binary string into escape PostgreSQL format.

+  @param Value a binary stream.

+  @return a string in PostgreSQL binary string escape format.

+}

+function EncodeBinaryString(Value: string): string;

+

+{**

   Determine the character code in terms of enumerated number.

   @param InputString the input string.

   @return the character code in terms of enumerated number.

@@ -618,7 +629,63 @@

   DestBuffer^ := '''';

 end;

 

+

 {**

+  add by Perger -> based on SourceForge:

+  [ 1520587 ] Fix for 1484704: bytea corrupted on post when not using utf8,

+  file: 1484704.patch

+

+  Converts a binary string into escape PostgreSQL format.

+  @param Value a binary stream.

+  @return a string in PostgreSQL binary string escape format.

+}

+function EncodeBinaryString(Value: string): string;

+var

+  I: Integer;

+  SrcLength, DestLength: Integer;

+  SrcBuffer, DestBuffer: PChar;

+begin

+  SrcLength := Length(Value);

+  SrcBuffer := PChar(Value);

+  DestLength := 2;

+  for I := 1 to SrcLength do

+  begin

+    if (Byte(SrcBuffer^) < 32) or (Byte(SrcBuffer^) > 126)

+    or (SrcBuffer^ in ['''', '\']) then

+      Inc(DestLength, 5)

+    else Inc(DestLength);

+    Inc(SrcBuffer);

+  end;

+

+  SrcBuffer := PChar(Value);

+  SetLength(Result, DestLength);

+  DestBuffer := PChar(Result);

+  DestBuffer^ := '''';

+  Inc(DestBuffer);

+

+  for I := 1 to SrcLength do

+  begin

+    if (Byte(SrcBuffer^) < 32) or (Byte(SrcBuffer^) > 126)

+    or (SrcBuffer^ in ['''', '\']) then

+    begin

+      DestBuffer[0] := '\';

+      DestBuffer[1] := '\';

+      DestBuffer[2] := Chr(Ord('0') + (Byte(SrcBuffer^) shr 6));

+      DestBuffer[3] := Chr(Ord('0') + ((Byte(SrcBuffer^) shr 3) and $07));

+      DestBuffer[4] := Chr(Ord('0') + (Byte(SrcBuffer^) and $07));

+      Inc(DestBuffer, 5);

+    end

+    else

+    begin

+      DestBuffer^ := SrcBuffer^;

+      Inc(DestBuffer);

+    end;

+    Inc(SrcBuffer);

+  end;

+  DestBuffer^ := '''';

+end;

+

+{**

   Converts an string from escape PostgreSQL format.

   @param Value a string in PostgreSQL escape format.

   @return a regular string.


Posted: 26.07.2006, 10:03
by Terence
hmm, i have no idea? i am using firefox 1.504 on WindowsXPSP2.
If i click on the paper clip nothing happens, indeed no link is showed by firfox there.
Maybe it has somesth. to do with rights of rank? Would be a bug..
Can any other team memeber click the download link successfully?

Btw this patch is not made for revision 59 of testing branch, i can't patch anything else (trunk),
Take also a look into this already on tesr-br. applied mods.
http://zeos.firmos.at/viewtopic.php?t=589

Posted: 26.07.2006, 10:34
by aperger
Hi. This patch was made for revision 86 (test branch) :-) Did you apply it or not?

Posted: 26.07.2006, 10:46
by Terence
well i tried, but it says "patch outdated"
Maybe stgh went wrong with copy/paste - i really need the the plain file!
That is the file i created from forum thread
http://cforce.dnsalias.org/files/Source ... ev86.patch
else mail me at "junk@vollbio.de"

Posted: 26.07.2006, 13:04
by mdaems
Fabian,

I applied the first patch without problems. Maybe you should try to get these links working in your browser. (For me it also works with Firefox1.0.7 as well,clickingonthe text 'download', not the paperclip)
I'll commit it soon.

Mark

Posted: 26.07.2006, 13:11
by Terence
Ahh, i found the solution with the non working download link! The Bug must be in language Profile "german" for Orion Forum soft, if switched to english it works.

Now patch works..


Patch applied to tesing branch!

Posted: 26.07.2006, 13:16
by mdaems
Did that before. Will probably give a collision... Sorry for that. I hoped you saw this:
mdaems wrote:Fabian,

I applied the first patch without problems.
....
I'll commit it soon.

Posted: 26.07.2006, 13:20
by Terence
Yep, i did - no problem.. 2 times holds better ;)

Posted: 26.07.2006, 13:29
by aperger
It is ok.

THANX

Attila

Posted: 02.08.2006, 18:33
by belerophon
Hello,

When a try to patch the files I got an error...

patch -pnum SourceForge_patch615stable_modified_for_rev86.patch
patch: **** strip count num is not a number

Do you know what I'm doing wrong?

Posted: 02.08.2006, 22:10
by belerophon
Hei all,

1) I have downloaded the latest version of zeoslib, the file zeosdbo-6.5.1-alpha_cvs_13-10-2005.zip.
I also use the C++Builder 6.

2) As per the instruction found in the file installation.html, I installed the package and started to build applications.

3) Only when I need to use characters as "é"...UTF-8, in a SQL statement, I got an error message. SQL error. Invalid UTF-8 character near byte xxx.

Can somebody explain me ho to aply the pacth? I'm running a WXP box, but I can boot linux and mount a fat partition where I can put the sources of the package and apply the patch trought the patch program.
Also, which parameters do I have to use to it work fine?
And a easy way, can I download the files patched?

Thaks and regards,

Posted: 03.08.2006, 08:22
by mdaems
Hi Belerophon,

The version you have downloaded is really outdated. All new development is done using a SVN repository. So patches are made against SVN versions.
However, I have packaged ZEOS myself (meaning : it's not 'official') yesterday and you can find it here : http://users.telenet.be/mdaems I update this once in a few weeks.

BTW : if you're using mysql there's a problem for C++ builder not yet included in the download. Youget an AV when compiling. Please see a recent bug report in the buglist forum. There you also find info about the SVN repository.

Mark

Posted: 03.08.2006, 14:43
by belerophon
Mark,

Thank you!!! I'll download it and install. Actually, I'm using PostgreSql.
In truth, I'm starting now to use zeoslib, so I'll have to read a lot to understand how development and new versions are.
Thanks again,
Roberto.