Bytea/blob - PostgreSQL - UNICODE/UTF8

aperger · Post by **aperger** » 26.07.2006, 01:46

Hi Developers,

I can find a solution to solve my problem

From the http://hup.hu pages I got a mail with 2 links:
http://sourceforge.net/tracker/index.ph ... tid=415824
http://sourceforge.net/tracker/index.ph ... tid=415826

These links provide me the solution. The patch was based on 6.1.5 stable release. I made a patch for revision 86 which based on this patch. Attached

Terence · Post by **Terence** » 26.07.2006, 08:16

sry, the download links don't work..

aperger · Post by **aperger** » 26.07.2006, 09:31

Hi

The link is working for me...
PATCH:

Code: Select all

Index: src/dbc/ZDbcPostgreSqlStatement.pas

===================================================================

--- src/dbc/ZDbcPostgreSqlStatement.pas	(revision 86)

+++ src/dbc/ZDbcPostgreSqlStatement.pas	(working copy)

@@ -377,9 +377,14 @@

             end

             else

             begin

+              (* removed by Perger -> based on SourceForge:

+              [ 1520587 ] Fix for 1484704: bytea corrupted on post when not using utf8,

+              file: 1484704.patch

               Result := EncodeString(TempBlob.GetString);

               Result := Copy(Result, 2, Length(Result) - 2);

               Result := EncodeString(Result);

+              *)

+              Result := EncodeBinaryString(TempBlob.GetString);

             end;

           end else

             Result := 'NULL';

Index: src/dbc/ZDbcPostgreSqlUtils.pas

===================================================================

--- src/dbc/ZDbcPostgreSqlUtils.pas	(revision 86)

+++ src/dbc/ZDbcPostgreSqlUtils.pas	(working copy)

@@ -87,6 +87,17 @@

 function EncodeString(Value: string): string; overload;

 

 {**

+  add by Perger -> based on SourceForge:

+  [ 1520587 ] Fix for 1484704: bytea corrupted on post when not using utf8,

+  file: 1484704.patch

+

+  Converts a binary string into escape PostgreSQL format.

+  @param Value a binary stream.

+  @return a string in PostgreSQL binary string escape format.

+}

+function EncodeBinaryString(Value: string): string;

+

+{**

   Determine the character code in terms of enumerated number.

   @param InputString the input string.

   @return the character code in terms of enumerated number.

@@ -618,7 +629,63 @@

   DestBuffer^ := '''';

 end;

 

+

 {**

+  add by Perger -> based on SourceForge:

+  [ 1520587 ] Fix for 1484704: bytea corrupted on post when not using utf8,

+  file: 1484704.patch

+

+  Converts a binary string into escape PostgreSQL format.

+  @param Value a binary stream.

+  @return a string in PostgreSQL binary string escape format.

+}

+function EncodeBinaryString(Value: string): string;

+var

+  I: Integer;

+  SrcLength, DestLength: Integer;

+  SrcBuffer, DestBuffer: PChar;

+begin

+  SrcLength := Length(Value);

+  SrcBuffer := PChar(Value);

+  DestLength := 2;

+  for I := 1 to SrcLength do

+  begin

+    if (Byte(SrcBuffer^) < 32) or (Byte(SrcBuffer^) > 126)

+    or (SrcBuffer^ in ['''', '\']) then

+      Inc(DestLength, 5)

+    else Inc(DestLength);

+    Inc(SrcBuffer);

+  end;

+

+  SrcBuffer := PChar(Value);

+  SetLength(Result, DestLength);

+  DestBuffer := PChar(Result);

+  DestBuffer^ := '''';

+  Inc(DestBuffer);

+

+  for I := 1 to SrcLength do

+  begin

+    if (Byte(SrcBuffer^) < 32) or (Byte(SrcBuffer^) > 126)

+    or (SrcBuffer^ in ['''', '\']) then

+    begin

+      DestBuffer[0] := '\';

+      DestBuffer[1] := '\';

+      DestBuffer[2] := Chr(Ord('0') + (Byte(SrcBuffer^) shr 6));

+      DestBuffer[3] := Chr(Ord('0') + ((Byte(SrcBuffer^) shr 3) and $07));

+      DestBuffer[4] := Chr(Ord('0') + (Byte(SrcBuffer^) and $07));

+      Inc(DestBuffer, 5);

+    end

+    else

+    begin

+      DestBuffer^ := SrcBuffer^;

+      Inc(DestBuffer);

+    end;

+    Inc(SrcBuffer);

+  end;

+  DestBuffer^ := '''';

+end;

+

+{**

   Converts an string from escape PostgreSQL format.

   @param Value a string in PostgreSQL escape format.

   @return a regular string.

Terence · Post by **Terence** » 26.07.2006, 10:03

hmm, i have no idea? i am using firefox 1.504 on WindowsXPSP2.
If i click on the paper clip nothing happens, indeed no link is showed by firfox there.
Maybe it has somesth. to do with rights of rank? Would be a bug..
Can any other team memeber click the download link successfully?

Btw this patch is not made for revision 59 of testing branch, i can't patch anything else (trunk),
Take also a look into this already on tesr-br. applied mods.
http://zeos.firmos.at/viewtopic.php?t=589

aperger · Post by **aperger** » 26.07.2006, 10:34

Hi. This patch was made for revision 86 (test branch) :-) Did you apply it or not?

Terence · Post by **Terence** » 26.07.2006, 10:46

well i tried, but it says "patch outdated"
Maybe stgh went wrong with copy/paste - i really need the the plain file!
That is the file i created from forum thread
http://cforce.dnsalias.org/files/Source ... ev86.patch
else mail me at "junk@vollbio.de"

Post by **mdaems** » 26.07.2006, 13:04

Fabian,

I applied the first patch without problems. Maybe you should try to get these links working in your browser. (For me it also works with Firefox1.0.7 as well,clickingonthe text 'download', not the paperclip)
I'll commit it soon.

Mark

Terence · Post by **Terence** » 26.07.2006, 13:11

Ahh, i found the solution with the non working download link! The Bug must be in language Profile "german" for Orion Forum soft, if switched to english it works.

Now patch works..

Patch applied to tesing branch!

Post by **mdaems** » 26.07.2006, 13:16

Did that before. Will probably give a collision... Sorry for that. I hoped you saw this:

mdaems wrote:Fabian,

I applied the first patch without problems.
....
I'll commit it soon.

Terence · Post by **Terence** » 26.07.2006, 13:20

Yep, i did - no problem.. 2 times holds better

aperger · Post by **aperger** » 26.07.2006, 13:29

It is ok.

THANX

Attila

belerophon · Post by **belerophon** » 02.08.2006, 18:33

Hello,

When a try to patch the files I got an error...

patch -pnum SourceForge_patch615stable_modified_for_rev86.patch
patch: **** strip count num is not a number

Do you know what I'm doing wrong?

belerophon · Post by **belerophon** » 02.08.2006, 22:10

Hei all,

1) I have downloaded the latest version of zeoslib, the file zeosdbo-6.5.1-alpha_cvs_13-10-2005.zip.
I also use the C++Builder 6.

2) As per the instruction found in the file installation.html, I installed the package and started to build applications.

3) Only when I need to use characters as "é"...UTF-8, in a SQL statement, I got an error message. SQL error. Invalid UTF-8 character near byte xxx.

Can somebody explain me ho to aply the pacth? I'm running a WXP box, but I can boot linux and mount a fat partition where I can put the sources of the package and apply the patch trought the patch program.
Also, which parameters do I have to use to it work fine?
And a easy way, can I download the files patched?

Thaks and regards,

Post by **mdaems** » 03.08.2006, 08:22

Hi Belerophon,

The version you have downloaded is really outdated. All new development is done using a SVN repository. So patches are made against SVN versions.
However, I have packaged ZEOS myself (meaning : it's not 'official') yesterday and you can find it here : http://users.telenet.be/mdaems I update this once in a few weeks.

BTW : if you're using mysql there's a problem for C++ builder not yet included in the download. Youget an AV when compiling. Please see a recent bug report in the buglist forum. There you also find info about the SVN repository.

Mark

belerophon · Post by **belerophon** » 03.08.2006, 14:43

Mark,

Thank you!!! I'll download it and install. Actually, I'm using PostgreSql.
In truth, I'm starting now to use zeoslib, so I'll have to read a lot to understand how development and new versions are.
Thanks again,
Roberto.