Database Unicode Support Table

The alpha/beta tester's forum for ZeosLib 7.0.x series

Report problems concerning our Delphi 2009+ version and new Zeoslib 7.0 features here.

This is a forum that will be removed once the 7.X version goes into stable!!

Moderators: gto, EgonHugeist, olehs

Locked
gto
Zeos Dev Team
Zeos Dev Team
Posts: 278
Joined: 11.11.2005, 18:35
Location: Porto Alegre / Brasil

Database Unicode Support Table

Post by gto »

We're trying to find out what databases support unicode and how they do that. Please help us!

Values for fields are: Unknown, Partial and Full, except for encoding, which can be UTF-8, UTF-16LE (Windows Widechar default), UTF-16BE, UTF-32, UCS-2 and UCS-4.

Protocol: ADO (Active Data Objects)
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: ASA11
  • Support for Unicode: Full
  • Field Names in Unicode: Full
  • Table Names in Unicode: Full
  • Database name in Unicode: Full
  • Buffers (SQL, commands) in Unicode: Full
  • Encodings supported for Buffers: UTF-8
  • String Field (or Codepage) Unicode: Full
  • Encodings supported for String Field Unicode: UTF-8
Protocol: Firebird 1.0
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: Firebird 1.5
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: Firebird 2.0
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: Firebird 2.1
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: Interbase 5.0
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: Interbase 5.0
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: Interbase 6.0
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: mssql (Microsoft SQL Server)
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: MySQL 4.1
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: MySQL 5.0
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: oracle
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown

Protocol: PostgreSQL 7
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: PostgreSQL 8
  • Support for Unicode: Full
  • Field Names in Unicode: Full
  • Table Names in Unicode: Full
  • Database name in Unicode: Full
  • Buffers (SQL, commands) in Unicode: Full
  • Encodings supported for Buffers: UTF-8
  • String Field (or Codepage) Unicode: Full
  • Encodings supported for String Field Unicode: UTF-8
Protocol: SQLite 2.8
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Protocol: PostgreSQL 8
  • Support for Unicode: Full
  • Field Names in Unicode: Full
  • Table Names in Unicode: Full
  • Database name in Unicode: Full
  • Buffers (SQL, commands) in Unicode: Full
  • Encodings supported for Buffers: UTF-8/UTF-16
  • String Field (or Codepage) Unicode: Full
  • Encodings supported for String Field Unicode: UTF-8/UTF-16
Protocol: sybase
  • Support for Unicode: Unknown
  • Field Names in Unicode: Unknown
  • Table Names in Unicode: Unknown
  • Database name in Unicode: Unknown
  • Buffers (SQL, commands) in Unicode: Unknown
  • Encodings supported for Buffers: Unknown
  • String Field (or Codepage) Unicode: Unknown
  • Encodings supported for String Field Unicode: Unknown
Last edited by gto on 10.12.2008, 12:49, edited 2 times in total.
Use the FU!!!!!IN Google !

gto's Zeos Quick Start Guide

Te Amo Taís!
Top
mariuszekpl
Senior Boarder
Senior Boarder
Posts: 54
Joined: 30.09.2008, 10:59

Post by mariuszekpl »

Is not easy check/find IT :(

What does it mean: "Encodings supported for Buffers" ?
List all Encodings supported by database ?

We REALLY NEED this list ? For all databases ?

Maybe we should think about abstract layer to convert encoding.
DATABASE -> ZEOS
ZEOS -> DATABASE ?
With support all encodings :)
Conversion is only when we get data from database and when we post data to database.




Protocol: ASA11

* Support for Unicode: Full (UTF-8 )
* Field Names in Unicode: Full (UTF-8 )
* Table Names in Unicode: Full (UTF-8 )
* Database name in Unicode: Unknown (I can't create/open database, but may be it work and only BUG is in my client)
* Buffers (SQL, commands) in Unicode: Full (UTF-8 )
* Encodings supported for Buffers: (UTF-8 )
* String Field (or Codepage) Unicode: UTF-8
* Encodings supported for String Field Unicode: UTF-8


Protocol: PostgreSQL 8

* Support for Unicode: Full (UTF-8 )
* Field Names in Unicode: Full (UTF-8 )
* Table Names in Unicode: Full (UTF-8 )
* Database name in Unicode: Full (UTF-8 )
* Buffers (SQL, commands) in Unicode: Full (UTF-8 )
* Encodings supported for Buffers: Full (UTF-8 )
* String Field (or Codepage) Unicode: Full (UTF-8 )
* Encodings supported for String Field Unicode: Full (UTF-8 )

Protocol: SQLite 3

* Support for Unicode: Full (UTF-8 )
* Field Names in Unicode: Full (UTF-8 )
* Table Names in Unicode: Full (UTF-8 )
* Database name in Unicode: Full (UTF-8 )
* Buffers (SQL, commands) in Unicode: Full (UTF-8 )
* Encodings supported for Buffers: Full (UTF-8 )
* String Field (or Codepage) Unicode: Full (UTF-8 )
* Encodings supported for String Field Unicode: Full (UTF-8 )
sqlite support utf-16 too :)
Top
User avatar
mdaems
Zeos Project Manager
Zeos Project Manager
Posts: 2766
Joined: 20.09.2005, 15:28
Location: Brussels, Belgium
Contact:
Contact mdaems

Post by mdaems »

Are you using ASA? Welcome aboard. You're the first one I know about ;)

Concerning 'Do we need this?' Not sure, but it could give us an indication of what's possible/needed.

I agree on 'We should think about it.'

My vision (and gto's, I believe) is we should stop working on 'Full unicode support' NOW. And start working on 'Get current functionality working on D2009'. When this accidentally helps forward the 'full support', right.
We must also make sure we don't break stuff for people who are now encoding theirs strings in an exotic encoding before putting them in a TZQuery and are setting the database client encoding on the connection (talking about mysql now). I believe that's the way people used to code when using non-evident character sets before D2009 was available?
So I believe best we could do is get a D2009 compliant version 7.0 available ASAP.

Next phase could be to make the internal encoding in Zeoslib UTF-8 (seems the best convention?) and make the drivers do the necessary conversions.
Do you think there could be a reason to let the user choose a client encoding instead of UTF-8 only because the database COULD accept other encodings?Let's call that version 7.1 for the moment.

Mark
Image
Top
mariuszekpl
Senior Boarder
Senior Boarder
Posts: 54
Joined: 30.09.2008, 10:59

Post by mariuszekpl »

I don't use ASA11 i only test zeos , but i can't find other version and i test only ASA11 :)

I think we don't need all databases.
We realy need sqlite2 ? Firebird 1 ? etc. this protocol is very OLD

We should thing about better zeos core.
Yes! we should stop working on 'Full unicode support' NOW "at a push"
Top
gto
Zeos Dev Team
Zeos Dev Team
Posts: 278
Joined: 11.11.2005, 18:35
Location: Porto Alegre / Brasil

Post by gto »

Hi there :)

Agreed with the conversion layer! On top of Plain, following the unicode mantra: "Convert to unicode as soon as possible".

And I say that when we support full unicode (Zeos 7.1, officialy?), the internal encoding should be done with UTF-16 Little Endian, which is Delphi 2009 AND Windows default.
These systems can handle UTF-16 without much impact on performance, while UTF-8 still simpler, it need to be converted from and to UTF-16 (which is, in fact, slow).
Use the FU!!!!!IN Google !

gto's Zeos Quick Start Guide

Te Amo Taís!
Top
mariuszekpl
Senior Boarder
Senior Boarder
Posts: 54
Joined: 30.09.2008, 10:59

Post by mariuszekpl »

But delphi before 2009 works with UTF-16 ?
Top
User avatar
mdaems
Zeos Project Manager
Zeos Project Manager
Posts: 2766
Joined: 20.09.2005, 15:28
Location: Brussels, Belgium
Contact:
Contact mdaems

Post by mdaems »

Let's discuss that later...
Now we must get it working 'as is'. I want to get the changes moved into trunk.

Mark
Image
Top
gto
Zeos Dev Team
Zeos Dev Team
Posts: 278
Joined: 11.11.2005, 18:35
Location: Porto Alegre / Brasil

Post by gto »

mariuszekpl wrote:But delphi before 2009 works with UTF-16 ?
before? Widestring, maybe.. UTF8 in D2007 and previous is just a mapping to regular string..
mdaems wrote:Let's discuss that later...
Now we must get it working 'as is'. I want to get the changes moved into trunk.

Mark
Right!

7.0 with ANSI support in D2009
7.1 with unicode, as deep as possible
Use the FU!!!!!IN Google !

gto's Zeos Quick Start Guide

Te Amo Taís!
Top
Locked

Return to “ZeosLib 7.0 Beta Forum”