Oracle & UTF16 - Fetched column value was truncated (solved...?)

Forum related to Oracle Server

Moderators: gto, EgonHugeist, mdaems

MJFShark
Expert Boarder
Expert Boarder
Posts: 218
Joined: 04.06.2020, 13:59

Re: Oracle & UTF16 - Fetched column value was truncated (solved...?)

Post by MJFShark »

In ZDbcOracleUtils NormalizeOracleTypeToSQLType change:

Code: Select all

    if (ScaleOrCharSetForm = SQLCS_NCHAR) or (ConSettings.ClientCodePage.Encoding = ceUTF16)
    then DataSize := Precision shl 1
to

Code: Select all

    if (ScaleOrCharSetForm = SQLCS_NCHAR) or (ConSettings.ClientCodePage.Encoding = ceUTF16)
    then DataSize := Precision shl 2
And see if that solves it. I mentioned this in viewtopic.php?p=259618#p259618 but I don't think the change was put in.

-Mark
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 800
Joined: 18.11.2018, 17:37
Location: Hungary

Re: Oracle & UTF16 - Fetched column value was truncated (solved...?)

Post by aehimself »

Unfortunately the result is the same, returned value is still truncated.
Delphi 12.2, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmysql.dll 8.0.40 x64 5.7.19 x68, libmariadb.dll 3.3.11
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.15
- MSSQL 2012, 2019; sybdb.dll FreeTDS_3102
- SQLite 3.47
MJFShark
Expert Boarder
Expert Boarder
Posts: 218
Joined: 04.06.2020, 13:59

Re: Oracle & UTF16 - Fetched column value was truncated (solved...?)

Post by MJFShark »

Interesting! Can you narrow it down to the exact record or column that causes the problem? I'll be happy to take a look if it can be replicated. I don't see that error on any of the databases I have access to.

-Mark
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 800
Joined: 18.11.2018, 17:37
Location: Hungary

Re: Oracle & UTF16 - Fetched column value was truncated (solved...?)

Post by aehimself »

MJFShark wrote: 14.11.2024, 12:22Can you narrow it down to the exact record or column that causes the problem?
I don't know how much you changed the buffer logic, but previously it was not possible. It was not one record, but the amount of data handled UNTIL a specific number of records... I remember changing the order by, adding new filtering before and the error always appeared at different cycles / records.

I'll see if I can pinpoint it this time but don't get your hopes up :)

Maybe to create a temp table and dump the data there... will run a round with the legal team if that is possible. Names of sequences shouldn't contain too much proprietary information after all.
Delphi 12.2, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmysql.dll 8.0.40 x64 5.7.19 x68, libmariadb.dll 3.3.11
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.15
- MSSQL 2012, 2019; sybdb.dll FreeTDS_3102
- SQLite 3.47
MJFShark
Expert Boarder
Expert Boarder
Posts: 218
Joined: 04.06.2020, 13:59

Re: Oracle & UTF16 - Fetched column value was truncated (solved...?)

Post by MJFShark »

The underlying buffer logic hasn't really changed. Is the error consistently repeatable? Can you remove columns from the query until you figure out which one causes the issue? I also have debug code you can put inside the TZOracleResultSet.Next() that I used to debug the buffers (Oracle reports overflows in the indicator array and the DataLengths array so we can narrow it down to the column causing the problem.) Note that you can set the FIteration value to 1 to force a single record at a time (which should help you find the data causing the problem.)

> Maybe to create a temp table and dump the data there... will run a round with the legal team if that is possible.
I'd keep that as a "last resort" option. Let's try to pinpoint the error to a column and that may give us a clue what's going on. After that it should be possible to recreate it with pure "dummy" data (or at least let's hope lol.)

-Mark
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 800
Joined: 18.11.2018, 17:37
Location: Hungary

Re: Oracle & UTF16 - Fetched column value was truncated (solved...?)

Post by aehimself »

Just wondering... the query which fails has the following fields. As I understand LONG is just for backwards compatibility now, essentially it's a LOB.
Is it possible that this will also have something to do with the LOB storage mode (BasicFiles truncates, SecureFiles doesn't)?

I'll confirm this theory later...
You do not have the required permissions to view the files attached to this post.
Delphi 12.2, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmysql.dll 8.0.40 x64 5.7.19 x68, libmariadb.dll 3.3.11
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.15
- MSSQL 2012, 2019; sybdb.dll FreeTDS_3102
- SQLite 3.47
Post Reply