Possible issue with 64bit Oracle

[MJFShark]

Hi all.

I use FastMM5's debug mode to check my programs for heap corruption and memory leaks. When I run any queries using the Oracle protocol in 64bit while using FastMM_EnterDebugMode I get a "Memory Corruption Detected" error. Here's the info on the error and the simple console mode program I used to test it. I tested with a few different Oracle Client versions with the same results. This is with the latest Zeos 8 source:

---------------------------
Memory Corruption Detected
---------------------------
A memory block footer has been corrupted.
The block size is 52.
The block was allocated by thread 0x30C0, and the stack trace (return addresses) at the time was:

0000000001028E7A [FastMM5.pas][FastMM5][_ZN7Fastmm532FastMM_DebugGetMem_GetDebugBlockExb][7793]
0000000001028F30 [FastMM5.pas][FastMM5][_ZN7Fastmm518FastMM_DebugGetMemEx][7820]
0000000001009984 [System.pas][System][_ZN6System7_GetMemEx][4923]
000000000149B813 [ZDbcOracleUtils.pas][ZDbcOracleUtils][_ZN15Zdbcoracleutils21AllocateOracleSQLVarsERPNS_9TZSQLVarsEi][1167]
0000000001491899 [ZDbcOracleResultSet.pas][ZDbcOracleResultSet][_ZN19Zdbcoracleresultset17TZOracleResultSet4OpenEv][2411]
0000000001019A7E [System.pas][System][_ZN6System10_IntfClearERNS_15DelphiInterfaceINS_10IInterfaceEEE][39015]
000000000101B7D9 [System.pas][System][_ZN6System17TInterfacedObject14QueryInterfaceERK5_GUIDPv][40152]
0000000001019AB5 [System.pas][System][_ZN6System9_IntfCopyERNS_15DelphiInterfaceINS_10IInterfaceEEES2_][39048]
0000000001019B22 [System.pas][System][_ZN6System9_IntfCastERNS_15DelphiInterfaceINS_10IInterfaceEEES2_RK5_GUID][39128]
000000000148B2A4 [ZDbcOracleResultSet.pas][ZDbcOracleResultSet][_ZN19Zdbcoracleresultset25TZOracleAbstractResultSetC3EN6System15DelphiInterfaceIN9Zdbcintfs11IZStatementEEENS1_13UnicodeStringEPvS7_i][738]
00000000014A6D56 [ZDbcOracleStatement.pas][ZDbcOracleStatement][_ZN19Zdbcoraclestatement25TZAbstractOracleStatement15CreateResultSetEv][437]
00000000012967B6 [ZDbcStatement.pas][ZDbcStatement][_ZN13Zdbcstatement27TZAbstractPreparedStatement28PrepareOpenResultSetForReUseEv][3842]
00000000014A77E9 [ZDbcOracleStatement.pas][ZDbcOracleStatement][_ZN19Zdbcoraclestatement25TZAbstractOracleStatement20ExecuteQueryPreparedEv][521]
00000000012983AC [ZDbcStatement.pas][ZDbcStatement][_ZN13Zdbcstatement27TZAbstractPreparedStatement7SetWSQLEN6System13UnicodeStringE][4296]
000000000129428C [ZDbcStatement.pas][ZDbcStatement][_ZN13Zdbcstatement27TZAbstractPreparedStatement12ExecuteQueryEN6System13UnicodeStringE][3306]
00000000014C445A [ZDbcOracle.pas][ZDbcOracle][_ZN10Zdbcoracle18TZOracleConnection10GetCatalogEv][1025]
000000000100E573 [System.pas][System][_ZN6System8_WriteLnERNS_8TTextRecE][15311]
0000000001688AC5 [ZOraCatTest.dpr][ZOraCatTest][_ZN11Zoracattest14initializationEv][41]

Here's the code:

Code: Select all

program ZOraCatTest;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  FastMM5,
  System.SysUtils,
  System.Diagnostics,
  ZConnection;

var
  ZConn: TZConnection;
  S: String;
begin
  {$IFDEF DEBUG}
  ReportMemoryLeaksOnShutdown := True;
  FastMM_EnterDebugMode;
  // FastMM_DebugMode_ScanForCorruptionBeforeEveryOperation := True;
  FastMM_MessageBoxEvents := FastMM_MessageBoxEvents + [mmetUnexpectedMemoryLeakSummary];
  {$ENDIF}

  try
    ZConn := TZConnection.Create(nil);

    ZConn.Protocol := 'Oracle';
    ZConn.User := 'SHARKDB';
    ZConn.Password := 'scalesDB4411';
    ZConn.Database := 'benthic_tp';
    {$IFDEF WIN64}
      ZConn.LibraryLocation := 'C:\U\DbClients\Oracle\21.8\64Bit\oci.dll';
    {$ELSE}
      ZConn.LibraryLocation := 'C:\U\DbClients\Oracle\21.8\32Bit\oci.dll';
    {$ENDIF}

    ZConn.Connect;

    Writeln('Connected!');
    Writeln('In 64bit with FastMM5''s Debug mode, this causes a block footer corruption.');
    S := ZConn.DbcConnection.GetCatalog;
    Writeln('  Connected Schema is: ' + S);
    Writeln('');
    Writeln('Done');

    ZConn.Disconnect;
    ZConn.Free;

  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;

  Writeln('');
  Write('Press Enter to quit...');
  ReadLn;

  {$IFDEF DEBUG}
  FastMM_ExitDebugMode;
  {$ENDIF}

end.

Note that this doesn't happen in 32bit mode and it also doesn't cause any noticeable issues when run in 64bit without FastMM. It may indicate an issue or perhaps someone more savvy can say that it's "as expected". Thanks!

-Mark

[aehimself]

Welcome to the club.

Something is definitely not right with Oracle.

viewtopic.php?f=37&t=175546
viewtopic.php?f=37&t=152220
viewtopic.php?f=37&t=137189
viewtopic.php?f=37&t=128973

I'm using the 64bit library as well.

[aehimself]

I can confirm, that building a 32bit application and using the 32 bit 21.7 oci.dll no AV and no "user requested..." exception is being raised when using LobCacheMode = lcmOnLoad when opening the exact same dataset.

[MJFShark]

I found out what's causing this particular heap issue. In AllocateOracleSQLVars it dynamically allocates memory for the TZSQLVars record, but I think it's assuming that the record is packed and so the allocation doesn't take into account the record field alignment setting (which defaults to Quad word in recent versions of Delphi.) Regardless, the issue can be "fixed" for now by just allocating more memory.
So change

Code: Select all

Size := SizeOf(ub4) + Max(1,Count) * SizeOf(TZSQLVar);

to

Code: Select all

Size := (2 * SizeOf(ub4)) + Max(1,Count) * SizeOf(TZSQLVar);

But note that that assumes Delphi's default quad word alignment. I'll do some more testing and see what else I can find.

-Mark

[aehimself]

This won't fix the AV. I already tried with 10000 the size of the whole equation, still being thrown.

Btw, I suspect that instead of multiplying by 2 here, ub4 has to be NativeUInt instead of Cardinal.
Changing that requires changing half of Zeos, though (including other protocols) so I'm either unsure if this is the correct thing to change or if half of intervals are wrongly declared throughout the package.

[MJFShark]

Note that this change only fixes the one issue I mentioned in my first post in this thread. I'm not suggesting it fixes anything beyond that. If there are other areas that are affected by memory allocations that are done in this same manner, they should be looked at for similar issues.

-Mark

[aehimself]

Either MadExpect's buffer underrun / overrun protection does not work or FastMM's is faulty.

First time in my life I tried FastMM with FullDebugMode and I'm getting popup after popup when connecting to an Oracle database using UTF16.
That has nothing to do with the x64 issue we both discovered but you identified - but is reinforcing my suspicions that Zeos has some serious issues with the Oracle protocol.

I'll attempt to fix stuff when I'm available until at least a query can be executed without any errors and see where we can go from there.

[MJFShark]

I can verify that what I fixed was an actual memory overrun (you can tell because the struct wasn't "zeroed" all the way to the end. If you have a small app that shows the UTF-16 charset issue I can take a quick look. Note that Zeos doesn't support the OCI_UTF16 connection mode setting (which is more comprehensive than using the OCI_UTF16ID charset.) I do all my stuff with zeos as utf8.

-Mark

[aehimself]

The problem is not the application - it's the data. From what I saw a large amount of VarChar fields in a large amount of records will trigger it however - the data I'm using for testing is however obfuscated it's still proprietary and might still resemble to real-world data... exporting and sharing it could get me in trouble which I clearly don't want.
Playing around with INTERNAL_BUFFER_SIZE makes the issue better (or worse, that is) as it gets rid of the exception but it might show fields as empty whereas there is actual data in the database.

When I first recognized it I offered that I can attempt to consult our legal team to check my options. I can offer the same now - I just can not guarantee results.

[miab3]

Hi,

Which version of Delphi?
It seems to me that there is a bug in Delphi11.1 upd1 Win64 that wasn't there before.

Michal

[aehimself]

Delphi 11.2 defaulted to the wrong DCP folder afaik which could be manually fixed (https://en.delphipraxis.net/topic/7434- ... ment=62811).

I had that issue fixed by hand, currently I'm on 11.3 where this was already fixed.

[miab3]

That's not the point.
Try to test the same in Delphi 10.3(Win64), 10.4(Win64) or 11.1(Win64) without update1 (if you can).

Nichal

[marsupilami]

When I first recognized it I offered that I can attempt to consult our legal team to check my options. I can offer the same now - I just can not guarantee results.

I am sorry - I didn't understand that you need some feedback. But I think it would be good if you could consult your legal team. Any help is apreciated :9

I have no clue about Oracle and what they do. But some things in created some questions for me:

Btw, I suspect that instead of multiplying by 2 here, ub4 has to be NativeUInt instead of Cardinal.

Hmm - I am unsure about that. According to this part of the Oracle documentation ub4 is declared in oratypes.h. And according to this unofficial copy of oratypes.h, ub4 is declared as "unsigned int". And according to this Microsoft specific documentation the maximum value of unsigned int is "4294967295 (0xffffffff)". So having ub4 as Cardinal seems right to me? Maybe it helps to check the size of ub4 using a small C program?

I found out what's causing this particular heap issue. In AllocateOracleSQLVars it dynamically allocates memory for the TZSQLVars record, but I think it's assuming that the record is packed and so the allocation doesn't take into account the record field alignment setting (which defaults to Quad word in recent versions of Delphi.) Regardless, the issue can be "fixed" for now by just allocating more memory.

I think that memory alignments could be a source of problems. As far as I remember, records were always packed in 32 bit C / C++ while they are aligned to some degree in 64 bit C / C++. At least on Windows?

[miab3]

I think that memory alignments could be a source of problems. As far as I remember, records were always packed in 32 bit C / C++ while they are aligned to some degree in 64 bit C / C++. At least on Windows?

It seems that in Delphi 11.1sp1 the issue of align for Win64 has been changed significantly.
In Delphi 11.1 sp1 appeared as new in System.pas, among others:

end align {$IFDEF CPU64BITS} 16 {$ELSE !CPU64BITS} 8 {$ENDIF CPU64BITS};

and I had problems in a large test which for Win32 passes without error and in Win64 there are a number of them.
In Delphi 11.1 without sp1 both tests(Win32 and Win64) pass without errors

A good example is the application:

https://github.com/synopse/mORMot/blob/ ... stSQL3.dpr

compiled with the library:

https://github.com/synopse/mORMot

Once for Win32 and then for Win64.
With Delphi 11.1 sp1 Win64, the application always ends with an AV error.
For earlier ones, including Delphi 11.1 without sp1, the Win64 version ends without this (critical) error.

I raised this issue with the author some time ago here:

https://synopse.info/forum/viewtopic.php?id=6228

And it seems that it is still unsolved in the current version(for version mORMot '1.18.6435').
I checked it a while ago.

In conclusion:
For mORMot 1.18.6435 and Delphi 11.1:
==================================
TestSQL3 Win32 - PASSED
TestSQL3(with FastMM5) Win32 - PASSED
TestSQL3 Win64 - PASSED
TestSQL3(with FastMM5) Win64 - PASSED

For mORMot 1.18.6435 and Delphi 11.1sp1:
====================================
TestSQL3 Win32 - PASSED
TestSQL3(with FastMM5) Win32 - PASSED
TestSQL3 Win64 - CRASH
TestSQL3(with FastMM5) Win64 - CRASH

I don't quite trust the Win64 compiler for Delphi as of 11.1 sp1.

Michal

[MJFShark]

If we find a bug we should report it, though in this specific case I'm not sure it's a bug in anything but the code I mentioned above. I could easily be wrong about this, but I'm not sure it's ever been a good idea to calculate the size of a record using the individual sizes of the datatypes. The original code doesn't work if there is any alignment other than it being packed, and my understanding is that the alignment setting is a compiler option. My suggested fix is in double quotes because I don't think this is how we should fix this issue at all. However, I don't have a lot of experience doing dynamic memory allocation for records, so take all of that with a grain of salt lol. I just know that there's a pitfall here. I'd probably change the def of TZSQLVars to "Variables: array[0..0] of TZSQLVar" and then use Sizeof to take care of the alignment issues (and then add the array mem calc.) (OR I'd try to do it in a more Delphi like way without using getmem, but that suggestion is a bit sophomoric on my part since I don't know the details of why it was done that way.) The stuff above about the datatypes in ZPlainOracleDriver is something that might need to be looked at. Some of the individual definitions look like they don't match the OCI header files exactly, however most of them and certainly the UB4, etc definitions are correct. I often use the open source ocilib project to see OCI code samples btw.

-Mark