DBLib errors are messages...?

[aehimself]

Using Zeos 7.3-d2b27638 and FreeTDS 1.1.6.0 I have the following logs in my application:

Execute MsgNo : [00226] : CREATE DATABASE statement not allowed within multi-statement transaction., affected 0 row(s)
Execute Statement 4 : CREATE DATABASE [test] in 3 ms
Query affected -1 row(s) in 158 milliseconds <- SQLQuery.RowsAffected property logged manually

Execute MsgNo : [00102] : Incorrect syntax near 'rewrwe'., affected 0 row(s)
Execute Statement 3 : CREATE DATABASE [test]rewrwe in 4 ms
Query affected -1 row(s) in 121 milliseconds <- SQLQuery.RowsAffected property logged manually

There are no exceptions raised, failed operations can only be seen in the SQL trace.

The problem MIGHT be TZDBLibConnection.CheckDBLibError, which ends like this:

Code: Select all

    if LogMessage <> ''
    then FLogMessage := Format(FormatStr, [FLogMessage, FirstError, LogMessage])
    else FLogMessage := Format(FormatStr, [FLogMessage, FirstError]);
    FLogMessage := '';
    FLastWarning := EZSQLWarning.CreateWithCode(FirstError, FLogMessage);

I modified the method a little bit to actually throw the error message created if FSQLErrors had entries before the clearing, but nothing happens as the error messages are coming as FSQLMessages.

Symptom is the same with FreeTDS 0.95.87.0.

Is this a Zeos or a FreeTDS issue?

[aehimself]

Okay, progress. I pulled the latest Git snapshot, and now it is throwing invalid pointer operations.
I rolled back the commit #a02669624c8df0fc22b04335391b2c1e813865de:

egonhugeist on 8/8/2020, 7:35:28 AM
(for the Docs) resolve ticket #348 by adding two TZConnection properties AddLogMsgToExceptionOrWarningMsg, RaiseWarningMessages. IZConenction also implements SetAddLogMsgToExceptionOrWarningMsg, SetRaiseWarningMessages in the interface so we can switch the option dynamically. The more the ConenctionLost is complete/prepared for all drivers. (OleDB the State is unknown, for SQLite it will be triggered by an disk-io error)

and discarded all changes (of the rollback!) except ZdbcDbLib.pas and the exception went away. When I was debugging I saw that the parameter "Error" is changing to "Unaccessible value" in the moment when we arrive to TZLoggingEvent.Create so I suspect a memory corruption somewhere.

I'll see what I can do to fix it.

[aehimself]

So, I pinpointed the issue, I just have no goddamn clue why it appears...

This is the ONLY difference now while rolling back the said commit:

Capture.PNG

Line is in TZDBLibConnection.CheckDBLibError. I guess the memory corruption will be because of thread safety, as I'm calling .Connect in a background thread using TZMethodInThread.

I'll try to dig deeper.

Edit: Don't get me wrong. Deleting these lines will only get rid of the Invalid Pointer Operation message; exceptions are still NOT raised when an error occures.

[aehimself]

I saw @EgonHugeist checked in some fixes for DBLib; Exceptions are back to life - thank you for that!

Although, I still do have issues with the invalid pointer operation. Different project, 32 bit instead of 64, same version of dblib.dll. Commenting out the two lines I mentioned above gets rid of the issue. Structure is different, but one thing is the same: both applications are attempting to open a connection in a worker thread with a custpm message pump.

As the error usually pops up when attempting to deallocate a record (or a string) I suspect a memory corruption somewhere. Sometimes the error pops up when exiting the application, sometimes as TZAbstractDbcConnection.LogError (called by TZDBLibConnection.CheckDBLibError) exits. As no other protocols I use (MySQL, Oracle) seems to be affected, I suspect the issue will be in DBLib somewhere.
Using no ClientCodePage instead of UTF-8 makes no difference, invalid pointer operation still pops up.

I'll try to make a minimalistic test case.

[aehimself]

I think it's about thread safety. See the minimalistic test case attached. Moving the creation of TZConnection object within .Execute makes no difference, so message pump has nothing to do with the issue.

Zeos:
50fa230a84f3d27cd968f904eee4803ad3748816
egonhugeist on 8/18/2020, 2:35:36 PM
compile TestSuite with the NO_AUTOENCODE define (the VariantManager tests need to be reviewed for the Ansi-Compilers)

Delphi 10.4 patch 3

sybdb.dll 0.95.87.0, 1-1-6-0

Microsoft SQL Server 2012 (SP4) (KB4018073) - 11.0.7001.0 (Intel X86)
Aug 15 2017 10:26:40
Copyright (c) Microsoft Corporation
Express Edition on Windows NT 6.0 <X86> (Build 6002: Service Pack 2) (Hypervisor)

[aehimself]

So, there was a memory leak in ZDbcDBLib, so I made some changes and guess what - the memory corruption disappeared too.
I have a guess on why that happened, but I cannot be sure.

Anyway - as usual - the fix is in a pull request on GitHub (along with some others now... :))
Please review and change/apply if necessary :)

[miab3]

@aehimself,

Can you see with this library?:
ftp://ftp.freepascal.org/pub/fpc/contri ... ib_1.1.zip

Michal

[aehimself]

@miab3,

I roll back my change and introduced a cycle to connect and disconnect from the database 100 times. Results are...

64bit:
All 3 bundled .DLLs (dblib, _2000, _2008) seem to be working fine

32bit:
_2000 crashes the application crashes silently at TZDBLIBPLainDriver.dbsetversion, at this line: "then Result := Fdbsetversion(Version)"
_2008 it seems to be working fine

I can even redirect the messages to a TMemo - no memory leak, no invalid pointer operation.

So it seems that the problem is not Zeos, but FreeTDS-related.
Where did this library come from? I remember downloading 1.1.6.0 from the URL Jan gave me a while ago, which seemed to be the official repository. Do we know the exact version number?

[aehimself]

I am a fucking idiot.

Period.

I spent the past 4 days still investigating the invalid pointer exception reported in this thread, as one (!!!) project was still throwing it after the sybdb.dll update @miab3 advised. Not even 10 minutes ago I realized I hard-wired an old version of the driver in the application settings :(

I can confirm, all worker thread methods (with logging) works fine with the "new" version of sybdb.dll.

Question, mainly to @EgonHugeist
As I realized not that long ago, the library used can be extracted by GetModuleName(ZConnection.DbcConnection.GetIZPlainDriver.GetInstance.Loader.Handle) which is extremely useful. Now, when I call ZConnection.Connect and it trows an exception for whatever reason, ZConnection.DbcConnection is already unassigned and therefore the .DLL used by the connection attempt can not be determined anymore. Is it possible to add a (hidden?) property to a TZAbstractConnection object, which will store the name of the module? Or include it in the exception message (or a separate property)?
This would have saved me 4 days of hitting my head in the wall.

@miab3:
I am still interested on where you got this .dll from. I'd keep re-visiting the repository to keep the library updated.

[miab3]

Hi,
Is it not enough to specify the path and/or library name in LibraryLocation?

@miab3:
I am still interested on where you got this .dll from. I'd keep re-visiting the repository to keep the library updated.

I have libraries from the link I provided (public fpc repository).

Michal

[EgonHugeist]

Hi aehimself, feel free to make a patch for the module-name. A IZConnection.GetModuleName would be good i think. (What's the name for ADO/OleDB? we have no lib there..) According the FreeTDS-lib: Jan also maintains a compiled version in our SVN, check the \lib\FreeTDS folder.

[marsupilami]

Jan also maintains a compiled version in our SVN, check the \lib\FreeTDS folder.

Honestly I don't maintain that anymore, because I think that client libraries don'tbelong into our SVN. I upload a compiled version of FreeTDS to SourceForge from time to time.

[aehimself]

As there were occasional "Invalid pointer operations" at some colleagues, I continued to investigate. For your information, the latest official build of dblib seems to work just fine, downloaded from https://ci.appveyor.com/project/FreeTDS/freetds. I used the VS2017 64 bit version.

Seems I tried a broken build the last time :)