Possible memory leak in TZCodePageConversionStream.Create

The offical for ZeosLib 7.3 Report problems, ask for help, post proposals for the new version of Zeoslib 7.3/v8
Quick Info:
-We made two new drivers: odbc(raw and unicode version) and oledb
-GUID domain/field-defined support for FB
-extended error infos of Firebird
-performance ups are still in queue
In future some more feature will arrive, so stay tuned and don't hassitate to help
Post Reply
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 797
Joined: 18.11.2018, 17:37
Location: Hungary

Possible memory leak in TZCodePageConversionStream.Create

Post by aehimself »

Today it seems I managed to call .AbortOperation in a lucky spot, as I got a memory leak caused by GetMem:
leak.PNG
I think they should be encapsulated by this:

Code: Select all

  if (SourceCodePage = DestCodePage) then begin
    ReadStreamToMem(Src, L, Owner);
    Try
      SetPointer(Src, L);
      FInConstructionState := True;
    FInally
      Capacity := L; //realloc mem
      FInConstructionState := False;
    End;
  end else begin
But I'm not good with pointers and don't really know what is going on here... but it can be a starting point :)
You do not have the required permissions to view the files attached to this post.
Delphi 12.2, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmysql.dll 8.0.40 x64 5.7.19 x68, libmariadb.dll 3.3.11
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.15
- MSSQL 2012, 2019; sybdb.dll FreeTDS_3102
- SQLite 3.47
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Re: Possible memory leak in TZCodePageConversionStream.Create

Post by EgonHugeist »

aehimself

looking to the stacktrace.. I think there is a general design problem in what you're doing i guess.

Why is there an FetchAll called while you wanna kill the fetches dynamic?
Shouldn't you add an event to the dataset and test if it was signaled? I.e.

Code: Select all

While not Aborted do DataSet.Next
?

IIRC did the AbortOperation kill a long query while waiting for server responce. You StackTrace shows me the Server is ready and you're already consuming the inputs. That way is always dangerous and of course it leads to leaking memory. It's a random game where it happens.. Yet it are the lob's later it may be the accessor objects, or the fields or ... etc.
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 797
Joined: 18.11.2018, 17:37
Location: Hungary

Re: Possible memory leak in TZCodePageConversionStream.Create

Post by aehimself »

EgonHugeist wrote: 24.01.2021, 07:38Why is there an FetchAll called while you wanna kill the fetches dynamic?
This worker thread collects a large amount of CLOB fields and does some calculations on their contents one-by-one. If I call .FetchAll before the actual data processing 2 things will happen:
- The application will report nothing but "Opening query" for ~20-30 minutes instead of almost immediately showing that it already processed some
- Memory consumption will be in the 4 GB-ish range instead of 150 MB
EgonHugeist wrote: 24.01.2021, 07:38looking to the stacktrace.. I think there is a general design problem in what you're doing i guess.
[...]
Shouldn't you add an event to the dataset and test if it was signaled? I.e.

Code: Select all

While not Aborted do DataSet.Next
I do have a cycle like this... in this case the question is valid... why am I calling .AbortOperation at the first place...? You got me thinking, I'll attempt to get around it.

The reason I reported is because in the constructor there is an if statement. One side has a Try..Finally block, the other does not and that looked suspicious.
Just take a look when you have some free time; I'll attempt to eliminate the .AbortOperation in the mean time.
Delphi 12.2, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmysql.dll 8.0.40 x64 5.7.19 x68, libmariadb.dll 3.3.11
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.15
- MSSQL 2012, 2019; sybdb.dll FreeTDS_3102
- SQLite 3.47
Post Reply