My LOBs aren't getting released...?

[aehimself]

I have a background thread which cycled through a number of CLOBs and searches in their contents. It is creating an SQLConnection object and sets:

Code: Select all

Self.SQLConnection.Properties.Values['CachedLob'] := '';

The read-only query is in a simple cycle:

Code: Select all

While Not query.Eof Do
Begin
  s := query.FieldByName('field').AsString
  [...]
  query.Next;
End;

What I realized is that the memory consumption of my application keeps increasing until the worker thread terminates... starts from 100 MB, currently at 700 MB after 2 hours of execution.

Correct me if I'm wrong, but isn't CachedLob = false should mean that we load the lob on access and then releasing (= freeing it up completely) once it's not needed anymore? Is this behavior can be controlled somewhere else, too?

In the mean time I'll run some tests while commenting out the processing part to make sure the memory increase is indeed from simply accessing the LOB. I'll get back once I have more info.

[MJFShark]

Are you using TryKeepDataOnDisconnect? That will switch CachedLobs on. Just a thought! Your assumption about CachedLobs = false is correct btw, it should just read the stream and then release it after. I'll test here using LobCachedMode and see if I see a problem.

-Mark

[aehimself]

I'm not touching TryKeepDataOnDisconnect. If the default is true, then yes, I'm using it :D

My assumption is correct. I made a snapshot with DeLeaker when ~3000 CLOBs were processed and the same amount of TZOracleCLOB objects were created and alive at the very same time.

Let me check TryKeepDataOnDIscnnect.

[MJFShark]

It's definitely not on by default so that's probably not it. Can you run your test before my commits related to LobCacheMode? If you aren't changing the value it shouldn't be affecting this, but I'd love confirmation. I use FastMM for leak detection and I haven't seen any lob related leak reports, but maybe it's something that wouldn't show up in my test cases so far.

-Mark

[aehimself]

No joy. Created queries have TryKeepDataOnDisconnect in False, Options do not contain cachedlobs.

I think the issue is that when we move the cursor, the current row's data is not being disposed anywhere... at least I can not find a reference to it...?

Edit: it's NOT a memory leak! They are just getting freed up when the datases closes (or being destroyed in my case) instead of when moving the cursor. I think.

Edit-edit: oh-oh. Going back to commit

c0f1f175a9f50cd59ab4b8719edf2e37c99e111e
add ORA_01089_immediate_shutdown_or_close_in_progress as provided by aehimself, thx!
egonhugeist on 4/1/2021, 6:31:53 AM

and my application sticks around 80 MB of used memory instead of hundreds... I'll let it to run to see if it will increase eventually.

[MJFShark]

I can't seem to cause the problem here (though granted I'm running with the "final" version of LobCacheMode instead of the test version.) I open an uncached dataset with lots of clobs, move up and down many times and I don't see memory increase at all. Any tips on causing it? Any settings that you use that I might not be using?

-Mark

[aehimself]

So, I changed the cycle to this:

Code: Select all

While Not query.Eof Do
 query.Next;

Just like this, cycling through 27.000 LOBs increased the memory usage of my application from 60 MB to 211 MB. Without accessing any LOBs whatsoever.

So, the increased memory is not coming from LOBs not being released, but the descriptions (TZOracleClob Delphi object?) not being freed when cursor position changes. I did the test by standing on the same commit, before the LobCacheMode change - but as not the LOBs are the problem it seems to be irrelevant.

Edit: Yes, that is going to be the issue. Changing the query from SELECT ID, CLOBFIELD to SELECT ID and cycling through the same 27.000 records my application stays on ~30 MB of memory usage.

Edit-edit: the object gets created at ZDbcCache : 1315 (TZRowAccessor.FillFromFromResultSet.InternalSetLob)

Code: Select all

    PIZLob(Dest)^ := ResultSet.GetBlob(ResultSetIndex);

which is called from ZDbcCache :1378 (TZRowAccessor.FillFromFromResultSet)

Code: Select all

        stAsciiStream, stUnicodeStream, stBinaryStream: InternalSetLob(PIZLob(Data), ResultSet, ResultsetIndex);

Data is a pointer, calculated like...

Code: Select all

    P := @FBuffer.Columns[FColumnOffsets[ColumnIndex]];
    {$IFDEF RangeCheckEnabled}{$R+}{$ENDIF}
    Data := Pointer(PAnsiChar(P)+1);

I think this should be freed up too if we are not caching LOBs, no?

[MJFShark]

Can you try the new version that has my full LobCacheMode conversion? It's possible it's something I introduced in the test code and then fixed in the final version. I don't see any issues with memory consumption here even when looping through thousands of pretty big clobs (with LobCacheMode set to None, which is the default if you're not using TryKeepDataOnDisconnect.)

I'll check the code you mentioned, though I'm breakpointing at
TZAbstracOracleLobStream.Destroy;
TZAbstractOracleBlob.Destroy;

And they both seem to be being called when moving through records as I'd expect.

-Mark

[aehimself]

I will, but Git doesn't show todays SVN commits yet so I have to wait :)

[marsupilami]

I synchronized GIT and SVN already because I wanted the code to show up on Github

[aehimself]

Synchronized to the latest Git version. Huge changes.

The very same query with the very same amount of data (27.000 records) takes about double time to load. Also, it seems it preallocates all objects as right after query.Open my application is already using >200 MB of memory instead of the previous 60-ish.

Yeah, no memory increase during the cycle, but the end result is almost the same. Except the loading time :)

[MJFShark]

Can you put some breakpoints in and make absolutely sure that LobCacheMode isn't being set to OnLoad anywhere? That much extra memory use can only mean that the streams are being cached, right?

I'm testing with all three LobCacheMode values and things seem to work the way I'd expect as far as when the streams are created and destroyed. The Lob objects themselves are certainly created on fetch, but they'll just have a locator. They only get destroyed when the query does, unless they get cached, in which case they are destroyed right after caching. That seems to work the way I'd expect, I don't think I changed that behavior.

Any thoughts appreciated!

-Mark

[marsupilami]

Maybe it is possible to create a small test application that demonstrates the problem? It shouldn't be too hard to insert some 20,000 records with LOBs and then see if the problem can be seen in that application?

[MJFShark]

I'm working on this. I'll report back when I learn something. Btw an easy way to test is just to use something like:

Code: Select all

    ZQuery.SQL.Text :=
      'select rownum, to_clob(dbms_random.string(''A'', trunc(dbms_random.value(50, 4000)))) ' +
      'from dual connect by level <= 5000';

-Mark

[MJFShark]

I'm stumped so far.

Using a branch at: add ORA_01089_immediate_shutdown_or_close_in_progress as provided by aehimself,
Which should be before any of my changes. I get:

Connected
Memory Used : 782 KB (801084 bytes)
Memory Reserved: 2.49 MB (2621408 bytes)
Query.Open: CachedLobs: False in 8731ms
Memory Used : 830 KB (849936 bytes)
Memory Reserved: 3.74 MB (3932112 bytes)
Query Next Loop: 5000 records in 44162ms
Memory Used : 1.74 MB (1831004 bytes)
Memory Reserved: 3.74 MB (3932112 bytes)
Query.Close in 2ms
Memory Used : 837 KB (857224 bytes)
Memory Reserved: 3.74 MB (3932112 bytes)

Using the most current master at commit: ec9ecdb

Connected
Memory Used : 782 KB (801084 bytes)
Memory Reserved: 2.49 MB (2621408 bytes)
Query.Open: LobCacheMode: None in 8740ms
Memory Used : 830 KB (849944 bytes)
Memory Reserved: 3.74 MB (3932112 bytes)
Query Next Loop: 5000 records in 34282ms
Memory Used : 1.73 MB (1823004 bytes)
Memory Reserved: 3.74 MB (3932112 bytes)
Query.Close in 2ms
Memory Used : 829 KB (849676 bytes)
Memory Reserved: 3.74 MB (3932112 bytes)

So I'm not seeing any real difference. I'm using the query that I gave above, connecting, opening, and then looping through the dataset without reading any values.

-Mark