I just sent a bunch of general and Oracle specific fixes

Code patches written by our users to solve certain "problems" that were not solved, yet.

Moderators: gto, cipto_kh, EgonHugeist, mdaems

Post Reply
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 766
Joined: 18.11.2018, 17:37
Location: Hungary

I just sent a bunch of general and Oracle specific fixes

Post by aehimself »

Hey guys,

I recently discovered some issues with how Zeos 8.0 and the Oracle protocol specifically handled connection losses so I made some changes and issued a pull request on GitHub. It is working fine on my side, but since I added a new property and touched some TZAbstractConnection .SetActive and .PingServer logic too please doublecheck if my solution fits the Zeos codebase.

https://github.com/marsupilami79/zeoslib/pull/22

Thanks!
Delphi 12.1, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmariadb.dll 3.3.8
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.13
- MSSQL 2012, 2019; sybdb.dll FreeTDS_2435
- SQLite 3.45.2
marsupilami
Platinum Boarder
Platinum Boarder
Posts: 1918
Joined: 17.01.2011, 14:17

Re: I just sent a bunch of general and Oracle specific fixes

Post by marsupilami »

I checked in the changes to trunk / master. I left it to EgonHugeist to apply these changes to Zeos 8.0 if he likes them. So - this basically means that Zeos (8.0) currently (without this patch) cannot use Oracle in in a multithreaded application in the usual way - one connection per thread? If so, I will have to add that to the release notes.

It seems to me that we need some (small) multithreading documentation. There now are at least two databases (Oracle, SQLite) who need special attention before they can be used safely in a multithreaded environment?
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 766
Joined: 18.11.2018, 17:37
Location: Hungary

Re: I just sent a bunch of general and Oracle specific fixes

Post by aehimself »

marsupilami wrote: 03.11.2020, 21:41I checked in the changes to trunk / master. I left it to EgonHugeist to apply these changes to Zeos 8.0 if he likes them.
The good thing about waking up late is that he didn't just merge it to 8.0; it's already visible on GitHub as well! Cheers for that :)
marsupilami wrote: 03.11.2020, 21:41So - this basically means that Zeos (8.0) currently (without this patch) cannot use Oracle in in a multithreaded application in the usual way - one connection per thread? If so, I will have to add that to the release notes.
Well, yes and no. The thing I realized is that the OCI driver was wildly throwing nullpointer exceptions here and there if certain - but unknown - conditions were met. Yesterday for example it threw a nullpointer exception instead of the timeout message when a connection could not be established. This was my starting point.
After adding this patch to my local Zeos repo the issue seems to have disappeared - however as it was wildly random, I'll need some time of continuous testing to be able to tell for sure.

I think that OCIMultiThreaded only tells the driver to use Mutexes inside, therefore the execution of different threads won't scramble.
marsupilami wrote: 03.11.2020, 21:41It seems to me that we need some (small) multithreading documentation. There now are at least two databases (Oracle, SQLite) who need special attention before they can be used safely in a multithreaded environment?
I personally weren't able to make SQLIte to work in my multithreaded application. It always threw the wait time error :)
Delphi 12.1, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmariadb.dll 3.3.8
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.13
- MSSQL 2012, 2019; sybdb.dll FreeTDS_2435
- SQLite 3.45.2
marsupilami
Platinum Boarder
Platinum Boarder
Posts: 1918
Joined: 17.01.2011, 14:17

Re: I just sent a bunch of general and Oracle specific fixes

Post by marsupilami »

aehimself wrote: 04.11.2020, 08:19 Well, yes and no. The thing I realized is that the OCI driver was wildly throwing nullpointer exceptions here and there if certain - but unknown - conditions were met. Yesterday for example it threw a nullpointer exception instead of the timeout message when a connection could not be established. This was my starting point.
After adding this patch to my local Zeos repo the issue seems to have disappeared - however as it was wildly random, I'll need some time of continuous testing to be able to tell for sure.

I think that OCIMultiThreaded only tells the driver to use Mutexes inside, therefore the execution of different threads won't scramble.
Yep - which basically means, using Zeos 7.2 for multi threaded applications on Oracle isn't safe. ;) So I will add that to the release notes as a known issue.
aehimself wrote: 04.11.2020, 08:19 I personally weren't able to make SQLIte to work in my multithreaded application. It always threw the wait time error :)
Ok - I don't have experience with SQLite but to me that still sounds like you set SQLiteOpen_Flags to SQLITE_OPEN_NOMUTEX or SQLITE_OPEN_FULLMUTEX? But even then - if I read The SQLite documentation on Multi-Threaded Applications correctly, SQLite needs to be compiled with Multi-Threading support to make this work:
Using SQLite In Multi-Threaded Applications / Run-time selection of threading mode wrote:Nor is it possible to escalate an individual database connection if the compile-time or start-time mode is single-thread.
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 766
Joined: 18.11.2018, 17:37
Location: Hungary

Re: I just sent a bunch of general and Oracle specific fixes

Post by aehimself »

marsupilami wrote: 04.11.2020, 09:01Yep - which basically means, using Zeos 7.2 for multi threaded applications on Oracle isn't safe. ;) So I will add that to the release notes as a known issue.
I guess it should be possible to add that change to 7.2 too...? Maybe even the whole commit...?
marsupilami wrote: 04.11.2020, 09:01Ok - I don't have experience with SQLite but to me that still sounds like you set SQLiteOpen_Flags to SQLITE_OPEN_NOMUTEX or SQLITE_OPEN_FULLMUTEX? But even then - if I read The SQLite documentation on Multi-Threaded Applications correctly, SQLite needs to be compiled with Multi-Threading support to make this work:
Using SQLite In Multi-Threaded Applications / Run-time selection of threading mode wrote:Nor is it possible to escalate an individual database connection if the compile-time or start-time mode is single-thread.
I did not change anything and I used the DLL available on their website. I never managed to get it to work but it was not important so I simply abandoned the idea :)
Delphi 12.1, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmariadb.dll 3.3.8
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.13
- MSSQL 2012, 2019; sybdb.dll FreeTDS_2435
- SQLite 3.45.2
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Re: I just sent a bunch of general and Oracle specific fixes

Post by EgonHugeist »

@aehimself
According your patch i just wanna say: Awesome job! That rocks! Thank you very much.
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
User avatar
aehimself
Zeos Dev Team
Zeos Dev Team
Posts: 766
Joined: 18.11.2018, 17:37
Location: Hungary

Re: I just sent a bunch of general and Oracle specific fixes

Post by aehimself »

Np, glad I could help.
I'm also glad that these mysterious oci.dll nullpointer AVs should be gone for good... they were driving me INSANE!
Delphi 12.1, Zeos 8 from latest GIT snapshot
Using:
- MySQL server 8.0.18; libmariadb.dll 3.3.8
- Oracle server 11.2.0, 12.1.0, 19.0.0; oci.dll 21.13
- MSSQL 2012, 2019; sybdb.dll FreeTDS_2435
- SQLite 3.45.2
Post Reply