Keep ZQuery active even if ZConnection is closed

[brunodelmondes]

Hello, I'm Bruno.

I'm using Zeos 7.2.6 with Lazarus 2.0.8 (FPC 3.0.4) and Firebird 2.1.7 as DBMS. I'm building a client - server application using dbware components. I searched the forum a lot before asking this but I didn't find any answer... my doubt is, is it possible, somehow, to keep a ZQuery active with CachedUpdates also active and ZConnection closed? My point is, I wish it could be possible, because I would be able to connect to the database only to commit changes e then disconnect from it.

I wrote a very simple code to test this but had no luck:

procedure TForm1.FormCreate(Sender: TObject);
begin
qryDados.CachedUpdates:= True;
qryDados.Open;
qryDados.Connection := nil;
conn.Disconnect;

if qryDados.Active then
begin
ShowMessage('qry active');
end;
end;

Any help is welcome. Thank you in advance !!!

Bruno Delmondes

[marsupilami]

Hello Bruno,

is it possible, somehow, to keep a ZQuery active with CachedUpdates also active and ZConnection closed? My point is, I wish it could be possible, because I would be able to connect to the database only to commit changes e then disconnect from it.

Zeos doesn't support disconnected operation. Why do you want to disconnect?

Best regards,

Jan

[MJFShark]

In general if you need to keep a dataset outside of the database connection the solutions are TClientDataset or TFDMemTable (there are some others as well btw.) I'd actually love if Zeoslib had a virtual dataset, or had some way to keep data after disconnect. It seems like all the pieces are there to do it, but I bet it's not a trivial coding job!

[brunodelmondes]

Hello Jan, thanks for replying!

Well, as long as I'm using dbware components, if my application loses connection with the server for some reason, I can't commit changes to database because, to get a connection with the server again, it is necessary to disconnect ZConnection and connect again and this closes all the datasets, so the changes that were cached are also lost. I also think it would be a way to prevent a database corruption, in case of a power loss, since I'm not keeping an active connection to it.


Bruno Delmondes

[brunodelmondes]

Hello MJFShark!

Those are really awesome components, but unfortunately I can't use them since I'm building my application with Lazarus. There is the BufDataset in Lazarus, witch is a virtual dataset, but I'm not sure (don't know how to do) if it is possible to use it along with a ZQuery.

[marsupilami]

Well, as long as I'm using dbware components, if my application loses connection with the server for some reason, I can't commit changes to database because, to get a connection with the server again, it is necessary to disconnect ZConnection and connect again and this closes all the datasets, so the changes that were cached are also lost.

I don't know your environment but with most environments / networks that I know about, connection loss happens very rarely. If this really is a concern to you, we might consider to find a way to extend the webservice driver in Zeos 7.3/8.0 so that it can work with FPC too.

I also think it would be a way to prevent a database corruption, in case of a power loss, since I'm not keeping an active connection to it.

With most modern databases that isn't a problem anymore. Firebird inherited a method called "careful writes" from its ancestor Interbase that makes sure that a database file doesn't get corrupted during power outage because the disk image is always valid.

[brunodelmondes]

I don't know your environment but with most environments / networks that I know about, connection loss happens very rarely. If this really is a concern to you, we might consider to find a way to extend the webservice driver in Zeos 7.3/8.0 so that it can work with FPC too.

Well, at my country, some regions are underprivileged about power stability. If there is a better way to prevent data loss considering this scenario with Zeos on current stable version, can you tell me how to accomplish it?
Can you explain me about this webservice driver? How does it work?

With most modern databases that isn't a problem anymore. Firebird inherited a method called "careful writes" from its ancestor Interbase that makes sure that a database file doesn't get corrupted during power outage because the disk image is always valid.

Didn't know about that feature. Is it necessary to manually activate it? We have experienced a few cases that the database was corrupted with power loss, in most cases we have been able to fix it using GFIX and other cases we haven't.

[marsupilami]

I don't know your environment but with most environments / networks that I know about, connection loss happens very rarely. If this really is a concern to you, we might consider to find a way to extend the webservice driver in Zeos 7.3/8.0 so that it can work with FPC too.

Well, at my country, some regions are underprivileged about power stability. If there is a better way to prevent data loss considering this scenario with Zeos on current stable version, can you tell me how to accomplish it?
Can you explain me about this webservice driver? How does it work?

Duh - so - the question is, what you really want to accomplish and need to accomplish.

If your primary concern is to make sure, the database file doesn't get corrupted, then this most probably isn't something you can solve by disconnecting from the server. You most probably cannot solve it by the design of the application software. In that case the traditional approach would be to use a UPS that can be used to shut down the server computer safely. If a traditional UPS for a traditinal server is not possible, then maybe it is an option to use a Raspberry PI as the firebird server and back it with a battery or something like that? If that also is not possible, maybe it is an idea to try to do backups often - to get a kind of nearline backup. nbackup and gbak come to mind there.

If your primary concern is for the application to survive a network outage, the webservice driver might help. The webservice driver is meant to allow Zeos applications to connect to database servers over the internet and from mobile devices. Basically there is a special Zeos Server application that keeps connections to the database servers open. Clients issue requests by using SOAP over HTTP(S) to the Zeos Server. This way, there is no need to keep a permanent connection between the client device and the server. Even if the client device disconnects and reconnects to the network or changes its IP address that is no problem because there is no permanent connection between the Zeos based application and the Zeos Server. Permanent connections only happen between the Zeos Server and the database servers.
But it still requires that the server keeps to be powered up because the Zeos Server manages connections to the database. If the server fails because of a power outage, sessions opened by clients will get invalid. So it only could help you if the server has a UPS and the netwokrs switches or the Wifi is not powered by the UPS.
Unfortunately the webservice driver in Zeos currently only compiles with Delphi. We would have to make it compile with FPC if you want to use it. Also these components are currently at an early beta stage. There are things to consider before using them.

Another option is to see what it takes to make a Zeos result set be able to keep open even if the connection is closed. Even though it should be possible and a lot of the components already are there, there are things to think through like how do we make sure that no changes to data get lost. What do we do if a record was deleted while we were disconnected? What do we do if fields of a record got changed while we were disconnected?

With most modern databases that isn't a problem anymore. Firebird inherited a method called "careful writes" from its ancestor Interbase that makes sure that a database file doesn't get corrupted during power outage because the disk image is always valid.

Didn't know about that feature. Is it necessary to manually activate it? We have experienced a few cases that the database was corrupted with power loss, in most cases we have been able to fix it using GFIX and other cases we haven't.

This doesn't need to be activated usually. But it requires that forced writes are set to on in the database. I think this can be checked with gstat or flamerobin. If forced writes are off, then you should set it to on.

[brunodelmondes]

Hey Jan!

Thank you for this great answer. Indeed, a computer server must be on all the time. The Zeos web service driver will be very useful, no doubts.

Another option is to see what it takes to make a Zeos result set be able to keep open even if the connection is closed. Even though it should be possible and a lot of the components already are there, there are things to think through like how do we make sure that no changes to data get lost. What do we do if a record was deleted while we were disconnected? What do we do if fields of a record got changed while we were disconnected?

I'd really love if Zeos could behave like this. I'd like to thank you guys, the hole team, everyone that some how had contributed for the Zeos development, for your awesome work on this great component.

This doesn't need to be activated usually. But it requires that forced writes are set to on in the database. I think this can be checked with gstat or flamerobin. If forced writes are off, then you should set it to on.

Yeah, I just confirmed, with flamerobin, forced writes are on.

Sorry if I said something wrong or if I wasn't clear enough. My English is not so good.

Best regards,