Page 1 of 1

How to master transaction?

Posted: 25.05.2009, 04:42
by lukezeoslib
Hi, guys, the transaction in FB is driving me crazy. :(

On top of this is the AutoCommit flag. Yes, I have read the quick guide in documentation area which has a brief explaination about it and has 4 URLs linking to 4 relative threads, but after I want over them, I was still hard to understand how to use it in a correcting way.

In fact, my problem is, my autocommit was set to true, and I used multithread to achieve the purpose, that was every threads could update or select data from a FB database file and they shared a TZConnection, but they had different TZQuery in their thread.

Sometimes they ran well, but sometimes, an exception raised from my program as below:
--------------
Project XXX.exe raised exception class EZDatabaseError with message 'Invalid operation in AutoCommit mode'. Process stopped. Use Step or Run to continue.
--------------

Here is the 'update' clauses.

Code: Select all

        try
          fdbquery.SQL.Clear();
          fdbquery.Close();
          fdbquery.Connection.StartTransaction();

          fdbquery.SQL.Add(sqlstr);
          fdbquery.ExecSQL();
          fdbquery.Connection.Commit();
          fdbquery.Close();
        except
          if fdbquery <> nil then begin
            fdbquery.Connection.Rollback();
            fdbquery.Close();
          end;
        end;
Here is the 'select' clauses.

Code: Select all

      try
        fdbquery.SQL.Clear();
        fdbquery.Close();
        fdbquery.Connection.StartTransaction;
        fdbquery.SQL.Add(sqlstr);
        fdbquery.Open;
        while not fdbquery.Eof do begin
          { fetch data }
          fdbquery.Next();
        end;
        fdbquery.Connection.Commit;
        fdbquery.Close();
      except
        if fdbquery <> nil then begin
          fdbquery.Connection.Rollback;
          fdbquery.Close();
        end;
        Result := false;
        exit;
      end;

Here is the process opening the database.

Code: Select all

      fdb := TZConnection.Create(nil);
      if not FileExists(fdbfullname) then begin
        if not inCreateDB(fdbfullname, fdbdllfullname) then begin
          Result := false;
          exit;
        end;
      end;
      fdb.Disconnect;
      fdb.Database := fdbfullname;
      fdb.User := 'SYSDBA';
      fdb.Password := 'masterkey';
      fdb.Protocol := 'firebirdd-2.0';
      fdb.Properties.Clear;
      fdb.Properties.Add('Dialect=3');
      fdb.Properties.Add('hard_commit=yes');
      fdb.Connect;
And here is the creating database clauses.

Code: Select all

  sqlstr := Format('CreateNewDatabase=CREATE DATABASE ''%s'' USER ''SYSDBA'' PASSWORD ''masterkey'' PAGE_SIZE 16384 DEFAULT CHARACTER SET NONE', [dbfullname]);
  fdb.Properties.Clear;
  fdb.Properties.Add(sqlstr);
  fdb.Properties.Add('Dialect=3');
  fdb.Properties.Add('hard_commit=yes');
  fdb.Database := dbfullname;
  fdb.Protocol := 'firebirdd-2.0';
  fdb.TransactIsolationLevel := tiReadCommitted;

  try
    fdb.Connect;
  except
    Result := false;
    exit;
  end;

  try
    try
      sc := TZSQLProcessor.Create(nil);
    except
      Result := false;
      exit;
    end;
    sc.Connection := fdb;
    sc.DelimiterType := dtSetTerm;

    fdb.StartTransaction;

    {something about sc.Script.Add(...) }

    try
      sc.Execute;
    except
      fdb.Rollback;
      Result := false;
      exit;
    end;
    fdb.Commit;
    Result := true;
  finally
    if sc <> nil then begin
      sc.Connection := nil;
      FreeAndNil(sc);
    end;
  end;

Posted: 25.05.2009, 07:54
by mdaems
Well, you're trying stretch it, I'm afraid...

Multithreading -> seems to work in general, but only with one connection per thread. So I think your approach only works when the threads are using the connection one by one.
hard_commit=yes -> this is an experimental option at most. I didn't even test it.

Concerning the exception... I'm not sure. It MAY be because of the multithreading. Internally the library plays a little with the autocommit switch during starttransaction/commit processing.

Some hint : no need to close your query after ExecSQL.

Another hint : as long as you're executing only one statement as in the example above, you don't need to use starttransaction. Using Starttransaction doesn't open a new transaction at API level! It just avoids a commit to be executed unless explicitly asked. So 2 queries using the same connection use the same transaction after starttransaction has been called. In your multithreaded case that might mean two threads are executing within the same transaction. Which makes your setup invalid anyway.

Mark

Posted: 25.05.2009, 09:40
by lukezeoslib
Thank you, Mark.

After I posted this post, I realized that maybe the source of problem is multithread(several Tzquery shared one tzconnection, is there a synchronizing mechanic in zeoslib? I need to read the sources), but I am not sure whether the zeoslib is thread-safety or not.

I had to use this pattern(many tzquery and one tzconnection) in my application, because I used a embeded FB database. At first, I had tested using a tzconnection with a tzquery in every threads, they failed. The first one can open the db, but the next one can not do it, for remaining the efficiency of parallel operation, I used the way I mentioned before. Is it correct? I had considered using one tzconnection and one tzquery in my application, but I warried about the operating DB would be the bottleneck.
:?

Posted: 25.05.2009, 11:07
by mdaems
No, as far as I know there's nog synchronisation mechanic provided.
Seems like the thread-safety might be allright when keeping the queries in the same thread as your connection.

Using an embedded server you're indeed limited to one connection component, I'm afraid. I'm not sure if multithreaded access to the embedded server is possible anyway. Is it possible to do another API call to the same handle while an API call is already running?

When you foresee the application will also work in a normal client-server way, you can make the threading approach optional, of course. Then you can use a connection per thread.

What I would advise is doing it all in one thread, but use multiple query objects to avoid changing the sql statement of the query component all time. I'm not sure that reduces overhead significantly on a 6.6 zeoslib version. But when the number of different queries is small it may make the code easier to read anyway.
When you're already using 7.x this should save some database interaction, because the same statement will effectively be reused. In that scenario the query is sent to the server only once and only the parameter values are sent at every execution. (but I'm looking into a related bug now, so beware...)

If there's also other work to do that can be threaded you could of course devise a system where one db-thread processes a queue that's filled by other threads. But I think that's an approach that only works when there's loads of non-db work to perform.


Mark

Posted: 31.05.2009, 23:37
by lukezeoslib
Thank you, Mark.

I have found the solution for my project. In fact, FB embedded suports that more connections in a program, I misread it, so gave up the pattern(one connection with more queries in different threads), and changed them to one connection with one query in a thread.

I think that is a best way for my program.

Posted: 03.06.2009, 21:36
by mdaems
Just for your information : when you're using SVN you could have a look at the new feature provided by Fabiano Bonin. (http://fisheye2.atlassian.com/changelog/zeos/?cs=662) It adds a new generic driver which uses pooled connections for all existing db specific drivers. Beware : it's highly experimental Transaction Isolation Level problems may still exist.

Mark