(solved)New bug created in Version 7.2.6?

[Soner]

Ich have big trouble friends.
I used Zeos 7.2.4-stable on windows "without" troubles. I wanted compile my program for MacOs. But TZIBEventAlerter from Zeos 7.2.4. had bug on Macos, it could not difference Events. So I updated to Zeos 7.2.6-trunk version, but the trunk version has new bug with TZIBEventAlerter too. Look here viewtopic.php?f=28&t=112301

So I am using Zeos 7.2.6-stable. Now, i found out that 7.2.6-stable has also "bug".
What is wrong with following code? It worked with Zeos 7.2.4 without error. Now, with Zeos 7.2.6 following error occurs at program closing:

"SQL ERROR: cannot disconnect database with open transactions (1.active). Error Code: -901.
Unsuccesful execution caused by system error that does not preclude succesful execution of subsequent statement"

This is ist the code:

Code: Select all

function TDocsList.SelectedDataAsStream: TMemoryStream; 
var aQ: TZReadOnlyQuery;
   aAcnr: string;
begin
  if QrMain.Active then aAcnr:=QrMain.FieldByName('ACNR').AsString
  else aAcnr:='';
  if aAcnr<>'' then  begin
    aQ:= TZReadOnlyQuery.Create(self); aQ.Connection:=QrMain.Connection;
   try
    aQ.SQL.Text:='SELECT ACDATA FROM AUFDOCS WHERE ACNR='+aAcnr;
    aQ.Open;
    
    // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    // FOLLOWING LINE CREATES ERROR AT APPLICATION END     
    if aQ.RecordCount>0 then Result:=TMemoryStream(aQ.CreateBlobStream(TBlobField(aQ.FieldByName('ACDATA')),bmRead)) 
    else Result:=nil;
   finally
    aQ.Free;
   end;
  end
  else Result:=nil;
end;    

[Soner]

This line creates Error:
Result:=TMemoryStream(aQ.CreateBlobStream(TBlobField(aQ.FieldByName('ACDATA')),bmRead))

When I free the result-stream in calling-function, then there is no error. But I want hold the stream and close close connection. In Zeos 7.2.6. you can not close connection without freeing the blob-stream (in my code above).
Is there better way to read blob field?

[Soner]

I inspected the execution with debugger. The error occurs in "procedure TZAbstractConnection.Disconnect;" when it calls FConnection.Close;

[Soner]

I have another Question.
I created a workaround for this error. I used same code, I removed only unnecessary parts. My Workaround returns first 13 bytes of the blob different, look at the picture:

difference.png

This is the workaround code. Important part is TZBlobStreamCreate.

Code: Select all

{$REGION 'Workaround for blob-field-bug' -fold}
// bug description: https://zeoslib.sourceforge.io/viewtopic.php?f=28&t=117054
type                           
  TMemoryStreamHack = class(TMemoryStream)
  end;

  // This function is intended to replace TZBlobStream.Create
  // It is copied from TZBlobStream.Create and the unnecessary functions are removed.
  // It returns TMemoryStream not TZBlobStream
  function TZBlobStreamCreate(Field: TBlobField; Blob: IZBlob; Mode: TBlobStreamMode; ConSettings: PZConSettings): TMemoryStream;
  var
    Buffer: Pointer;
    ASize: Integer;
  begin
    Result:= TMemoryStream.Create;

    if (Mode in [bmRead, bmReadWrite] ) and not Blob.IsEmpty then
    begin
      if Blob.IsClob then
        case Field.DataType of
          ftMemo, ftFmtMemo:
            if ConSettings^.AutoEncode then
              Buffer := Blob.GetPAnsiChar(ConSettings^.CTRL_CP)
            else
              Buffer := Blob.GetPAnsiChar(ConSettings^.ClientCodePage^.CP);
          {$IFDEF WITH_WIDEMEMO}
          ftWideMemo:
            Buffer := Blob.GetPWideChar;
          {$ENDIF}
          else
            Buffer := Blob.GetBuffer;
        end
      else
        Buffer := Blob.GetBuffer;
      ASize := Blob.Length;
      {$IFNDEF WITH_MM_CAN_REALLOC_EXTERNAL_MEM}
      if Mode = bmReadWrite then
      begin
        Result.WriteBuffer(Buffer^, ASize); //something courrupts the FPC-Memory-Manager here??? D7??
        Result.Position := 0;
      end
      else
      {$ENDIF}
        TMemoryStreamHack(Result).SetPointer(Buffer, ASize);
    end;
  end;

// TZAbstractRODataset.CreateBlobStreamMy is same as TZAbstractRODataset.CreateBlobStream
// except the line TZBlobStreamCreate.
// it calls TZBlobStreamCreate not TZBlobStream.Create
function TZAbstractRODataset.CreateBlobStreamMy(Field: TField; Mode: TBlobStreamMode): TStream;
var
  ColumnIndex: Integer;
  RowBuffer: PZRowBuffer;
  Blob: IZBlob;
  WasNull: Boolean;
begin
  WasNull := False;
  CheckActive;

  Result := nil;
  if (Field.DataType in [ftBlob, ftMemo, ftGraphic, ftFmtMemo {$IFDEF WITH_WIDEMEMO},ftWideMemo{$ENDIF}])
    and GetActiveBuffer(RowBuffer) then
  begin
    ColumnIndex := DefineFieldIndex(FieldsLookupTable, Field);
    RowAccessor.RowBuffer := RowBuffer;

    if Mode = bmRead then
    begin
      Blob := RowAccessor.GetBlob(ColumnIndex, WasNull);
      Result := TZBlobStreamCreate(Field as TBlobField, Blob, Mode,
       FConnection.DbcConnection.GetConSettings);
    end
    else
    begin
      Blob := RowAccessor.GetBlob(ColumnIndex, WasNull);
      if Blob <> nil then
        Blob := Blob.Clone(Mode =  bmWrite);
        RowAccessor.SetBlob(ColumnIndex, Blob);
      Result := TZBlobStreamCreate(Field as TBlobField, Blob, Mode,
        FConnection.DbcConnection.GetConSettings);
    end;
  end;
  if Result = nil then
    Result := TMemoryStream.Create;
end;
{$ENDREGION}

[marsupilami]

Hello Soner,

"SQL ERROR: cannot disconnect database with open transactions (1.active). Error Code: -901.
Unsuccesful execution caused by system error that does not preclude succesful execution of subsequent statement"

Zeos 7.2.4 had a bug that required us to keep a reference to previous transactions on Firebird. It seems that is what is happening here. The transaction is kept alive by the blob and that is why Firebird complains.

Casting the result of CreateBlobStream to TMemoryStream can be problematic. I am not 100% sure, if you will always get a TMemoryStream object from Zeos. This is even more true for the future. There are plans to add special streams that don't keep all the data in memory but fetch it from the database as needed. If you want to close the database connection and keep the blob contents around, you will have to copy it to a separate stream:

Code: Select all

function CopyStreamToMemory(DataSet: TDataSet; FieldName: String): TMemoryStream;
var
  Field: TField;
  BlobStream: TStream;
begin
  Field := DataSet.FieldByName(FieldName);
  BlobStream := DataSet.CreateBlobStream(Field, bmRead);
  try
    Result := TMemoryStream.Create;
    try
      Result.CopyFrom(BlobStream, 0);
      Result.Seek(0, soBeginning);
    except
      FreeAndNil(Result);
    end;
  finally
    FreeAndNil(BlobStream);
  end;
end;

You also could try to set doCachedLobs in TZQuery.Options. It will make Zeos cache all blobs in memory. But Zeos still might keep the reference to the originating transaction. So I am not sure if this solves your problem.

Regarding the TZIBEventAlerter: I will post an answer in that thread later on.

Best regards,

Jan

[Soner]

Copy of the stream is no option, this makes application unnecessarily slow.
I think my workaround solution is good for me, it does not reference connection, I can close it without error.
I must solve, why die first thierteen bytes of the blob are different then as ZEOS-function.

EDIT:
I found out why the resulting stream is corrupted. It happens after freeing TZQuery (aQ.Free;).
It is very strange, why this does not happen with version 7.2.2.

[Soner]

I found out that this error has nothing to do with ZEOS. It is Freepascal/Lazarus error.
My Zeos 7.2.4 was on Lazarus 1.8.4 with Freepascal 3.0.4. both 32 Bit.
I tried 7.2.6 only on my new system Lazarus 2.0.7 with Freepascal 3.0.4. both 64 Bit.

When use 7.2.6 on my old system then the error in this topic doesn't appear.

[marsupilami]

Hello,

Copy of the stream is no option, this makes application unnecessarily slow.

Since I don't know your use case it is hard to make suggestions. But I have to admit, that I don't understand, why it would be necessary to close the database connections.

I think my workaround solution is good for me, it does not reference connection, I can close it without error.

In any way - I have to repeat my warning: Simply casting a stream is _not_ supported. We don't guarantee that streams returned are of the type TMemoryStream. At a minimum you should check the type of the strem that gets returned. Chances are high that this will change in Zeos 7.3. It might even change for Zeos 7.2. So your workaround will most probably break in the future.

Best regards,

Jan

[Soner]

Jan, thanks for your suggestions.

Forget what i wrote in this topic. I made stupid error, I forgot to free the stream.
The application also works in new Zeos version without copying the stream.