[patch_done] mssql Blobs are limited to 4096 bytes

Forum related to MS SQL Server

Moderators: gto, cipto_kh, EgonHugeist

User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Post by EgonHugeist »

ludob,

pm sendet.
That looks like missing debug info in some of the units. When all debug info is available, the debugger stops nicely on the line that causes the problem. Win64 can be a little more complex with exceptions raised from external non-fpc code because of the incompatible SEH but there also some progress has been made recently. Let me know if you have again such a difficult debugging case.
Don't know many things about it. I had this issue on the Zeos-Code with our testsuites. I'll report you an issue if i've got one. But i think if you add the Firbird protocol to the tests then the dbc test will crash and i'm not able to debug them.
ludob
Access granted. Ludo if you've got the time then you can patch the time issue by your selves or i do it tonight. One thing would be very kind: Please merge the patches on \testing and \testing-egonhugeist(as long as this branch exists).

Michael
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
ludob
Fresh Boarder
Fresh Boarder
Posts: 17
Joined: 04.07.2012, 09:35

Post by ludob »

Just checked out \testing-egonhugeist and got with firebird a SigSegv in TZTestDbcInterbaseCase.TestBlobs at the end; statement. Started debugging it and the problem is that the destruction of ResultSet:IZResultSet is apparently overwriting the stack. That is indeed a particularly nasty bug to pinpoint. I'll keep you posted.
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Post by EgonHugeist »

I think i can help you a little bit, Ludo. I remember this happens in the ZDbcIntebaseUtils.pas somewhere where we do access floating-point values. The most parts of this unit are without range-check {$R-} and someone did add an own range-check function. The strange thing is that i was able to read the value with the debug-window bud the app wasn't. Firebird does deliver the most values as PPValue (point to a pointer of a value). I can not understand why the FPC makes trouble there.

Next point is that not only the FPC has such strange behavior. Changing some lines of Zeos "can" result this strange issue on the other compilers too. They are coming an going and i'm not able to understand why. But the FPC trouble is constant not so strange like the Delphi issues i was talking about...

These are the real issues i'm fighting with, not the small bugs the people do report me. Getting help would be great!!

Michael
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
ludob
Fresh Boarder
Fresh Boarder
Posts: 17
Joined: 04.07.2012, 09:35

Post by ludob »

The stack is being corrupted by isc_detach_database. Tried another fbclient.dll (2.1.5.18496) with the same problem. Added some dummy variables in TZInterbase6Connection.Close to change the distance to the stack location being overwritten but the same location is overwritten. So no data overrun in isc_detach_database but apparently somewhere a pointer is handed over to the fbclient that is on the stack and that is being zeroed in the call to isc_detach_database. I'll continue tomorrow with a fresh head;)
miab3
Zeos Test Team
Zeos Test Team
Posts: 1309
Joined: 11.05.2012, 12:32
Location: Poland

Post by miab3 »

You may want to download versions of Firebird-2.5.2.26510:
http://www.firebirdsql.org/en/snapshot-builds/
a lot of bugs version 2.1 removed.

Michal
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Post by EgonHugeist »

@miab3,

i do run my test with FB2.5 and the behavior is the same. Hmpff. I can not say what happens exactly. Realy strange is that our testsuites do report this trouble but we had never ever one thread where somebody reported something like this.

@ludob,

the isc_detach_databes issue is a small issue. It was solvable on adding the try except end; around. But you're right. 'Sometimes' the handle is invalid. Either the stack was overriden or FireBird does result this trouble. Going several tests deeper the suites do totaly crash with the FPC. Delphi raises an Exception which we see on the report and is going on to run the tests. Did you run the test with the GUI or in -b Batch mode? IF you use the GUI then also the GUI is seval times crashing with the FPC. Don't know if Marco ever used the suites...


Michael

Btw. introduced WITH_FPC_FTTIME_BUG define. Patch done Rev.1440eh and Rev1441testing
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
ludob
Fresh Boarder
Fresh Boarder
Posts: 17
Joined: 04.07.2012, 09:35

Post by ludob »

Tried a FB2.5 client before downloading the latest 2.1.5 but that one crashed even earlier, this time inside isc_detach_database, not zeos code. A try except would only mask the problem and, as you said before, changing a few lines of code elsewhere could make the problem disappear or appear somewhere else. I'll spend some more time to find the root cause of this problem. Some of the other problems encountered could be just other symptoms of the same problem. If need be I'll build the fbclient with debug info and try that path.

I use the GUI. It raises sometimes SIGFPE errors. I haven't dug into that one but at first sight it is in a calculation of the progress bar. IIRC it happened when I unchecked the test to run (instead of just selecting) and then click on the "run xyz" button. The exception is uncaught and the program just dies.
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Post by EgonHugeist »

ludob,

Yes i know the try, except does only mask the issue. But i was forced to do this to see what happens with the other tests. Like i wrote is that a strange not 100% reproduceable exception which does make me slightly grazy. I was wondering very long times why and how this can happen. But i'm not a compiler hero which does understand asm code and i was equal not able to find the origin of thes strange issues. My impressen since long times are the disabled range-checks or the huge count of connections. It could also be an issue with Zeos 'em selves. I can not realy say what happens there and why either the stack is vinny nilly or firebird. I love the FB engine. And i use it in several project of my companies with Zeos. Everything is stable. I never ever had issues like this.

Here we've the problem that the FPC (in my mind) has a different behavior to delphi with such exception handling. Yes indeed the GUI makes trouble with the progressbar. So i commented this line out on my sources(no solution, i know) and later i switched to the batch-mode because i use WinMerge to check the fails and exception with previous releases. Mark the project-Manager does the same weekly. But i was equal not able to locate the issue with this progressbar. I made a try and except around and did declare some vars to copy the values in the except parts. This was no problem. All values where easiely able to copy but again in exceuting the algorithym for the position of the bar it crashing again and again. I don't know why.. The only thing i know is that this trouble only happens if firebird+fpc is involved.

There are other things too which i can't explain like this:

Code: Select all

function TZRowAccessor.GetBlobObject(Buffer: PZRowBuffer;
  ColumnIndex: Integer): IZBlob;
var
  BlobPtr: PPointer;
  NullPtr: {$IFDEF WIN64}PBoolean{$ELSE}PByte{$ENDIF};
begin
  BlobPtr := PPointer(@Buffer.Columns[FColumnOffsets[ColumnIndex - 1] + 1]);
  NullPtr := {$IFDEF WIN64}PBoolean{$ELSE}PByte{$ENDIF}(@Buffer.Columns[FColumnOffsets[ColumnIndex - 1]]);

  {$IFNDEF FPC}
  if NullPtr^ = {$IFDEF WIN64}false{$ELSE}0{$ENDIF} then  //M.A. if NullPtr^ = 0 then
  {$ELSE}
  if NullPtr^ = 0 then
  {$ENDIF}
    Result := IZBlob(BlobPtr^)
  else
    Result := nil;
end;

{**
  Sets a blob into the specified columns.
  @param Buffer a row buffer.
  @param ColumnIndex an index of the column.
  @param Value a stream object to be set.
}
procedure TZRowAccessor.SetBlobObject(Buffer: PZRowBuffer; ColumnIndex: Integer;
  Value: IZBlob);
var
  BlobPtr: PPointer;
  NullPtr: {$IFDEF WIN64}PBoolean{$ELSE}PByte{$ENDIF};
begin
  BlobPtr := PPointer(@Buffer.Columns[FColumnOffsets[ColumnIndex - 1] + 1]);
  NullPtr := {$IFDEF WIN64}PBoolean{$ELSE}PByte{$ENDIF}(@Buffer.Columns[FColumnOffsets[ColumnIndex - 1]]);

  {$IFNDEF FPC}
  if NullPtr^ = {$IFDEF WIN64}false{$ELSE}0{$ENDIF} then  //M.A. if NullPtr^ = 0 then
  {$ELSE}
  if NullPtr^ = 0 then
  {$ENDIF}
    IZBlob(BlobPtr^) := nil
  else
    BlobPtr^ := nil;

  IZBlob(BlobPtr^) := Value;

  if Value <> nil then
  {$IFNDEF FPC}
    NullPtr^ := {$IFDEF WIN64}false{$ELSE}0{$ENDIF}  //M.A. NullPtr^ := 0
  else
    NullPtr^ := {$IFDEF WIN64}true{$ELSE}1{$ENDIF};  //M.A. NullPtr^ := 1;
  {$ELSE}
    NullPtr^ := 0
  else
    NullPtr^ := 1;
  {$ENDIF}
end;
This is an code extraction of the ZDbcCache unit. Totaly grazy here is the the Null byte of our rowaccessor field offsets was not readable corectly in Win64+FPC. A thread for more details: http://zeos.firmos.at/viewtopic.php?t=3 ... c&start=45
Here i was running the delphi and fpc debugger to find the issue. I was wondering why the IsNull byte delivers always false which means i had the crash on trying to access a nilled interface. It was a very long night. But successful and Zeos was now prepared for Win64.

Ludo thank you for investing some time into these strange things. But it could be equal save your time if you point somone who is a compiler hero to this issues. They are strange, realy strange. For me not realy understandable. Eventually do we find another leak on the FPC or an Zeos. But knowing the reason would be great. I'm nobody who likes it to close his eyes and play the blind man. If i can understand an issue then i can solve this. But here i was forced to give uo a while until i find more time for this.

Michael
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
ludob
Fresh Boarder
Fresh Boarder
Posts: 17
Joined: 04.07.2012, 09:35

Post by ludob »

Got it:)
isc_dsql_alloc_statement2 will null the StmtHandle when freed. TZInterbase6Statement.ExecuteQuery uses a local variable to hold the StmtHandle but doesn't call isc_dsql_free_statement on it. The handle is kept in fbclient and when isc_detach_database is called all pending statement handles are nulled. The pointer to StmtHandle is not valid any more and isc_detach_database nulls a "random" value on the stack.
I inserted a
FreeStatement(GetPlainDriver, StmtHandle, DSQL_drop);
at ZDbcInterbase6Statement:238 and the crash in TestBlobs is gone.

The question is: why use isc_dsql_alloc_statement2 and not just isc_dsql_alloc_statement? Nulling the handle is not useful when it is a local variable and when it is a property, nulling it in code after freeing is good practice. In this case a simple potential memory leak (statement handle not freed) strikes back as a particular nasty bug (had to use visual studio and debug fbclient to find who writes on the stack and then where the "user_handle" was passed on to fbclient).

I'll continue to run other FB tests using isc_dsql_alloc_statement2 and when no other related problems pop up I'll commit the fix.


EDIT: ZDbcInterbase6Statement:238 is not a good solution because it throws away the resultset also. So I'm now using isc_dsql_allocate_statement. Full test drops to 2 errors and 18 failures now. I haven't updated the WITH_FPC_FTTIME_BUG yet.
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Post by EgonHugeist »

ludob,

What can i say? Great job! Great mind! You can't believe how happy these news do making me. And i'm sure MDaems will be happy too! So it was a Zeos-bug! We need somebody like you here on the Zeos front!
I'll test it this afternoon if i'm back on my computer.
According the patch of yesterday night: I think no untit where you did work yet is touched with my small patch. It's only the ZDataSetUtils.pas and the Lazarus.inc where the new define is introduced. So updating seems not like a problem.

Trank you, Ludo!
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
ludob
Fresh Boarder
Fresh Boarder
Posts: 17
Joined: 04.07.2012, 09:35

Post by ludob »

Patch committed in testing and testing-egonhugeist.

Also solved the SIGFPE in GUITestRunner. Patch at http://bugs.freepascal.org/view.php?id=22375
ludob
Fresh Boarder
Fresh Boarder
Posts: 17
Joined: 04.07.2012, 09:35

Post by ludob »

Michael,

Committed another patch for TZAbstractRODataset.GetFieldData and ftWideString which caused an exception in the Firebird tests. I don't have the necessary Delphi tools to do the testing against Delphi. I try to use non-FPC specific code for fixes but I can't test it fully. I suppose you do regular regression testing.

This mssql thread is becoming a mess. Do you prefer email, pm or opening another thread for other problems fixed? There no exception type of errors any more with FB. I noticed one with Postgres yesterday. I'm going to look at that one now.

BTW I sent you an email yesterday. Did you receive it?

Ludo
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Post by EgonHugeist »

ludob,

you right! Lol, what you propose is what i'm thinking! Point 1 is that we do talk here about things which had been mostly hidden and i do not want to have users which becomming worried again about Zeos7. The bugs you've killed where present since the Zeos6 and now you did run the test's without instruction and do solve the most issues we had since several years!

Ludo yes i got your mail. Didn't i answer from my smartphone? I was on tour with my employes while answering so i can't say if i realy send a repley. Let's switch to eMail. I'll attach you my test-protocols after the two fixes you've made. Yes i saw the second one and i'm equal not able to say how do the Ansi-Delphis now behave because i cant force one of the test for these compilers into the WideString-mode. The reason is that my TZConnection.UTF8AsWideString property will not be set via the TZConnection.Properties (Info). Let's see what the users do say. If we've got a new issue then we'll all a {$IFDEF FPC} around and everything is fine.

Now i swith to eMail, Ludo. Thanks again.

Michael
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
User avatar
EgonHugeist
Zeos Project Manager
Zeos Project Manager
Posts: 1936
Joined: 31.03.2011, 22:38

Post by EgonHugeist »

ludob,

did you receive any of my mails?

Michael
Best regards, Michael

You want to help? http://zeoslib.sourceforge.net/viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image
Post Reply