Page 1 of 1

Accessing large tables

Posted: 25.09.2006, 12:23
by sandeep_c24
Hi

I have a table with 100,000+ records in it and every 2nd day I have to read all the records in that table to do some reports. If I use TZTable to read this table will it fetch all the records at once in memory or will it read as it goes along?

Is it possible to have Unidirectional mode like in dbExpress?

Regards

Sandeep

Posted: 25.09.2006, 14:19
by mdaems
Hi Sandeep,

As far as I know, when you arrive at the end of your dataset, everything will be in memory. Whether it will be read at once or not depends partially on your database implementation. For mysql you can choose, I think.
For the Unidirectional mode : that's a TDataset property, so it should be available. I only don't know if it affects the behaviour the way you expect.

I think you know : avoid processing all data from a table when you can ask your database to do it.
If you just need summaries you can usually write these in a query.
Sometimes you can create a temporary table to summarize data a little before you actually get them into your app.
If you have to write all this data to a file, there might be better solutions to do that.

Mark

Posted: 25.09.2006, 14:36
by gto
Ugly 'n Dirty fix: Use a TZQuery (or TZReadOnlyQuery), set the filtered property to true (don't add any filter, left it empty) and them use the query.

This makes the query to just fetch what is needed, so you will be "fetching at demand" ;)

Posted: 26.09.2006, 10:44
by sandeep_c24
Thanks, I'll try that.

Can Unidirectional mode be added to Zeos?

Regards

Sandeep

Posted: 26.09.2006, 12:17
by gto
sandeep_c24 wrote:Thanks, I'll try that.

Can Unidirectional mode be added to Zeos?

Regards

Sandeep
Not quick 'n dirty :D

Some changes must be made, like a property to set the Unidirectional condition and all the paths to avoid if it's set (like no caching). In other words: It's possible, but must be made ;)