Zeoslib Portal

Hello there!

Since when I started using Zeos, I've been thinking in optimization and new features. I've switched from BDE, and since then, Zeos helped me a lot.

I've been saying that along posts, in my opinion, Zeos claim for a major rework. In this post, I'll try to brainstorm these ideas, and hear what you think of them.

[align=center][/align]


Architecture: This is a picture from what I suggest to be Architecture of packages in Zeos. Today we have ZCore, ZParseSQL, ZPlain, ZDbc and ZComponents. My suggestions:

• ZPlain, today, have only the intafaces to each driver (without coding, only translation/exposition to pascal). This is a very important acting, but it can be improved. Each DB have it's own implementation at ZPlain layer. It's a real mess. And all the layer makes intense use of inheritante (from one version to another, of the same driver/lib). This makes the thing even more confuse in some points. My suggestion is to make "standalone" drivers, which could encapsulate all specific DB operations and expose them to a commom object, to be used then. Sounds like: Joining ZPlain and ZParseSQL with the DB-specific parts of actual ZDbc, making this layer the only one which is DB-Dependant. Along with this, making a driver will become really easy, not to mention the mantaining of actual drivers. Even the linking can be done at protocol level (to be discussed in this post), only embedding in the application the desired drivers, blocking the increase of EXE size with the removing of driver inheritance.
• ZDbc will become really thin. Removing the DB-specific operations and caching things (next topic), this will be the layer which exposes the commom database taks, just like it do it now. These two layers seem to be bad designed (ZPlain overloaded and ZDbc much thin), and this is a thing I wan't some suggestions..
• ZCache should be a new layer, removing the caching functions from Dbc and making them non DB-specific. This layer should inherit and provide the exactly objects/methods from ZDbc, but cached and bi-directional.
• ZComponents stays as it's today, without DB-specific things.
• ZUtils will be the "others", utilities, common functions, common classes.. all non DB-specific.

Topics I think we should take a look (the numbers are for reference, only):

1. I've been thinking in linking the protocol through a component. The old Zeos 5 have one component to each DB (ZInterbaseQuery, ZMySQLQuery), and that was heavy to keep. We can expose the DB's (or protocols?) with a component, removing the needing of keep the a ZDriverManager instance always on. Doing that way, exposing DB-really-specific things (like MySQL ping) will be really easy.
2. Should we create and cast TZFields with ZDatasets? This will open a wide margin to operate with more "freedom", but should break some compatibilities. Including retro-compatibility. A topic to think about, as I don't have enought knowledge to suggest anything.
3. The caching layer really deserves a rework. There are countless improvements we can do in this, as lower memory using, multithreaded caching (one thread show's the data, another caches it), better searching (using trees)... I'm a bit newbie in this terrain, but open to suggestions
4. With this layout, we can easily expose a non-cached dataset, removing the ZCache weight for who don't need it.
5. I really think we should redesign the sequence component. I wish I can create a sequence in DB (doesn't matter what, since it support's it), see it in IDE and link it to a field, so I can get/set and use it in an AutoInc field.
6. An optional connection pool will be wonderfull. This can be done at Dbc level, exposing the option to use it, or not, and their parameters. Doesn't seem a hard work.
7. Is Zeos thread safe? Until what level? We should grant this somehow. Or not ?
8. ZSQLMonitor can be easily improved. I think it should give us more details on what's going on, that these details should have a standard output so we can filter than (like options to show me all updates, but not the selects). And I wish I can link it to a connection, if I want, then it will show me only the details from this specific connection. And thread safe, too!
9. Someone started a project in which Zeos does not load the metadata. That was good! We, normally, wan't metadata, but sometimes remving it can easily improbe speed. I think it should be optional, at the ZDataSet level. And, why not, expose the metadata in a component, with the ability to export it (to SQL)
10. The DB-specific operations (like ping in MySQL) can be exposed through the driver, in a standard way. This topic needs discussion, but there are operations we cannot have in all DB's, or which are specific to this DB (or driver). These should be managed somehow.
11. Can we have a query builder?
12. Stored Procedures are like normal selects? There are suites which have they own component to StoredProcs. I don't know if it's the case, but it can be inherited directly from ZCache.
13. The actual filter and filtered properties can be improved in speed and syntax.
14. The locate method should be as fast as possible. In a cached dataset or sorted ResultSets, this can be improved.
15. Transactions may have they're own component. There are DBs which support multi-transaction (it can be emulated with a connection pool). This can be done in this way: ZDataset with a transaction property. If nothing specified, it will get the transaction in ZConnection (wihch is a property). This way we can export the TIL, timeout, default action, commit strategy and other configs to the transaction object. Thread safe!
16. I think we should expose the driver capabilities, like supporting one feature or another. This will be informed by the driver, anyway.
17. Server-started notification (like postgres and firebrid) are important? They deserves their own component, or should we expose them through the driver interface?
18. A really charmy feature is to detect a broke connection, make the reconnection and continue to operate as before.
19. If we're caching data in ZCache, why can't we work without being connected to DB ? Ok, there are issues, but should be a good feature to who need it. Connect, get the data, disconnect (or "close" the dataset), change the data, reconnnect and then send the updates).
20. In unicode support, we can use the actual work in Zeos 7 Alpha. This is a very tricky terrain, but it should be done.
21. We really should take care on ClientDataSet support. A lot of people use it.
22. If someone need it, isn't hard to create a BatchMove component, which moves data from one dataset to an equal one, without coding.

There are my ideas and now I'll be pleased to hear yours!

Talking about revolution. That's what you get when you invite young hot-blooded latin types in your team, I suppose.

ZDbc will become really thin. Removing the DB-specific operations and caching things (next topic), this will be the layer which exposes the commom database taks, just like it do it now. These two layers seem to be bad designed (ZPlain overloaded and ZDbc much thin), and this is a thing I wan't some suggestions..

Dbc was intended to do the database connectivity, actually, the part you want to move to the plain layer.

We really should take care on ClientDataSet support. A lot of people use it.

You don't have to wait with that part. Can you get bug 100 fixed? That's a show stopper for the moment.

Server-started notification (like postgres and firebrid) are important? They deserves their own component, or should we expose them through the driver interface?

I think they deserve ther own component, but the current FB implementation should be split into a component part for 'all' databases and a dbc part.

Stored Procedures are like normal selects? There are suites which have they own component to StoredProcs. I don't know if it's the case, but it can be inherited directly from ZCache.

Zeoslib does have this component too... Only, it's not implemented for all databases (eg mysql ) There's a danger in returning resultsets from storedprocs too : not all databases guarantee you get only one result even not a uniform result format. (eg same mysql )

I really think we should redesign the sequence component. I wish I can create a sequence in DB (doesn't matter what, since it support's it), see it in IDE and link it to a field, so I can get/set and use it in an AutoInc field.

Sure this doesn't work already like that??

Someone started a project in which Zeos does not load the metadata. That was good! We, normally, wan't metadata, but sometimes remving it can easily improbe speed. I think it should be optional, at the ZDataSet level. And, why not, expose the metadata in a component, with the ability to export it (to SQL)

Exposition in a component already exists, but if you want to export the database/table/objects/... creation statements you're going to have a tough job. this is different for every database. Unless you ignore the non-standard features of every database and just generatie plain ANSI SQL DDL.

The actual filter and filtered properties can be improved in speed and syntax.

Just don't hesitate

Can we have a query builder?

Michael is trying to build this now.

I think we should expose the driver capabilities, like supporting one feature or another. This will be informed by the driver, anyway.

This is what the DatabaseInfo object was made for.

I'll stop now... Let others think about it too...

Mark

heheh right!

better to have someone to initiate the revolution, even if it don't happen

Let's think, maybe I'm completely wrong, but I want to know it!

I think we must find and improve weak points of ZEOS.
I find many opinion about zeos: Zeos is slow , very slow

mariuszekpl,

I agree. So let's add full unicode support with loads of string operations .

Now seriously : I agree on zeoslib being slow. Maybe even VERY slow but that's all a matter of what you're comparing with.

On the other hand, that's (partly) the price of it's general purpose functionallity. If you want top speed you should write your program using the plain drivers for your specific database. If you want easy updatable TDataset objects, ...

Conclusion : if you see removable bottlenecks that do not impact current functionality, please tell us. Send the description of the problem and the patch to the bug tracker.

Now something practical : I've been trying to contact you directly using mail and private messages (last one still unread). Con you please give me some way to contact you privately?

Mark

I talk about opinions which we can find in internet : "news Group" and forums.
I think we should make delphi code (better) but also MUST think about other matter(s) like:
- web page - now is not easy download zeos
- SEO (Search engine optimization) , if somebody search "zeos+delphi" he must find as
-etc. Public relations

I don't test speed. I don't need it. (for the present)

4 me its more simple:

* do unicode

* do drivers only for Firebird and Postgree

* ClientDataSet support

the rest of drivers people can use other stuff like, DBX, ADO etc etc.
u need to do the best where codegear can't do.

Just a few cents. All databases have their own specific features. It would be great if it was possible to have a 'db' component allowing this. Adding db's to the form allows a dbgeneral component to list the available db's. This way, the footprint can be as low/high as needed. If there's an 'empty' dbgeneral with the necessary routines, then it would be more-easy to implement new drivers for newbees. The structure then becomes:

-dbmysql
-dbprogress ---> dbgeneral -> dbtquery/dbttable -> useual stuff.
-dbsqlite
-dbempty

I had started to write a DBX4 driver using ZEOS Plain, it is almost done, I stoped when I dont see Delphi 2009 progress, so Im using corelabs by while.

The ZEOS licence allow me to do that?

Maybe create this DBX4 driver will be a way for who whants TClientDataSet support.

What you guys think?

As long as you're keeping to the licence (see main directory of the zeoslib distribution) there's no problem. I think copying can be done in any open source component/project, as long as you keep the references intact. Not sure how it exactly works in commercial apps.

If you want to test our new (pre-alpha) 7.0 version, please do so. It's the only way we can get it running. Any help is appreciated.

Mark

BTW : what exactly is DBX4? Is it also known under a less cryptic name?

Where can I get the pre-alpha 7.0 version?

DBX4 = DBExpress Version 4 Framework, for Delphi 2007 and Delphi 2009.

cesarliws wrote:Where can I get the pre-alpha 7.0 version?

DBX4 = DBExpress Version 4 Framework, for Delphi 2007 and Delphi 2009.

SVN at svn://zeos.firmos.at/zeos/branches/testing

Some of my opinion:

1. Improvement on Sort perfomance, I open a query which is have 420 record and make it sorted using Sorted Field property and the performance is slow about 2 second (no BLOB or unicode involved).

2. Improvement of IBEventAlerter, I think the name of this component should be general like just TZEvent

3. The concept of Zeos since version 6 is to make one component for many database connection, so I think we should keep this feature & philosophy.

4. The sequence component is doing right thing now, I don't see that it need to change

5. Unicode is important but we also must think about performace, don't make unicode feature cause Zeos more slower

6. Make Zeos more popular in open source area

1. Improvement on Sort perfomance, I open a query which is have 420 record and make it sorted using Sorted Field property and the performance is slow about 2 second (no BLOB or unicode involved).

All idea's are welcome. This change seems like something everybody with a good Delphi/Pascal knowledge could do.

2. Improvement of IBEventAlerter, I think the name of this component should be general like just TZEvent

This was one of the changes we wanted for version 7.0. Here we need somebody who wants to split of the IB/FB specific implementation to the dbc layer.

3. The concept of Zeos since version 6 is to make one component for many database connection, so I think we should keep this feature & philosophy.

Finally, somebody who agrees with me this is a good thing. I personally believe this is one of the things which sets zeoslib apart from other suites.
If somebody finds an easy way to compile out specific database drivers on a 'per project' basis this is welcomed however.

5. Unicode is important but we also must think about performace, don't make unicode feature cause Zeos more slower

Unfortunately D2009 forces us to take unicode into account... I don't expect current changes to make performance worse on other compilers.

6. Make Zeos more popular in open source area

Idea's?

Mark