I found that ZSQLMonitor is quite useful for logging-purposes. The only thing I find very unpracticle is that it logs events from simply all ZConnection's I am using in a application. Even when I have 2 MDI-windows each containing 1 ZConnection + 1 ZSQLMonitor, both of the ZSQLMonitor's log also events from the other windows/connection...
Is this a bug? It is likely intended to work so, because in a ZSQLMonitor-Object there is no property like "Connection" or similar, which says that this monitor is linked to a connection. This would make more sense than the current behaviour of the 6.5.1 (latest SVN).
Can i somehow restrict event-logging to one connection with a workaround?
ZSQLMonitor always logs application-wide
Moderators: gto, cipto_kh, EgonHugeist
Hello! Yes, this is not a bug, is a feature! 
ZSQLMonitor links to every ZConnection found in application, so there's no way to limit to only one (as far I know).
But, ZSQLMonitor code is quite simple: There's a common variable called DriverManager, that does the communication between ZSQLMonitor and events. Every time that something needs to be logged, it's logged to DriverManager, and all ZSQLMonitors links to DriverManager. What you can do is inspect the source and create something to limit to only a certain ZConnection.
ZSQLMonitor code should be in zeosRoot\Src\Component\ZSQLMonitor.pas
DriverManager code should be in zeosRoot\Src\Dbc\ZDbcIntfs.pas
The logging stuff should be in zeosRoot\Src\Dbc\ZDbcLogging.pas
ZSQLMonitor links to every ZConnection found in application, so there's no way to limit to only one (as far I know).
But, ZSQLMonitor code is quite simple: There's a common variable called DriverManager, that does the communication between ZSQLMonitor and events. Every time that something needs to be logged, it's logged to DriverManager, and all ZSQLMonitors links to DriverManager. What you can do is inspect the source and create something to limit to only a certain ZConnection.
ZSQLMonitor code should be in zeosRoot\Src\Component\ZSQLMonitor.pas
DriverManager code should be in zeosRoot\Src\Dbc\ZDbcIntfs.pas
The logging stuff should be in zeosRoot\Src\Dbc\ZDbcLogging.pas
-
mdaems
- Zeos Project Manager
- Posts: 2766
- Joined: 20.09.2005, 15:28
- Location: Brussels, Belgium
- Contact:
I just had a quick look, and I don't think the events that are dispatched contain enough information to split by connection.
So I think this feature would require a connection field to be added and all calls to TZDriverManager.LogError should all be extended to contain a connection parameter.
Sounds like 'not to difficult but hell of a job'. Once this is done it should be fairly easy to implement some filtering features to the sqlmonitor component.
Anybody feels like doing it (or at least step one)?
Of course, if somebody sees an other solution, we all want to hear about it.
Mark
Code: Select all
TZLoggingEvent = class (TObject)
private
FCategory: TZLoggingCategory;
FProtocol: string;
FMessage: string;
FErrorCode: Integer;
FError: string;
FTimestamp: TDateTime;
public
constructor Create(Category: TZLoggingCategory; Protocol: string;
Msg: string; ErrorCode: Integer; Error: string);
function AsString: string;
property Category: TZLoggingCategory read FCategory;
property Protocol: string read FProtocol;
property Message: string read FMessage;
property ErrorCode: Integer read FErrorCode;
property Error: string read FError;
property Timestamp: TDateTime read FTimestamp;
end;
Sounds like 'not to difficult but hell of a job'. Once this is done it should be fairly easy to implement some filtering features to the sqlmonitor component.
Anybody feels like doing it (or at least step one)?
Of course, if somebody sees an other solution, we all want to hear about it.
Mark
-
mdaems
- Zeos Project Manager
- Posts: 2766
- Joined: 20.09.2005, 15:28
- Location: Brussels, Belgium
- Contact:
I use TortoiseSvn and rightclick on the src-directory of my testing branch copy. There is some svn command 'create patch' or something like that.
Some catches : new files need an 'svn add' before they are included in a patch. Idem for deletes to remove them. It's almost like a commit, it just writes to a file instead of the directory. When I receive this file format it's a one click merge .
Command line svn also has a diff command. For syntax you should check svn documentation. If you pipe the output to a file that should work.
Thanks for applying for the job!! Please check out a fresh testing branch version before doing the changes. It saves me a lot of work when integrating the patch.
BTW, Thanks for your jedi link. Due to time limitations I can't check it for the moment. I hoped I could write a few lines like
I hope to see the patch soon. This is a request that I've seen since a long time.
Will you add a pointer to the connection object (TZabstractConnection or IZConnection type) in the TZLoggingEvent? That could be more usefull than just the name of the object when we want to implement filters in the SQLMonitor (filter by protocol/host/port/...). Also, that way we could get the protocol from the connection instead of passing it to the logging event every time. (Or is that to difficult to get out of a connection object?)
Mark
Some catches : new files need an 'svn add' before they are included in a patch. Idem for deletes to remove them. It's almost like a commit, it just writes to a file instead of the directory. When I receive this file format it's a one click merge .
Command line svn also has a diff command. For syntax you should check svn documentation. If you pipe the output to a file that should work.
Thanks for applying for the job!! Please check out a fresh testing branch version before doing the changes. It saves me a lot of work when integrating the patch.
BTW, Thanks for your jedi link. Due to time limitations I can't check it for the moment. I hoped I could write a few lines like
Code: Select all
c:\borland\delphi7\bin\<compiler> <option option option>
c:\borland\delphi7\bin\<installcommand> <option>Will you add a pointer to the connection object (TZabstractConnection or IZConnection type) in the TZLoggingEvent? That could be more usefull than just the name of the object when we want to implement filters in the SQLMonitor (filter by protocol/host/port/...). Also, that way we could get the protocol from the connection instead of passing it to the logging event every time. (Or is that to difficult to get out of a connection object?)
Mark
Hello people!
First of all, some info:
If I can say what structure Zeos is build on, I'll show this:
The Core package have procudures, types and definitions that are common to all others. Common functions, basic types and other "core" stuff.
ParseSQL and Plain work together. Plain package have all low level, platform specific stuff. This is the interface where client library is made.
ParseSQL is almost the same thing, but it does the platform specific SQL parsing. That package does the SQL text that will be sent to server, and check your SQL to catch errors.
Dbc layer is the second in the top. I may call it the most important package. This is where Plain meets ParseSQL and the functions of Core are used. It join all lower packages into one "API".
Component is the top-most layer, responsable to talk with IDE specific things. It does the "middle-man-work" between user IDE and Dbc API. This is where all components are defined, and this is the layer that you talk with.
If I'm wrong, please, let me know it.
Today, the log is created usgin two procedures, LogMessage and LogError. The parameters of procedures are almost equal. As LogMessage redirects for LogError with ErrorCode (integer) = 0 and Error (String) = EmptyString, it don't have this two parameters. The others, common to both proceudres, are:
Category: TZLoggingCategory; A Category for the "Log Packet". TZLogginCategory have this options: lcConnect, lcDisconnect, lcTransaction, lcExecute, lcOther.
Protocol: String; The name of protocol used to generate this Log Packet. It's the same same that apeear into ZConnection protocol field, like 'firebird-1.5' or 'mysql-4.1'.
Msg: string; A message for the Log Packet. It changes much to where the log is generated. Can be a full SQL script or just a few words, explaining whats going on.
Joining all, you'll have the type called TZLoggingEvent, which is passed to SQLMonitor .
The logging subsystem is all into the Dbc layer. Outside, there's only SQLMonitor, that receives Log Packets from DriverManager, and object located into ZDbcIntfs.pas.
The LogError procedure:
As you may see, it loops through all FLoggingListeners and send them the just-created Log Packet (TZLoggingEvent). That behavior permits SQLMonitor to receive log form *anywhere*, as Dbc layer is always the same, independent from Component layer.
Then, I've started the job, but there's a little problems on the way:
1st Problem: Standard Coding ?
I've never get that deep into zeos code. The first thing I've seen: LOTS of differences beween loging calls. Take by example LogError, it's commony called by CheckMyProtocolError, into ZMyProtocolUtils.pas, only. This is true for DbLib, Interbase, Oracle and SQLite units. A problem is that in DbLib, CheckMyProtocolError is defined as a procudure from TZDBLibConnection and in all other protocols it's a "solitary proceudre".
For MySql and PostgreSQL, there's a call for LogError into ZMyProtocolConnection.Open! And they're allways called! It supposed to be a LogMessage call, not LogError.
In ASA subsystem, there are four calls to LogError, Two in Open/Close of TZASAConnection object, one in CheckASAError and one in TZASASQLDA.CreateException (!).
Even worse is ADO. There are nine calls to LogError, from different places, and no "ADOCheckError".
It's worse becouse you cant say that there's a standard coding template for Dbc layer. Most of them are standard, but some aren't. So, it's difficult to say what data will be avaliable to Log, as the log is invoked from different places, into different protocols. LogMessage has the same problem, but much more worse, as it have much more calls than LogError.
2nd Problem: What data to pass? Can I do a "Cross-Layer" parameter?
My original idea is to change the LogError and LogMessage procedure to be different things. Both will reach SQLMonitor, but with different data. LogError is aways "main called" from CheckMyProtocolError, a place that you don't have much information. LogMessage is called from many places, and you have different information, relative to the place you're calling. A problem: What information to Log ? There goes one idea:
procedure TZDriverManager.LogMessage(Category: TZLoggingCategory;
Driver: IZDriver; Connection: IZConnection; const Msg: string);
Leave LogError as now, and alter LogMessage to include Driver and Connection. These variables are mostly avaliable when LogMessage is called, one or other places they're not, but it can be fixed. But here's another problem: Passing them as parameters will be legal? I know that Zeos is based on JDBC (some commentaries still mention this), and JDBC, as the most Java projects is very problematic with this "OO stuff". It may (will) work well, but I dunno if it's "legal".
3rd Problem: Why not redo all?
Well, thats the end. After this all, my final idea: Rebuild the "log subsystem", change LogError and LogMessage to be the same thing (they're different things, from different places, that end up into the same place (SQLMonitor)), change all that stuff to make it "standard", or the closest of it. I can do this, no problemo. But I need two answers:
Can I do that deep changes?
What data may reach SQLMonitor?
For the second question: On errors, is not possible to pass many details, as they don't exist where LogError is called. It's possible to have a special key with TZLoggingEvent, saying that it's an error or not, and enhance it to have new fields, like the ones present in IZDriver and IZConnection. But, it's NOT currently possible to say from what TZConnection it belongs. We may pass the entire IZConnection for TZLoggingEvent, and them compare with the IZConnection acessible through TZConnection, but there's the "OO conformance" hole again.
Well, I'm on wait of comments.
First of all, some info:
If I can say what structure Zeos is build on, I'll show this:
Code: Select all
component
------------------
dbc
------------------
parsesql - plain
------------------
coreThe Core package have procudures, types and definitions that are common to all others. Common functions, basic types and other "core" stuff.
ParseSQL and Plain work together. Plain package have all low level, platform specific stuff. This is the interface where client library is made.
ParseSQL is almost the same thing, but it does the platform specific SQL parsing. That package does the SQL text that will be sent to server, and check your SQL to catch errors.
Dbc layer is the second in the top. I may call it the most important package. This is where Plain meets ParseSQL and the functions of Core are used. It join all lower packages into one "API".
Component is the top-most layer, responsable to talk with IDE specific things. It does the "middle-man-work" between user IDE and Dbc API. This is where all components are defined, and this is the layer that you talk with.
If I'm wrong, please, let me know it.
Today, the log is created usgin two procedures, LogMessage and LogError. The parameters of procedures are almost equal. As LogMessage redirects for LogError with ErrorCode (integer) = 0 and Error (String) = EmptyString, it don't have this two parameters. The others, common to both proceudres, are:
Category: TZLoggingCategory; A Category for the "Log Packet". TZLogginCategory have this options: lcConnect, lcDisconnect, lcTransaction, lcExecute, lcOther.
Protocol: String; The name of protocol used to generate this Log Packet. It's the same same that apeear into ZConnection protocol field, like 'firebird-1.5' or 'mysql-4.1'.
Msg: string; A message for the Log Packet. It changes much to where the log is generated. Can be a full SQL script or just a few words, explaining whats going on.
Joining all, you'll have the type called TZLoggingEvent, which is passed to SQLMonitor .
The logging subsystem is all into the Dbc layer. Outside, there's only SQLMonitor, that receives Log Packets from DriverManager, and object located into ZDbcIntfs.pas.
The LogError procedure:
Code: Select all
procedure TZDriverManager.LogError(Category: TZLoggingCategory; const Protocol: string; const Msg: string; ErrorCode: Integer; const Error: string);
var
I: Integer;
Listener: IZLoggingListener;
Event: TZLoggingEvent;
begin
if FLoggingListeners.Count = 0 then
exit;
Event := TZLoggingEvent.Create(Category, Protocol, Msg, ErrorCode, Error);
try
for I := 0 to FLoggingListeners.Count - 1 do
begin
Listener := FLoggingListeners[I] as IZLoggingListener;
try
Listener.LogEvent(Event);
except
end;
end;
finally
Event.Destroy;
end;
end;Then, I've started the job, but there's a little problems on the way:
1st Problem: Standard Coding ?
I've never get that deep into zeos code. The first thing I've seen: LOTS of differences beween loging calls. Take by example LogError, it's commony called by CheckMyProtocolError, into ZMyProtocolUtils.pas, only. This is true for DbLib, Interbase, Oracle and SQLite units. A problem is that in DbLib, CheckMyProtocolError is defined as a procudure from TZDBLibConnection and in all other protocols it's a "solitary proceudre".
For MySql and PostgreSQL, there's a call for LogError into ZMyProtocolConnection.Open! And they're allways called! It supposed to be a LogMessage call, not LogError.
In ASA subsystem, there are four calls to LogError, Two in Open/Close of TZASAConnection object, one in CheckASAError and one in TZASASQLDA.CreateException (!).
Even worse is ADO. There are nine calls to LogError, from different places, and no "ADOCheckError".
It's worse becouse you cant say that there's a standard coding template for Dbc layer. Most of them are standard, but some aren't. So, it's difficult to say what data will be avaliable to Log, as the log is invoked from different places, into different protocols. LogMessage has the same problem, but much more worse, as it have much more calls than LogError.
2nd Problem: What data to pass? Can I do a "Cross-Layer" parameter?
My original idea is to change the LogError and LogMessage procedure to be different things. Both will reach SQLMonitor, but with different data. LogError is aways "main called" from CheckMyProtocolError, a place that you don't have much information. LogMessage is called from many places, and you have different information, relative to the place you're calling. A problem: What information to Log ? There goes one idea:
procedure TZDriverManager.LogMessage(Category: TZLoggingCategory;
Driver: IZDriver; Connection: IZConnection; const Msg: string);
Leave LogError as now, and alter LogMessage to include Driver and Connection. These variables are mostly avaliable when LogMessage is called, one or other places they're not, but it can be fixed. But here's another problem: Passing them as parameters will be legal? I know that Zeos is based on JDBC (some commentaries still mention this), and JDBC, as the most Java projects is very problematic with this "OO stuff". It may (will) work well, but I dunno if it's "legal".
3rd Problem: Why not redo all?
Well, thats the end. After this all, my final idea: Rebuild the "log subsystem", change LogError and LogMessage to be the same thing (they're different things, from different places, that end up into the same place (SQLMonitor)), change all that stuff to make it "standard", or the closest of it. I can do this, no problemo. But I need two answers:
Can I do that deep changes?
What data may reach SQLMonitor?
For the second question: On errors, is not possible to pass many details, as they don't exist where LogError is called. It's possible to have a special key with TZLoggingEvent, saying that it's an error or not, and enhance it to have new fields, like the ones present in IZDriver and IZConnection. But, it's NOT currently possible to say from what TZConnection it belongs. We may pass the entire IZConnection for TZLoggingEvent, and them compare with the IZConnection acessible through TZConnection, but there's the "OO conformance" hole again.
Well, I'm on wait of comments.
-
mdaems
- Zeos Project Manager
- Posts: 2766
- Joined: 20.09.2005, 15:28
- Location: Brussels, Belgium
- Contact:
Hi Gto,
I've the impression you did your studywork very well. Your conclusions seem right.
Standard Coding
I think the origin of the problem is the different developers that wrote the drivers.
Best solution for the future would be to make (error)Log a Connection method. That way you'll always have a Connection you can pass to the Monitor component. Every driver can in turn extend this basic method to add his specific requirements when needed. Have you seen events to be logged where no connection object is available?
Practically you can implement this for 1 database only and set the 'connection' parameter for the TZLoggingEvent to nil by default when DriverManager.LogError is used. For the new way of working we introduce Drivermanager.Log(...), basically the same function, but with more input parameters. We can rewrite LogError and LogMessage as a call to Log.
What data to pass
Why would you add a IZDriver parameter? From the connection you can always derive the Driver using the GetDriver function.
Concerning JDBC : Zeos was derived from it, but I don't think we should think about java troubles. So, if there's no problem for Object Pascal... fine. I don't think we're talking about 'cross-layer' data. There's only one interface involved as far as I see, but I can be wrong. We only pass data from the Components to the DBC interface and viceversa. And I think it can all be done using Interfaces instead of Classes, but I'm a very beginner with the interfaces concept. Are all connections descendents of TZConnection? Maybe we better use the class as that makes it easier to cast to TZMyConnection objects later?
Why not redo it all
Yes, why not... the only problem I see is that we don't "know" the drivers we don't use and we can't test them. Therefore we might need some 'in between' solution as I mentionned before. You choose the database driver you want to start with and change it to the new way of working. If you need to change interfaces, just make dummy methods for the drivers that are not used by you to get the compilation allright. (Add some easy hint for colleages implementing the other drivers.) As long as we don't break the old logging interface definitions this won't break up anything.
Of course, the more you can implement on the generic level (eg TZConnection level), the better. That should make adapting the other drivers easier.
If you're not using Mysql I'll give the mysql driver a rework to see if it all fits in before commiting to SVN. If you use mysql... let's hope it turns out good for the other drivers too.
What should reach SqlMonitor
Everything. (hmmm...) When we add the connection parameter this should enable most of usefull data, I think.
Actually, it's the task of the SqlMonitor to filter the data himself.
I would provide two ways to filter : Some component properties like 'ConnectionFilter','ProtocolFilter',.... (regexp?) and the OnTrace event which can be used by a user to write the filter himself. OnTrace is already present, but can only use the limited information that's present in the current TZLoggingEvent.
To be able to write this filters it might be necessary to add some more methods to the IZConnection interface, but that's not a big deal (and useless if we transfer a TZConnection object).
So, just give it a try. If nobody complains within now and and a week, I think we should go on and commit it to SVN.
Mark
I've the impression you did your studywork very well. Your conclusions seem right.
Standard Coding
I think the origin of the problem is the different developers that wrote the drivers.
Best solution for the future would be to make (error)Log a Connection method. That way you'll always have a Connection you can pass to the Monitor component. Every driver can in turn extend this basic method to add his specific requirements when needed. Have you seen events to be logged where no connection object is available?
Practically you can implement this for 1 database only and set the 'connection' parameter for the TZLoggingEvent to nil by default when DriverManager.LogError is used. For the new way of working we introduce Drivermanager.Log(...), basically the same function, but with more input parameters. We can rewrite LogError and LogMessage as a call to Log.
What data to pass
Why would you add a IZDriver parameter? From the connection you can always derive the Driver using the GetDriver function.
Concerning JDBC : Zeos was derived from it, but I don't think we should think about java troubles. So, if there's no problem for Object Pascal... fine. I don't think we're talking about 'cross-layer' data. There's only one interface involved as far as I see, but I can be wrong. We only pass data from the Components to the DBC interface and viceversa. And I think it can all be done using Interfaces instead of Classes, but I'm a very beginner with the interfaces concept. Are all connections descendents of TZConnection? Maybe we better use the class as that makes it easier to cast to TZMyConnection objects later?
Why not redo it all
Yes, why not... the only problem I see is that we don't "know" the drivers we don't use and we can't test them. Therefore we might need some 'in between' solution as I mentionned before. You choose the database driver you want to start with and change it to the new way of working. If you need to change interfaces, just make dummy methods for the drivers that are not used by you to get the compilation allright. (Add some easy hint for colleages implementing the other drivers.) As long as we don't break the old logging interface definitions this won't break up anything.
Of course, the more you can implement on the generic level (eg TZConnection level), the better. That should make adapting the other drivers easier.
If you're not using Mysql I'll give the mysql driver a rework to see if it all fits in before commiting to SVN. If you use mysql... let's hope it turns out good for the other drivers too.
What should reach SqlMonitor
Everything. (hmmm...) When we add the connection parameter this should enable most of usefull data, I think.
Actually, it's the task of the SqlMonitor to filter the data himself.
I would provide two ways to filter : Some component properties like 'ConnectionFilter','ProtocolFilter',.... (regexp?) and the OnTrace event which can be used by a user to write the filter himself. OnTrace is already present, but can only use the limited information that's present in the current TZLoggingEvent.
To be able to write this filters it might be necessary to add some more methods to the IZConnection interface, but that's not a big deal (and useless if we transfer a TZConnection object).
So, just give it a try. If nobody complains within now and and a week, I think we should go on and commit it to SVN.
Mark
Well, just a post to say: I'm not dead yet! 
Many problems this and past week, but back to bussines now.
The work to get more details reaching SQLMonitor is almost done. The TZLoggingEvent now have the following properties:
This procedure take all details of current connection, create the TZLoggingEvent and spread it through DriverManager, just like the LogMessage and LogError do today.
One problem that i'm trying to solve: I've already converted all LogMessage calls, all places they're called are descendant or have access to the connection object. But LogError have calls into CheckMyDbError, which have no direct access to the connection, just the driver.
Someone would like to suggest me what is better: Find a way to expose the connection object into MyDbUtils files (through a variable or something) or make procedure to create logs from the driver (which MyDbUtils already have access) and them forward it to the connection.
I'm not posting the code becouse it's very unclear yet. I'm almost at the end of it, just this calls to LogError (six or seven) to convert.
While this, i'll develop the filtering options for SQL monitor.
Many problems this and past week, but back to bussines now.
The work to get more details reaching SQLMonitor is almost done. The TZLoggingEvent now have the following properties:
- Category: TZLoggingCategory
- Protocol: string
- Message: string
- ErrorCode: Integer
- Error: string
- Timestamp: TDateTime
- ConnectionOpen: boolean
- ConnectionReadOnly: boolean
- ConnectionAutoCommit: boolean
- HostName: string
- Port: integer
- Database: string
- User: string
Code: Select all
procedure CreateLog(LogCategory: TZLoggingCategory; const LogMessage: string; ErrorCode: integer = 0; ErrorMessage: string = '');One problem that i'm trying to solve: I've already converted all LogMessage calls, all places they're called are descendant or have access to the connection object. But LogError have calls into CheckMyDbError, which have no direct access to the connection, just the driver.
Someone would like to suggest me what is better: Find a way to expose the connection object into MyDbUtils files (through a variable or something) or make procedure to create logs from the driver (which MyDbUtils already have access) and them forward it to the connection.
I'm not posting the code becouse it's very unclear yet. I'm almost at the end of it, just this calls to LogError (six or seven) to convert.
While this, i'll develop the filtering options for SQL monitor.