Event Log should rotate or at least have some maximum number of records

Hello,

MODX version : 0.9.5

This morning I tried to export my MODX database.
This was not easy since it grew to a size of 180 Mb (!!)
while it is a fairly small site I have.

Investigating the problem it became clear that the Event_Log
table has grown to almost 500000 (500 thousand!!) records.

While I do not check all 500000 records if there are any sensible
log entries, it leaves me with no other option that to truncate
the table.

What should be done is one of the following:
1) maximize the Event_Log table. Maximum number of records in
the config.

2) rotate the entries. If more than 1000 entries, remove the
100 oldest ones. (900 entries is also already too much to read,
but makes the table in the database easier to handle)

3) (Maybe nicest solution) Run an MD5 hash on your message,
if EventLogger wants to add a message, first check the md5 of
the message if it’s already there and keep a counter of the
number of same messages.

(The error which appeared 500000 times will be left in another
topic of course)

Thanks,

Bert Catsburg

Delete completely the template switcher plugin and your event record table will stop growing like that. There’s a reason we removed it from the 096 install/distribution completely.

Good idea on truncating the event log table though. If you have sample code we’d be glad to look at it and commit it for an upcoming release.

Hello !

I had the same problem with the 0.9.6 modX french version.
I didn’t have the ’template switcher plugin’ installed in my modX, but I found it’s equivalent : ’EditArea’. Deleting this one, the problem was solved.

Thanks to you

My previous idea about automatically cleaning up the even_log after X records sounds nice, but the problem is that MODX CMS does not have any Batch Processing. So, what should trigger the cleaning up. We don’t want every Write action to the event log made slower because it has to do a SELECT COUNT(*) of the table.

I would suggest the following flow of the cleaning-up routine:
1) Have a System Parameter indicating how many records the Site Owner wants in the even log. Default to 1000.
2) Every time a record is written, increase a counter in another table.
3) Check the counter every time (probably this system setting is loaded in memory anyways, I don’t know the core of MODX to know this)
4) If counter reaches MAX + 10%, remove 20% of the records.

OR

1) Have a System Parameter indication how often the event log should be checked and cleaned. (in Number of Days)
2) Records the last time checking and cleaning was done.
3) If Current Time > (Last Time + number of days) --> Check and Clean.

Since I don’t know the core code of MODX, I have no clue where to look for the code which writes the Event Log. Probably in /manager/actions somewhere?

You could write a script to clean out the events table and have a cron scheduled to run the script monthly or weekly.
http://en.wikipedia.org/wiki/Cron

A cron is no option since most of our customers are running on Shared Hosting environments. There is no access to Cron in Shared Hosting.
I can imagine that most of the website built with MODX are running in Shared Hosting.

Quote from: bertcatsburg at Dec 27, 2007, 07:59 AM

A cron is no option since most of our customers are running on Shared Hosting environments. There is no access to Cron in Shared Hosting.
I can imagine that most of the website built with MODX are running in Shared Hosting.

This is not really true in any regard: many of us deploy MODx on VPS or dedicated servers (I would never host a business web site on a shared host if there is any requirement for data security or uptime reliability) and many shared hosting control panels (notably CPanel and Plesk) do in fact offer cron job configuration.

Regardless, a new feature in 0.9.7 (not yet on the roadmap) called the modRegistry will make scheduling and polling processes much easier to implement purely in MODx/PHP. modRegistry will provide a "message queue" that essentially serves as a centralized place for applications to communicate asynchronously, and checking for messages will mean a bit less overhead than running a query or similar action on every request to determine if the cache needs to be cleared.

Speaking from Europe (Netherlands in fact) I can safely say that most medium sized business websites run on shared hosts. There might be a connection (XML, HTTP-Post, SOAP) connection to a backend ERP system or something, but the frontends are mostly running in shared hosts in the big datacenters in Amsterdam.

The ModRegistry sounds very interesting, but I hope it will be implemented in such a way that Shared Hosting websites can also benefit from this functionality.

modRegistry should work in any server configuration.