Proposal: Nuke site stats from the manager entirely

The built-in stats from MODx that were inherited from Eto can rapidly lead to performance degradation and bloated databases in even moderately visited sites. I definitely don’t think this is a "best practice" way of handling things and personally goes against my vision for the project. MODx should not be about mediocrity.

In working on replacing them with SlimStats, which works great now, I realized that SlimStats doesn’t do a good job of relaying basic page view information. Hits, yes, but PageViews, no. Slimstats is also prone to the same database bloat of the default stats. I’m not quite the fan of it I was originally.

If I were to recommend a solution to someone at this point, it would be to implement server log stats (AW Stats for instance), Google Analytics, Measure Map (when they open it back up), BB Clone or Mint.

WIth that said, I’d like to see our Slimstats work moved to a module that is NOT installed by default with the core distribution. I’d really like to see it get smaller, not larger. I’d like to likewise see a BB Clone module made.

We could easily release plugins and snippets for SlimStats, Google Analytics and Mint. A Google Analytics plugin is already in the repository and I’m about to update it again to prevent logging when a manager session is established. Jeff has a Mint plugin pretty much ready to go as well. We need to release these as snippets also so that we can give devs the flexibility of using them in templates as well as pervasively as plugins.

This would ultimately make the database smaller, the distribution smaller, and keep us focused on providing a great framework and not mediocre stats.

Thoughts?

I’ve always felt that logging and statistics should be done by some third-party application or another. It is a whole ’nother ball game of itself, and there is no need to waste time and energy trying to reinvent the wheel. Not to mention that logging and caching is not actually what MODx is all about. There should be a framework for plugging in whatever logging and caching utilities might be appropriate for a given site. Even none at all if the designer so wishes. Besides that, the current loggin and caching systems both leave much to be desired, not to mention being a vector for hacking.

I concur completely. I think just about everyone can agree that functionality like this should not be hard coded into the functionality of MODx and the manager. After all, that’s what modules and plugins are for. The philosophy should be that the only functionality that should be hard coded is functionality that is absolutely necessary to MODx itself. Everything else should be able to be implemented as a plugin or module...site stats included.

On that note, if we do this then its safe to say that we can do away with the "Log Visits" system configuration parameter. After all if you want to disable logging you can simply disable the plugin or module. The main question though is do we need a system event for logging? If so, how would such a thing be implemented? As it stands, scripts can be written to take advantage of the "Log visits" parameter for each page. A typical script would check to see if a manager session exists, whether the page is being loaded on the frontend, if the page is a standard HTML page, and if the page itself is set to allow logging. I’m wondering if we need to create a system event and/or additional methods to the documentObject class to aid in script production and cut down on repetition.

Jeff

Sounds great to me. We need to stay focused on doing what MODx does best and not try to be a jack-of-all trades right out of the box. There is a lot of stiff competition in the statistics market and we don’t have the time or resources to try to stay compeitive.

I tend to lean away from including any third-party stuff in the core distribution (aside from some of the necessities like an RTE and maybe a couple of others). Once you do that you basically have to support it because it’s "part of MODx." I’d rather focus on making it simple for people to write modules and connectors to third-party stuff so that we don’t get bogged down supporting a lot of other peoples stuff.

I would love to see a stats module written specifically for MODx down the road. I think it could be very helpful with things like, tag clouds for most visited pages and other purposes. But it should be a module because most people won’t want that and there are other more pressing priorities at this point.

Ditto ditto ditto...

I’ve had my built in stats turned off since the "PhaseCMS" days! I’ve always thought they were under-informative and over-cumbersome. Bad ROI. I’ve said it before and am happy to reiterate that I’d be happy to see them gone by default. If someone wanted to put a stats package in as an optional plugin/module - super. But I’m good letting AWStats do the analysis for me at the server log level.

I think that for "upgrade" installations we need to at least have some sort of method to give the users a copy of their old stats since there will no-longer be a method to keep stats in MODx.

Ryan - Nice idea with the "don’t log manager views". If you’re updating the Google analytics one, can you please double check that it loads Google’s .js from https:// when the site is on a secure server?

I can get on board the "leave stats out" train, even though I’m likely to have some form of stats in my custom default install, but only assuming the following:

Essential:
People upgrading from 0.9.2.1 are given the option to keep the stats module in. An upgrade path is essential.

Nice-To-Have:
Existing stats modularised for people who want to install 0.9.5 fresh and keep their data. Although I’d instantly deprecate it, to show that it’s not being continued.

The Campaign Starts Here To Keep A Simple ’Log Visits’ (’Count Visits’):
I’d really really like to see PageViews field retained/added and a very basic page view incrementer included within MODx (which can be switched off). Page views is probably the only stat field that is useful to have accessible to MODx - every thing else is best viewed and analysed in another application on the server or on the internet. There’s a snippet released that uses the page view count as a popularity check. There is clear demand for this. If there was a common field, other modules could write to it (possibly hourly or any other interval) and there would be some interoperability on this simple thing. Heck, you could probably have something query Google Analytics for numbers to save into the field.

Surely updating 1 field doesn’t have any server bloat, and I’d rather avoid the module overhead for a pageview increment as it’s such a simple little thing.

Further, Google Analytics is no use for RSS feeds. Having PageViews on that document would therefore be useful.

Quote from: PaulGregory at Aug 02, 2006, 11:34 AM

I can get on board the "leave stats out" train, even though I’m likely to have some form of stats in my custom default install, but only assuming the following:

Essential:
People upgrading from 0.9.2.1 are given the option to keep the stats module in. An upgrade path is essential.

Progress demands that we remove this completely; there is no option to continue using the legacy stats when upgrading. The tables will be left alone, but will be ignored from 0.9.5 on without an add-on.

Quote from: PaulGregory at Aug 02, 2006, 11:34 AM

Nice-To-Have:
Existing stats modularised for people who want to install 0.9.5 fresh and keep their data. Although I’d instantly deprecate it, to show that it’s not being continued.

Anyone want to work on a completely isolated add-on that could serve as an upgrade path to 0.9.5 if they must have the existing stats continue to be tracked?

Quote from: PaulGregory at Aug 02, 2006, 11:34 AM

The Campaign Starts Here To Keep A Simple ’Log Visits’ (’Count Visits’):
I’d really really like to see PageViews field retained/added and a very basic page view incrementer included within MODx (which can be switched off). Page views is probably the only stat field that is useful to have accessible to MODx - every thing else is best viewed and analysed in another application on the server or on the internet. There’s a snippet released that uses the page view count as a popularity check. There is clear demand for this. If there was a common field, other modules could write to it (possibly hourly or any other interval) and there would be some interoperability on this simple thing. Heck, you could probably have something query Google Analytics for numbers to save into the field.

Surely updating 1 field doesn’t have any server bloat, and I’d rather avoid the module overhead for a pageview increment as it’s such a simple little thing.

Great idea and yes, we could have a single table with document id, a field for tracking total hits, and an auto-update timestamp much less overhead than the current stats, and no bloat at all. Scalable and it could completely exist as an add-on.

BTW, I think the existing configuration options should stay, to support a modularized legacy stats add-on, and to be used by any other stats add-on packages people may choose to integrate via a plugin or otherwise.

Quote from: PaulGregory at Aug 02, 2006, 11:34 AM

I’d really really like to see PageViews field retained/added and a very basic page view incrementer included within MODx

I can understand where you’re coming from and I agree with much of what you’ve said. However, my hesitation with this is the difference in user perception. If we do inculde page hits only I fear that we’ll go from "MODx doesn’t have built-in stats" to "MODx’s built-in stats are pathetic" and we find ourselves with lots of requests to improve stats in MODx.

Now a lot of this is just semantics, for instance making it a module and calling it PageViewCounter instead of calling it statistics might do the trick. But I think we need to consider how this will play out in the long run to the end user. Personally I think I lean toward let’s do it right or not do it at all.

I’d be very happy with a basic page view counter as well. It’s not stats, as Adam mentioned. It would make it easy to do a "most popular pages" or "recently updated" snippet, which is honestly the only reason I’d like to keep the existing stuff.

Progress sometimes involves pain, and it’s time to hurt a bit. I’m certainly not opposed to a module to do the existing stats, which should actually work much better now that the header is not being butchered by the parser any longer. Anyone want to make a stab at modularizing it?

I’d also like to see an option to drop the existing stats tables in the installer, unchecked by default.