Snippet Cache Changes

I am not really happy with the entire caching mechanism of snippets.

First, it is hideously inefficient. Not only due to memory usage (why on earth do we load EVERY snippet into memory when we may only use five/two/one/none on the page??). Then the base64 encoding-- not only does it increase the size by about 133% (which we the load into memory on every request!), but it then requires a decode AND an eval (also very bad!) to run.

I would like to change this so that each snippet is cached individually in a PHP file on the filesystem. These would be automatically loaded on demand thereby decreasing memory usage and eliminating the usage of base64 and eval() entirely.

The interface in the admin and the usage of the snippets would not be changed, only the internals on how they are handled.

What does everyone else think?

(Also, on re-reading: I'm actually volunteering to do this, not asking that it be done. )

Hi infoclipper,

This sounds very good.

I was thinking about the use of base64 and was planning to use mysql_escape_string() instead. This would have reduce the size of the cache file immensely.

Now that you have raised the idea of using separate "cache pages" to store snippet data. How about doing the same for chunks, etc?

I would say go right ahead and implement it for the snippets at first then do a speed test comparison to see how much faster the new "cache page" system will perform under loads of 10, 50, 100 snippets - (should be able to just copy one or more snippet multiple times on a page).

Can't wait to hear how well this new caching method compares the the old one

Am I understanding that the idea is to store snippets in the filesystem instead of the database? Why not have the first pass of the engine collect the snippets being used, then only gather those from the database, and store them into here doc variables? I usually use here docs when I want to "echo" a lot of HTML layout tags; you don't have to worry about escaping stuff.


http://www.jvmultimedia.com/home/articles.php?articleId=8

Am I understanding that the idea is to store snippets in the filesystem instead of the database? Why not have the first pass of the engine collect the snippets being used, then only gather those from the database, and store them into here doc variables? I usually use here docs when I want to "echo" a lot of HTML layout tags; you don't have to worry about escaping stuff.


http://www.jvmultimedia.com/home/articles.php?articleId=8

Currently, the snippets aren't stored in the database except to create a cache file on the filesystem. Unfortunately the way the caching has been implemented you would probably be better off not having the cache at all.

Its that shortcoming that I'm concerned about.

There is a problem with just switching to doc variables, though: snippets can contain new PHP functions. Putting those in a doc variable throws a parse error on most, if not all, versions of PHP.

(Which is why at the moment I am only looking at the caching inefficiencies: it will give us the most bang while still allowing us to be 100% compatible with any snippet for Etomite. I do agree there are probably improvements that could be made to the entire system-- from organization to coding to the way they function in general...)

I don't know if there are any opensource caching systems that can be implement, but it might be useful to have a somewhat open caching API so it's possible to create caching plugins for different cache systems and/or extend the default caching.

ie. a cacheing plugin for Turck MMCache / eAccelorator which caches everything in memory (mmcache_ api calls)

ie. a plugin which maches use of smarty's cache api(smarty_core_read_cache_file, smarty_core_write_cache_file) [sidenote: i like smarty, and the caching is AFAIK very good, however I don't know if we can use it for our purpose)