automatic cache "clean and then auto-rebuild" plugin?

125 Posts

ad2003 Reply #1, 13 years, 4 months ago

Hi there,

everytime when I change something in my ressources, I want the cache to be cleared, to see the result. But then, somebody needs to call the webpage, to let modx generate content the first time, which can take up to 10-20 seconds.

Is there a plugin (I couldn't find one) which automatically rebuild the cache after clearing it - like automatically "opens" the website in the background, to refill the cache?

I found this here, which is the right direction:

http://www.cloudtec.ch/blog/web/modx-cache-rebuild-refresh.html

but it would be great if it could be integrated into modx, without using any cronjob stuff...

Thank you!

AD

This question has been answered by multiple community members. See the first response.

discuss.answer

534 Posts

exside Reply #2, 13 years, 4 months ago

Basically I think if your website needs 10-20 seconds to regenerate the cache, there could be some other problems (had a similar problem with my last project) with caching (does everything change or can it maybe be cached with getCache and cleared occasionally?)...so check out:

http://devries.jp/blog/2013/03/23/cash-in-big-with-modx-cacheing/

Then there are other extras doing something similar as the one you linked to:

http://bobsguides.com/refreshcache-tutorial.html
https://github.com/myfreeweb/modx-uncacher

discuss.answer

☆ A M B ☆
318 Posts

freejung Reply #3, 13 years, 4 months ago

I use a bit of JavaScript to request all of the pages of the site and regenerate the cache. I could post it if you prefer that solution, although the extras mentioned should do it for you.

However, I agree that it's a problem that your resources take so long to render uncached. Do you know what's taking so long? The most likely culprits are getResources, Wayfinder and phpthumbof, in my experience.

If you're using phpthumbof, you might want to disable the plugin that clears the phpthumof cache every time the cache is cleared. That way cached images will survive the cache clear (assuming that's acceptable).

If Wayfinder is the problem, definitely use getCache to separate the menu from the rest of the cache, and only clear it when you need to.

If it's getResources, post your code and we can look at how to optimize the call. You can sometimes get a lot of performance improvements by not using template variables, or by restricting it to only the template variables you absolutely need. It's also often possible to improve speed a lot by removing conditional code and output modifiers from your templates, and using a custom snippet to perform the logic instead.

discuss.answer

24,544 Posts

BobRay Reply #4, 13 years, 4 months ago

There is also CacheClear, which deletes all files in the core/cache directory http://bobsguides.com/cache-clear-tutorial.html.

RefreshCache will rebuild the cache, but OpenGeek has posted code that does it too.

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting

125 Posts

ad2003 Reply #5, 13 years, 4 months ago

Hi people,

thanks for your nice suggestions and help.

It's a one page design, so all my content is on only one page, therefore a bit longer processing than a seperated page is maybe normal.

But I will check the phpthumb cache, and of course the getCache stuff. Thanks for that.

here is my Template:

[[$pages.header]]
[[$pages.toparea]]
[[$pages.nav]]
<div class="wrapper">
<div id="main" class="container">
<div id="content"> <!-- Content start --> 
[[$jumbo]]
[[getResources? &parents=`0`  &limit=`10` &includeContent=`1`  &includeTVs=`1` &processTVs=`1`  &tpl=`pages.block`  &sortdir=`ASC` &sortby=`{"menuindex":"ASC"}`]] 
 

[[$pages.footer]]

Thank you! AD

125 Posts

ad2003 Reply #6, 13 years, 4 months ago

Quote from: esnyder at May 09, 2013, 11:19 AM

If you're using phpthumbof, you might want to disable the plugin that clears the phpthumof cache every time the cache is cleared. That way cached images will survive the cache clear (assuming that's acceptable).

how can I set this up?
Thank you!

discuss.answer

☆ A M B ☆
318 Posts

freejung Reply #7, 13 years, 4 months ago

how can I set this up?

Go to Elements, and look under Plugins for the phpThumbOfCacheManager plugin. Click on it, check "plugin disabled" and save.

The disadvantage of this is that if you change one of the images without changing its name, the thumbnail for that image won't update. You can get around that by simply making sure to change the filename of the image.

The getResources call is going to be very inefficient because it includes content and template variables and processes all template variables.

You could probably improve that quite a bit by using the parameters includeTVList and processTVList to limit it to just the TVs you need - see here: http://rtfm.modx.com/display/ADDON/getResources#getResources-OtherProperties

Could you post the content of the chunks "pages.nav," "jumbo" and "pages.block"? There might be other things in there we could improve on.

Definitely look into using getCache to create different cache partitions and only clear the ones you need. See this blog: http://modx.com/blog/2012/10/29/optimization-with-getcache-and-custom-cache-partitions/ [ed. note: esnyder last edited this post 13 years, 4 months ago.]

125 Posts

ad2003 Reply #8, 13 years, 4 months ago

Hi FreeJung,

thank you for your reply!

I am pretty sure, it's not the best code so far. I was using a lot of chunks, to seperate content, to make it easier to edit single passages of the website later on. I was working with wordpress before, used php "include" at several places, to make the editing easier, which worked out...
In the rftm docs, I read, that you should never use chunks inside of chunks.

Do not include other Chunks in Chunks! Things can get very confusing if you start going down that rabbit-hole.

Is it for performance reasons, or just because it could get confusing?

Eg. jumbo - this stuff could absolutely just be inside the
header area, so I would have one chunk less to be rendered.
So here's the code I am using inside pages.nav of course with typical HTML stuff surrounded...:-)

[[Wayfinder?
  &startId=`0`
  &outerTpl=`pages.nav.outer`
  &rowTpl=`pages.nav.row`
  &firstClass=``
  &lastClass =`` ]]

in jumbo it's for the HeroSlogan I want to use on my site:

 <a href="#home" class="scroll hidden-phone"><div  id="subline" class="jumo"><div class="container">
  <p>[[$mySlogan]]</p>  </div></div> </a>

the pages.block contains the html of every page in my onepage designed website:

 <div  id="[[+longtitle]]"></div>
 <div class="content-holder">  
 <div class="container">
 [[+tv.myImg:notempty=`<div class="text-center"><a href="#[[+longtitle]]" class="scroll" ><img src="[[+tv.myImg:phpthumbof=`w=140&h=140& zc=1`]]" title="[[+description]]" alt="[[+longtitle]]" /></div></a> `]] 
[[+id:isnot=`1`:then=`<h2 class="text-center">[[+pagetitle]]</h2>`:else=``]]
   [[!+modx.user.id:is=`1`:then=`
<a class="login" target="_blank" href="/manager/?a=30&id=[[+id]]" title="edit">Edit Page</a>`]]
         [[+content]] </div>
 [[$connectbutton? &introtext=`[[+introtext]]`]] 
   </div>
[[+id:isnot=`2`:then=`<hr class="soften">`:else=``]]

Very thankful for optimazation!

AD

☆ A M B ☆
318 Posts

freejung Reply #9, 13 years, 4 months ago

Hmmm... well, generally speaking, I wouldn't advise designing your site this way. This page is going to be a bit slow uncached no matter what we do, because it has to retrieve and process the content of all those resources. I would consider using MIGX to store your "pages" (which are actually page elements) as MIGX grid items, using an Image TV for the image and a RichText TV for the content. That would be way faster, especially if you use MIGXdb.

However, for these purposes we'll assume you have compelling reasons to do it this way, and try to make that work.

I bet the main thing causing the slowdown is the phpthumbof call inside the getResources template. Disabling the phpThumbOfCacheManager should help a lot with that.

Regarding chunk nesting - I think you'll get a performance hit if you do it inside a template chunk, such as a getResources template. Otherwise I don't think the slowdown will be noticeable, it's just a best practice to make your site editable. I will admit I actually do it all the time. I'm trying to cut down ;-) I've actually even passed chunks into other chunks as parameters before - probably not the best idea, but it seemed to make sense at the time. I didn't notice any performance problems with that.

I would advise to use getCache to separate out the Wayfinder call into its own cache partition. Obviously you'll have to install getCache first. Then replace your Wayfinder call with this:

[[getCache?
  &element = `Wayfinder`
  &cacheKey = `menu_cache`
  &cacheExpires = `0`
  &startId=`0`
  &outerTpl=`pages.nav.outer`
  &rowTpl=`pages.nav.row`
  &firstClass=``
  &lastClass =`` ]]

Then you'll want to create a menu item to clear the menu_cache partition. Just go to System->Actions, right-click on (for example) Site on the right side and choose Place Action Here. Name it whatever you like, choose "no action" for the action and don't put anything in parameters, icon or permissions. Use the following for the handler:

var topic = '/getcache/cache/partition/refresh/menu_cache/';
if (this.console == null || this.console == undefined) {
    this.console = MODx.load({
       xtype: 'modx-console'
       ,register: 'mgr'
       ,topic: topic
       ,show_filename: 0
    });
} else {
    this.console.setRegister('mgr', topic);
}
this.console.show(Ext.getBody());

MODx.Ajax.request({
    url: MODx.config.assets_url+'components/getcache/connector.php'
    ,params: { action: 'cache/partition/refresh', partitions: 'menu_cache', register: 'mgr' , topic: topic }
    ,listeners: {
        'success':{fn:function() {
            this.console.fireEvent('complete');
        },scope:this}
    }
});
return false;

Now selecting this menu item will clear only the cache for the Wayfinder menu, and clearing the normal cache will not refresh the Wayfinder menu. So only clear the menu cache when you make changes that modify the menu. This alone should help a lot.

Regarding the getResources call - yeah, that's going to be very slow, it has several conditionals, a phpthumbof call and a nested chunk.

Get rid of the nested chunk - just put whatever code is in connectbutton directly in the template chunk.

I'm pretty sure you can get away with turning processTVs off.

Limit getResources to only including the one TV you need by adding this parameter:

&includeTVList = `myImg`

It would be nice if we could get rid of the other conditionals too, but I don't think that's the major source of slowness.

125 Posts

ad2003 Reply #10, 13 years, 4 months ago

Dear FreeJung,

thank you very much for the detailed infos.
I have set the getCache plugin up and tested it. The impact is not that huge - about one second faster loading. So there is something better.

But it's still around 7-8sec for the first load, after cache completely cleaned.
I changed the getResources with "&includeTVList=`myImg`" like this:

[[getResources? &parents=`0` &includeTVList=`myImg` &limit=`10` &includeContent=`1` &includeTVs=`1` &processTVs=`0` &tpl=`pages.block` &sortdir=`ASC` &sortby=`{"menuindex":"ASC"}`]]

What would be the best way to get "data" from TV items generally?
For every Resource page I have one field from a TV myImg where I put in an image for representing the main topic of that resource.

E.g: I have an overview on the top of the page, what my page is about - so from every resource I want to get the image and the introtext and show it.

[[getResources? &parents=`0` &limit=`10` &includeContent=`1` &resources=`-1,-2,-3,-7,-18,-9` &includeTVs=`1` &processTVs=`1` &tpl=`my.list` &sortdir=`ASC` &sortby=`{"menuindex":"ASC"}`]]

code of my.list:

<div class="text-center">
<a class="scroll" href="#[[+longtitle]]">
<img src="[[+tv.myImg:phpthumbof=`w=120&h=120&zc=1&q=100 `]]" title="[[+description]]" alt="[[+longtitle]]" />
</a>
<h4 >[[+pagetitle]]</h4>
<p class="text-left">[[+description]]
[[$my.liste? &longtitle=`[[+longtitle]]`]] </p>
</div>
[[+idx:mod=`2`:eq=`0`:then=`<div class="clearfix"></div>`:else=``]]

Than I have a small container on the bottom of the page, where also the small images should show up - which I also call with a "getResource" snippet and a template - including a phpthumbof instance the same way like the code above.

I guess a problem could be, that I am calling several getResources instances with different templates, which will slow down the page loading. What would be the best practice then? Can I kind of "recycle" one getResources call?

Thank you very much!

Answered automatic cache "clean and then auto-rebuild" plugin?