Answered MODx Revo White Screen Caching Issue on load balanced servers

Hi all,

We've recently developed a large project in MODx Revo and we've come across an issue which is proving really difficult to solve but quite critical, so I wondered if anyone has had a similar experience and knows how to or where we could start to look to solve it.

The issue is that occasionally, the site is returning just a blank white page, with no errors. This is on the front end only, but affects all pages. We can access the manager. It appears the cache is corrupt in some way, as clearing it will fix the problem every time. There are no errors logged either, which is making it very difficult to debug.

We've disabled the cache in system settings, and the problem stops, so we've fairly confident the issue is somewhere here. I've looked around the forums for some answers and a lexicon caching error seems to closest fit, however we're not getting that error. We have a load-balanced set up and wonder if this may affect it? I realise that is quite a broad area but if anyone has experienced similar issues it may help. The issue does not appear on our non load-balanced development server. Currently we have a cron set up to check if the page is down and clear the cache if so, but obviously it'd be ideal to be able to solve this.

The load balancer is sharing the files in real-time.

Server is a load balanced set up (2 servers):

• The site is running on MODx Revolution 2.2.15-pl traditional.
• PHP Version: 5.4.27
• Mysql Version Version: 5.1.73
• Server experiences high loads (there is a UK National ad campaign running with the site so there are large spikes), however the issue has happened with minimal load too
• Installed MODX Add-ons: getResourceField, getResources, If, Login, MIGx, minifyx (used in development only), phpThumbOf, revoSSL, siblingnav, TinyMCE, Wayfinder

As you can see we haven't upgraded from 2.2.15 yet as there were phpthumb issues in 2.3.0 when we tried. I see these are fixed in 2.3.1 and it's in our next development stage to upgrade, so if upgrading is a potential fix that'd be good to know too.

Thanks in advance! Any other info just let me know.





[ed. note: toasteddigital last edited this post 12 years, 1 month ago.]

I've experienced the same issue with one site. As you say clearing the cache (which I guess is getting corrupted) solves the issue, but I've never been able to track down what's causing it.

You are not alone.

2.2.15 is probably not the issue.

Check your error logs to see if there is a consistent item that is stalling the site. If random, it may be that the error displaying is simply the last task occurring when the site ran out of resources or used up all it's connections to the database.
Confirm whether the site is down for just you, or for everyone else on the internet (from other IPs).
Make sure your add-ons are up to date.
Make sure your site has not been hacked. (Is there something else running that is using up the resources?) (Look for users that you do not recognize. Look in your assets directory for suspicious files you don't recognize.)

I've had this problem occur on two sites hosted at goDaddy.
On one, the site had been hacked. Cleaning out the nefarious files and updating to 2.3.1 seems to have resolved the issue.
On another, the site was on older generation servers. Updating to 2.3.1 and migrating to newer cPanel hosting at goDaddy seemed to have fix the blank screen issue

I have sites at too other hosting companies that have never experienced the blank screen issue.

Thanks for the feedback.

The servers are a dedicated set up (we use two servers load balanced) and we've confirmed it happens when the server resources are not being pushed at all, so I don't think it's running out of memory. It happened again in the middle of the night with almost no load at all. While all evening before it didn't white screen with 350+ concurrent users. Like I mentioned there are no errors logged either unfortunately. The site hasn't been hacked, and the problem does go away when we disable the cache.

To help debug, with the clear cache script we wrote, we've now logged the output of the home page white screen page cache file when it happens (/resource/web/resources/1.cache.php) - interestingly this cache file is completely fine. It's identical to the cache file when the site is working and contains no errors, but the screen still renders blank. I should point out that when this happens it isn't just the home page, it's the entire front end of the website which goes blank. Given the resource cache file looks fine I'm wondering if it's a more globally used one such as database or even settings which is causing the issue.

Have you tried grabbing the system setting/lexicon/context etc cache when this happens to see if anything's amiss with them?

I should note that the one site where I had this happening on was also on a load balanced "cloud" hosting platform. I since switched to non-load balanced servers and haven't had this issue on another site.

If you have the time when it happened, the server error log might have something useful at or before that time.

I went backwards and forwards with a high level tech guy at the host over other persistent caching errors in MODX. What we ended up thinking was happening was that on a rare occasion, two or more servers try to write the same cache file at the same time which corrupts it. It should be locked by one but occasionally wasn't for some reason. They were developing their load-balanced cloud platform at the time so were interested in looking into it. I don't know if it was resolved in the end because they moved me onto non-load balanced servers, with a beefier deal at the same price.

Thanks guys.

I've actually been working with a high level tech at the host today too and we came to the exact same conclusion, Dan. We have a share set up instead of rsync, so what we've now tried is creating a symlink between the cache folders on each server and a new folder outside of the share to keep the caches completely independent of each other, and so far we haven't had the issue come up.

I'll keep you posted as to whether this resolves it.

That's good to know. I don't know if this is something that could possibly be resolved with alterations to the MODX core, or solely an issue with how hosts implement load-balancing, but it's maybe something for the dev team to consider.

Unfortunately we got the same issue early this morning - so no resolution yet. Just happened the one time, low load on the site (20-30 users). It was running completely fine last night with a higher load of around 700-800 concurrent users.

I'd be really interested to know of successful MODx load balanced set ups which don't experience this issue so I can pass it on to the server techs, anyone? This one looked interesting: http://www.modx360.com/assets/posts/12/load_balancing_modx.pdf

Could using memcached potentially fix this issue?