If you have dozens or hundreds of users hitting the site simultaneously with a cold cache, you
will be putting a lot of load on the database. That load is killing the database server(s), as shown by the MySQL Server has gone away errors. So saying there is no hardware problem is not entirely correct as you evidently don't have sufficient resources during those peaks, but I agree the solution is not to throw more hardware at it - lowering the stress on the server is the way to go.
If you are open for it, I would advise to get in touch with the MODX team. They often work with large companies and have quite some experience dealing with the complex sites that need a bit more than what the core handles out of the box. You can work with them to evaluate the set up and to find out what would really help. Arguably fixing the issues you have would be a lot more cost effective than moving to a different platform.
There is no "one size fits all" solution for sites with major traffic, but there are a bunch of tricks that can be used and when used together they can resolve the problems you are seeing.
- Prevent the cache from getting cleared when editing pages which have not yet been published - if your publishing workflow includes editing stuff on the master site directly, this can be a quick win. I believe there is a plugin written by Bob Ray available that will do this for you but don't recall the name. Bob might pitch in with the name later; he's very active on these forums.
- If you're seeing heavy I/O, switching to a different cache handler (memcache(d) / redis) instead of the default file one could help with that, however be careful to make sure there is a lot of I/O causing bottlenecks before switching this as it could bring new problems of its own when not done right. It mostly sounds like a database load issue to me, but wanted to throw this out there as it could help.
- Use getCache
with a custom cache partition to keep the fixed bits and pieces that can be heavy to load/parse (such as menu structures, external feeds) separate from the resource cache. While people often say MODX "clears the entire cache" after a resource save, that is not technically true as it only clears the resource cache while other partitions remain. Taking advantage of that can be a huge win when done right.
- Use microcache to instruct browsers to cache the page output. This can be a big boost for the user while also lowering the load on the server, but must be done right again. See
http://www.sepiariver.ca/blog/modx-web/modx-quick-tip-microcache and
https://github.com/opengeek/microcache/wiki
- Find a way (read: custom development) to prime the cache on your slaves. This is super-specific to your environment and publishing process, but if you would have a small master server that is only used by the editors for generating content, you could create a button in the manager they can click that will automatically generate the cache on the master server, and then push that cache to the slaves along with the updated database. That way you would avoid having a cold cache to begin with.
I don't know too much about how other CMSs handle their caches and cache clears, but I am pretty sure other systems will also encounter application specific issues when scaling.
☆ A M B ☆