MODX Staff
Again, the point everyone is missing is that Resources are not exactly equal to the data it is you are presenting in a 1 to 1 relationship. It is a distinct view of data that is alike. This is how dynamic PHP web sites are built to scale. Contexts ar great for dividing up the load, but it still isn’t going to help you if you have 2000 magazine articles or 10000 recipes you want to present. Do you create 2000 or 10,000 Resources, one for each article or recipe, or whatever? No. You create various views of those pieces of data. Those views are Resources. The rest needs to be optimized for the job at hand, including scalability requirements.
By default there is just one context (web), right? What is the recommended number of contexts for a large site, and how does one go about creating them?
That is exactly what disabling the cache_context setting does.
I think that modx’s performance would be greatly improved if there were just one cache file per document.
When a document is saved, just one small cache file (containing all its basic + TV data) would have to be re-built.
Template variables provide a very convenient way to extend built-in document fields without having to rely on writing custom SQL and PHP. I think that most people who use modx do so because of the convenience that TVs provide. If we needed to model data using custom tables, most of that appeal would be lost.
I think that by tweaking how data is cached internally, we could still store hundreds of thousands of documents (each with tens of TVs) effortlessly. Expensive queries would only have to be run once, after which the relevant data would be cached. As long as cache files can be cleared on a granular (as opposed to global) level, and only when their underlying data has changed, then modx should turn out to be a lot faster, and more scalable.
Again, the point everyone is missing is that Resources are not exactly equal to the data it is you are presenting in a 1 to 1 relationship. It is a distinct view of data that is alike.Thats the whole point to use resources as the data points. Yes, i will create a folder and have a ditto or getresources snippet on it to display last 100 blog posts, but those 100 posts need to live somewhere, dont they?
Do you create 2000 or 10,000 Resources, one for each article or recipe, or whatever? No. You create various views of those pieces of data. Those views are Resources.- Is this how you manage your MODx site? All articles in a completely separate DB table(s) (entered directly into MySQL or through custom admin pages) and MODx resources used only for distinct views? If yes, i think i have a major misconception about MODx in general (and probably not only me).
That is exactly what disabling the cache_context setting does.the setting is off and the site is at the standstill - made it unusable.
Is this thing on?
MODX Staff
My apologies; did not mean for that to come across like it did. I am a bit frustrated over the topic, which I’ve been discussing for years in these forums, in regards to both Evolution and Revolution code. But no excuse to be an ass.
That is exactly what disabling the cache_context setting does.
Is this thing on?
I guess I’m surprised as well; I’ve discussed this exact topic and held the same stance for years. If you have 1000’s of anything, creating MODx Resources is simply not the most efficient way to handle presentation of that data. And if you want to reuse that data in other views, think of how complicated and memory intensive it will be to load a whole Resource and worry about applying different formatting, etc. I’ve tried to make this a key point when talking about Resource limits and caching and scalability, but I’ve not yet been able to make it as important an issue as it should be. There is a difference between raw content (what I call data), and content that is prepared for presentation, be it in HTML, XML, JSON, PDF, or any other format.
I’ve been using MODx for years already and its very surprising to hear things like:
Again, the point everyone is missing is that Resources are not exactly equal to the data it is you are presenting in a 1 to 1 relationship. It is a distinct view of data that is alike.Thats the whole point to use resources as the data points. Yes, i will create a folder and have a ditto or getresources snippet on it to display last 100 blog posts, but those 100 posts need to live somewhere, dont they?
Do you create 2000 or 10,000 Resources, one for each article or recipe, or whatever? No. You create various views of those pieces of data. Those views are Resources.- Is this how you manage your MODx site? All articles in a completely separate DB table(s) (entered directly into MySQL or through custom admin pages) and MODx resources used only for distinct views? If yes, i think i have a major misconception about MODx in general (and probably not only me).
Let’s explore how we can address this; but I’m afraid this working better is going to require some optimization on your side and on the part of the core. Being able to query the entire tree structure of a Context in a single query will improve this situation immensely, but that simply is not possible without some additional data structures that create their own overhead. These are not challenges we take lightly either; regardless of my insistance on separating raw content or data from your presentation layer, we absolutely want MODx to scale as much as possible, even if you create 20,000 Resources in a single Context.
the setting is off and the site is at the standstill - made it unusable.
I think that is a good thing. If you had 20,000 Resources in Evolution, these should have been an issue, even before now.
This thread is raising more and more questions for me.
My apologies; did not mean for that to come across like it did. I am a bit frustrated over the topic, which I’ve been discussing for years in these forums, in regards to both Evolution and Revolution code. But no excuse to be an ass. Embarrassedall good here, i dont think you were out of the line at all, so no apologies needed )) I think my sarcasm was more inappropriate.
If the conception is that MODx is going to be able to handle, out of the box, without any customization or development, all site requirements with regards to content quantity and scalability when using Resources to represent all kinds of data, not just a few distinct web views, then I would love to destroy that right here and now.i clearly understand this - its the FAQ page (http://modxcms.com/learn/faq.html) that might confuse other users (especially these parts: Do I have to know how to code in order to use MODx? and How big of a site can I build with MODx?
I suggest we start by filing a bug report covering the situation so we can address this directly, and not in casual conversation in the forums. Is this something you are willing to do bakalek?We emailed RT access to our development site and can file a bug report as well
I think that is a good thing. If you had 20,000 Resources in Evolution, these should have been an issue, even before now.Believe it or not, we are way past the 20000 document ID mark (which probably resolves in like 15,000 to 17,000 resources)
i clearly understand this - its the FAQ page (http://modxcms.com/learn/faq.html) that might confuse other users (especially these parts: Do I have to know how to code in order to use MODx? and How big of a site can I build with MODx?+1 to update the FAQ section. most people think they won’t run into problems with revolution (out of the box) having thousands of resources
For the MODx Evolution/0.9.6.x branch of software, and depending on your server configuration, the caching system may run into size and performance limitations around 5,000 documents, but this is only a rough guideline. The Revolution branch has no limits.
I’m assuming this means build some custom tables for the articles or recipes? If so the current SimpleSearch functionality won’t include this custom data. Or am I also missing the point?
... if you have 2000 magazine articles or 10000 recipes you want to present. Do you create 2000 or 10,000 Resources, one for each article or recipe, or whatever? No. You create various views of those pieces of data.
Actually, it kind of will:
I’m assuming this means build some custom tables for the articles or recipes? If so the current SimpleSearch functionality won’t include this custom data. Or am I also missing the point?
[[!SimpleSearch?
&customPackages=`QuipComment:body:quip:{core_path}components/quip/model/:QuipComment.resource = modResource.id`
]]
/* In the format: 0: class name, 1: field name(s) (csl), 2: package name, 3: package path, 4: criteria */