MODx's problem with 10k+ resources

Hi.

I am facing a problem, which apparently is related with URI saving on update.
My case is that the (so called) "Ultimate Parent" has thousands of children, and the children has hundreds of grandchildren.

Imagine what would be happened if I save that Ultimate Parent.
The refreshURIs() method tries to update the entire descendants recursively.
Server heats up:
-CPU 100%
-Memory 100%

This is a nightmare.

I am pretty sure, we should not use this method.
If MODX's requires URI for something, then it should be done programmatically, not by expecting a static data from DB.
The problem with the later is that MODX needs to refresh every URI on the nested resources all the way down.

This is a critical problem for MODX for a large site.

I believe, this must be resolved to make MODX reliable on handling a large site.

Oh, btw...
My case is not about 10k+ resources.
Not 20k+ either.
It's 30k+.
Please, feel my pain.

Any suggestion?

Problem also occurs if I delete the Ultimate Parent.
MODX tries to mark all the descendants as deleted as well.
Then the process will be timed out.

1) Try to disable caching resource maps
2) Stop using UltimateParent (in most cases it can be done)
3) Do not use the file cache

I do not know about you, but I was in the resource tree holds more than 500k + resources and following these recommendations, it works very fast.

I don't think goldsky was referring to use of the actual UltimateParent snippet, he is referring to the top-level resource (ultimate parent resource) in the Tree.

Quote from: sottwell at Mar 12, 2014, 12:59 AM

I don't think goldsky was referring to use of the actual UltimateParent snippet, he is referring to the top-level resource (ultimate parent resource) in the Tree.

In this case, use the checkbox "Freeze URL". This should help to solve the problem with a change in the parent document...

@sottwell,
thanks for the clarification. Yes, I was referring to the top resources.

@Agel_Nash,
Quote from: Agel_Nash at Mar 12, 2014, 01:01 AM

In this case, use the checkbox "Freeze URL". This should help to solve the problem with a change in the parent document...

That also crossed in my mind.
But from user point of view, this is not practical.
For a big website, the manager user will not only 1 person.
There'll be employees.
Those employees can be replaced again at anytime.

Quote from: Agel_Nash at Mar 12, 2014, 12:56 AM

1) Try to disable caching resource maps
3) Do not use the file cache

1. which key?
3. what's the connection with the issue? Because at this point, we're thinking to cache everything for the speed improvement.

Quote from: goldsky at Mar 12, 2014, 02:57 AM

That also crossed in my mind.
But from user point of view, this is not practical.
For a big website, the manager user will not only 1 person.
There'll be employees.
Those employees can be replaced again at anytime.

small plugin will solve your problem

Quote from: goldsky at Mar 12, 2014, 02:57 AM

1. which key?
3. what's the connection with the issue? Because at this point, we're thinking to cache everything for the speed improvement.

1. cache_alias_map
2. To retrieve data from a large cache requires no less server resources than the one requested data directly from the database. Moreover, the method of caching xPDOFileCache creates a lot of files which, when fully clearing the cache long enough removed.

Thanks for the suggestion, @Agel_Nash.
I've implemented it, I'll check further about the improvement.

Still, I wish there's a routing code to replace this URI thing.
This really is a bottleneck.