Optimising / caching TV values

When using getResources to list resources which all have assigned several TVs and they are being processed to retrieve their display / inherited / default values, ModX can slow down very quickly. I believe this is because every time renderOutput() is called, a lot of database queries, directories read, files opened, etc. If you’re listing 100 documents, each with 10 TVs, this rapidly slows things down.

Is there are form of caching used on TV values which could help this process? Or optimisations to the renderOutput function?

When a resource is cached, are it’s TV values cached too? If not, would ModX benefit from rendered TV values being stored in a cache database (I reckon this could be done as a plugin?)

I see a recent commit which improved getting the raw value - could something similar be done for rendered values?

Any input welcome.

I’d be very interested in this as well! I would love to know more about this.

Going by recent posts on these forums I think a lot of people are having performance issues with getResources and TVs.
Any suggestions to speed things up would be very welcome!

I’ve recently rewritten large chunks of getResources to allow more control over filtering by TV, and in the process have optimised it so using processTVs is far quicker (in my case 2x faster) - but this is the main bottleneck.

Out of curiosity: have you submitted these changed to getResources’ Github repository?

I did but pulled them as I found a bug - I’ve put the latest version on my Github fork if you want to take a look, although it contains lots of debugging code. One of the big changes is instead include all TVs or none, you can specify which TVs to include - which can massively reduce the overhead (if a template has 10 TVs, but you only want to display 1 in your getResources template, for example).

specify which TVs to include - which can massively reduce the overhead (if a template has 10 TVs, but you only want to display 1 in your getResources template

That sounds brilliant and a big step in the right direction - I’ve found myself in this situation quite often!

Quote from: lucas at Apr 20, 2011, 07:01 AM

specify which TVs to include - which can massively reduce the overhead (if a template has 10 TVs, but you only want to display 1 in your getResources template

That sounds brilliant and a big step in the right direction - I’ve found myself in this situation quite often!

Agreed—this is a great idea...I look forward to a pull request

Quote from: OpenGeek at Apr 20, 2011, 12:29 PM

Quote from: lucas at Apr 20, 2011, 07:01 AM

specify which TVs to include - which can massively reduce the overhead (if a template has 10 TVs, but you only want to display 1 in your getResources template

That sounds brilliant and a big step in the right direction - I’ve found myself in this situation quite often!

Agreed—this is a great idea...I look forward to a pull request

Will do...

On the other questions - is there any room for optimising TV retrieval in 2.2?

On the site I’ve been testing on (which has quite a few resources with around 12 TVs) getResources was taking >7 seconds to run. With my optimisations its down to <4 seconds, but the ultimate delay is the calls to renderOutput

I think I have to agree with ncrossland here: the heart of the matter seems to be the calls in Revolution itself, in my opinion.