Wayfinder Slowness on a site with 1235 pages & 6 main menu items - PHP 1 second!

Ok, just got to the bottom of this one this morning at 6AM.

rev 6981
Modx ver 1.0.4
Apache/2.2.3 (Red Hat)
Mysql 5.0.45

It would appear wayfinder has some code in need of optimisation, or I have done something in order to screw this up:
There is about 50-60 root pages in the main menu in manager. Out of these, some break down into 2-3 levels, totaling around 1200 pages.
Out of all these pages, only 6 are used for the main site menu (some breaking down 1 level into a few more pages, so not a lot). The wayfinder call for these pages in the template is simply
[!Wayfinder?startId=`0`!]
All the other pages are set NOT to show in the menu, and are either just content pages or various functionality pages within the site.
The reason why the call for wayfinder is uncached is due to dynamic "login"/"logout" link in the menu, depending on whether an user is logged in (some other pages appear in main menu if user is logged in).

Now, having recently noticed some of these pages took a whopping 2-3 seconds to parse & render (using "[^qt^], [^q^], [^p^], [^t^], [^s^]." to display exec time) i set out to optimise the site code in order to cut this down. I have to point out no caching is used, most pages render live, containing various submenus and link lists rendered from custom database.

I’ve managed to reduce the MYSQL exec time down to miliceconds, however one thing i noticed straight away - PHP exec time was always around a second no matter what i did with my code!
Disabled all modules, plugins and had a good look for any loose snippets which might cause this. Nothing.

Then just for a laugh i created a new page, with no custom php code on it and an old default modx template, and checked exec times - still around a second! Then i disabled the call for the wayfinder and BINGO - PHP exec time went down to under 0.1 secs.
Go back to my custom pages, disable the wayfinder call:

With wayfinder - PHP Exec 1.5 secs
Without wayfinder - PHP Exec 0.3 secs

My question is, apologies if this sounds harsh, but im not that great of a programmer - how can all my bloated code, querying lots of custom tables, some having milions of rows, doing loop-de loops and crazy array transformations all take under 0.3 secs (uncached), yet the wayfinder snippet with a mere 1200 pages, only 5 of which composing the actual menu be brought down to its knees in this fashion? I would have expected the wayfinder snippet to easily handle tens of thousands of pages AND create menus in minimal time "out of the box" so to say. Growing the number of pages within the modx site is natural evolution of a site and i would have expected the wayfinder to handle this, in uncached mode, without issues, at least up to the theoretical limit given in documentation.

Perhaps I am being stupid and not using the wayfinder correctly? I know i could easily write a snippet which reads the content table, filters out and builds my links, together with a custom function to show login/logout in short time but then the whole reason why i chose modx (still the best framework!) was to cut down on dev time needed for implementation and benefit from built in functionality as much as i can.

I attempted to use the great advice given on http://www.devtrench.com/how-to-use-a-modx-plugin-to-cache-snippets/ to cut down exec time of wayfinder, which works, however i need an uncached, fresh version of menu each time the page renders.

Any advice appreciated - and once again apologies if above comments sound harsh, no ill feelings towards devs of wayfinder!

The only other comments i would leave after developing heavily within modx environment for the past year would be, perhaps the whole concept of chunks/snippets brings its own set of problems when it comes to organising your code & dev - it can be a chore sometimes to repeat - open page, find snippet & chunk responsible for page in question, then open snippets to find the code, look through the code to find "php file include" containing more code, etc. Visually it can be pretty daunting to find your way around when there is a lot of snippets & chunks to deal with, especially for the inexperienced. Im not saying get rid of snippets & chunks, just that visually they could perhaps be represented better within the manager, tied into the pages on which they are used in such a way as to aid organisation & quick referencing out of the box.

Great system by the way, hoping to start using the new branches when they are stable!

Hello,

If you only need one level in the menu try adding &level=`1` or however many levels you need, this should help cut down on things a little. Also I am guessing you have tried using wayfinder cached to determine if that helps the problem?

Also could you explain your menu structure a tad more, perhaps we can move some of it to a cached wayfinder call and other aspects not?

Ok, adding level filter:

with wayfinder = 0.8 sec
without wayfinder = 0.1 sec
with level=4 = 0.25 sec

Site menu has 6 items in it, 2 out of those items have submenus (only 1 level) with another 5-6 items in, that is it.

The root menu in manager has about 60 pages. 20% of these pages have children. Children usually go 2-3 levels deep at most. There is one page which contains 20 children, all going around 5 levels deep. None of it is set to be used in the menu.

Using wayfinder cached obviously speeds things up considerably, however as i said i cannot use it cached. I might need to do this and separate my login/logout buttons into a secondary snippet for the menu.
It seems like wayfinder processes all pages even though they might not be set to be used in the menu generated by wayfinder?

ABC; Always Be Caching.

Have you considered having two templates? One for logged in visitors and one for guests? Without knowing more of your menu needs, it seems like you could cache a separate version of each and rather than having the menu me un-cached and dynamic, you have the CMS it’self do the dynamic things.


Maybe the concept is too far in the .NET direction.

OTH, the main issue with Wayfinder in particular, is that the legacy data structures of MODx do not provide a way to query an infinite hierarchy in a single query; you have to query one level at a time. This is being addressed in the core and when this is available, a lot of Snippets that have traditionally been the performance bottlenecks (Wayfinder, Ditto, etc.) can take advantage of a very efficient single query mechanism for grabbing any tree of Resources.

One of the main reasons for the creation of Revo, other than making MODx database agnostic, was to be able address changing data structures without completely destroying backwards compatibility.

Nicely explained OpenGeek, i had a hunch my slowness had to do with something along the lines of what you said.
If i get some time i will look into wayfinder code and see if anything can be done to improve the state of things with current data structures in place - no time at the moment!

Suggested fix of specifying levels=x had given me enough performance improvements to ignore the problem for the time being.

Hi OpenGeek,
Quote from: OpenGeek at Nov 21, 2010, 11:13 AM

OTH, the main issue with Wayfinder in particular, is that the legacy data structures of MODx do not provide a way to query an infinite hierarchy in a single query; you have to query one level at a time. This is being addressed in the core and when this is available, a lot of Snippets that have traditionally been the performance bottlenecks (Wayfinder, Ditto, etc.) can take advantage of a very efficient single query mechanism for grabbing any tree of Resources.

Are there any tips how to fasten wayfinder. In our setting wayfinder works very slowly. We have about 200 menu-items (distributed to 3 levels) on our webpage which takes a very long time to be processed by wayfinder.

Is it for example useful to put all level 1-resources which are labeled as "don´t show in menu" to level 4 when we limit wayfinder to level=`3`? Or does this have no effects on the speed of wayfinder? Or is it useful to add &excludeDocs with all resources which are already labeled as "don´t show in menu" or labeled as "unpublished"? Is there any way to make wayfinder faster?

Letti