Ditto: I feel the need for speed

Hi all

There’s no doubt that Ditto is totally awesome, however, I am finding the page loading speed real sluggish (7.5 seconds in all according to MySQL: [^qt^], [^q^] request(s), PHP: [^p^], total: [^t^])

The page concerned is <url removed> which contains 2 Ditto calls, one ’Featured’ used a filter - but now uses tags, the other ’Latest Items’ is a straight up summarized list in editedon order.

Stripping these 2 calls out the page increases the page loading speed to an incredible 0.26 seconds (page cachable is checked btw) - but of course, without Ditto, is pretty boring innit!

I’m thinking of hardcoding the ’Featured’ links, but thought I’d run the whole thing by you guys (and gals) first...

Any comments are always appreciated.

0

What version of Ditto are you using? If not newest (1.0.2) I would suggest to try it. There’s some changes how documents are retrieved what should speed it up incase you are not allready using it.

Ditto 1.0.2 yes, sorry for not including that...

Wowzers... I’ve never seen that happen. Can you run one call and then the other and note the times. Then can you run them both with &debug=1 and send me the output. Also, try setting a &total value to limit the number of documents retrieved.

debug attached, thanks Mark - login details PM’ed as well.

cheers =)

Ok, I took a look at the site (created a hidden test page called Homepage Test) and came up with the following conclusions:

1. Your server is running very slow. Manager generation times were in excess of 5 seconds.
2. You are having Ditto retrieve EVERY document in your site for each of the two homepage calls, which it then retrieves all associated document data for. Given that you have almost 400 documents, with a minimum of 3 queries each (one for basic doc data, 1 for tvs, and one of author information).... you get the idea that it takes PHP a while.

Solutions:
1. Cache all of the output on that page. When I cached all the snippet calls, the page rendered in less than a second.
2. Move to a faster server (<1 second manager gen time, mediatemple.net offers this for $7.99 a month).
3. I need to recode &total to use LIMIT TOTAL for use in situations like this. This would mean that the actual number of documents returned would be less than total (because filtering would remove some) but still, this would help.

Quote from: Mark at Jul 25, 2006, 07:52 PM

2. You are having Ditto retrieve EVERY document in your site for each of the two homepage calls, which it then retrieves all associated document data for. Given that you have almost 400 documents, with a minimum of 3 queries each (one for basic doc data, 1 for tvs, and one of author information).... you get the idea that it takes PHP a while.

Yes, I was just wondering this when I peeked at the debug log and saw the timing info:

For the first snippet call it’s MySQL: 3.2972 s, 431 request(s) and for the second MySQL: 3.2368 s, 355 request(s)

Sure it takes long when those are done.. so setting the &total doesn’t help reducing the needed mysql queries? what is it used for?

Eek. I’d prefer adding &limit rather than reusing &total, so that it can fine-tuned for individual circumstance to a number that is high enough to return sufficient content.

But woh.. what’s going on here???

I am however concerned that none of your suggestions involve changing the snippet call. EDIT: Although it looks like you tweaked it so it’s not as bad as it was...

Are you saying that pulling 5 featured pages using a Ditto search on the TV "IsFeatured" requires every page to run through Ditto?

Are we saying that it is significantly less efficient to query using the TV data structure than if you just had an extra IsFeatured field in the content table? If so, that’s of horrendous concerning.

Yeah sure, "get a faster server" always works, but the inference that 400 documents is a lot is hugely concerning. I’m loving Ditto now, but this does raise serious concerns.

@ Mark: Thanks for looking, you’ve halved the loading speed - I’m stoked!

Yeah, we’re looking at server reaction times too

I am however concerned that none of your suggestions involve changing the snippet call.

There are no current parameters (hint, hint) that would help.

Are you saying that pulling 5 featured pages using a Ditto search on the TV "IsFeatured" requires every page to run through Ditto?

Yes, that is how Ditto was coded. I never saw Ditto being used to pull documents from the entire site. Knowing this, I will see about possibly recoding the TV filter to only pull the documents that have a TV value...

I’m loving Ditto now, but this does raise serious concerns.

This shouldn’t raise concerns. This is just Ditto being used in a case I had not coded it for. With a few clever speed optimizations I should be able to reduce the time required to generate a large listing like this one.