Intelligent Parallel Parsing (P^2)

Hi All,

In an effort to resolve caching issues with the new parser I’ve developed what I would like to call Parallel Parsing.

The idea is that the content to be cached is parsed separately from the content to be displayed on the browser. This means that we only parse the document once and all widgets will be executed during that parser sweep.

The parser will then output the content cache source and the content display source.

See more information here:
http://modxcms.com/forums/index.php/topic,1776.msg16234.html#msg16234

Sounds interesting indeed... but I’m not qualified to make such judgments

I am not sure I understand to be honest. If content is cached, why does it need to be reparsed?

Do you have code done for this, or maybe even a flowchart or a state diagram or something to show how it’d work? Sounds interesting.

Quote from: vbrilon at Dec 09, 2005, 02:23 PM

I am not sure I understand to be honest. If content is cached, why does it need to be reparsed?

True for most systems but in MODx not all items inside the content will be cached. Kinda like a partially cached content.

Do you have code done for this, or maybe even a flowchart or a state diagram or something to show how it’d work? Sounds interesting.

It’s in the making.

Hello Raymond,

I had some ideas about this too a while ago.

Things which came into mind was indeed to seperately parse and cache the widgets.
Problem with the output of widget is that in can dependend on the user/document permissions.

Then I thought, why not always cache a widget if the document is parsed in the public domain, unless the snippet itself explicitely has disabled it, and do it again when the same page is requested, but with different access permissions, like, a user x was logged in...

Now, you can compare the output of the widgets, and if equal, this widgets output can be cached too with access permission xxxx at which the document was parsed.
I hope you follow me

It perhaps makes sense too, to make the cached output php code, e.g. store it into an array.
Then in the cached template, you only have to replace an array index instead of using string search and replacement?
In fact, the whole cache could be stored into an array of strings hehe

Not sure if thats faster though...

Greetings,

Remon

Hi Remon,

I see what you’re saying. In the current (x9.1) build MODx needs to parse the content twice if non-cacheable snippets exists. This is of course the easiest way to do it but can cause problems if you want to pass the output of a snippet to another snippet. If we were to do the samething then here’s what would happen.

Consider:

[[Widget1? &param=`[!Widget2!]` ]]

In the above Widget1 will be processed before Widget2.

New method:

[[Widget1? &param=`[[Widget2]]` ]]

In parallel parsing the content is not parsed twice. The system is designed to execute Widget2 followed Widget1 in the same way it would normally.

Widget2 Code:


In the Widget2 example the developer can choose whether or not the widget should be cached. This will override the Widgets Cacheable Settings.

All Widgets, ContentFields, Templates should have a Cacheable option on their property page. This option should be enabled by default.

Hi everyone,

I’ve completed the parallel parsing pseudo code.

Here’s an example. Consider the follow:

[[A? &p1=`[[B? &p1=`[[C]]` &p2=`[[D]]`]]` &p2=`[[B? &p1=`[[E]]` &p2=`[[F]]`]]`]]

When A,B and E has caching disabled the parser will generate a cache output of:

[[A? &p1=`[[B? &p1=`1` &p2=`2`]]` &p2=`[[B? &p1=`[[E]]` &p2=`5`]]`]]

and for display it would render:

7

What do you guys think about this?

Wow! That’s pretty crazy. You know it’s way to late for me to even try to figure this out. It’ll have to wait ’till tomorrow. Unitl then, I’m impressed.

Any performance comparisons yet?

Quote from: aNoble at Dec 12, 2005, 10:42 PM

Any performance comparisons yet?

From what I’ve seen (from the naked eye) the timing seems to be the same as before.

It’s fast. Even with a boatload of iterations. Raymond’s example above simply adds a count+1 for each of  the pseudo snippets above. For a  reminder, please keep the snippet call in mind when viewing this explanation:



So in the code demo, it evals C first, returning 1. This is because it is the first most deeply nested snippet that is truly "closed". Next D (2), then back up to the first B because it can then "close that one out" (3).

It continues to the right with A still an open snippet call. It then backs up and finds another open snippet,  B. It parses out E/F (4/5), then figures out it can close out B (6). Finally, it can close out A.

Hence the 7 — simply a count of the parsing passes for the pseudo-snippets.

If you’re interested, here’s a copy of the entire code sample output:

Content Source:
[[A? &p1=`[[B? &p1=`[[C]]` &p2=`[[D]]`]]` &p2=`[[B? &p1=`[[E]]` &p2=`[[F]]`]]`]]

1) [[C]] --> output = 1
2) [[D]] --> output = 2
3) [[B? &p1=`1` &p2=`2`]] --> output = 3
4) [[E]] --> output = 4
5) [[F]] --> output = 5
6) [[B? &p1=`4` &p2=`5`]] --> output = 6
7) [[A? &p1=`3` &p2=`6`]] --> output = 7

Content for display:
7

Content for cache:
[[A? &p1=`[[B? &p1=`1` &p2=`2`]]` &p2=`[[B? &p1=`[[E]]` &p2=`5`]]`]]

Template processed within 0.0007479190826416 seconds

Very cool stuff Raymond. Now it would cache what could be cached and use that more quickly the next time through (but not in the pseudo-snippet demonstration if you were wondering)!