AjaxSearch Results Showing HTML Formatting?

@VanMeter: Your server seems to be down at the moment...

@tuatara (Matt) and VanMeter

Following the post of Matt, I checked the code and i (re)found the following issue: Whithout any &documents or &parents parameter the getListIDs function run in any case, even it’is not necessary! >:(
I have fixed this issue some time ago and i thougth that it was now ok. Unfortunately it wasn’t the case.
Obviously with 1200 documents the getListIds function which get all the ID of the site go out in error with a timeout.
Now without &parents or &documents parameter, the doSearch fonction (which build the sql request) doesn’t take care of the listIDs. So the behaviour should be the same as the 1.6 version.

FYI the changes regarding this bug fixe named ’listIDs =’’ means all documents’ are the following :

function getListIds in AjaxSearch.inc.php(line 271)
if (!strlen($IDs)) return $IDs; // listIDs =’’ means all documents

function doSearch in AjaxSearch.inc.php (lines 145 & 146):
$qry_sql =""; // listIDs =’’ means all documents
if (strlen($listIDs)) $qry_sql = "sc.id IN ({$listIDs}) AND ";

So find enclosed as described some posts above the version 1.6.3.
Thanks in advance for your new feedbacks. VanMeter, not sure that this correct your problem. But possibly.
Sorry for all these troubles.

Get the version 1.7 from the repository

WOOHOOOOO!!!!!!!!!

It works Coroico!! Thank you VERY much for this fix. I appreciate your persistence through this problem so much! It is awesome to have people like you dedicate their time to this forum and to addon’s to this CMS system.

As I am a graphic designer, please let me know if there is ever anything I can help you with designing!!! I would be more than happy to dedicate my time to helping you in return for this.

Thanks so much!!!!!!!!!!

Thank You VanMeter. Some time stubbornness could be also a quality

Nevertheless regarding the following troubles :
- Ublix’s feedback :http://modxcms.com/forums/index.php/topic,5357.msg125737.html#msg125737
- Andy’s feedback :http://modxcms.com/forums/index.php/topic,5357.msg126758.html#msg126758

and Tuatara (Matt)’s feedback (before the fix: listIDs =’’ means all documents)
http://modxcms.com/forums/index.php/topic,21047.msg131559.html#msg131559

the correction provided will not resolve the problem of memory fault, if you use $parents or $documents with a large number of pages.

But, as indicated by Matt, I suspect that array_merge allocation in the Ditto function getChildIDs is never
released, so with an important number of documents (like 1200), the memory failed.
This should explain the message: Fatal error: Allowed memory size of xxxxxxx bytes exhausted got by Ublix,Andy and Matt before the correction "listIDs =’’ means all documents"

In AjaxSearch.inc.php the fix seems to be the use of the unset($kids())
foreach ($IDs AS $seed) {
if (!empty($kids[intval($seed)])) {
$docIDs = array_merge($docIDs,$kids[intval($seed)]);
unset($kids[intval($seed)]);
}

I am awaiting confirmation from Mark (Ditto) before updating (again, >:( sorry) the zip file AjaxSearch 1.6.3
With this fix, I hope that everybody could use the &parents and &documents features safely.

Updated done. Get the version 1.7 from the repository.

Concerning your offer about designing, I appreciate. Thks.
On other hand, I am a very poor designer (look at my personal site and you understand )
I keep you informed when I will do this new update.

My issue is that AjaxSearch bypasses the MODx document output cycle (which is obviously needed for speed) without offering too many ways of fixing the problems that are created by the fact that the content displayed to the client is not fully evaluated.

To its credit, It has two parameters ($stripHTML and $stripSnip) to avoid displaying HTML or Snippet code, but what about sites like mine with a custom markup language resolved by a plugin? AjaxSearch displays all my tags, this is why I cannot use it.

One of the features it could offer is the ability to strip tags specified by strings pairs. For example [!AjaxSearch? &strip=`[[,]],[!,!],<,>`!] but also the option to perform a search and replace on the content. For example [!AjaxSearch? &replace=`[t~, `!] (replace "[t~" with a space) which would solve my issue.

Tools like AjaxSearch should attempt to be as generic as possible and not make too many assumptions as to how people use MODx. Since it does most of its work on the records of the table site_content, why not have parameters defining which field of the table is searched and which field is displayed in the search results? for example [!AjaxSearch? &input=`content` &output=`description`] to search the document’s main content and display the document’s description instead.

There’s a few key distinctions here with AjaxSearch that really applies to any MODx add-on jgestiot: 1) it’s not the MODx core, and 2) it was made to take advantage of parts of the core by accessing parts of it, and then 3) it fit a particular need then was made freely available for everyone to use. And for many of the sites out there it works great, albeit not without some problems that coroico is doing a great job of fixing. Let’s assume you want to return 20 search results per page: The AjaxSearch activation would need to parse 20 pages to return your results. :/

The search methodologies used in AjaxSearch probably aren’t the best in your application since you have a custom parser/markup language implementation as well — as you correctly identified. You need a "real" search indexing solution like Search Lucene or ht:dig (if it’s even maintained any more) with a custom Ajax-driven front end search/search results form implementation in order to maintain scalable and respectable performance. "Genericizing" a bit of code to the point where it’d handle your situation would have probably lead to unacceptable performance and extra codethat adds to complexity and maintenance requirements.

In your case, I’d suggest in the interim coding the suggested changes you need and trying to keep up with any changes to the snippet you need as they’re released.

Quote from: rthrash at Jan 04, 2008, 08:42 AM

There’s a few key distinctions here with AjaxSearch that really applies to any MODx add-on jgestiot: 1) it’s not the MODx core, and 2) it was made to take advantage of parts of the core by accessing parts of it, and then 3) it fit a particular need then was made freely available for everyone to use. And for many of the sites out there it works great, albeit not without some problems that coroico is doing a great job of fixing. Let’s assume you want to return 20 search results per page: The AjaxSearch activation would need to parse 20 pages to return your results. :/

The search methodologies used in AjaxSearch probably aren’t the best in your application since you have a custom parser/markup language implementation as well — as you correctly identified. You need a "real" search indexing solution like Search Lucene or ht:dig (if it’s even maintained any more) with a custom Ajax-driven front end search/search results form implementation in order to maintain scalable and respectable performance. "Genericizing" a bit of code to the point where it’d handle your situation would have probably lead to unacceptable performance and extra codethat adds to complexity and maintenance requirements.

In your case, I’d suggest in the interim coding the suggested changes you need and trying to keep up with any changes to the snippet you need as they’re released.

I agree that AjaxSearch fits a particular need and works well for most sites. I do not agree that the extra coding does lead to unacceptable performance. it takes a single if statement to check if there is a need for a search and replace and another if statement to check if stripping is required. Those who do not need the feature would not see a degeneration in performance. As for the idea of specifying the two fields used in the search and the output, the extra coding is negligeable.

The sites needing the extra features would need to make a decision regarding the degradation in performance resulting from multiple search and replace and stripping.

JG

Then it sounds like it’d be a pretty straightforward bit of code to add a few extra params.

I’d encourage you to just do it and collaborating with the other folks here that have been making additional fixes to AjaxSearch. coirico is the latest individual that just stepped up and volunteered to maintain this bit of code that was started by Kyle. Kyle’s working a new job and doesn’t have the time to work on this right now and this is what MODx is about: everyone contributing to make a better solution for all and welcoming contributions.

Thanks for your feedback and ideas.

Hi jgestiot,

Your remarks are very accurate. Thanks.

1/ offer the option to specify the list of tags to remove before outputing the content

I haven’t studied the impacts regarding security, but if this "strip_mytags" occurs in addition to stripHtml and stripSnip. Why not

Could you send me, by Modx email, as example, the list of your tags and an example of document

But, as suggested, the use of a regexp could be the solution.

2/ for your second request, the better way to treat that is probably to offer a parameter like &outputTpl=’mytemplate’ and inside this template use or not the table_site_content field (Title, Longtitle, content ...) with placeholders

But also as suggested by staed a possible breadcrumb :
http://modxcms.com/forums/index.php/topic,5357.msg125790.html#msg125790

Regarding the output of TV content when the searchword is found in the TV table is different (and not solved for the moment, but any suggestion is also welcome)
The addition of the content of the TV (stc.value) in the list of fields returned by the SELECT statement increase the number of record set returned (due to the LEFT JOIN statement, we will get one record set by TV linked with the content value). So it could be an option, but this will decrease the performance (more record sets returned + a new filtering to keep only the TV content which include the searchword). Again, I am open to any suggestion about that.

A new option is in preparation, the &config parameter to specify an external configuration file :

&config [config_name | "default"] (optional)
Load a custom configuration
config_name - Other config installed in the configs folder or in any folder within the MODx base path via @FILE
Configuration files are named in the form: <config_name>.config.php
Snippet call parameter values overwrite the config file parameter values
The "default" config file is always read. So keep it empty or take care of its content

e.g: [!AjaxSearch? &ajaxMax=`3` &config=`mycfg` ].
If &ajaxMax is set to 5 in the file mycfg.config.php then the &ajaxMax used will be 3.
If parameter is not set in the snippet call neither in the config file, the default value apply

As the snippet call value parameter overwrite the value of the config file, this will permit to define global config file with general parameters like the &replace parameter or &regexp parameter

And I work to change the extract mechanism, wich is very rude, when multi searchword are used.
For the moment if you search e.g : china wall you get one extract for china and one other extract for wall, which could be the same . In any case the extact ouput are merged to facilitate the reading. And the split of the sentence could occur anywhere in the sentence. Even between two bytes of a multi-byte UTF-8 character

Thks for your suggestions, sometime i prefer discuss about new features rather than how to fix the trouble ...
And as I am not fluent in English, sorry if sometime I am not very readable

As said by Ryan, any suggestion to improve the AjaxSearch snippet is welcome

I’ve read your reply and also read again the comments by Ryan. Here are a few comments based on that:

Stripping tags

Whatever class of tags you are stripping from the searched content, it makes sense to strip them before doing the matching. Unfortunately, it is also the slowest option but if you don’t and you have a home-made tag called "navigation" in your content and somebody searches for the word "navigation", you get a positive match, then strip the navigation tag from the output and present a document to the user that does not have the keyword he was searching for.

Security

I cannot see any obvious flaws in security in allowing stripping and search/replace, but nothing is truly secure on the web.

Search method

This is a difficult one. If we only needed single-keyword searches, then it would be easy to create an extra table and store in there all the non-common words contained in the content pages (using a language file), have a cron job refresh the keywords table once in a while (or activate/schedule it from the Manager) and have all search tools use the keywords table rather than search the document’s content. That would be a half-way solution with what Ryan suggested before regarding Search Lucene or ht:dig.

This topic is really wide and fiddly. You mention "china wall" but people can also enter "the great wall of china", "wall of china", "the china wall" and so on.

Since Google is a good search engine, a creative but twisted solution might involve searching using google and then parse the results and format them for your own site. I have not really looked into this but it sounds devious.

Config file

It is so easy for a Snippet to check for a config file parameter and load the file that all major snippets should have this option. The Snippet can have base options that would suit the majority and the rest of us can use the config file to override options.


My Tags

My tags are defined from a config file so I can define some tags for one site and other tags for another site. The tags have custom parameters, they are recursive and can self-modify. I found them very useful because they save me from using other plugins and snippets. For example, if I have the tags [img~flower[random~1|40].jpg|[justify~[random~1,2]]], I will display a random flower picture from a total of 40, from a pre-set folder and justify the image left or right randomly. [img] and [justify] are custom tags and [random] is a system tag handled by my plugin. I put all that cryptic stuff into a Chunk called {{random_flower}} and we are done.

On my photography site I use tags for the random photo and also the navigation. It is my tags that make the decision on what navigation image to show (full or shadows) based on the presence of neighboring pages (go to the last page using the double right arrow to check this). I just drop the Chunk {{nav}} on the page and the rest is done automatically. They save me a lot of time and this is why I am keen to have other tools such as AjaxSearch work with them.

Thanks for all the work that you and others have put into AjaxSearch. It is much appreciated.

Cheers,

JG