Strip snippets from results

79 Posts

tobice Reply #1, 13 years, 5 months ago

This has been already posted several times in this forum but never answered so I am asking again. When a searchable document contains some snippets, then these snippets are normally displayed in ajaxSearch results (in so called "extracts"). As far as I known there isn’t any "stripSnippets" option in the documentation and I can’t use pHx either because [! !] are very often missing in extracts and without these tags I can’t detect what is snippet and what is not.

I wanted to avoid changing the original code so the "cleanest" solution I found was to write my own stripOutput functions as it is described in the documentation. However to avoid copying code I had to call the original stripOutput which turned out to be a little bit problematic. After all I ended up with following function in my config file:

if (!function_exists('ajaxSearchStripSnippets')) {
    function ajaxSearchStripSnippets($string) {
        $as = new AjaxSearchResults;
        $string = $as->defaultStripOutput($string);
        $string = preg_replace('~\[\[[^\]]*\]\]~is', '', $string);
        $string = preg_replace('~\[\![^!]*\!\]~is', '', $string);
        return $string;
    }
}

It would be nice if this feature could be included in the next release

Or maybe it is already there and I just don’t see it so in that case please help me, I will welcome any better solution

-- sorry for my english, bad teacher

1,717 Posts

coroico Reply #2, 13 years, 4 months ago

When a searchable document contains some snippets, then these snippets are normally displayed in ajaxSearch results (in so called "extracts").

With AS 1.9.1, if you search something like "snip" and you have a content page with for instance [!Snippet!], then the sql search found this document.
But, before to display the results, the search results are filtering when the search terms are found inside HTML or MODx tags.

79 Posts

tobice Reply #3, 13 years, 4 months ago

Well, that’s cool, I’ll definitely upgrade to 1.9.1. But if I understand you correctly, my problem was still a little bit different. I was solving the situation when someone searched for some "keyword" and this "keyword" was just next to the snippet in the document, for example "keyword [!snippet!]". Then the extract contained something like "...keyword [!snipp...". And I fixed it by the code I sent in my first post.

-- sorry for my english, bad teacher

1,717 Posts

coroico Reply #4, 13 years, 4 months ago

and this "keyword" was just next to the snippet in the document, for example "keyword [!snippet!]". Then the extract contained something like "...keyword [!snipp...".

I haven’t done this test, but the [!snippet!] call only will be remove. If the searchterm is near a snippet call, it should remain. The snippet call is parsed and stripped before to set up the extract.

I’ll definitely upgrade to 1.9.1

1.9.2 is coming in few days, so ...

79 Posts

tobice Reply #5, 13 years, 4 months ago

The snippet call is parsed and stripped before to set up the extract.

This is new in 1.9.1? My experience with 1.9.0 is different, I had to write my own function to strip these snippets. I skimmed through the code of AjaxSearch and found the responsible function/method (defaultStripOutput) but it didn’t do anything with snippets.

I’ll wait for 1.9.2

-- sorry for my english, bad teacher

1,717 Posts

coroico Reply #6, 13 years, 4 months ago

This is new in 1.9.1?

No. The code is the same for 1.9.0 and 1.9.1

ajaxSearchResult.class:

                    $results = $this->_asRequest->doSearch();
                    $results = $this->_doFilter();
                    $this->_setSearchResults($site, $subsite, $results);

and then in ajaxSearchOutput.class:

                    _displayResults

_doFilter() {
...
    $results = $this->_doFilterTags($results, $searchString, $advSearch);
...
}

    /*
    *  Filter the search results when the search terms are found inside HTML or MODx tags
    */
_doFilterTags(){
...
$text = $this->defaultStripOutput($text);
...
}

79 Posts

tobice Reply #7, 13 years, 4 months ago

Ok, I am sorry, you’re right, I didn’t read your code really carefully

Probably I thought when I was reading it, that method stripTags only strips HTML tags, not also MODx tags. However I tested it again and the snippet tags were still displayed in search results. But I finally hopefully found where is the problem

It’s in the function stripTags.

$modRegExArray[] = '~\[!(.*?)!\]~';

For some reason this doesn’t work in all cases. I am not a regexp expert so I don’t know why. I just replaced it with my code and it works! (not actually my code, I found it somewhere on this forum).

$modRegExArray[] = '~\[\![^!]*\!\]~is';

-- sorry for my english, bad teacher

1,717 Posts

coroico Reply #8, 13 years, 4 months ago

Thanks Tobyce for this feedback. You are right the regexp doesn’t work in all cases.

I did the test with this regexp tester. With "xxxx[!Snippet?
&id=`24`
&extract=`10`
!]aaaaa " as text to test.

With ’~\[!(.*?)!\]~’ ,the snippet call is not found

With ’~\[\![^!]*\!\]~is’; the snippet call is found

When the snippet call is on the same line, the both rexgexp works, so i think that the explanation comes from the use of ’s’ modifier
If this modifier is set, a dot metacharacter in the pattern matches all characters, including newlines. Without it, newlines are excluded. This modifier is equivalent to Perl’s /s modifier. A negative class such as [^a] always matches a newline character, independent of the setting of this modifier.

Issue registered as AJAXSEARCH-80. I will integrate this regexp in the 1.9.2 release.

79 Posts

tobice Reply #9, 13 years, 4 months ago

You’re welcome

Well, in my regexp the ’s’ modifier is actually useless as [^!] already matches a newline character. BUT using [^!] is not a good solution after all because it doesn’t match any snippet calls containing ’!’, for example:

[!snippet? &param=`Hello world!`!]

I did some tests and I ended up with following regexp:

~\[\!.+\!\]~Us

I added an extra Ungreedy modificator in case there were two snippets on the same line. This last regexp fails for snippet calls with an argument containing ’!]’, for example &param=`!] test` but this is the correct behavior for MODx.

Btw. it would be nice if the MODx API provided an official function for clearing tags from text

-- sorry for my english, bad teacher

1,717 Posts

coroico Reply #10, 13 years, 4 months ago

For the moment, I get the better results with : ’~\[!(.*?)!\]~is’

<?php	
    $text1= '[!snippet? &param=`Hello world!`!]';
	$text1 = preg_replace('~\[!([^!]*)!\]~is', '', $text1);
	echo "text1=".$text1;
	echo "<br />";

    $text2= '[!snippet? &param=`Hello world!`!]';
	$text2 = preg_replace('~\[!(.*?)!\]~is', '', $text2);
	echo "text2=".$text2;
	echo "<br />";
	
	$text3= '[!snippet? 
	  &param=`Hello world!`
	  !]
	';
	$text3 = preg_replace('~\[!(.*?)!\]~is', '', $text3);
	echo "text3=".$text3;	
	echo "<br />";
	
	$text4= '[!Snippet1? &param1=`xx` &id=`[!Snippet2!]` &param2=`zz`!]';
	$text4 = preg_replace('~\[!(.*?)!\]~is', '', $text4);
	echo "text4=".$text4;	
	echo "<br />";	
?>

But it doesn’t work If I have an imbricated call like: [!Snippet1? &param1=`xx` &id=`[!Snippet2!]` &param2=`zz`!]

I think I need to use assertions (lookahead) to detect if the next !] belongs or not to the same snippet call. Not easy for a sunday laugh

Could you post your first snippet call which oblige you to improve the stripOutput. Thanks.

EDIT: I didn’t read carefully your post undecided

If I use ’~\[!(.*?)!\]~Us’, I remove all the snippets calls

<?php	
	$text4= '[!Snippet1? &param1=`xx` &id=`[!Snippet2!]` &param2=`zz`!]';
	$text4 = preg_replace('~\[!(.*?)!\]~Us', '', $text4);
	echo "text4=".$text4;	
	echo "<br />";
	
	$text5= '[!Snippet1? &param1=`xx` 
	  &id=`[!Snippet2!]` &param2=`zz`!]
	  xxxxxxxxxxxxxxx 
	  [!snippet3!]
	';
	$text5 = preg_replace('~\[!(.*?)!\]~Us', '', $text5);
	echo "text5=".$text5;	
	echo "<br />";	
?>