We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 1841
    • 141 Posts
    Hi everyone,
    I have struggled to get the AJAX search snippet to work very well for my site, it runs very slow with AJAX on or off.

    As a result I have got a replacement for this that has been written in PHP outside of MODX, at present it Connects to the DB, queries the content table and returns search results in a nice AJAX format with paginated results.

    It’s very fast and to me is pretty sweet, you can see it in action here:

    http://www.sciencefictionandfantasy.co.uk/search.htm

    The only problem is that it pulls out any results in the Content table regardless of them being published or not.

    I have tried to turn this into a snippet so I could then filter the results but I am in over my head a bit.

    If anyone could help I am sure it could be turned into a nice alternative modx search snippet.

    Let me know and i`ll post the code.

    cheers.
      • 5811
      • 1,717 Posts
      I have struggled to get the AJAX search snippet to work very well for my site, it runs very slow with AJAX on or off.
      When you say that ajaxSearch runs very slow with your datas, how does it take to get a result and among how many records the search occurs ?

      As a result I have got a replacement for this that has been written in PHP outside of MODX, at present it Connects to the DB, queries the content table and returns search results in a nice AJAX format with paginated results.
      Could you explain how you have improved the SQL request to improve the performance ?

      The only problem is that it pulls out any results in the Content table regardless of them being published or not.
      So you need to add a where clause with the field "published" from the site-content table.
        • 1841
        • 141 Posts
        Hi Coroico,
        Thanks for the response, yes I did understand adding a published clause would help but I am still learning PHP and so my knowledge is limited. I know how I would do this in modx (ie using PHX) but I haven’t been able to get a clause like that to work in PHP.

        The AjaxSearch snippet would take 30-40 seconds (or even more sometimes) to process a search, which is a long time for a visitor to wait, and it does load a fairly large library.

        The version I am trying to use has a much smaller library and returns results as you type, not sure why it’s so much faster but it is!

        The PHP used is:

        <?php

        $username = "username";
        $password = "password";
        $hostname = "localhost";
        $database = "dbase";

        mysql_connect($hostname, $username, $password) or die(mysql_error());
        mysql_select_db($database) or die(mysql_error());

        $table_name = "modx_site_content"; //which table from database
        $first_field = "pagetitle"; //which field from table
        $second_field = "introtext"; //which field from table
        $limitchar = 4; //minimum of characters
        $records_number = 10; // Number of records to show per page (different from 0)
        $page_number = 1;// default start page

        // file for database connection
        include(’../assets/templates/clean/inc/db.inc.php’);

        // configuration file
        include(’../assets/templates/clean/inc/config.inc.php’);

        if(isset($_GET[’p’])) {
        $page_number = $_GET[’p’];
        $arraySearch = $_GET[’terms’];
        $show_count = $_GET[’count’];
        settype($page_number, ’integer’);
        }
        $nospaces = substr($_GET[’terms’],0,4);
        $offset = ($page_number - 1) * $records_number;
        // check for an empty string and display a message.
        if ($_GET[’terms’] == "") {
        echo ’<div id="counter">eg. write &acute;book and&acute; or &acute;science fiction&acute; without quotes.</div>’;
        // minim 3 characters condition
        } else if(strlen($_GET[’terms’]) < $limitchar) {
        echo ’<div id="counter">’. $limitchar .’ characters minimum</div>’;
        // no spaces in first 4 letters
        } else if(preg_replace(’/[a-zA-Z0-9]/’, ’’, $nospaces)) {
        echo ’<div id="counter">Please use letters or numbers in first 4 characters</div>’;
        } else {

        // explode search words into an array
        $arraySearch = explode(" ", $_GET[’terms’]);
        // table fields to search
        $arrayFields = array(0 => $first_field, 1 => $second_field);
        $countSearch = count($arraySearch);
        $a = 0;
        $b = 0;
        $query = "SELECT * FROM $table_name WHERE (";
        $countFields = count($arrayFields);
        while ($a < $countFields)
        {
        while ($b < $countSearch)
        {
        $query = $query."$arrayFields[$a] LIKE ’%$arraySearch[$b]%’";
        $b++;
        if ($b < $countSearch)
        {
        $query = $query." AND ";
        }
        }
        $b = 0;
        $a++;
        if ($a < $countFields)
        {
        $query = $query.") OR (";
        }
        }
        $query = $query.") LIMIT $offset, $records_number;";
        $search = mysql_query($query);


        // get number of search results
        $arrayFields = array(0 => $first_field, 1 => $second_field);
        $countSearch = count($arraySearch);
        $a = 0;
        $b = 0;
        $query = "SELECT * FROM $table_name WHERE (";
        $countFields = count($arrayFields);
        while ($a < $countFields)
        {
        while ($b < $countSearch)
        {
        $query = $query."$arrayFields[$a] LIKE ’%$arraySearch[$b]%’";
        $b++;
        if ($b < $countSearch)
        {
        $query = $query." AND ";
        }
        }
        $b = 0;
        $a++;
        if ($a < $countFields)
        {
        $query = $query.") OR (";
        }
        }
        $query = $query.")";
        $count_results = mysql_query($query);
        $numrows = mysql_num_rows($count_results);

        // no results
        if($numrows == 0) {
        echo ’<div id="counter">No results found</div>’;

        // show results
        } else {

        echo ’<div id="results">
        <div id="results_top"><p><b>’. $_GET[’terms’] .’</b> - ’. $numrows .’ results found</p></div>
        ’;

        while($row = mysql_fetch_assoc($search)) {

        $urltitle = str_replace(" ","_", $row[’alias’]);

        echo ’<div class="item">
        <div class="details">
        <a href="’.$urltitle.’.htm" class="title">’.$row[’pagetitle’].’</a>

        ’.$row[’introtext’].’</div>
        <div style="clear:both;"></div></div>’;
        }
        // pagination
        $maxPage = ceil($numrows/$records_number);

        $nav = ’’;
        for($page = 1; $page <= $maxPage; $page++) {
        if ($page == $page_number) {
        $nav .= "$page";
        }
        else
        {
        $nav .= "<a href=\"javascript:htmlData(’search.php’,’terms=".$_GET[’terms’]."&amp;p=$page’)\">$page</a>";
        }
        }

        if ($page_number > 1) {

        $page = $page_number - 1;
        $prev = "<a href=\"javascript:htmlData(’search.php’,’terms=".$_GET[’terms’]."&amp;p=$page’)\">&laquo;</a>";

        $first = "<a href=\"javascript:htmlData(’search.php’,’terms=".$_GET[’terms’]."&amp;p=1’)\">First</a>";
        }
        else {
        $prev = ’’;
        $first = ’’;
        }

        if ($page_number < $maxPage) {
        $page = $page_number + 1;
        $next = "<a href=\"javascript:htmlData(’search.php’,’terms=".$_GET[’terms’]."&amp;p=$page’)\">&raquo;</a>";

        $last = "<a href=\"javascript:htmlData(’search.php’,’terms=".$_GET[’terms’]."&amp;p=$maxPage’)\">Last</a>";
        }
        else {
        $next = ’’;
        $last = ’’;
        }
        echo $data;
        echo "<div id=\"results_bottom\"><p>$first $prev $nav $next $last</p></div>
        </div>";
        }
        }
        ?>

        Load 2 JavaScript files:

        function GetXmlHttpObject(handler)
        {
        var objXMLHttp=null
        if (window.XMLHttpRequest)
        {
        objXMLHttp=new XMLHttpRequest()
        }
        else if (window.ActiveXObject)
        {
        objXMLHttp=new ActiveXObject("Microsoft.XMLHTTP")
        }
        return objXMLHttp
        }

        function stateChanged()
        {
        if (xmlHttp.readyState==4 || xmlHttp.readyState=="complete")
        {
        // show_results will be filled with new page
        document.getElementById("show_results").innerHTML = xmlHttp.responseText
        }
        else {
        //alert(xmlHttp.status);
        }
        }

        function htmlData(url, terms, pag)
        {
        if (url.length==0)
        {
        document.getElementById("show_results").innerHTML = "";
        return;
        }

        xmlHttp = GetXmlHttpObject();

        if (xmlHttp==null)
        {
        alert ("Browser does not support HTTP Request");
        return;
        }

        url=url+"?"+terms+"&amp;"+pag;
        url=url+"&sid="+Math.random();
        xmlHttp.onreadystatechange = stateChanged;
        xmlHttp.open("GET",url,true) ;
        xmlHttp.send(null);
        }

        search_id = ’’;
        function handleHttpResponse() {
        if (http.readyState == 4) {
        if (search_id != ’’) {
        document.getElementById(search_id).innerHTML = http.responseText;
        }
        }
        }
        function getHTTPObject() {
        var xmlhttp;
        if (!xmlhttp && typeof XMLHttpRequest != ’undefined’) {
        try {
        xmlhttp = new XMLHttpRequest();
        } catch (e) {
        xmlhttp = false;
        }
        }
        return xmlhttp;
        }
        var http = getHTTPObject();

        function getScriptPage(div_id,terms_id,get_count,get_p) {
        search_id = div_id;
        zearch = document.getElementById(terms_id).value;
        http.open("GET", "search.php?terms=" + escape(zearch)+"&count="+get_count+"&page="+get_p, true);
        http.onreadystatechange = handleHttpResponse;
        http.send(null);
        }

        and then simply call the search form on the page:

        <!-- begin content -->
        <div id="searchcontent">
        <img src="../assets/templates/clean/images/search_form.png" width="240" height="32" alt="search form" />
        <!-- begin form -->
        <div id="form">
        <input type="text" id="terms" size="40" onkeyup="getScriptPage(’show_results’,’terms’,’1’)" onblur="if ( this.value == ’’ ) this.value = this.defaultValue" onfocus="if ( this.value == this.defaultValue ) this.value = ’’" value="type your search here" /></div><!-- end form -->
        </div><!-- end content -->

          • 5811
          • 1,717 Posts
          You need to replace
          $query = "SELECT * FROM $table_name WHERE (";

          by
          $query = "SELECT * FROM $table_name WHERE (published=1) AND (searchable=1) AND (deleted=0) AND (privateweb=0) AND (";

          (published=1) => published document

          (searchable=1) => searcheable document

          (deleted=0) => not a deleted document

          (privateweb=0) => not a private document

          For searching in private document you need to test before that the user is a connected user.


          The AjaxSearch snippet would take 30-40 seconds (or even more sometimes) to process a search, which is a long time for a visitor to wait, and it does load a fairly large library.
          This is not a normal time. With which release of AS did you run these tests ? Could you post the ajaxSearch snippet call used for these tests. And I am interested to get the debug file, if you run again a search with &debug=`3`

          AjaxSearch runs only one request, which could be longer with a very large database. And then the time depends of the number of the displayed results and of the selected options (withTvs, tvPhx, ...). You can improve the speed of the search by adjusting lot of parameters.
            • 1841
            • 141 Posts
            Hi Coroico,
            Thank you very much for your help, that worked a treat. The version of AjaxSearch I was running is 1.90. I tried it first with Ajax on (which was a bit faster) then with Ajax off (as the libraries loaded were slowing the pages down).

            I do try and keep the database as trim as possible but the site has in excess of 2000 pages, which I guess could be an issue (about 1/2 are unpublished).

            I have setup an AjaxSearch debug as you suggested, the resulting log though is about 327k and too long to list here.

            I have kept the AjaxSearch test URL published, you can see it here:

            http://www.sciencefictionandfantasy.co.uk/search-test.htm

            This run with no other parameters set other than debug, as you can see it’s pretty slow, including going to the next / previous page, but also the results it displays aren’t always the best either, for example if you search for "hylton smith" you get 43 results, but in reality (and reflected in the search I am currently using) there are only 5 pages that are actually relevant.

            Thanks again for all your help.

              • 5811
              • 1,717 Posts
              This run with no other parameters set other than debug, as you can see it’s pretty slow, including going to the next / previous page,
              For a correct appreciation of the performance, set the debug mode to 0.

              but also the results it displays aren’t always the best either, for example if you search for "hylton smith" you get 43 results, but in reality (and reflected in the search I am currently using) there are only 5 pages that are actually relevant.
              For which reason do you get 43 results if only 5 are relevant ? Do you have the term "hylton smith" somewhere in a tv ?

              To optimize your search adds: &whereSearch=`content:pagetitle,content`. This will avoid to search in TV and some extra content fields.
                • 1841
                • 141 Posts
                Ok I set debug to 0 and the speed was the same, however when I put in the params: &whereSearch=`content:pagetitle,content` it’s much much faster, strangly though I still get the irrelevant results, there isn’t any mention of the search terms on the pages in question either.
                  • 5811
                  • 1,717 Posts
                  To understand why you get these results, do as follow:

                  1/ Run again this search with &debug=`1`. Then open the ajaxSearch/debug/ajaxSearch_log.txt file. Search in the file the SELECT request executed. Something like:
                  SELECT sc.id, sc.pagetitle, sc.longtitle, sc.description, sc.alias, sc.introtext, sc.template, sc.menutitle, sc.content, sc.publishedon, 
                  ...
                  ORDER BY sc.publishedon,sc.pagetitle
                  Copy this SQL order from the SELECT to the end of the ORDER clause.

                  2/Then open your PHPMyAdmin tool, open a SQL window, paste the previous copied SQL order and execute the order. Then export the results to a file.

                  3/ Open this file result with an editor and search "hylton" or/and "smith".


                    • 1841
                    • 141 Posts
                    Ahah! that solved it, it appears the search finds a broad match, rather than an exact match, so it was finding the name "smith" in the other 38 documents.

                    Thanks again for all the help.
                      • 5811
                      • 1,717 Posts

                      In this case add &advSearch=`exactphrase` in your snippet calls
                      ---- &advSearch [ ’exactphrase’ | ’allwords’ | ’nowords’ | ’oneword’ ] (optional)

                      Advanced search
                      - exactphrase : provides the documents which contain the exact phrase
                      - allwords : provides the documents which contain all the words
                      - nowords : provides the documents which do not contain the words
                      - oneword : provides the document which contain at least one word [default]
                      But keep care, I fixed an issue yesterday regarding the "exact phrase" option. See this post.