We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 26503
    • 620 Posts
    Sean Kimball Reply #1, 14 years ago
    I have a little reporting tool that does some lookups on modx users, currently about 2000.

    one of the lookups needs to return all users, problem is that it does not, it will only return up to 700 or so, I put the exact query into a snippet and run it, it works fine. returns all users.

    is there some sort of limit or execution time maximum in the manager pages or with extjs? it seems that if the page takes longer than 7 or 8 seconds, no results will be returned?

    UPDATE
    I've upped all the php settings to allow more memory, longer execution, post times, limits etc. I've also tweaked mysql to allow larger queries, caches, tmp tables etc..etc.. still nothing. Best I got was an increase in the initial search from 300 -> 700 results. the only thing that made a difference was adjusting the memory to 64M... if I up that to 1024M I still cannot get more than 700 records initially.

    I think we might be hitting something with the extjs? maybe there is a max execution time or amount of data that can be held?

    here is the code that is initially executed:

    $query = $modx->newQuery('modUser');
    // if I set this to > 700 no results are returned.
    $query->limit(700);
    $modx->lexicon->load('user');
    $users = $modx->getCollection('modUser', $query);
    [ed. note: sean69 last edited this post 14 years ago.]
      *** Not just websites, we also create signage, banners, print, trade show displays and more! ***

      Sean Kimball CLP, CLS.
      Technical Director / Sr. Developer | BigBlock Studios
      ._______________________________________________.
      Bigblock Studios http://www.bigblockstudios.ca Web site design & development.
      27-1300 King Street East. Box 167 Oshawa, Ontario L1H8J4 Canada.
      phone/fax: 905-426-5525
      • 18373 ☆ A M B ☆
      • 3,141 Posts
      Mark Hamstra Reply #2, 14 years ago
      Do you get your records through the connector response at all?

      In other words; does the query do what you need and is it properly sent to the browser and is it something in ExtJS, or does the query fail before even getting to that stage.
        Mark Hamstra • Developer spending his days working on Premium Extras and a MODX Site Dashboard with the ability to remotely upgrade MODX and extras to make the MODX world a little better.

        Tweet me @mark_hamstra, check my infrequent blog at markhamstra.com, my slightly more frequent ramblings at MODX.today or see code at Github.
        • 26503
        • 620 Posts
        Sean Kimball Reply #3, 14 years ago
        I don't think it is the query, I've tweaked mysql to allow larger queries, tmp tables etc... I can run that little block of code in a snippet and count or dump the full result set.
        if I limit the query to under 700 results I can search & filter it normally.

        so 'yes' the connector does return records. just not all of them.
          *** Not just websites, we also create signage, banners, print, trade show displays and more! ***

          Sean Kimball CLP, CLS.
          Technical Director / Sr. Developer | BigBlock Studios
          ._______________________________________________.
          Bigblock Studios http://www.bigblockstudios.ca Web site design & development.
          27-1300 King Street East. Box 167 Oshawa, Ontario L1H8J4 Canada.
          phone/fax: 905-426-5525
          • 18373 ☆ A M B ☆
          • 3,141 Posts
          Mark Hamstra Reply #4, 14 years ago
          The question is; does the connector return all results and can't ExtJS handle that, or does the connector fail to handle 700 records.
            Mark Hamstra • Developer spending his days working on Premium Extras and a MODX Site Dashboard with the ability to remotely upgrade MODX and extras to make the MODX world a little better.

            Tweet me @mark_hamstra, check my infrequent blog at markhamstra.com, my slightly more frequent ramblings at MODX.today or see code at Github.
            • 26503
            • 620 Posts
            Sean Kimball Reply #5, 14 years ago
            sorry, the connector will return up to 719 results.
              *** Not just websites, we also create signage, banners, print, trade show displays and more! ***

              Sean Kimball CLP, CLS.
              Technical Director / Sr. Developer | BigBlock Studios
              ._______________________________________________.
              Bigblock Studios http://www.bigblockstudios.ca Web site design & development.
              27-1300 King Street East. Box 167 Oshawa, Ontario L1H8J4 Canada.
              phone/fax: 905-426-5525
              • 22303 MODX Staff
              • 10,725 Posts
              If you inspect the requests to the connector, do you get a response with the records you expect in JSON format, or is it empty? Maybe full of PHP errors? Any PHP errors being logged on your server?
                • 22303 MODX Staff
                • 10,725 Posts
                Also, regarding the memory_limit, have you tried using getIterator() instead of getCollection()? There is a HUGE difference in memory usage between iterating through the result set one object at a time and returning an entire collection of data, especially when working with large result sets like this.
                  • 26503
                  • 620 Posts
                  No php errors at all.

                  Going to try the getIterator method. but not sure if that is the issue - as I said, I only saw a difference with ram usage the one time - then even if I gave it up to a gig to work with no more records were returned.

                  this used to work, it stopped working when we updated to 2.2.4
                    *** Not just websites, we also create signage, banners, print, trade show displays and more! ***

                    Sean Kimball CLP, CLS.
                    Technical Director / Sr. Developer | BigBlock Studios
                    ._______________________________________________.
                    Bigblock Studios http://www.bigblockstudios.ca Web site design & development.
                    27-1300 King Street East. Box 167 Oshawa, Ontario L1H8J4 Canada.
                    phone/fax: 905-426-5525