We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 14349
    • 44 Posts
    I have a large product database - each product is a modx "resource" - I have about 2000 under a landing resource container

    When I select the dropdown arrow to display all of my product resources - modx loads for a while but then doesn’t show any resources under my landing resource container.

    Things worked fine at about 1,000 resources - but now no go.


    System specs - php 5 on centos 5 mysql 5.0.67 community
      • 9207 ☆ A M B ☆
      • 2,475 Posts
      Which version of PHP?

      Is this a shared host? You might be getting clipped -- often they throttle queries that return large numbers of rows. Do you have access to the MySQL logs? Normal debugging procedure here for suspected slow queries is to turn on the slow query log, restart MySQL and instigate the suspicious queries. On a shared host, you might have to issue ’SET OPTION SQL_BIG_SELECTS = 1’ prior to running your big queries.
        • 14349
        • 44 Posts
        Good advice,

        I own my own servers I keep my clients on - so I have full control of everything

        What do you think the best way to handle it is - A mysql setting in php.ini config that needs to be changed ?
          • 9207 ☆ A M B ☆
          • 2,475 Posts
          You can try downloading mysqltuner.pl
          http://mysqltuner.pl/mysqltuner.pl
          -- save it on your server, then run it using perl:

          perl mysqltuner.pl


          It gives a lot of good info about optimizing your database... but that’s not the only thing that could be going on here.
            • 26903
            • 1,336 Posts
            This could be a facet of the tree implementation of modext/extjs, it may not be able to draw that many nodes. I seem to recall splittingred mentioning something about this on another thread, are you getting an JS errors?
              Use MODx, or the cat gets it!
              • 14349
              • 44 Posts
              Nope, no javascript errors that I can see in firefox error console.


              Here is the Statistics from the Mysql Tuner

              -------- General Statistics --------------------------------------------------
              [--] Skipped version check for MySQLTuner script
              [OK] Currently running supported MySQL version 5.0.67-community
              [OK] Operating on 32-bit architecture with less than 2GB RAM
              
              -------- Storage Engine Statistics -------------------------------------------
              [--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster 
              [--] Data in MyISAM tables: 2M (Tables: 290)
              [!!] InnoDB is enabled but isn't being used
              [!!] Total fragmented tables: 27
              
              -------- Performance Metrics -------------------------------------------------
              [--] Up for: 2m 51s (15K q [92.766 qps], 47 conn, TX: 6M, RX: 5M)
              [--] Reads / Writes: 99% / 1%
              [--] Total buffers: 34.0M global + 2.6M per thread (100 max threads)
              [OK] Maximum possible memory usage: 296.1M (29% of installed RAM)
              [OK] Slow queries: 0% (0/15K)
              [OK] Highest usage of available connections: 4% (4/100)
              [OK] Key buffer size / total MyISAM indexes: 8.0M/905.0K
              [OK] Key buffer hit rate: 99.5% (21K cached / 97 reads)
              [!!] Query cache is disabled
              [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 36 sorts)
              [OK] Temporary tables created on disk: 0% (20 on disk / 2K total)
              [!!] Thread cache is disabled
              [OK] Table cache hit rate: 89% (43 open / 48 opened)
              [OK] Open file limit used: 8% (85/1K)
              [OK] Table locks acquired immediately: 100% (28K immediate / 28K locks)
              
              -------- Recommendations -----------------------------------------------------
              General recommendations:
                  Add skip-innodb to MySQL configuration to disable InnoDB
                  Run OPTIMIZE TABLE to defragment tables for better performance
                  MySQL started within last 24 hours - recommendations may be inaccurate
                  Enable the slow query log to troubleshoot bad queries
                  Set thread_cache_size to 4 as a starting value
              Variables to adjust:
                  query_cache_size (>= 8M)
                  thread_cache_size (start at 4)
              



              Maybe something will stand out to someone.

              I think what I will do for now is just split these resources into different subnodes - but eventually even those might get to vast
                • 9207 ☆ A M B ☆
                • 2,475 Posts
                That doesn’t look too bad for mySQL. If I were you, I’d disable InnoDB for now and up the 2 recommended settings:
                query_cache_size (>= 8M)
                thread_cache_size (start at 4)

                Then restart mysqld.

                I’d take a look at your PHP memory settings in /etc/php.ini -- how much memory is allocated to PHP. Check your error logs too.

                I just looked at a Revo site with ~100k pages, and the poor performance had more to do with the limited amount of RAM available on the machine.
                  • 14349
                  • 44 Posts
                  Thanks,

                  Checking on php.ini settings - I think I’ve maxed them out.

                  Created a snippet that separates every 1000 products into a "folder" - the tree can handel about 1000ish without flipping out.
                    • 9207 ☆ A M B ☆
                    • 2,475 Posts
                    Good thinking. This would make for a fascinating benchmarking article...
                      • 32699 ☆ A M B ☆
                      • 427 Posts
                      I recently built a massive website (20 times the topic size) and had content disappearing from manger.

                      The easy fix: allocate more memory in php.ini.

                        Get your copy of MODX Revolution Building the Web Your Way http://www.sanitypress.com/books/modx-revolution-building-the-web-your-way.html

                        Check out my MODX || xPDO resources here: http://www.shawnwilkerson.com