We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 23027
    • 7 Posts
    Hi!
    I try build with MODx shop-like catalog. I’ve imported >16000 resourses. I have encountered the following problems:

    1) Resourse tree is not displayed until I increased the amount of memory on server from 64 to 128MB. Let’s think, it’s problem solved. But I don’t think CMF should use such big amount of memory. The next one

    2) File tree is not displayed too. It’s blank. I spent a few tests and found the dependence between this problem, such amount of resourses and context’s cache. For example: if i change any system option and click "refresh file tree", i can view files once. But if i click refresh file tree again, i cann’t view files - File’s tab is blank.
    Request screenshot: http://www.webpagescreenshot.info/img/731909-1115201085814PM

    Why files depends on pages (resourses)? Seems to be architectural error.

    I think that 16000 resources should not be great for the modern CMF sad
    Is there any way to solve this problem?

    MODx Revo 2.0.4 pl-2 (advanced distribution)
    Mysql 5.0.67-percona-highperf-b7-log
    Mysql Client API version 5.0.67
    Linux server18.jino.ru 2.6.18-194.11.1.el5 #1 SMP Tue Aug 10 19:05:06 EDT 2010 x86_64
    PDO drivers mysql, sqlite, sqlite2
    PHP Version 5.2.13
    Browser: Chrome 7, FF 3.6
    memory_limit 128M
    execution_time: 30

    P.S. I can provide my mysql dump, if necessary.
      • 3749
      • 24,544 Posts
      Using an entire resource to hold each product is fairly inefficient -- resources have a lot of overhead. You’ll want to consider using one or more custom database tables to hold the product data and using a small number of resources just to display them.
        Did I help you? Buy me a beer
        Get my Book: MODX:The Official Guide
        MODX info for everyone: http://bobsguides.com/modx.html
        My MODX Extras
        Bob's Guides is now hosted at A2 MODX Hosting
        • 22629
        • 194 Posts
        Quote from: Perceive at Nov 15, 2010, 01:26 PM
        I’ve imported >16000 resourses. I have encountered the following problems:

        1) Resourse tree is not displayed until I increased the amount of memory on server from 64 to 128MB. Let’s think, it’s problem solved. But I don’t think CMF should use such big amount of memory.

        That’s not much at all. On another project I’m using Sitecore (a large .NET CMS) with a little over 1,000 resources and it’s taking up 170MB with no more than 20 visitors, and over 500MB with a few thousand visitors.

        I concur with BobRay, you need a custom database table to hold your products. I wouldn’t class product details (price, etc) as being content - it’s more data.
          Andy Shellam | www.networkmail.eu | @Pandy06269 @NetworkMail

          modx Revolution 2.2.6
          Windows 2012 | IIS 8 | php 5.4.11 | MySQL 5.5.29

          Content-Managed Websites Built on MODX
          • 23027
          • 7 Posts
          Quote from: Pandy06269 at Nov 16, 2010, 03:43 AM

          That’s not much at all. On another project I’m using Sitecore (a large .NET CMS) with a little over 1,000 resources and it’s taking up 170MB with no more than 20 visitors.
          Yes, it’s not much.. It’s very much). I’m think that this is a bad code, that needed 170MB per 20 users. I willn’t ever use Sitecore grin But not about this..

          You said that i should use custom tables. But there is very small amount of information about this in documentation and, especially, in examples. That’s why I have more questions:
          1) how i can implement system Friendly url’s functionality in my snippet, that uses custom tables for bilding big catalog?
          2) how i can edit my catalog’s item? CMP?
          3) can i use other system functionality, e.g. while editing item in my CMP i need call File Manager. Or in that way I have to write own file manager?
          4) can my custom tables participate in search on site?
          5) what about access permissions? How i can determine group of resourses (my catalog’s items) and access policies to this groups?
            • 22629
            • 194 Posts
            Apologies for my delay in replying - I’ve only just checked my previous posts smiley

            Quote from: Perceive at Nov 17, 2010, 05:18 AM

            Yes, it’s not much.. It’s very much). I’m think that this is a bad code, that needed 170MB per 20 users. I willn’t ever use Sitecore grin But not about this..

            The fact is Sitecore is a huge system with a lot of functionality (like modx.) We’ve hit similar issues with Sitecore - when treating "data" (e.g. a list of products) as separate resources it really struggles.

            Quote from: Perceive at Nov 17, 2010, 05:18 AM

            You said that i should use custom tables. But there is very small amount of information about this in documentation and, especially, in examples. That’s why I have more questions:

            I’ve implemented about 3 components in my modx installation which use custom tables - and I gleaned all my knowledge from the modx documentation, and found it to be really decent.

            Have you followed this tutorial? It goes through an example from start to finish and I followed it to the letter to create my custom tables - I had a few minor issues but mostly it was a snap.

            Quote from: Perceive at Nov 17, 2010, 05:18 AM

            1) how i can implement system Friendly url’s functionality in my snippet, that uses custom tables for bilding big catalog?

            However you wish. I’ve done it by creating friendly URLs from within my snippet and using Apache’s mod_rewrite to map them back to the file system, similar to how modx does.

            for example:

            <a href="[[~23]]/2">View product 2</a>

            Might create a URL like: http://www.example.com/viewproduct/2
            (resource 23 is your "view product" page, "2" is your product ID.)

            A rewrite rule could then catch this and rewrite it to http://www.example.com/viewproduct?productId=2:

            RewriteRule ^(/viewproduct)/[\d]+$ $1?productId=2 [DPI]


            (may not be perfectly exact but hopefully you get the idea.)

            The standard modx friendly URL rewrite rule would then also grab this and convert it to something like http://www.example.com/index.php?q=/viewproduct&productId=2

            Your snippet could then retrieve $_GET[’productId’] and use this and your model to look up the product from the database.

            Quote from: Perceive at Nov 17, 2010, 05:18 AM

            2) how i can edit my catalog’s item? CMP?

            Yes.

            Quote from: Perceive at Nov 17, 2010, 05:18 AM

            3) can i use other system functionality, e.g. while editing item in my CMP i need call File Manager. Or in that way I have to write own file manager?

            I’ve not done it myself but I’m sure you could. Perhaps create a separate thread to see how you can call the file manager from a 3rd party component - someone may already have done it, so search around as well.

            Quote from: Perceive at Nov 17, 2010, 05:18 AM

            4) can my custom tables participate in search on site?

            Depends which search engine you’re using. I’m using Sphinx Search which indexes directly from MySQL tables, so yes, this could also index your custom tables.

            Quote from: Perceive at Nov 17, 2010, 05:18 AM

            5) what about access permissions? How i can determine group of resourses (my catalog’s items) and access policies to this groups?

            The modx ecosystem is very flexible and re-usable (that’s the way it’s been designed.) Therefore, again, I’ve not done it myself but I’m sure it is possible to implement checking access policies against your own models.
              Andy Shellam | www.networkmail.eu | @Pandy06269 @NetworkMail

              modx Revolution 2.2.6
              Windows 2012 | IIS 8 | php 5.4.11 | MySQL 5.5.29

              Content-Managed Websites Built on MODX
              • 4172
              • 5,888 Posts
              1) how i can implement system Friendly url’s functionality in my snippet, that uses custom tables for bilding big catalog?
              you can make a plugin for onPageNotFound, see gallery-component
              2) how i can edit my catalog’s item? CMP?
              I made an example with xdbedit: https://github.com/Bruno17/xdbedit
              3) can i use other system functionality, e.g. while editing item in my CMP i need call File Manager. Or in that way I have to write own file manager?
              with xdbedit I can use the most TVs in my forms, also image-TV with filemanager or tinyMCE with filemanager.
              you can also create custom-TV-input-types and use them for custom-table-fields in xdbedit.
              you only need a config-file and say which fields you want to edit with which TV-input: https://github.com/Bruno17/xdbedit/blob/master/core/components/xdbedit/configs/example.config.inc.php
                -------------------------------

                you can buy me a beer, if you like MIGX

                http://webcmsolutions.de/migx.html

                Thanks!
                • 3749
                • 24,544 Posts
                You might also find this helpful: http://bobsguides.com/custom-db-tables.html
                  Did I help you? Buy me a beer
                  Get my Book: MODX:The Official Guide
                  MODX info for everyone: http://bobsguides.com/modx.html
                  My MODX Extras
                  Bob's Guides is now hosted at A2 MODX Hosting