We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 33175
    • 711 Posts
    If Modx database is modified fot next realesed, I think it will be great to modify thinking query optimization (by optimizing database structure).
      Sorry for my english. I'm french... My dictionary is near me, but it's only a dictionary !
      • 22303 MODX Staff
      • 10,725 Posts
      Quote from: Guillaume at Mar 16, 2006, 07:04 PM

      If Modx database is modified fot next realesed, I think it will be great to modify thinking query optimization (by optimizing database structure).
      The database structures will definitely be changing. As will the way MODx developers interact with them. And as for the concern about query optimization, do not worry; I was a database administrator before I ever learned to code, and have a lot of experience developing and maintaining multi-lingual web applications for telemarketing. Database optimization and overall optimization of requests served by MODx, are definitely among my top priorities for this effort.
        • 32241
        • 1,495 Posts
        Could you give us a few shots on what’s the plan for the multi lingual site database Jason?
        I would love to know more about best practices and etc with this kind of problem. I did built a multi lingual site before, but I won’t say that that’s the best approach out there, so if you can give me an input on it, it might help me to understand a better solution to this problem. I like learning a new stuff, even though I won’t be able to get indepth into it, but by knowing the big picture will give me satistaction smiley
          Wendy Novianto
          [font=Verdana]PT DJAMOER Technology Media
          [font=Verdana]Xituz Media
          • 33175
          • 711 Posts
          Quote from: OpenGeek at Mar 16, 2006, 07:14 PM

          I was a database administrator before I ever learned to code, and have a lot of experience developing and maintaining multi-lingual web applications for telemarketing. Database optimization and overall optimization of requests served by MODx, are definitely among my top priorities for this effort.
          Cool !! Great news !!

            Sorry for my english. I'm french... My dictionary is near me, but it's only a dictionary !
            • 33175
            • 711 Posts
            Quote from: Djamoer at Mar 16, 2006, 07:00 PM
            If we need to replicate each chunk, site content, template variables, and templates, soon or later we will have a lot of tables filled up in the database. But considering that we are going to have only 2-4 translations at one site, so we need to have 8-16 tables to support the translation, compare to 4 tables, when we are not replicating the tables.
            For multi language I think language files are required. So, it is not too difficult to maintain and it permits to not replicate all tables. Datas should be duplicate be only chunk and site content.
            Templates can be writed only with snippets and chuncks : there is no necessary to duplicate each.
            I think a great functionnality is chunck can contain language variables which will be replace by a core modx function from language files. For translated a chunck, we will not need to duplicate chunck but only add a new entry in a language file. With cache, I think Modx uses not more ressources of server.
            Quote from: Djamoer at Mar 16, 2006, 07:00 PM

            Now lets talk about the amount of rows needed. Lets say that each site usually contain 200 site contents, 5-15 template variables, which sum up as 1000-3000 rows, 1-5 templates, and 100-150 chunks. So if we want to duplicate into 4 languages without replicating table, here is the number of rows needed.
            site contents: 800 rows
            template variables: 4000-12000 rows
            templates: 4-20 rows
            chunks: 400-600 rows
            It is enormous, it is right. With my previous proposition I think there will be much less duplicated rows.

            All tables need not to be duplicated and it’s the same thing for all field.
            For example, the table "site_content" can be splitted nearly like bs said
            I propose that :

            [table]
            [tr][td]altered: modx_site_content[/td][td] [/td][td]new: modx_site_content_[lang][/td][/tr]
            [tr][td]docid[/td][td] [/td][td]docid[/td][/tr]
            [tr][td]type[/td][td] [/td][td]published[/td][/tr]
            [tr][td]contentType[/td][td] [/td][td]pub_date[/td][/tr]
            [tr][td]parent[/td][td] [/td][td]unpub_date[/td][/tr]
            [tr][td]isfolder[/td][td] [/td][td]createdby[/td][/tr]
            [tr][td]richtext[/td][td] [/td][td]createdon[/td][/tr]
            [tr][td]template[/td][td] [/td][td]editedby[/td][/tr]
            [tr][td]menuindex[/td][td] [/td][td]editedon[/td][/tr]
            [tr][td]searchable[/td][td] [/td][td]deleted[/td][/tr]
            [tr][td]cacheable[/td][td] [/td][td]deletedby[/td][/tr]
            [tr][td]donthit[/td][td] [/td][td]haskeywords[/td][/tr]
            [tr][td]privateweb[/td][td] [/td][td]hasmetatags[/td][/tr]
            [tr][td]privatemgr[/td][td] [/td][td]pagetitle[/td][/tr]
            [tr][td]content_dispo[/td][td] [/td][td]longtitle[/td][/tr]
            [tr][td]hidemenu[/td][td] [/td][td]description[/td][/tr]
            [tr][td][/td][td] [/td][td]alias[/td][/tr]
            [tr][td][/td][td] [/td][td]introtext[/td][/tr]
            [tr][td][/td][td] [/td][td]content[/td][/tr]
            [tr][td][/td][td] [/td][td]menutitle[/td][/tr]
            [/table]

            If you need to find a tractioon on only one language you can use a request with UNION.
            If you need to list all available translated, another table is required to optimize :

            [table]
            [tr][td]content_language[/td][/tr]
            [tr][td]docid[/td][/tr]
            [tr][td]langid[/td][/tr]
            [/table]

            A little request as this is very fast :
            "SELECT COUNT(langid) FROM content_language WHERE docid=5"


            I knox, I know, I’m very headstrong grin
              Sorry for my english. I'm french... My dictionary is near me, but it's only a dictionary !
              • 17895
              • 209 Posts
              Well then,
              if Jason says that the whole database structure should change, I think that here, the best thing to do (unless you core coders decide to take the multi table solution) is to have the content primary id as (id, lang), so that we can get rid of the refId field and all that this means for snippets and modules and so on...

              everything should become more easy...

              I do not know what this will result in the whole php code... should I try on my own?

              the multi-table solution is query-quicker, guillame is right (the problem with queries is the table size, not the whole database size)... but there is not only the content table to duplicate, there are also TV, metatags, and so on... while with "lang field solution" we have only a new field in these tables: it’s much easier to mantain!

              for queries, I cannot figure out now how we can easily (i.e. with one query) filter the content in order to have all pages in the language preferred and, when these are lacking, the default language counterpart

              I think that finding a way in which we can have that query, will solve if not all, a large part of the problems

              for jason: I am always around here.. ;-)... do you know a way to have day of 30 hours? ;-)
                Daniele "MadMage" Calisi
                • 22303 MODX Staff
                • 10,725 Posts
                Quote from: madmage at Mar 18, 2006, 06:44 AM

                I do not know what this will result in the whole php code... should I try on my own?

                the multi-table solution is query-quicker, guillame is right (the problem with queries is the table size, not the whole database size)... but there is not only the content table to duplicate, there are also TV, metatags, and so on... while with "lang field solution" we have only a new field in these tables: it’s much easier to mantain!

                for queries, I cannot figure out now how we can easily (i.e. with one query) filter the content in order to have all pages in the language preferred and, when these are lacking, the default language counterpart

                I think that finding a way in which we can have that query, will solve if not all, a large part of the problems
                I would hold off if you can on coding this yourself; I already have the data structure and new parser design to solve this implemented in PHP 5 (part of the development cycle of Tattoo), which I am in the process of porting back down to PHP 4 so MODx can utilize it. It is also going to greatly simplify the core code and API’s in the process, hopefully, further improving MODx page rendering performance.

                Quote from: madmage at Mar 18, 2006, 06:44 AM

                for jason: I am always around here.. ;-)... do you know a way to have day of 30 hours? ;-)
                On my roadmap already grin
                  • 6726
                  • 7,075 Posts
                  Quote from: OpenGeek at Mar 18, 2006, 02:10 PM
                  (...) I am in the process of porting back down to PHP 4 so MODx can utilize it. It is also going to greatly simplify the core code and API’s in the process, hopefully, further improving MODx page rendering performance.

                  Awesome !
                  That’s GREAT news laugh

                  Quote from: OpenGeek
                  Quote from: madmage
                  for jason: I am always around here.. ;-)... do you know a way to have day of 30 hours? ;-)
                  On my roadmap already grin

                  I’ll buy it !!!

                  LOL
                    .: COO - Commerce Guys - Community Driven Innovation :.


                    MODx est l'outil id
                    • 17895
                    • 209 Posts
                    Quote from: OpenGeek at Mar 18, 2006, 02:10 PM

                    I would hold off if you can on coding this yourself; I already have the data structure and new parser design to solve this implemented in PHP 5 (part of the development cycle of Tattoo), which I am in the process of porting back down to PHP 4 so MODx can utilize it. It is also going to greatly simplify the core code and API’s in the process, hopefully, further improving MODx page rendering performance.

                    ahem... it’s not so clear to me (damned babel of languages!)... do you have some big modification (data structure and new parser to solve WHAT?) that I should know before trying to close this issue? i.e. you WILL modify the MODx porting back Tattoo stuff or you are not sure about this?
                      Daniele "MadMage" Calisi
                      • 22303 MODX Staff
                      • 10,725 Posts
                      Quote from: madmage at Mar 20, 2006, 03:36 PM

                      ahem... it’s not so clear to me (damned babel of languages!)... do you have some big modification (data structure and new parser to solve WHAT?) that I should know before trying to close this issue?
                      Yes madmage, I am about 25% complete with an effort to re-organize the entire MODx core for 1.0 release (i.e. big modification), and it will support multi-cultural content, content revisioning (i.e. track changes made to content and be able to rollback to a previous version), subsites, and will feature an object-oriented (OO) API that will virtually eliminate the need to write SQL queries against the MODx database.
                      Quote from: madmage at Mar 20, 2006, 03:36 PM

                      i.e. you WILL modify the MODx porting back Tattoo stuff or you are not sure about this?
                      And yes, I am sure of this --> many of the database changes I am making are based on my existing prototypes for Tattoo. A little more work on the new PDO-based database layer for MODx, and I will show you the upcoming structure changes in detail.