We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 3749
    • 24,544 Posts
    Now that I’m actually writing about Lexicon Management, I find I’m not sure what a Lexicon actually is. undecided

    At one point, I implied that there was a Lexicon for each language/culture but the more I think about it, that doesn’t seem right because, in that case, we wouldn’t need the term "Lexicon."

    This is where I am now:

    A Lexicon is a MODx object into which you can load a set of language strings, divided into various namespaces and topics. Once loaded, the language strings can be retrieved by their keys and displayed in the Manager or on pages in the front end.

    That would mean there’s only one Lexicon in a typical MODx install, but it can contain one or more languages/cultures.

    Does that sound right?

    BTW, are we switching "Language" to "Culture" or have we decided against it?

    Also, if it’s not too late, "en-gb" should probably be "en-uk," "en" should probably be "en-us," and something like "en-ca" should be added for Canada (I don’t think we need "en-li" for Liberia yet). wink
      Did I help you? Buy me a beer
      Get my Book: MODX:The Official Guide
      MODX info for everyone: http://bobsguides.com/modx.html
      My MODX Extras
      Bob's Guides is now hosted at A2 MODX Hosting
      • 22303 MODX Staff
      • 10,725 Posts
      The Lexicon is the "dictionary" of loaded strings in whatever namespaces, topics, and language( s ) they were loaded from. Typically, in a single request, this lexicon will be loaded from a single culture indicated at the beginning of the request, though it is possible to load strings from various cultures as needed, on demand.

      Culture can be more specific than language, combining language, region, and/or script information; it can be used to determine the appropriate lexicon entries to load from as well as to make other important internationalization decisions within a request.

      Ultimately, en and en-us (or en-uk, etc) will coexist, with overrides coming from the more specific culture indication and falling back to the less specific. This way, more specific cultures with a few differences will only have to provide overrides for specific lexicon keys rather than providing a full translation. These strings should based on the official IANA codes as used in the language-accept headers and HTML/XML language tags. This is not yet implemented, but is definitely what my vision was intending with these features.

      IMO, it should be "Culture", as the region and/or script information is not really conveying just Language, but potentially something more specific.
        • 3749
        • 24,544 Posts
        Quote from: OpenGeek at Aug 23, 2009, 11:04 AM

        The Lexicon is the "dictionary" of loaded strings in whatever namespaces, topics, and language( s ) they were loaded from. Typically, in a single request, this lexicon will be loaded from a single culture indicated at the beginning of the request, though it is possible to load strings from various cultures as needed, on demand.

        I think that’s consistent with my definition above. Let me know if you disagree.


        IMO, it should be "Culture", as the region and/or script information is not really conveying just Language, but potentially something more specific.

        I don’t have strong feelings either way. "Language" is a more familiar term and, IMO, easier for new users but, as you say, somewhat inaccurate. I thnk, technically speaking, that en-uk and en-us are "forms" of the English language (called "American English" and "British English"). American English includes all "dialects" used in the US.

        OTOH, francais and francais-utf8 aren’t really different "Cultures." So there’s no perfect solution for us. I think it’s common to see "What language is it written in?" answered with "American English" so I’d probably lean toward using "Language," especially since "Culture" includes so much other stuff that has little or nothing to do with Lexicons or MODx.


        My main concern is that if we’re going to change to "Culture," the sooner we change it in the Manager, the better.
          Did I help you? Buy me a beer
          Get my Book: MODX:The Official Guide
          MODX info for everyone: http://bobsguides.com/modx.html
          My MODX Extras
          Bob's Guides is now hosted at A2 MODX Hosting
          • 25663 MODX Staff
          • 12,272 Posts
          How do the IANA folks and other RFC-producing bodies refer to such topics? I love the idea of standardizing on certain naming conventions like pt-pt and pt-br for the actual language files themselves. From one of the such RFC-producting bodies: http://tools.ietf.org/html/draft-ietf-ltru-registry-14

          Related: http://svn.modxcms.com/jira/browse/MODX-391

            Ryan Thrash, MODX Co-Founder
            Follow me on Twitter at @rthrash or catch my occasional unofficial thoughts at thrash.me
            • 22303 MODX Staff
            • 10,725 Posts
            Quote from: BobRay at Aug 23, 2009, 07:47 PM


            IMO, it should be "Culture", as the region and/or script information is not really conveying just Language, but potentially something more specific.

            I don’t have strong feelings either way. "Language" is a more familiar term and, IMO, easier for new users but, as you say, somewhat inaccurate. I thnk, technically speaking, that en-uk and en-us are "forms" of the English language (called "American English" and "British English"). American English includes all "dialects" used in the US.

            OTOH, francais and francais-utf8 aren’t really different "Cultures." So there’s no perfect solution for us. I think it’s common to see "What language is it written in?" answered with "American English" so I’d probably lean toward using "Language," especially since "Culture" includes so much other stuff that has little or nothing to do with Lexicons or MODx.


            My main concern is that if we’re going to change to "Culture," the sooner we change it in the Manager, the better.
            First, there will be no francais and francais-utf8 in Revo; Revo only supports utf8 encodings at this time and likely will for final release as well. This may change in the future releases, but no guarantees...

            Second, defining culture for internationalization conveys more than language, it conveys localization information as well, for formatting currency, date strings, and even behavior of a site. This is above and beyond what the language indicated for lexicon strings is, though it may also affect those (thus my discussion of culture-specific overrides to languages, i.e. fr might be a base language being used for fr-CA, with overrides only in the more specific lexicon strings defined for fr-CA.

            IMO, the culture is what is indicated by a request to indicate the context information about the client request related to i18n. The language partitions the specific set of lexicon strings defined for a culture or part of a culture (again fr and fr-CA would both apply for fr-CA requests). I don’t see any incompatibility between using the MODx "Culture" indication in conjunction with defined "Languages" for MODx Lexicons. But if you want to call it Languages all the way around in the manager, I don’t have a problem with that; as long as the developers know the code refers to it as Culture. Personally, I think the Culture/Language help differentiate the parts of the core involved in the i18n features of Revo.
              • 3749
              • 24,544 Posts
              Quote from: OpenGeek at Aug 24, 2009, 09:23 AM

              But if you want to call it Languages all the way around in the manager, I don’t have a problem with that; as long as the developers know the code refers to it as Culture. Personally, I think the Culture/Language help differentiate the parts of the core involved in the i18n features of Revo.

              I think using "language" everywhere there could be a naive user around would be best.

              I dont think it’s too much of a stretch to say that the date and time formats are part of the language (along with ltr, rtl and other differences). It will be a lot easier to explain that to developers than to make it clear to newbies what we mean by "culture" since they think of that word as including religion, food, art, daily customs, music, etc.

              [EDIT] I just re-read your message and want to make sure we’re talking about the same things. I was primarily referring to the "Language" term on the lexicon screens. I wouldn’t want that changed to "culture."

              When you talk about "culture," (as distinct from lexicon language) how is that specified by the user/developer?
                Did I help you? Buy me a beer
                Get my Book: MODX:The Official Guide
                MODX info for everyone: http://bobsguides.com/modx.html
                My MODX Extras
                Bob's Guides is now hosted at A2 MODX Hosting
                • 22303 MODX Staff
                • 10,725 Posts
                Quote from: BobRay at Aug 24, 2009, 02:53 PM

                I dont think it’s too much of a stretch to say that the date and time formats are part of the language (along with ltr, rtl and other differences). It will be a lot easier to explain that to developers than to make it clear to newbies what we mean by "culture" since they think of that word as including religion, food, art, daily customs, music, etc.
                Culture does include all those things, which can all be involved in i18n, above and beyond language determination.

                Quote from: BobRay at Aug 24, 2009, 02:53 PM

                [EDIT] I just re-read your message and want to make sure we’re talking about the same things. I was primarily referring to the "Language" term on the lexicon screens. I wouldn’t want that changed to "culture."

                When you talk about "culture," (as distinct from lexicon language) how is that specified by the user/developer?
                Right, Language fits in the Lexicon, but I’m talking about the part of initialization of the modX instance that includes execution of the event OnInitCulture, which would be used (via plugins) to auto-determine a language based on client browser headers, user preferences, or any other way developers come up with. By setting the $modx->cultureKey in this event, everything in the request now uses this cultureKey to determine what Lexicon language to load, and could be used to make any other cultural-specific decisions within that request.

                BTW, I don’t make up these terms, they are standard terms for discussing i18n and used extensively by the W3C. I always tend towards standards and not restricting comprehension of MODx services by dumbing down terminology for the sake of inexperienced users; more technical users/developers would then be confused IMO. Ultimately it should be up to add-on developers to make configuration and usage easy for less savvy users.
                  • 3749
                  • 24,544 Posts
                  Ah. I wasn’t aware of the OnInitCulture event. I think we’re in agreement here too. With few exceptions, I also object to "dumbing down" terminology, but some tech-related language is *unnecessarily* arcane (eg. "foci" versus "topics"). That said, in many cases where I’ve suspected MODx of that, I’ve been wrong (like now). tongue

                  I do still have a couple of lexicon-related issues. One is about namespaces (I’ll start a new topic for that).

                  The other is the use of "lexicon" in the manager. My feeling is that the term "lexicon" should be reserved for the service (can I call it an object?) created with $this->getService(’lexicon’,’modLexicon’); in the modx class (ironically, in the _initCulture() method). I would say that language strings are loaded into the Lexicon with lexicon->load() and retrieved from the Lexicon via language tags or $modx->lexicon(’key’).

                  An alternative would be to say that the lexicon_entries table is the lexicon, but I think that would be misleading because the table fields aren’t available with $modx->lexicon(’key’) until they are loaded.

                  In the manager and some docs, however, lexicon is sometimes used for any collection of langage-related key=>value pairs. In "Import Lexicon," for example, it’s used to refer to a topic file, which I think is quite misleading.

                  The manager also refers to "lexicon strings" in several places, which I can live with but would rather call "language strings" because they exist outside the lexicon until they are loaded.

                  I’d also like things a lot better if there were a simple, automatic mapping of Lexicon stuff to physical files, which there definitely isn’t now. That way, MODx could look in the files if a language key wasn’t found in the DB and you could easily back up, restore, and update the lexicon_entries table without any manual intervention.
                    Did I help you? Buy me a beer
                    Get my Book: MODX:The Official Guide
                    MODX info for everyone: http://bobsguides.com/modx.html
                    My MODX Extras
                    Bob's Guides is now hosted at A2 MODX Hosting
                    • 28215
                    • 4,149 Posts
                    Quote from: BobRay at Aug 24, 2009, 05:36 PM

                    The other is the use of "lexicon" in the manager. My feeling is that the term "lexicon" should be reserved for the service (can I call it an object?)
                    It’s an object loaded as a Service. tongue


                    In the manager and some docs, however, lexicon is sometimes used for any collection of langage-related key=>value pairs. In "Import Lexicon," for example, it’s used to refer to a topic file, which I think is quite misleading.
                    I think you’re right; we have been kind of haphazard with the terminology. Lexicon should refer to the aggregate collection of Lexicon Topics and Entries loaded. I’ll try and iron this out when I get home.

                    The manager also refers to "lexicon strings" in several places, which I can live with but would rather call "language strings" because they exist outside the lexicon until they are loaded.
                    Ack, it does? We’ll try and fix that too. tongue


                    I’d also like things a lot better if there were a simple, automatic mapping of Lexicon stuff to physical files, which there definitely isn’t now. That way, MODx could look in the files if a language key wasn’t found in the DB and you could easily back up, restore, and update the lexicon_entries table without any manual intervention.

                    Hm. How would that work? The idea of having it in a DB is so it can be editable; making it file-based removes that functionality (see: Trying to change a language string without hacking core in Evo).

                    So are you suggesting that if a lang string isn’t found, that it should search the corresponding file? Should it do this every time it doesn’t find a string? This might cause some latency...hm. What do you think?
                      shaun mccormick | bigcommerce mgr of software engineering, former modx co-architect | github | splittingred.com
                      • 3749
                      • 24,544 Posts
                      Quote from: splittingred at Aug 26, 2009, 11:25 PM


                      I’d also like things a lot better if there were a simple, automatic mapping of Lexicon stuff to physical files, which there definitely isn’t now. That way, MODx could look in the files if a language key wasn’t found in the DB and you could easily back up, restore, and update the lexicon_entries table without any manual intervention.

                      Hm. How would that work? The idea of having it in a DB is so it can be editable; making it file-based removes that functionality (see: Trying to change a language string without hacking core in Evo).

                      So are you suggesting that if a lang string isn’t found, that it should search the corresponding file? Should it do this every time it doesn’t find a string? This might cause some latency...hm. What do you think?

                      Yes, that’s what I was envisioning, although I’m not sure it’s actually possible unless language tags can also specify the language (or manager_language is used -- but that has its own problems). I guess whether MODx searched for the corresponding file automatically and whether it logged that event could be two system settings.

                      Whether we do that or not, I’d like to see a standard place for language files. I don’t think it’s an unreasonable imposition on the developer since it shouldn’t create any significant performance penalty and it would allow a lot of options down the road that aren’t possible with language files in unpredictable places.

                      I guess it could all be below the /core, but I think I’d rather see a main /lexicon directory -- something like this unless there are security issues that make it desireable to put it under the core:

                      /lexicon  /* or core/lexicon/ */
                         /core
                            /en
                               topic1.php
                               topic2.php
                            /fr
                               topic1.php
                               topic2.php
                         /namespace2
                            /en
                               topic1.php
                               topic2.php
                            /fr
                               topic1.php
                               topic2.php
                         /namespace3
                            /en
                               topic1.php
                               topic2.php
                            /fr
                               topic1.php
                      
                        
                         /country
                            en.inc.php
                            fr.inc.php



                        Did I help you? Buy me a beer
                        Get my Book: MODX:The Official Guide
                        MODX info for everyone: http://bobsguides.com/modx.html
                        My MODX Extras
                        Bob's Guides is now hosted at A2 MODX Hosting