We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
MODX Community Forums Archive

Answered How to display Japanese characters properly?

    • 4628
    • 58 Posts
    carlina Reply #1, 9 years, 9 months ago
    Hi! I want to build a MODX site (latest MODX version) in Japanese and German. Now I struggle with the Japanese characters. With the HTML-Templates I had no problems. I used <meta charset="UTF-8"> in the <head> and everything was fine. Not so in MODX. The MySQL connection collation is utf8_unicode_ci and the MySQL charset as well as the MODX charset is utf8. In the MODX editor (default Editor, no TinyMCE) are question marks every where and in the front end something like this: j%C%/$7$F$/$@$5$$!#(B

    I don't know if it's relevant but though I have the database settings above I found some collations set to latin1_swedish_ci i.e. in the modx_site_content table. I didn't do this.

    As it's the first time a have to deal with Japanese content I'm at a loss a little bit. Are there any online sources or tutorials about the use of the japanese language and the proper (MODX/ database) settings? I couldn't find something really helpful so far. Or do you have any other tips to solve this problem?

    Thanks and regards!

    This question has been answered by BobRay. See the first response.

    • discuss.answer
      • 3749
      • 24,544 Posts
      BobRay Reply #2, 9 years, 9 months ago
      I think utf8 would be the correct choice.

      http://stackoverflow.com/questions/1045338/which-is-the-best-character-encoding-for-japanese-language-for-db-php-and-html

      If you do have a mismatch, you should look at this: http://bobsguides.com/convert-db-utf8.html, but I don't know if the script provided there works on the current version of MODX. Be sure to back up the current DB before doing anything!
        Did I help you? Buy me a beer
        Get my Book: MODX:The Official Guide
        MODX info for everyone: http://bobsguides.com/modx.html
        My MODX Extras
        Bob's Guides is now hosted at A2 MODX Hosting
        • 44195
        • 293 Posts
        Murray Reply #3, 9 years, 9 months ago
        Hi,

        As I'm based in Hong Kong I deal with Chinese characters a lot and have faced similar problems. As Bob said above everything should be UTF8. (Either utf8_unicode_ci or utf8_general_ci).

        If there are collation mismatches you will likely get garbled characters displayed as they won't be stored correctly.
        (Actually the best collation to use in the future will be UTF8mb4 as some less common characters require it but MODX doesn't support that yet. I understand it'll be supported in version 3.0!)

        I've used Bob's converting script above and it works great.
          I'm lead developer at Digital Penguin Creative Studio in Hong Kong. https://www.digitalpenguin.hk
          Check out the MODX tutorial series on my blog at https://www.hkwebdeveloper.com
          • 3749
          • 24,544 Posts
          BobRay Reply #4, 9 years, 9 months ago
          @Murray - Do you remember which version of MODX you ran it for?
            Did I help you? Buy me a beer
            Get my Book: MODX:The Official Guide
            MODX info for everyone: http://bobsguides.com/modx.html
            My MODX Extras
            Bob's Guides is now hosted at A2 MODX Hosting
            • 44195
            • 293 Posts
            Murray Reply #5, 9 years, 9 months ago
            Quote from: BobRay at Dec 12, 2016, 11:00 PM
            @Murray - Do you remember which version of MODX you ran it for?

            Hmm I've used it a few times, probably the latest was version 2.3.x
            It shouldn't matter too much though as it's manipulating the database directly rather than via MODX isn't it? [ed. note: muzzstick last edited this post 9 years, 9 months ago.]
              I'm lead developer at Digital Penguin Creative Studio in Hong Kong. https://www.digitalpenguin.hk
              Check out the MODX tutorial series on my blog at https://www.hkwebdeveloper.com
              • 4628
              • 58 Posts
              carlina Reply #6, 9 years, 9 months ago
              Oh, wow, thank you very much, Bob! Your script is great and helped me to solve this character problem. And yes, it works perfectly with the current 2.5.2 MODX version smiley

              Thanks, Murray, for the hint with the missing characters. I'll keep that in mind. At least at the moment everything works well.
                • 3749
                • 24,544 Posts
                BobRay Reply #7, 9 years, 9 months ago
                @Carlina - I'm glad I could help. Thanks for reporting back.

                @murray -
                Hmm I've used it a few times, probably the latest was version 3.x
                It shouldn't matter too much though as it's manipulating the database directly rather than via MODX isn't it?

                Thanks. It is working on the DB, but the MODX DB has some complicated compound indexes which the script has to manually remove, then put back after the operation, or the script crashes. If new ones get created in a new version of MODX, I have to rewrite the script.
                  Did I help you? Buy me a beer
                  Get my Book: MODX:The Official Guide
                  MODX info for everyone: http://bobsguides.com/modx.html
                  My MODX Extras
                  Bob's Guides is now hosted at A2 MODX Hosting
                  • 44195
                  • 293 Posts
                  Murray Reply #8, 9 years, 9 months ago
                  Quote from: BobRay at Dec 13, 2016, 08:30 PM
                  @Carlina - I'm glad I could help. Thanks for reporting back.

                  @murray -
                  Hmm I've used it a few times, probably the latest was version 3.x
                  It shouldn't matter too much though as it's manipulating the database directly rather than via MODX isn't it?

                  Thanks. It is working on the DB, but the MODX DB has some complicated compound indexes which the script has to manually remove, then put back after the operation, or the script crashes. If new ones get created in a new version of MODX, I have to rewrite the script.

                  Sorry that should have been the latest version I tried it on was 2.3.x (edited)
                  Ah I see. Thanks I'll keep an eye on that for future versions.
                    I'm lead developer at Digital Penguin Creative Studio in Hong Kong. https://www.digitalpenguin.hk
                    Check out the MODX tutorial series on my blog at https://www.hkwebdeveloper.com