We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 5091
    • 332 Posts
    Zi, Ryan or any admin team member, can you please look into this problem? When I post in Chinese,  and type the characters directly into message textarea, my Chinese texts look fine but as soon as I clicked ’post’, the characters turn to ’?’ and this " huh" expression icons. I am able to rescue it by copying the texts to my email client in plain text format, then copy back.

    Wonder if Japanese and Farsi users have the similar problem. Sometimes I see posts from the Modx RSS feed turns to ’?’ too.

    thanks!
    tgp
      • 22303 MODX Staff
      • 10,725 Posts
      Sounds like you do not have the proper character encoding set on the database. Make sure it is compatible with the character set of the content you are trying to save, and that you are presenting the web site with the proper character set setting, as well. The character set defined in the System Settings (modx_charset) is used to set the charset on the header of pages being served from your site, and must match the database encoding, as well as the manager language file encoding.
        • 25663 MODX Staff
        • 12,272 Posts
        He’s talking about in the message boards here. I’ve seen one of those posts and wondered what it was! Maybe a browser setting for character encoding? Perhaps we should convert the content here to UTF-8... Not sure...
          Ryan Thrash, MODX Co-Founder
          Follow me on Twitter at @rthrash or catch my occasional unofficial thoughts at thrash.me
          • 5091
          • 332 Posts
          Hi Ryan, thanks and yes I was talking about this forum.

          So it has nothing to do with the SMF? I am considering to use the SMF for a site that requires both Simplified and Traditional Chinese support, and the UTF-8 is the only way to go, therefor I really like to know how well the SMF supports is for unicode Chinese characters.

          My browser default is UTF-8.

          tgp

            • 25663 MODX Staff
            • 12,272 Posts
            I’ll need to run an upgrade script to convert the tables to UTF-8, which is available on the admin... I’ll need to do that in about 17 hours from now, when this forum is the least busy, and hope it doesn’t die!
              Ryan Thrash, MODX Co-Founder
              Follow me on Twitter at @rthrash or catch my occasional unofficial thoughts at thrash.me
              • 20289
              • 958 Posts
              ASCII only allows a set of 256 characters to be used for alphanumeric characters & note the size of byte 8 ones and zeros!, in 0-1 terminology, As long as Chineese, Korean, Japanese language have special characters means that you need more than 256 and therefore more than one byte of space- you need multibyte character unicode based encodings such as utf-8, dealing with multibyte characters is slightly different with normal ones, because functions like substr( ) and strtoupper( ) except preciesly one byte per character and will corrupt a multibyte string. instead you should use a multibyte equivalents of these functions, such as mb_strtoupper( ), mb_ereg_match( ) mb_strlen( ), here the parameters are the same as their original exept that most accept an optional extra parameter to force specific encoding....

              there is a special php.ini setting you can change : mbstring.func_overload. by default this is set to 0, wich means functions behave as you would expect, if you change it to 1 mail( ) function gets rerouted to the mb_send-mail( ) function, same when you set it to 2 all the functions starting with "str" gets rerouted for multibyte!... 4 all the "ereg" functions get rerouted. if you combine all ie:... 1(mail) + 2(str) + 4(ereg) ... set 7 to overload everything!. laugh
                [img]http://i10.tinypic.com/52c4eir.gif[/img][/td]
                [td][Wiki] [Persian support forum]
                [SVN] [RTL SVN Branch] [bugs] [FishEye+Crucible] [Learn MODx!] | [My Google Code]
                [font=tahoma][برای دسترسی به راهنمای فارسی به [url=http://www.modxcms.ir]
                • 25663 MODX Staff
                • 12,272 Posts
                The Japanese forums here are working fine though... so perhaps it is a local browser problem?
                  Ryan Thrash, MODX Co-Founder
                  Follow me on Twitter at @rthrash or catch my occasional unofficial thoughts at thrash.me
                  • 5091
                  • 332 Posts
                  Ryan, not sure though. I see the forum charaset is "ISO-8859-1". All my local settings are UTF-8 because I work on different languages everyday and really depend on it.
                  When I type Chinese directly in this forum, it looks fine but goes wrong after the message is posted, so my guess is because of the charaset the forum has, so my Chinese input system auto convert the text to ASCII instead of unicode . Apple Mail is a good UTF-8 converter, and my exerience with it is, any Chinese texts created there, always get the right charaset no matter what browser/program I paste the texts to.
                  tgp
                    • 25663 MODX Staff
                    • 12,272 Posts
                    Do you use Safari or Firefox as your primary browser for the forum? I’ve noticed that Safari has bugs on some of the admin functions with non-ASCII characters that Firefox does not. (Woohoo! Another Mac user! smiley )
                      Ryan Thrash, MODX Co-Founder
                      Follow me on Twitter at @rthrash or catch my occasional unofficial thoughts at thrash.me
                      • 5091
                      • 332 Posts
                      Mainly Safari.  I am not aware of Safari problem with admin fuctions, but yes, you probably is right. Just did a test with Fireforx and it shows up fine.

                      By the way, Firefox and other Gecko based browsers (maybe just for Mac version) actually have display problem with UTF-8 Chinese and this is the main reason I still use Safari as my default browser. Gotta change to Opera  smiley