MODX Staff
If by smart quotes you mean Windows extended ASCII characters, these are not valid utf8 characters and will not translate properly from MySQL. If you use utf8 and set names, you should be fine, but you’ll need to replace all those invalid characters. Otherwise, you need to start with a clean database in latin1 and use latin1 all the way around so you can have your extended ASCII characters.
I have a problem that seems related to character sets, although I can’t figure out exactly how.
I’m pulling data from a UTF-8 database (modx is also set to utf8) on the same server as the modx database. When I pull and display the data through a modx page, all smart quotes are displayed as unknown characters (�). If I view the source text, however, all of the characters are present and displaying correctly.
If I enter smart quotes directly into a modx page via the text editor, they display correctly. If I make the database call from a straight PHP page (not parsed through modx) they also display correctly, albeit only if I specify ’latin1’ or ’utf8_unicode_ci’ as the collation. (Using the utf8 collation returns unknown characters for smart quotes. )
I’m using MySQL version 5.1.30 (charset utf8, connection collation: utf8_unicode_ci) and Modx version 0.9.6.2. Modx manager shows the modx charset as utf8 and seems to have no option for switching to latin1 (although latin3 is present). I’ve tried modifying manager/includes/config.inc.php to use latin1 and SET NAMES for the database connection, but that seems to have no effect.
these are not valid utf8 characters and will not translate properly from MySQL. If you use utf8 and set names, you should be fine, but you’ll need to replace all those invalid characters.
MODX Staff
You said your database was utf8; you simply cannot store those characters in a utf8 database and pull them out properly. Your database needs to be latin1 and you front-end needs to use ISO-8859-1 to display those characters properly. Also, it’s not just the database, but the individual table columns’ collations settings that matter, and if those don’t match your database, you are going to further confuse the real issue. Without actually seeing all of your database, php, and modx settings, it’s very difficult to diagnose these problems accurately.
Yes, I’m referring to extended ASCII characters.
these are not valid utf8 characters and will not translate properly from MySQL. If you use utf8 and set names, you should be fine, but you’ll need to replace all those invalid characters.
Let me make sure I’ve understood correctly: You’re saying that if I use utf8 and set names, the characters will come through the connection, but will need to be converted before they’ll display correctly? Are you referring to a character-by-character conversion in PHP?
Also, if the non-utf8 characters will not translate properly from MySQL, why do they display correctly when I view the source code from my browser?
I don’t know what you mean by source code from browser. Using what application?