I get '?' mark instead of 'ő' and 'ű' characters (Hungarian)

hi guys

i have a minor problem. I say minor since I have two installations of ModxCMS on my local server.
I am building a Hungarian version website and everything goes pretty well, just finished configuring the blog modules and so on...

Still, it looks like wherever the content (Page Title or body content)contains ’ő’ or ’ű’ characters, when previwed in the browser, a ’?’ mark appears.

I’m sure it is not a bug since on my other Modx site, these characters show correctly. I noticed this problem only with these two characters, other similar ones like ’á’, ’é’, ’ü’ or ’ö’ are OK.

And here are my settings on the Modx where the above mentioned two characters appear as ’?’ mark:
MODx 2.0.5 - pl ver
In System settings Character Encoding set to UTF-8
MySQL db collation set to: utf8_hungarian_ci

The header part:


Strangely, on my other MODx installation on which the MODx sample site is installed, everything is the default ie no hungarian collation, no hu lang set .... still, all Hungarian characters show correctly.

I would appreciate if someone could give me a tip what causes this ’?’ mark and what I should reconfigure to make my characters show correctly.

Thanx in advance.

Check the actual collation of your database tables. MODx will create the tables with whatever charset/collation you set when you create the database. If you let MODx create the database, it will set the charset/collation according to your selection during setup, but otherwise, you could possibly have a database with the default MySQL collation of latin1_swedish_ci.

Hi OpenGeek

As I have already mentioned in my post, all table’s collation are set to ’utf8_hungarian_ci’
On my other MODx installation on which the characters show correctly, all tables are set ’utf8_general_ci’, so basically colation shouldn’t be a problem

I can’t figure out what else should I check

Make sure the text field inside the tables are also set that way.

hi BobRay

I think you are right.
Initially I used a PHP script which automatically changed all my tables’ collation to ’utf8_hungarian_ci’ but didn’t do the same with all fields within the tables.
I don’t know exactly how all fields within all tables could be changed to UTF ot whatever other encoding. Is it a step-by-step process or can it be done automatically for all fields in all tables?

Now I’m almost sure this is the problem as when I created the Hungarian version of all lexicon components (in a new folder named ’hu’), all files were in ANSI and had to change them to UTF in order to be able writing Hungarian characters.

Now I checked the fields within my tables and you are right. All of them are set to ’latin1_swedish_ci’ and on my other MODx site all are set to ’utf8_general_ci’.

Now, it remains the solution to solve it automatically. If you have one, I would appreciate sending me over. In the meantime I’m Googleing for a quick solution to convert all fields within all tables automatically.

Anyway, lots of thanks BobRay

You can use your domain’s MySQL client (usually phpMyAdmin) to set the default collation to whatever you need, so that all subsequent entries will be in the correct charset, but there’s no easy way to convert data already entered.

thanx ’sittwell’ for the tip

indeed, i use phpMyAdmin but your suggestions relates mostly when entering new data into the db
there is no any worry as there isn’t any valuable content on my website, i’m just making all functionalities to be OK and only afterwards the content

so, basically now i need an automatic php script which changes all fields collation to utf8 and i think, I already found one which at first glance looks to be exactly what i need

eventually you guys could take a look on it tinyurl.com/44wmfo and have your opinion

i used something similar, much shorter which changed only tables’ collation and worked pretty well

of course, i will make a backup of all my fields prior anything

thanx a lot, you are awesome

How about this: http://bobsguides.com/convert-db-utf8.html ?

Hi BobRay

Well, first of all thanx for the script you sent me. Without you, I wouldn’t have managed to solve this issue. I admit, I sweat like a lion under the heat sun when at first attempt all my pages looked very weird and lot of parts were missing.

But no worries! After all I managed to solve this problem however I think I should share my story with others who might face similar issue with converting characters of already created db.

The script I sent you, I think was also ok, however I wasn’t smart enough to realise the generated SQL queries should be copy pasted in phpMySQL and hit the Go button so I started with yours in which the step by step guideline says the generated queries should be copy paste into phpMySQL and so on...

Since my templates and chunks already contained Hungarian characters, after conversion, pages didn’t load correctly and this was because starting from first Hungarian character in any template or chunk, all that was afterwards, was cut off

so, I deleted the db and imported the backup then copied in an HTML file all my chunks and templates then did the process again and recopied all templ and chunks in th CMS.

So, after all this struggle I managed to solve this, my characters show correctly, i’m happy of course and thanx for all your supports ....

sherlock

I’m glad it helped. Thanks for reporting back.