Invalid UTF-8 sequence in argument?

87 Posts

graves007 Reply #1, 14 years, 1 month ago

So I migrated a modx install to a new server and everything seems to be working OK except for this one thing.

When I access resources in the manager whose content has any special character(s), the content doesn't display in tinymce. The tinymce window is blank. The content will display if I visit the page in a browser except the special character is replaced by weird characters. This only happens on the new server.

Whenever I access a page with special characters, an error is logged:
PHP warning: json_encode(): Invalid UTF-8 sequence in argument

These special characters display fine on our old server:
http://www.meta-wave.com/goodwoodfunds-casestudy

But on new server, you can see replacement chars:
http://tempmetawave.meta-wave.com/goodwoodfunds-casestudy

I can fix it by opening up the database and fixing directly there, but how do I prevent it from happening in the first place? It's not like adding content with fancy quotes is uncommon.

Specs:
revo 2.2.0
MySQL 5.1 (latin1 charset)
PHP 5.3
Win2k8 r2

[ed. note: graves007 last edited this post 14 years, 1 month ago.]

87 Posts

graves007 Reply #2, 14 years, 1 month ago

So I changed PHP version to 5.4. MODx error log has piles of this:

[2012-08-22 16:12:00] (ERROR @ C:\inetpub\wwwroot\metawave\core\xpdo\xpdo.class.php : 2285) PHP warning: json_encode(): Invalid UTF-8 sequence in argument

PHP error log has many many many instances of this:

[23-Aug-2012 14:32:15 UTC] PHP Notice: Array to string conversion in C:\inetpub\wwwroot\metawave\core\xpdo\xpdo.class.php on line 575

Is this an issue with MODx or IIS7? Any help greatly appreciated

MODX Staff
10,725 Posts

opengeek Reply #3, 14 years, 1 month ago

I would first upgrade to the latest Revo, which addresses the array to string conversion errors.

The json_encode issue is because you are using a latin1 encoding for your database. I recommend always using UTF8 with Revolution since it depends heavily on JSON, and JSON only supports UTF-8 encodings.

Jason Coward
Chief Architect @ MODX
http://www.jasoncoward.com | http://twitter.com/drumshaman | https://github.com/opengeek

☆ A M B ☆
24,524 Posts

sottwell Reply #4, 14 years, 1 month ago

According to the PHP documentation on json_encode()

This function only works with UTF-8 encoded data.

Since your database is latin 1, it's not going to be happy.

Studying MODX in the desert - http://sottwell.com
Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
Join the Slack Community - http://modx.org

87 Posts

graves007 Reply #5, 14 years, 1 month ago

Awesome thanks. Seeing as this seems pretty important, perhaps the Revo documentation should reflect this, as part of MySQL requirements? Thanks again.

87 Posts

graves007 Reply #6, 14 years, 1 month ago

Well updating is a disaster, although Im getting used to updates breaking everything. Hasnt gone smooth once since Ive been using this beast. Anyway thats my scorching frustration speaking so I digress.

When I try to access Resources...I get a white screen, sometimes...or I get this error...sometimes:

500 - Internal server error.
There is a problem with the resource you are looking for, and it cannot be displayed.

Or MODx manager interface gets all squirrly, sometimes, and loads my 404 resource in a ajax popup. Or if I keep refreshing the Manager shows up...only to repeat the above crap.

MODx error log is blank. PHP error log is filled with this:

[23-Aug-2012 21:29:02 UTC] PHP Notice:  Undefined property: TinyMCE::$context in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29
[23-Aug-2012 21:29:02 UTC] PHP Fatal error:  Call to a member function getOption() on a non-object in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29
[23-Aug-2012 21:29:06 UTC] PHP Notice:  Undefined property: TinyMCE::$context in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29
[23-Aug-2012 21:29:06 UTC] PHP Fatal error:  Call to a member function getOption() on a non-object in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29
[23-Aug-2012 21:29:50 UTC] PHP Warning:  json_encode(): Invalid UTF-8 sequence in argument in C:\inetpub\wwwroot\metawave\core\xpdo\xpdo.class.php on line 2327
[23-Aug-2012 21:29:50 UTC] PHP Notice:  Undefined property: TinyMCE::$context in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29
[23-Aug-2012 21:29:50 UTC] PHP Fatal error:  Call to a member function getOption() on a non-object in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29
[23-Aug-2012 21:29:53 UTC] PHP Warning:  json_encode(): Invalid UTF-8 sequence in argument in C:\inetpub\wwwroot\metawave\core\xpdo\xpdo.class.php on line 2327
[23-Aug-2012 21:29:53 UTC] PHP Notice:  Undefined property: TinyMCE::$context in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29
[23-Aug-2012 21:29:53 UTC] PHP Fatal error:  Call to a member function getOption() on a non-object in C:\inetpub\wwwroot\metawave\core\cache\includes\elements\modplugin\3.include.cache.php on line 29

Accessing any other screen in the Manager seems to be fine...so far. Just when accessing Resources.

Help. please.

[ed. note: graves007 last edited this post 14 years, 1 month ago.]

☆ A M B ☆
24,524 Posts

sottwell Reply #7, 14 years, 1 month ago

Looks like you need to manually delete all of the folders inside of the core/cache folder. MODx will re-generate them properly (one hopes, anyway)

Studying MODX in the desert - http://sottwell.com
Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
Join the Slack Community - http://modx.org

87 Posts

graves007 Reply #8, 14 years, 1 month ago

Thanks Susan, but no go. Just a white screen...and more of that strange inconsistent behaviour as above sad

[ed. note: graves007 last edited this post 14 years, 1 month ago.]

24,544 Posts

BobRay Reply #9, 14 years, 1 month ago

You might find this useful: http://bobsguides.com/convert-db-utf8.html.

------------------------------------------------------------------------------------------
PLEASE, PLEASE specify the version of MODX you are using.
MODX info for everyone: http://bobsguides.com/modx.html

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting

87 Posts

graves007 Reply #10, 14 years, 1 month ago

I thought that my tinkering with charset conversion caused these problems, so I reverted everything back to Revo 2.2.0, including my latin1 database. My plan was from this point, to re-run 2.2.4 updater, then make database conversion fixes.

Nothin doin, still not working. When trying to access resources I get the following error, except it's totally inconsistent. Sometimes it shows up, other times page loads OK.

syntax error:
Server Error
500 - Internal server error.
There is a problem with the resource you are looking for, and it cannot be displayed.

I don't know if this helps at all, but it seems that this behaviour doesnt happen on Resources that don't have a template selected (ie: 'Uses Template' is set to 0). For example I have resources that work as includes, like a footer. Then from within a file on the filesystem (footer.html) I have:

[[getResourceField? &id=`25` &field=`content`]]

I've been clicking through these they work as expected (so far). Soon as I try Resources with templates selected. Blammo. Error (sometimes).

I checked my templates and everything looks in order. They are Static and pointing to respective files on Filesystem. Again, don't know if ths helps. Just trying to provide as much info as I can.

Any other suggestions? Any other info I can provide?