Hi everybody,
I am completely new to MODx and FeedX and I ran into a problem which seems to be a bug in FeedX. Maybe I am wrong and you are able to set me straight
I wanted to include an iso-8859-1 encoded feed into a MODx-page, which is utf-8 encoded. Doing that the feed lost the german umlauts. Here is the data I am working with:
Feed-URL:
http://www.photomed.de/index.php?id=240&type=100
Snippet-call:
[!FeedX? &url=`http://www.photomed.de/index.php|xq|id|xe|240|xa|type|xe|100` &preset=`rss` &cacheTime=`3600` &debug=`0`!]
I am using an MODx 1.0.2 with FeedX 0.1.2 on utf-8
After looking into the FeedX code I found that in line 104 in the feedx.class:
return mb_convert_encoding($output, $modx->config['modx_charset'], 'auto');
The input-encoding is set to ’auto’ which expands to ’ASCII,JIS,UTF-8,EUC-JP,SJIS’ according to the php-doc. As iso-8859-1 is not part of that collection I tried to introduce a feedx-class property called ’inputEncoding’ which stored the encoding of the feed (read from the original feed around line 336). That was just before I discovered that FeedX caches the parsed xml-tree and _not_ the feed-xml itself. So the encoding wouldn’t have been available if cached data would have been used. I decided that the parsed xml-data itself was the problem. When the above code gets executed the code does not now what it’s input is encoded with, nor does any other code know.
Some php documentation later I found:
xml_parser_set_option($parser, XML_OPTION_TARGET_ENCODING, 'myencoding');
So far FeedX only defined the input-encoding for the xml-parser. The output-encoding defaults to the input-encoding (iso-8859-1 in my case) which gets converted with mb_convert_encoding later on (without success). Now it is possible to convert the xml-parser result to something more convenient. Unfortunately only these encodings are supported by the xml-parser as output-encoding:
ISO-8859-1, US-ASCII and UTF-8
We are aiming for a support of all encodings that are supported by MODx, so that wouldn’t work. My current solution is to translate the parsed xml to utf-8. That should work in any case as any MODx-supported char-encoding fits into utf-8 (as far as I know). The later call to mb_convert_encoding now knows that its input is utf-8 under any circumstances (cached / non-cached). The call to mb_convert_encoding is only made if the output-encoding is anything but utf-8.
Disadvantages:
- Feed encoded in a and output encoding is b (a and b not beeing utf-8) will always be converted over utf-8 which might result in an useless encoding-run
Advanatages:
- Should work with any MODx-supported encodings now (hopefully)
It is my first code-contribution and I don’t know what format fits best. I tried to create a patch for my changes. Source-version was feedx-0.1.2.
Index: feedx.class.inc.php
===================================================================
--- feedx.class.inc.php (working copy)
+++ feedx.class.inc.php (revision 4)
@@ -100,8 +100,9 @@
}
}
- if (function_exists('mb_convert_encoding')) // Convert output encoding, if available
- return mb_convert_encoding($output, $modx->config['modx_charset'], 'auto');
+ // Only convert encoding if output-encoding differs from utf-8 and encoding-function is available
+ if ($modx->config['modx_charset'] != 'UTF-8' && function_exists('mb_convert_encoding'))
+ return mb_convert_encoding($output, $modx->config['modx_charset'], 'UTF-8');
else
return $output;
}
@@ -336,6 +337,8 @@
$parser = xml_parser_create($encType);
xml_parser_set_option($parser, XML_OPTION_CASE_FOLDING, 1);
+ // Always decode to UTF-8 as the parser does not support enough charset-encodings to fit all supported MODx-encodings
+ xml_parser_set_option($parser, XML_OPTION_TARGET_ENCODING, 'UTF-8');
xml_set_object($parser, $this);
xml_set_element_handler($parser, 'tagStart', 'tagEnd');
xml_set_character_data_handler($parser, 'tagData');
Quite easy changes. Took me 4 hours anyway. I’m eager to hear your feedback!
So long
Marc
☆ A M B ☆
