Hi friends,
Bug "FS 307 fix" in tmplvars.format.inc adds htmlentities() handling to all TV content. Probably someone reported a bug and this was a fix.
} elseif($tvtype==’rawtext’||$tvtype==’rawtextarea’) {
$value = $value;
} elseif($tvtype==’text’||$tvtype==’textarea’||$tvtype==’textareamini’) {
$value = htmlentities($value);
}
But, another bug is getting into with this solution. Simply, htmlentities() is only iso-8859-1 aware. Result: some no-english languages can’t use TV’s properly. It converts the local chars into &entities; and they are then no longer shown as a local characters.
I’m not comfortable with using some "raw textarea" and "raw text" in Czech - if we have a feature it must either work properly in all cases or it’s a "implemented bug".
I’m not sure about the solution now =(. Also I’m not sure about a reason for the fix above...I don’t see we can get it work properly this way, using htmlentities()...
Hi,
Many thanks for the heads up on this fix.
I’m thinking this solution can be implemented as a TV widget?
Nice catch !
It will avoid me some head scratching, I have a multilingual website with a few TVs coming up...
.: COO - Commerce Guys - Community Driven Innovation :.
MODx est l'outil id
I really vote for removing the htmlentities(), unless someone provide some REALLY strong reason to have it there. The reason for removing is simply a fact, that it is specific, language dependent thing. These kind of things should definitely not be in the CORE (!) source code.
The author of the original bug report wanted to convert some German chars to entities, I would suggest to do this by a language-specific plug-in. It’s the less evil.
There is also a problem in Ditto:
$placeholders[’[+rsspagetitle+]’] = htmlentities($resource[$x][’pagetitle’], ENT_QUOTES, $modx->config[’etomite_charset’]);
if $modx->config[’etomite_charset’] is iso-8859-2 and others, this would fail and produce PHP error/warning. This is the case when RSS output is used, so RSS becomes disfunctional.
-
MODX Staff
- 10,725 Posts
I’m a little confused here. htmlentities works fine with all the following encodings...but must be specified to match whatever encoding you are using for your site, and your database.
ISO-8859-1 ISO8859-1 Western European, Latin-1
ISO-8859-15 ISO8859-15 Western European, Latin-9. Adds the Euro sign, French and Finnish letters missing in Latin-1(ISO-8859-1).
UTF-8 ASCII compatible multi-byte 8-bit Unicode.
cp866 ibm866, 866 DOS-specific Cyrillic charset. This charset is supported in 4.3.2.
cp1251 Windows-1251, win-1251, 1251 Windows-specific Cyrillic charset. This charset is supported in 4.3.2.
cp1252 Windows-1252, 1252 Windows specific charset for Western European.
KOI8-R koi8-ru, koi8r Russian. This charset is supported in 4.3.2.
BIG5 950 Traditional Chinese, mainly used in Taiwan.
GB2312 936 Simplified Chinese, national standard character set.
BIG5-HKSCS Big5 with Hong Kong extensions, Traditional Chinese.
Shift_JIS SJIS, 932 Japanese
EUC-JP EUCJP Japanese
Since 4.3.0 according the documentation.
???
(NOTE: ugh, ignore me, just saw the issue with 8859-2 missing from the list -- what a horrible character set anyway -- never had anything but problems using 8859-2 or 1252 for that matter, but you might try 1252 to see if it can handle the missing 8859-2 chars).
Hi,
I’m not an expert in encodings, but wikipedia says:
...These (ISO-8859-2) code values can be used in almost any data interchange system to communicate in the following European languages: Bosnian, Croatian, Czech, Hungarian, Polish, Romanian, Serbian (in Latin transcription), Serbocroatian, Slovak, Slovenian, Upper Sorbian and Lower Sorbian. Furthermore it is suitable to represent some western European languages like Finnish (with the exception of å used in Swedish-Finnish names) or German.
Another big family is Windows-1250, also not supported. Wikipedia:
Windows-1250 is a code page used under Microsoft Windows to represent texts in Central European languages that use Latin script, such as Polish, Czech, Slovak, Hungarian, Slovene, Croatian, Romanian and Albanian. It may also be used with the German language; German-language texts encoded with Windows-1250 and Windows-1252 are identical.
I’m looking at the fix again and it appears to me that the user wants to all convert text based TVs that has all applicable characters to html entities, correct?
IMO this is not a good idea as some users will want to render TVs containing html as is.
The problem with htmlentities() and encoding is that php will revert to the default encoding (ISO-8859-1) if it does not recognise the charset.
Quote from: xwisdom at Jun 27, 2006, 09:17 AM
The problem with htmlentities() and encoding is that php will revert to the default encoding (ISO-8859-1) if it does not recognise the charset.
Exactly.
Probably, there should be a small config (like one or two checkboxes) in the TV management dialogs, that would provide additional settings for handling special characters.
Quote from: tillda at Jun 27, 2006, 09:22 AM
Probably, there should be a small config (like one or two checkboxes) in the TV management dialogs, that would provide additional settings for handling special characters.
Sounds ok to my.
