[Ditto] RSS Encoding

Great snippet, Mark.

However, I seem to have problems the special characters in the <title>-tags in the RSS feeds seem to be parsed oddly. This happens if I use either ISO or UTF-encodings. Any suggestions?

Cheers.

What characters?

Hey Mark,

Padon the late reply. It happens on most non alpha-numeric characters such as % and scadinavian letters such as æ, ø or å. However, I seem to have tracked down an issue with our set-up of php and MySQL that could be the root of some of our qualms. All in all I think it has to do more with how the manager handles these characters than Ditto’s handling of the characters.

If it’s still quirky once we fix these issues, I’ll let you know.

Ok, thanks for the update.

This seems to be a problem with RSS in general rather than Ditto specifically.

When I had Ditto create an RSS feed for a whole site, the page with an ’&’ in the title ("Swim & Sauna") caused my favoured RSS-reader (the Sage plug-in for Firefox) not to parse the feed at all because of the illegal character.

I don’t actually see the point of RSS-ing a whole site; I have a "Site Updates" folder with news in that is where I now get my RSS feed from. Thus an easy workaround is for me to just avoid putting an & in the titles of pages in this section, but at some point I will need to be able to let other people add news without this restriction.

Combined with what Tore says, it therefore looks at first glance that Ditto isn’t HTML-encoding the title field.
However, it seems no-one is quite sure how the title feed should be encoded (it is not in the spec).
See http://thresholdstate.com/threshold/4163/rss-feeds-valid-useful-or-accurate

Therefore it is seems that the feeds will appear differently in different RSS readers, and indeed in some might not work at all. It may be useful to know which RSS readers Ditto’s RSS output was/is tested with.

Tore, what readers are you using?

If there really isn’t any agreed method of having special characters in RSS titles, I can see myself adding a hack to change "&" to "and" so that it works on all readers.
Replacing foreign characters sounds more complicated - it might even be the case that multiple versions of the feed need to be made available, one HTML encoded, one not. (Make whichever one works for most people the default and add somewhere "Problems with accent characters in your RSS reader? Try our alternate feed".) For that to work, it would be useful to have the ability to toggle whether Ditto encodes the title.

I will probably do further research to see what the best practice is, and if anything can be shared I’ll let you all know. It could be that we’re better off raising the issue with the authors of RSS readers rather than the authors of Ditto!

Quote from: PaulGregory at May 28, 2006, 04:48 PM

This seems to be a problem with RSS in general rather than Ditto specifically.

This is indeed a common issue; one I’ve run into myself in developing a book review site -- TheVirtualHandshake.com, which uses WordPress for the blog and feed (though it is wrapped by FeedBurner), and MODx for the rest. I tried wrapping the title in CDATA at one point, but many readers and feed services didn’t like that. WP has (or used to anyway) a habit of turning the special characters into entities, which although they do seem to work on most readers, just looks freakin’ horrible!

Maybe someone has or will design a regexp or translation map that can help us translate characters to something acceptable for this problem (or other similar XML character data issues)

@PaulGregory: Thanks a bunch for the info. I did some snooping around, and it seems that Bloglines chews up the feed just fine and dandy. The vanilla reader in Firefox 1.5.0.2 chockes on it, though. Looking forward to seeing if you dig up anything else of value.

@OpenGeek: Great idea. I’ll cross my fingers for a great regxp.

Has anyone found a solution to this problem ?

I have found a temp. solution.

I have hacked the ditto.class.inc.php and replaced:



with:


I know that im not removing any quotes from my pagetitle but im not usign quotes in my pagetitles anyway. Its not a clean solution but it works for me.

The htmlentities function removes all characters that most RSS readers freak out with.

I’m going to check out how http://www.bitfolge.de/rsscreator-en.html does it and maybe come up with a solution.