Collecting all multi-language content requests and proposals

Let’s try to have a clear view of what we all need from multi-language content and what we have proposed, just to avoid to reinvent the wheel.

I think that it’s possible that we cannot find a solution that is good for everyone, but collecting ideas and needs we can have a good solution for most.

What I want:
I would like to give the possibility to choose the language of the content of the pages. The user make only once this choice and then go on visiting the pages in the language he chose. There is the possibility that some page isn’t (yet) translated, in this case, he will see a message ("sorry, no italian translation") and the page in the "default language" (e.g. english).
In the manager we should have some mechanism to see all translations while editing the content, or at least some easy way to switch between them (this is useful when we translate, for example).

Here are my ideas:
Idea 1: put a new field in the site_content table, we call it "lang", it contains the language of the content; in this way we have a separate page for each language content and we can arrange the directory structure as in Susan’s hack... all things like search, etc will work properly without any change, but we need to make changes to Wayfinder, for example.
This need also some change in the manager.
What to do with document Id? we can have different ids or the same id (and the primary key will become id+lang). What do you think?

Idea2: create a TV for each language and put the content in the TV. Some mechanism (e.g. plug-in) will show the correct content. This require changes in everything that has to do with contents (e.g. search), but we do not have to change Wayfinder. It seems we need also a TV for the title, one for the description, and so on... this is the reason for which I prefer Idea1.

Wouldn’t the newly implemented "category" fields be useful for this?

@ idea 1 :
That’s what my hack already does.
It adds a field that allows you choose what language you want to use. Then you follow Susan’s directory setup.
I even hacked the admin side to allow easy language selection.
The only problem was that I didn’t know how to hack the parser (errr, was too busy at the time to figure out) to pull the db data for the lanague out so I ended up using 1 more SQL query for the TV instead.

@ idea 2 :
hmmm, I thought about this a lot and ended up liking the idea of different directories. I think that Susan hit it right from the start with her cookie solution.

The reason why is because then you can use different templates and different layouts based on the language’s cultural preferences. Of course you can always use the same template but just in case is always a good solution to have.

However!
IF you:
Set up the admin so that you can choose: template and language for the same document (ID)
Have the TV/parser do the heavy lifting rather than run multiple directories for each language.
Also, at the same time when you edit/create the document you can tab through the different lanague content for the document.

This would leviate the problem of having to edit 3 (if 3 languages) documents in different directories.
in practice: I have a site that has 3 languages and this really is a pain in the arse since somebody always seems to forget something along the way. PLUS, the amount of time it takes to load the dam editor (even without FCK gong) is really a horrendous amount of clicks to go through.

BUT, for statistical purposes this method shines since you can have a better idea where people are coming to.

dammed if you do and dammed if you don’t

@ idea 3
Anybody?

If we’re collecting requirements, a pretty big one that will affect the structure is SEO.

I’m not particularly keen on the use of cookies to do anything much other than redirect the domain.com homepage to the correct language homepage. All links should pass through the correct language/culture at that point, because the French language page and the English language equivalent should be separately indexable by Google etc.

The means by which this is done should be selectable, as there are many possible methods:

For example, a simple language-based solution might use fake directories:
domain.com/fr/faq.html
domain.com/en/faq.html

or a prefix:
domain.com/fr-faq.html
domain.com/en-faq.html

Or the domain itself could dictate a lot of the content.
domain.fr/faq.html
domain.co.uk/faq.html

And some areas might use both approaches:
domain.fr/faq.html
domain.co.uk/faq.html
domain.fr/faq.html
domain.ca/fr/faq.html
domain.ca/en/faq.html

The point should be made is that not everything has a direct equivalent or is appropriate to share between two languages. I don’t know how blog posts would or should be handled across two languages, or their comments.

Excellent points Paul. Indeed using the dynamic features of a content management system for internationalization tempts us to want to handle the same content in multiple languages through a single gateway to deliver culture-specific content.¹ It seems like the least effort for managing the content, translation efforts, etc. But the reality is that internationalization is at best an emerging technology area on the web, and we would do well to follow closely the best practices that are emerging with it.

And just so everyone is a little clearer on the motivations and approaches that will actually be implemented in the upcoming MODx core, they are all based on the concepts in the current working draft of W3C Internationalization Best Practices: Specifying Language in XHTML/HTML Content. For instance, pages/documents should declare the language (or languages in some cases) of the intended audience, while individual pieces of content (think chunks, snippets, TV’s, etc) within a document may declare very specifically the language to use for text-processing/styling on that range of content. Further, there is no one-to-one mapping between character encoding, directionality of text, and a preferred language, so these important aspects have to be managed across all of these details.

Contexts are going to be available in the next minor release of MODx and this will help organize your site into these logical sections as suggested by PaulGregory, whether it be using subdirectories, subdomains, or context-specific alias prefixes/suffixes. For now, most of these things mentioned can still be accomplished in MODx, using various add-ons and organizational approaches.

I think my overall point is this: there will not be any changes made to core logic or tables to support multi-lingual authoring until it designed and implemented properly. I am 100% against rolling out important new features like this in haste, but rest assured that the development team is working hard to provide robust internationalization support that parallels these best practices.

¹ -- NOTE: I use culture to distinguish also the variations of language and other internationalization attributes within a country, as country or language alone are not enough to identify and isolate content in a way that is meaningful to your target audience, though best-practices dictate only using subtags when it actually provides benefit to your audience (i.e. use the shortest language tag applicable)

Hi,

I’m more for Paul’s idea:

For example, a simple language-based solution might use fake directories:
domain.com/fr/faq.html
domain.com/en/faq.html

or a prefix:
domain.com/fr-faq.html
domain.com/en-faq.html

My way of doing multi-language site is in the order domain/locale/page

nasty nice stuff!

Excellent Paul, I never thought of it that way.
Now, we just have to wait for 1.0.

One comment though, I do agree on rolling out when it’s ready but I hope that "not in haste" does not mean "wait for a standards baord to approve something that should be standard" that would be a real bummer.

Of course, the W3C specifications are about the final (X)HTML page created by MODx - they are about the content document structure, not the content management. All we need to ensure is that the structure used will generate pages accordingly. I can well imagine that a multilingual site will have one or more multilingual pages - the "Choose language" page, or even a bilingual home age. It’s great to hear that the MODx core will cope with such a page.

W3C technicalities are important to bear in mind but they don’t shape the usability or best practice concerns of multilingual content management (And I’ll have to reread that document properly at some point, at first glance some of the ’best practices’ contradict one other). Indeed, that document only really covers one best practice that any web visitor would notice : "Don’t use flags to represent languages". So the usability or best practice concerns of multilingual site structure aren’t really covered there.

W3C don’t care where document content comes from, they just want it marked up right. We MODx users *do* care where we’re inputting document content. As such, requests and proposals are still important to collect to ensure a system that is both on the right track and flexible enough to cover multiple sensible working practices.

I’m still very pleased that we’re getting context-based stuff rather than language-based stuff. I’m quite excited by the possiblilities of contexts, and most of them have nothing to do with language.

Thus the "standards board" is MODx, not W3C. OpenGeek is basically saying "There will be a big, well-thought-out core feature". Which is good news, but that doesn’t mean we all have to stop dreaming and start waiting. Additional ideas are always good - particularly ones that will affect the implementation on a site, rather than the core structure.

So I’d love to see examples of multilingual or multicontextual sites that people think are good.

One way of doing domain/lang/page.html is to get the parser to extract the lang portion of the url and then load the appropriate documents/folder.

Might the new "link attributes" field in the document editor page be useful in this way?