Multi language content

I often wonder if a database-heavy application would be significantly slower using the API over directly making the database calls. Going through two levels of wrapping (the modx class and the dbapi class) must have some impact; perhaps insignificant for simple use, but what about heavy use? Really, the only advantage to the wrappers is to make it easy to add another wrapper class for different databases. Is that potential advantage ever overshadowed by performance loss? When does it become performance vs. convenience?

Good call Susan, I don’t remember where but I recall we were talking of the good it would do to have a "performance person" in the testing team, to evaluate this kind of stuff (just like NetNoise took on security).

Anyway, Wendy’s last post to me this raises a new question : how do we set up built-in multilinguism in MODx if snippet are not designed to easily make it true ?

Should there be guidelines for snippet developper to make them easily fit with a multilingual system ? And first and foremost, Ryan mentionned earlier multilingual capabilities would go on top of the list. It means doing some brainstorming about it...

Few system are able to really do multilinguism, maybe we should do a quick tour and make a brief about the solutions used here and there...

If we are ever going to have an auto-installation feature for templates, snippets, etc, as well as internationalization capabilities built-in, yes, we will have to go to a standard way of developing snippets as well as templates. Of course, the way MODx framework itself works, anyone can do as they please. But they will need to conform to whatever standards are decided on if their snippets and templates are to work with the automated systems.

Personally, I would prefer seeing the internationalization/multilingual feature be an optional module. Again, the performance issue is at question. Anything along those lines will have a performance impact, which will be quite acceptable in exchange for the functionality it provides. However, someone who does not need the functionality should not be obliged to carry the performance penalty.

An example of this is the web user system. It is a major feature that is heavily used, but if a site doesn’t need it there is no "wasted" overhead involved.

A multilanguage system could be implemented with two tables, site_languages and content_language. The site_languages table could include fields for name and location of language files, character set, whatever relates to the row’s language. The content_language table would connect content with language. The functionality would be in snippets, and/or plugins and modules, and include files, just as it is for the weblogin system. It would also probably require a new menu listing snippet, keyed to the main language section of the tree, so "empty" spots in a given language’s tree would be filled by links to the corresponding document in the main language’s tree.

If you wanted the display text to be in the database rather than text files, a third table, language_values, with three fields, lang_id, key, value would do the job nicely. The key would be the value to replace, the value the expression to replace it with. Basically it would be the same thing as pulling up a text file, the query would pull up the key-value pairs for a given language and the resulting array would work the same way, only more like placeholders. I suppose a plugin would do that. The only advantage I see is the ability to have a more secure method of adding and editing new languages and values.

Thanks a lot Susan for your post, it helps me put things in perspective !

The faster we choose an orientation there, the better for snippet developpers... I mean, even if the multilingual thing is a module of some kind, snippets have to be localized and doing it by hand is a real time consumer... I hope future plugin/snippet/module will use a language folder like QuickEdit does... it will greatly help.

Also, we talked with Wendy about some tool to edit those languages files from the admin (like e107 or bitweaver has...). Would be great !

Quote from: sottwell at Feb 28, 2006, 01:16 AM

I often wonder if a database-heavy application would be significantly slower using the API over directly making the database calls. Going through two levels of wrapping (the modx class and the dbapi class) must have some impact; perhaps insignificant for simple use, but what about heavy use? Really, the only advantage to the wrappers is to make it easy to add another wrapper class for different databases. Is that potential advantage ever overshadowed by performance loss? When does it become performance vs. convenience?

From my own perspective, adding at least one layer of abstraction befoe going into the database will be a great way to go. Considering about the performance hit, it’s so true, some times it bogged down the system, but if you start comparing it with all the things that can be achieved with having 1 more abstraction layer, it overweight the performance issue. Let say, server computer right now has a lot more juice on it compare to a few years ago, and I believe PHP has their own advantage in terms of speed. So I don’t think it will cost too much having this one layer of abstraction.

A good example will be this Multi Language solution. Consider if everybody use the DB abstraction provided by MODx to access the doc structure. If madmage decided to have his own interpretation of the database, than all he need to do is change all the current db api that have access directly to the doc database to display the right logic that he is implementing in his custom solution. With this, everything wil works just fine with no compatibility issue. Another good example will be my hack for subsites, because everybody using [~~] or makeUrl API to build their snippet, now all I need to do is make sure the generation of link with [~~] is correct and makeUrl outputed the right link for what I want it. That’s it, it will not affect other resources that uses this features, because of this level of abstraction.

@David,

I believe Garyn will take on the project. We’ll see what he will do, for now I’ll seat back and work on my other things first. When I have a chance, I’ll look into this again. Time to setup my priority for now, but still can’t figure out which needs to be done first

Quote from: Djamoer at Feb 28, 2006, 09:44 AM

@David,
I believe Garyn will take on the project. We’ll see what he will do, for now I’ll seat back and work on my other things first. When I have a chance, I’ll look into this again. Time to setup my priority for now, but still can’t figure out which needs to be done first

Yeah you sure are working on a lot of simulanneous front right now !
That’s wise...

About Garryn taking over the project : great news !

Hi there,
I am working on the "links" rebuilding feature, that is necessary to keep the structure consistent after a move document or a deletion...

I stopped a little because (apart from heavy non-modx work ;-)) multilanguage stuff seems to be really complex, for snippets and so on... I was waiting for, at least, a stable API... but I am realizing that the API cannot be stable until multilanguage stuff is decided, because it probably change the way some API behaves...

I am thinking, like you all, about this issue

Can’t we think of any other solution?
Like general page variables injection using TV maybe. This will allow the use of other snippets normally in default language, while for other translation, they can access the document normally, but if they need to use snippets such as newslisting, it needs to be specially configured, like for example using the translation TV, instead of content var.

Any other thought on this? Honestly, I forgot the reason why I didn’t try this soluition before, but aside from that weaknesses that I described above, it has major weakness that kinda hold me back not to implement it. I think it’ll be best for us to discuss this further for the right solution. Having several people interested in this integration, I think it will give the core developer or users/coders like me to have the right solution for MODx.

I acknowledge not to have had the courage to read all messages, english is not my "cup of tea" and I have not very time too translate all that I don’t understand.
If my message is out of topic, thanks to indicate me to move it.

A solution for content with multi language is to create new tables. It is simply to make evolution when a new language must be added and also delete it.
I talk to create new tables for contents, metatags... every tables which contains localized string. I know it is not easy to do this but I think it’s the better way. Table name must contains a reference to the language like "modx_site_content_fr", "modx_site_content_en", etc... In this way, it is possible to translate the major part of string. It permits to select easly content localized, without weigh down default tables. Request on the database will be faster than one table with a field containing a reference to the language.
This solution is in the absolute. I don’t think it is possible currently to make it without too much difficulties.

I talk about this solution because I developed a website from scratch with this solution, after many tests to evaluate performance and simplicity of implementation.

Can you describe it further about the new table that you’re talking about?
To be honest, back then when I built a multi language website using ASP .Net, I’m using the same idea with madmage, basically by adding 2 more columns on the table, one to refer to the parent id and the other one is to refer to culture id, I get my site up and running, but I would love to know why we need another table to do this, and why it’s better than using single table. I believe with your detailed reasoning, we can bring the diiscussion further