Tags:
internationalization1Add my vote for this tag create new tag
, view all tags

Internationalization Support

Most of this page is discussing the ability to support multiple locales (including languages) within a single TWiki - a big part of this would require additional I18N infrastructure enabling translation of TWiki pages as well as skins/templates. See the Localisation and Translation section of InternationalisationEnhancements for some links to other pages on this issue. This page should probably be renamed to InternationalisationTranslations or TranslationSupport, since it's a subset of InternationalisationEnhancements as a whole. -- RichardDonkin

How would you like Internationalization support in TWiki?

I am porting to last alpha FrancoBagnoli's code (from WebTeach project) for Internationalization.

Implementation

  • each web/user has a LANGUAGE preference (usual $LANG codes)
  • a non-english template is named <name>.%LANGUAGE%.tmpl (is it does not exists TWiki falls back to the normal one)
  • a non-english topic is named <name>.%LANGUAGE%.txt

Desirable features

  • WikiNames should accept also accented characters
  • links to non-english pages have a small icon showing language's flag
  • automatic translation of topics via babelfish

Problems

  • Keeping in sync multi-language topics is a mess!!!
    • just show the flags in reverse save order (freshest version is the first icon)
    • double-window edit?
  • topic names
    • all topic have an english name
    • all topic keep translated names as a meta information (to be shown/recognized as a link in that language?)
  • different languages pluralize words differently ...
    • Is there some information on pluralization for european languages available online? -- MichaelUtech

-- AndreaSterbini - 17 Apr 2001

Zillions of open source programs in particular - newsloggers, content managers, community-type packages among others - are routinely offering support for 5-10-15-20 or more languages!

The implementation is different in each case, and so far nothing I've seen is really great. Mostly, it's substituting out of the box installation system text from different language files - if you set up a system and picked one of the available lang modules, all the intro text would be translated. Beyond that, once things get rolling, the main user language - usually, english, predominates: picking another lang mixes in bits and pieces but hardly translates properly.

There are a few translate-on-the-fly services, freebies from a couple of big translation companies. There, you got a full-text trans, real-time on demand, per page. The trans wasn't perfect - automated - but usuful. Langs limited to Eng-Fre-Ger, maybe Spa and one or two others. And this may've died down if free offers dried up.

Does anyone have a quick overview - something they know already and can pointform - of the prevailing pro end globalization options for multilingual sites?

-- MikeMannix - 28 Sep 2001

I don't think automatically translating contents is a good idea. The quality is so bad, that It's not really worth the hype.

Translating the documentation of TWiki is a lot of work, I'm on it for german (http://schnoffos.utech.de).

I also translated most of the templates and other places where english appears, this is just a matter of hours, so I feel it's not too bad to redo the work every one or two times a year. I'ld be good though if there were no hardcoded texts in bin/*, lib/* and such places.

From my experiences I would prefer a hierarchical message catalogue, over skin-like techniques or such. This would reduce the work to nearly none.

Best regards,

-- MichaelUtech - 27 Oct 2001

Regarding

I think it might be enough to add a per-web locale preference, as I suggested in LocalizationIsNeeded. Such a preference could be used to generate the proper header as suggested in NoWarningWhenEnteringNonStdChars. (Would that help with CharsetSelection, too?).

This supposes that the filesystem used by the server can use accented file names, since topic names map to similarly-named files. Perhaps that file-storage code should normalize topic names to e.g. utf-8 and back?

-- DavidSherman - 24 Nov 2001

I hacked my installation to accept german umlaut's. This was a change spreading through serveral files (I also modified the WikiWordSyntax). As off Sep 2001, there is no support for regex configurations and my guess is, that TWiki needs some general cleanup before this could be integrated.

There are problems with IE5 when setup to always encode urls to UTF-8, which is the default setting. AFAIK this is not supported in TWiki, when 8-bit characters are used.

On multi language topics: Whether such topics should be in synch or not depends on the policy of a web. I can think of applications, where the contents may well differ. I the usage of a wiki would require synchronization, I'ld suggest to invent a maintainer scheme that notifies maintainers about topic changes and asks them for a translation. Additionally a language selection scheme based on both user preferences and currency (how current a translation is) would be usefull, so that for example i get the german version if it's newest or else english, then french, if I set these languages in my preferences, but never italian, if I don't speak that language.

This is quite complex, but I think the complexity comes from demands, not from the solution.

On topic names: I don't like the idea to set up a policy to suggest or require english topic names, since that would mean non-english speakers would probably not understand them and thus not use the name. So there would be no automatic linkage to these topics from such users. Proper multi language support will require synonyms. Implementing these would probably leed to keyed storage (store topics in folders, named by number. Contents could be (en,de,it,fr).txt, and maybe more. Then we'ld need a dictionary to map topic names to (storage key, language) pairs.

Given that's not a trivial task, but as a side effect we could implement automatic linking to non-wikiwords, thus extending the definition of wikiwords by the namespace found in the dictionary (See also WikiWordSyntax).

So this approach would add both, some programming complexity as well as additional flexibility for possible future enhancements, so it may be worth it.

-- MichaelUtech - 23 Nov 2001

After reading all the options, I think that the best way to start supporting i18n is step by step. A non-english wiki site does not need something like:

  • each web/user has a LANGUAGE preference (usual $LANG codes)
  • links to non-english pages have a small icon showing language's flag
  • automatic translation of topics via babelfish

and so on, but some basic functionality like:

This two (and most important) features should not be so hard to do.

-- LucasDiPentima - 27 Jan 2002

I am actively porting TWiki to Italian. This is not a multi-language porting, just a translation.

Key points:

  • Lot of English strings are still hard-coded inside scripts. I'm extracting and replacing them with a hash, initialized in TWiki.cfg.
  • I needed to "abstract" some more topics, for instance
$wikiGroupsTopicname
  • I translated the name of monts, but the date menagement is more complex.

I will put my contributions into the ItalianTWiki page.

Comments: The template system is rather weak. It could be improved by adding IF and FOR loops. This would allow removing all strings from the scripts. The simplest thing would be to replace templates with perl scripts, and put variables to be interpolated into an hash, similar to what happens in the Template Toolkit (http://www.tt2.org). this would also speed up a lot twiki under mod_perl (since the templated would be cached). Compatibility is assured by post-processing the output in the usual way.

Let us consider the view template, and assume that variables are contained in the hash %context. One starts with

$output="";

and continues with

$output.="&lt;html&gt;&lt;head&gt;&lt;title&gt;$context{WEB}.$context{TOPIC}&lt;/title&gt;&lt;/head&gt;&lt;body&gt;";....

I do not think that it is much more complex than learning the template syntax. It allows to write

$output.="&lt;table&gt;&lt;tr&gt;";
for my $a (@$context{HEADINGS}) {
   $output .= "&lt;th&gt;$a&lt;/th&gt;";
}
...
and also to use high-level libraries.

But one could simply write

$output=<<TheEnd;
..old template stuff..
TheEnd

and everything goes as before.

-- FrancoBagnoli - 06 Apr 2002

I would like to work in an alternative implementation of the TWiki localization, based on Locale::Maketext module (thanks, Richard). Any ideas, any work already done?

-- EstebanManchado - 11 Aug 2003

It might be worth emailing the people who posted on TWikiInSpanish and GermanTranslation - I suspect they have done work on translations rather than internationalisation infrastructure. Perhaps a few volunteer translators could work with you to test out the CPAN:Locale::Maketext based code? Also, JoelPinckheard posted on TWikiInSpanish saying he was working on something along these lines, so I'd suggest contacting him as well. Google:twiki+translation may help too.

You could also put a brief request in CoffeeBreak, which is read by many people - hmmm, maybe we should have a Support.TestersWanted page for people who want to develop something and need people to test? Also, Support.TranslatorsWanted might get more readership from people who are not developers but are interested in doing translations, compared to using a Codev topic.

I saw an article recently about an improved module within Locale::Maketext, I think it's a sub-module - let me know if you're interested as I can probably dig it out, but I'm going on holiday tomorrow until near end of August so it may be a while!

I put a comment at the top about naming of this page and how it fits into InternationalisationEnhancements.

As for the content of this work - how about putting together TranslationRequirements to define what you want to work on and have some discussion on what's needed? For example, are you just working on L10N of templates/skins, or also on multi-language TWiki topics? Is per-user locale setup required?

-- RichardDonkin - 12 Aug 2003

Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r12 - 2006-02-15 - PeterThoeny
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.