On Dec 25, 2003, at 10:47 PM, bulia byak wrote:
Can anyone give any advice on how to deal with this problem? I know about g_locale_to/from_utf8() but the question is, where is the best place to put these functions. It looks like all standard file-related functions (gtk_file_selection_get_filename, file opening and saving, etc) require the filename in locale encoding, but all display functions (setting window title, display in XML editor, recent files menu) require UTF-8. And of course, serialized XML must get UTF-8 too. Looks like there are quite a lot of places where I'll need to translate between the encodings, and I'm afraid it will become pretty messy.
I'll have to look into the codebase with charset issues in mind (I've worked with international software for around 15 years now).
In general, the simplest way is to keep all strings in a program in a canonical form, and only translate when required. Best is either UTF-16/UCS-2 or UTF-8 for the internal format (depending on the platform, etc). For GTK+ and Qt/KDE programs, it's generally good to go with UTF-8. However, keeping Win32 compatibility complicates things somewhat.
On file versus display encodings, off hand I think you're correct. However... for the local filenames, things might get tricky. It might be that things need to work in the local encoding, or... it might be that the encoding of the filesystem needs to be used. Much depends on the OS and the means of accessing the file systems in question.
Hiding these issues for many people is the recent change to UTF-8 for the local encoding setting (such as with RedHat 8.0). Since the local encoding is the same as the encoding needed for internal/display calls, programmers don't notice the potential problems. Also... if users and programmers aren't actually testing with non-ASCII names (i.e. names with characters outside of $00-$7f) then they will also not see any problems even if they exist for their configurations.
I was planning on looking into this soon, but will try to get to it a little sooner. It would also help if some developers who are setup to do Win32 building and testing could give me a hand (I have MSVC 5.0 on Win98 dual-boot, but almost never leave Linux)