468,457 Members | 1,620 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,457 developers. It's quick & easy.

wcout, wprintf() only print English

Has anyone actually managed to print non-English text by using wcout or
wprintf and the rest of standard, wide character functions?
Feb 23 '08 #1
44 3849
Ioannis Vranos wrote:
Has anyone actually managed to print non-English text by using wcout or
wprintf and the rest of standard, wide character functions?

For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L" \n";
}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$
Feb 23 '08 #2
Ioannis Vranos wrote:
Ioannis Vranos wrote:
>Has anyone actually managed to print non-English text by using wcout or
wprintf and the rest of standard, wide character functions?


For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L"Δοκιμαστικό μήνυμα\n";
Are you sure that you stored your source file in the same encoding the
compiler expects as source character set?
}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$
Feb 23 '08 #3
Rolf Magnus wrote:
Ioannis Vranos wrote:
>Ioannis Vranos wrote:
>>Has anyone actually managed to print non-English text by using wcout or
wprintf and the rest of standard, wide character functions?

For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L"Δοκιμαστικό μήνυμα\n";

Are you sure that you stored your source file in the same encoding the
compiler expects as source character set?
>}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$

Well I created the file with anjuta editor with the message being a
Greek one. The Greek message also appears the same when I display the
source file in the console.

I suppose it is saved as UTF8.
Also the code

#include <iostream>
#include <string>

int main()
{
using namespace std;

wstring s;

wcin>s;
wcout<< s<< endl;
}
displays nothing when I enter greek text.
Should I mess with locales?
Feb 23 '08 #4
Ioannis Vranos wrote:
Rolf Magnus wrote:
>Ioannis Vranos wrote:
>>Ioannis Vranos wrote:
Has anyone actually managed to print non-English text by using wcout or
wprintf and the rest of standard, wide character functions?

For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L"Δοκιμαστικό μήνυμα\n";

Are you sure that you stored your source file in the same encoding the
compiler expects as source character set?
>>}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$


Well I created the file with anjuta editor with the message being a
Greek one. The Greek message also appears the same when I display the
source file in the console.

I suppose it is saved as UTF8.
Also the code

#include <iostream>
#include <string>

int main()
{
using namespace std;

wstring s;

wcin>s;
wcout<< s<< endl;
}
displays nothing when I enter greek text.

both in g++ under Linux and VC++ 2008 Express under Windows, with the
latest saving the source code file as Unicode after it detected
non-english text.

Should I mess with locales?
Feb 23 '08 #5
Made more precise:

Ioannis Vranos wrote:
>>>For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L" \n";

Are you sure that you stored your source file in the same encoding the
compiler expects as source character set?

}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$


Well I created the file with anjuta editor with the message being a
Greek one. The Greek message also appears the same when I display the
source file in the console.

I suppose it is saved as UTF8.
Also the code

#include <iostream>
#include <string>

int main()
{
using namespace std;

wstring s;
wcin>s;

wcout<< s<< endl;
}

displays the Greek text when I enter it, but outputs nothing. With
English text, the text is displayed both when entered and outputed.
[john@localhost src]$ ./foobar-cpp


[john@localhost src]$ ./foobar-cpp
Test
Test
[john@localhost src]$


both in g++ under Linux and VC++ 2008 Express under Windows, with the
latest saving the source code file as Unicode after it detected
non-english text.

>Should I mess with locales?
Feb 23 '08 #6
Ioannis Vranos wrote:
Ioannis Vranos wrote:
>Has anyone actually managed to print non-English text by using wcout
or wprintf and the rest of standard, wide character functions?


For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L" \n";
}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$
Hmmm... I work almost entirely in English, so this error message is new
to me:

$ make
g++ -ansi -pedantic -Wall main.cc -o main
main.cc: In function 'int main()':
main.cc:4: error: converting to execution character set: Invalid or
incomplete multibyte or wide character
make: *** [main] Error 1
Feb 23 '08 #7
On Sat, 23 Feb 2008 13:11:09 +0200, Ioannis Vranos
<iv*****@nospam.no.spamfreemail.grwrote:
[...]
>>Also the code
[...]displays the Greek text when I enter it, but outputs nothing. With
English text, the text is displayed both when entered and outputed.
I don't remember anymore the details but the problem has something to do
with codecvt: Your wide characters are automatically converted to narrow
characters by wcout. This is something you might not want (and even if you
want it the conversion might not work automatically the way you expect :).

Try writing to wstringstream and converting to UTF-8 explicitly (storing
the result eg. in string). If your console supports UTF-8 you can print to
cout (otherwise print to a file so you can test the output in an editor).

HTH,
Boris
Feb 23 '08 #8
Jeff Schwab wrote:
Ioannis Vranos wrote:
>Ioannis Vranos wrote:
>>Has anyone actually managed to print non-English text by using wcout
or wprintf and the rest of standard, wide character functions?


For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L"Äïêéìáóôéêü ìÞ*õìá\n";
}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$

Hmmm... I work almost entirely in English, so this error message is new
to me:

$ make
g++ -ansi -pedantic -Wall main.cc -o main
main.cc: In function 'int main()':
main.cc:4: error: converting to execution character set: Invalid or
incomplete multibyte or wide character
make: *** [main] Error 1

I tried the same:

[john@localhost src]$ g++ -ansi -pedantic-errors -Wall main.cc -o
foobar-cpp

[john@localhost src]$
Perhaps when you copy and paste the greek text, you copy garbage (that
is, not viewing the message in the correct character set in your
newsgroup reader).
So, I repost the code in this message which is encoded to Unicode (UTF-8):
#include <iostream>

int main()
{
using namespace std;

wcout<< L"Δοκιμαστικό μήνυμα\n";
}
Feb 23 '08 #9
Ioannis Vranos wrote:
Jeff Schwab wrote:
>Ioannis Vranos wrote:
>>Ioannis Vranos wrote:
Has anyone actually managed to print non-English text by using wcout
or wprintf and the rest of standard, wide character functions?
For example:

[john@localhost src]$ cat main.cc
#include <iostream>

int main()
{
using namespace std;

wcout<< L"Äïêéìáóôéêü ìÞ*õìá\n";
}

[john@localhost src]$ ./foobar-cpp
??????????? ??????

[john@localhost src]$

Hmmm... I work almost entirely in English, so this error message is
new to me:

$ make
g++ -ansi -pedantic -Wall main.cc -o main
main.cc: In function 'int main()':
main.cc:4: error: converting to execution character set: Invalid or
incomplete multibyte or wide character
make: *** [main] Error 1


I tried the same:

[john@localhost src]$ g++ -ansi -pedantic-errors -Wall main.cc -o
foobar-cpp

[john@localhost src]$
Perhaps when you copy and paste the greek text, you copy garbage (that
is, not viewing the message in the correct character set in your
newsgroup reader).
So, I repost the code in this message which is encoded to Unicode (UTF-8):
#include <iostream>

int main()
{
using namespace std;

wcout<< L"Δοκιμαστικό μήνυμα\n";
}
Thanks, you were correct.

Here's what I thought was "supposed" to be the portable solution:

#include <iostream>
#include <locale>

int main() {
std::wcout.imbue(std::locale("el_GR.UTF-8"));
std::wcout << L"Δοκιμαστικό μήνυμα\n";
}

However, my system still shows question marks for this. For whatever
it's worth, here's the (probably incorrect) way that appears to work on
my system:

#include <iostream>
#include <locale>

int main() {
std::cout.imbue(std::locale(""));
std::cout << "Δοκιμαστικό μήνυμα\n";
}
Feb 23 '08 #10
Based on the MSDN example:
// basic_ios_imbue.cpp
// compile with: /EHsc
#include <iostream>
#include <locale>

int main( )
{
using namespace std;

cout.imbue( locale( "french_france" ) );
double x = 1234567.123456;
cout << x << endl;
}
that doesn't work in my GCC, this works:

#include <iostream>
#include <limits>

int main()
{
using namespace std;

cout.imbue( locale( "greek" ) );

cout<< "Δοκιμαστικό\n";
}
This also works:

#include <iostream>
#include <limits>

int main()
{
using namespace std;

cout.imbue( locale( "en_US" ) );

cout<< "Δοκιμαστικό\n";
}


Crazy stuff.
Feb 23 '08 #11
It looks like GCC has the opposite stuff, cout, cin, string work as
wcout, wcin, wstring and vice versa! Bug?

#include <iostream>

int main()
{
using namespace std;

wstring ws;

wcin>ws;

cout<< ws.size()<< endl;
}

[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
0
[john@localhost src]$

#include <iostream>

int main()
{
using namespace std;

string s;

cin>s;

cout<< s.size()<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
22
[john@localhost src]$
#include <iostream>

int main()
{
using namespace std;

string s;

cin>s;

cout<< s<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$

#include <iostream>

int main()
{
using namespace std;

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό

[john@localhost src]$

#include <iostream>

int main()
{
using namespace std;

cout<< "Δοκιμαστικό-11\n";

wcout<< "Δοκιμαστικό-22\n";

cout<< L"Δοκιμαστικό-33\n";

wcout<< L"Δοκιμαστικό-44\n";
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό-11
-22
0x80488c8���������-44
[john@localhost src]$

Conclusion: It appears GCC has the wide character stuff messed up, or I
am missing important knowledge.

Feb 23 '08 #12
Ioannis Vranos wrote:
It looks like GCC has the opposite stuff, cout, cin, string work as
wcout, wcin, wstring and vice versa! Bug?
....
Conclusion: It appears GCC has the wide character stuff messed up, or I
am missing important knowledge.
You and me both. I would be very surprised if this were a GCC bug (I'm
using 4.2.4 pre-release), but I'm guessing somebody here knows a lot
more about this than we do, and is willing to enlighten us. :)
Feb 23 '08 #13
I posted the following to c.l.c., and I think it is useful to post it
here too:
[The current message encoding is set to Unicode (UTF-8) because it
contains Greek]
The following code does not work as expected:
#include <wchar.h>
#include <locale.h>
#include <stdio.h>
#include <stddef.h>

int main()
{
char *p= setlocale( LC_ALL, "Greek" );

wchar_t input[50];

if (!p)
printf("NULL returned!\n");

fgetws(input, 50, stdin);

wprintf(L"%s\n", input);

return 0;
}
Under Linux:
[john@localhost src]$ ./foobar-cpp
Test
T
[john@localhost src]$
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
�
[john@localhost src]$


Under MS Visual C++ 2008 Express:

Test
Test

Press any key to continue . . .
Δοκιμαστικό
??????ε????

Press any key to continue . . .
Am I missing something?
Feb 23 '08 #14
Reply I posted in c.l.c.:
Ioannis Vranos wrote:
>
It works OK under Terminal UTF-8 default encoding too. So "%ls" is what
was really needed.

Actually the code:

#include <wchar.h>
#include <locale.h>
#include <stdio.h>
#include <stddef.h>

int main()
{
char *p= setlocale( LC_ALL, "Greek" );

wprintf(L"Δοκιμαστικό\n");

return 0;
}

works only when I set the Terminal encoding to Greek (ISO-8859-7).
>

BTW, how can we define UTF-8 as the locale?
Thanks a lot.
Feb 23 '08 #15
How can we convert the C subset C++ code:
#include <wchar.h>
#include <locale.h>
#include <stdio.h>
#include <stddef.h>

int main()
{
char *p= setlocale( LC_ALL, "Greek" );

wchar_t input[50];

if (!p)
printf("NULL returned!\n");

fgetws(input, 50, stdin);

wprintf(L"%ls", input);

return 0;
}

that works, to use the newest and greatest C++ facilities? :-)
Feb 23 '08 #16
James Kanze wrote:
>
>Also I have MS Visual C++ 2008 Express installed.

Under Linux ! :-)

Linux::VMWare::Windows::VC++2008 Express.
Feb 24 '08 #17
Ioannis Vranos wrote:
How can we convert the C subset C++ code:
#include <wchar.h>
#include <locale.h>
#include <stdio.h>
#include <stddef.h>

int main()
{
char *p= setlocale( LC_ALL, "greek" );

wchar_t input[50];

if (!p)
printf("NULL returned!\n");

fgetws(input, 50, stdin);

wprintf(L"%ls", input);

return 0;
}

that works, to use the newest and greatest C++ facilities? :-)

The next best thing after this, is to use the C-subset setlocale with
wcin, wcout, wstring and stuff, and it works indeed:
#include <iostream>
#include <clocale>
#include <string>

int main()
{
using namespace std;

char *p= setlocale( LC_ALL, "greek" );

if (!p)
cerr<< "NULL returned!\n";

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$

The following works out of the box too:

#include <iostream>
#include <clocale>

int main()
{
using namespace std;

char *p= setlocale( LC_ALL, "greek" );

wcout<< L"Δοκιμαστικό\n";
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
[john@localhost src]$

Now how can we move from the setlocale() to the newer C++ facilities?
Feb 24 '08 #18
On Feb 24, 1:00 am, Ioannis Vranos <ivra...@nospam.no.spamfreemail.gr>
wrote:
James Kanze wrote:
Also I have MS Visual C++ 2008 Express installed.
Under Linux ! :-)
Linux::VMWare::Windows::VC++2008 Express.
Thanks. I'll give it a try myself. (Of course, the executables
it generates will also require VMWare to run, but it will allow
at least verifying that my code compiles with VC++ before trying
to port it to Windows.)

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique oriente objet/
Beratung in objektorientierter Datenverarbeitung
9 place Smard, 78210 St.-Cyr-l'cole, France, +33 (0)1 30 23 00 34
Feb 24 '08 #19
On Sat, 23 Feb 2008 23:19:57 +0200, Ioannis Vranos
<iv*****@nospam.no.spamfreemail.grwrote:
Alf P. Steinbach wrote:
[...]
>Ans as has also been remarked else-thread, by Boris, one issue,
relevant for i/o, is that the wide character streams convert to and
from narrow characters. wcout converts to narrow characters, and wcin
converts from narrow characters. They're not wide character streams,
they're wide character converters.

I am not sure I understand this.

Isn't L"some text" a wide character string literal? Don't wcout, wcin
and wstring provide operator<< and operator>overloads for wide
characters and wide character strings?
wcout and wcin represent external devices. When you read from or write to
external devices the facet codecvt is used. The C++ standard says there
are only two: codecvt<char, char, mbstate_twhich doesn't do anything and
codecvt<wchar_t, char, mbstate_twhich converts from wchar_t to char. As
you see there is an implicit conversion to char even if you actually use
wchar_t in your program. You don't know either how the conversion of
codecvt<wchar_t, char, mbstate_tworks (there is no guarantee that it's
UTF-16 to UTF-8 for example). Either you convert to UTF-8 explicitly and
write to cout or you define or use a codecvt from a third-party library
(like http://www.boost.org/libs/serializat.../codecvt.html).

Boris
[...]
Feb 24 '08 #20
Boris wrote:
On Sat, 23 Feb 2008 23:19:57 +0200, Ioannis Vranos
<iv*****@nospam.no.spamfreemail.grwrote:
>Alf P. Steinbach wrote:
[...]
>>Ans as has also been remarked else-thread, by Boris, one issue,
relevant for i/o, is that the wide character streams convert to and
from narrow characters. wcout converts to narrow characters, and
wcin converts from narrow characters. They're not wide character
streams, they're wide character converters.

I am not sure I understand this.

Isn't L"some text" a wide character string literal? Don't wcout, wcin
and wstring provide operator<< and operator>overloads for wide
characters and wide character strings?

wcout and wcin represent external devices. When you read from or write
to external devices the facet codecvt is used. The C++ standard says
there are only two: codecvt<char, char, mbstate_twhich doesn't do
anything and codecvt<wchar_t, char, mbstate_twhich converts from
wchar_t to char. As you see there is an implicit conversion to char even
if you actually use wchar_t in your program. You don't know either how
the conversion of codecvt<wchar_t, char, mbstate_tworks (there is no
guarantee that it's UTF-16 to UTF-8 for example). Either you convert to
UTF-8 explicitly and write to cout or you define or use a codecvt from a
third-party library (like
http://www.boost.org/libs/serializat.../codecvt.html).

Instead of messing with these details, perhaps we should accept that the
C subset setlocale() function defined in <clocaleis simpler (and thus
better)?
The following code works:
#include <iostream>
#include <clocale>
#include <string>
#include <cstdlib>
int main()
{
using namespace std;

if (!setlocale( LC_ALL, "greek" ))
{
cerr<< "NULL returned!\n";

return EXIT_FAILURE;
}
wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
Feb 24 '08 #21
I filed a bug in GCC Bugzilla:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35353
May anyone explain me how should I apply the suggested solution
"sync_with_stdio (false)", and the fstream suggested solution to the
following failing code?

#include <iostream>
#include <locale>
#include <string>

int main()
{
using namespace std;

wcout.imbue(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}

Feb 24 '08 #22
On Sun, 24 Feb 2008 16:46:26 +0200, Ioannis Vranos
<iv*****@nospam.no.spamfreemail.grwrote:
[...]Instead of messing with these details, perhaps we should accept
that the C subset setlocale() function defined in <clocaleis simpler
(and thus better)?
The following code works:
#include <iostream>
#include <clocale>
#include <string>
#include <cstdlib>
int main()
{
using namespace std;

if (!setlocale( LC_ALL, "greek" ))
{
cerr<< "NULL returned!\n";

return EXIT_FAILURE;
}
wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
If the locale name "greek" means an eight-bit character set is used you
don't need to use wstring, wcin and wcout at all? What character set do
you actually plan to use in your program?

Boris
Feb 24 '08 #23
Boris wrote:
On Sun, 24 Feb 2008 16:46:26 +0200, Ioannis Vranos
<iv*****@nospam.no.spamfreemail.grwrote:
>[...]Instead of messing with these details, perhaps we should accept
that the C subset setlocale() function defined in <clocaleis simpler
(and thus better)?
The following code works:
#include <iostream>
#include <clocale>
#include <string>
#include <cstdlib>
int main()
{
using namespace std;

if (!setlocale( LC_ALL, "greek" ))
{
cerr<< "NULL returned!\n";

return EXIT_FAILURE;
}
wstring ws;

wcin>ws;

wcout<< ws<< endl;
}

If the locale name "greek" means an eight-bit character set is used you
don't need to use wstring, wcin and wcout at all? What character set do
you actually plan to use in your program?

Boris

I am only experimenting with the wide character support. Is there any
possibility the "greek" locale to be only a 8-bit one? And if it is, how
can it also display english (latin characters)? By using something like
ASCII extended (0-127 latin characters and 128-255 greek characters)?
"locale -a" also displays the following greek encodings:
el_GR
el_GR.iso88597
el_GR.utf8

The following also works:
#include <iostream>
#include <clocale>
#include <string>
#include <cstdlib>

int main()
{
using namespace std;

if (!setlocale( LC_ALL, "el_GR.utf8" ))
{
cerr<< "NULL returned!\n";

return EXIT_FAILURE;
}

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$
but the following doesn't:

#include <iostream>
#include <locale>
#include <string>
#include <cstdlib>
int main()
{
using namespace std;

wcout.imbue(locale("el_GR.utf8"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}

[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό

[john@localhost src]$
Feb 24 '08 #24
I managed to make it work under the newer C++ facilities, after having a
look in ISO C++ example code:
The following codes work:
#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

locale::global(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}

[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$
along with el_GR, el_GR.iso88597, el_GR.utf8. For example the following
code also works:
#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

locale::global(locale("el_GR.utf8"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$
Feb 24 '08 #25
Ioannis Vranos wrote:
I filed a bug in GCC Bugzilla:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35353
Ahh, sanity from the Gnu. :)

May anyone explain me how should I apply the suggested solution
"sync_with_stdio (false)", and the fstream suggested solution to the
following failing code?
fstream has nothing to do with it. The Gnu developer just thought you
might have used sync_with_stdio before with file streams. What he means
is that before you do any I/O, you should make a call like this:

std::ios_base::sync_with_stdio(false);

The following program, as far as I can tell, works find on my system.

#include <iostream>
#include <locale>

int main()
{
std::ios_base::sync_with_stdio(false);

std::wcout.imbue(std::locale("el_GR.UTF-8"));

std::wcout<< L"Δοκιμαστικό μήνυμα\n";
std::wcout.flush();
}
Feb 24 '08 #26
Jeff Schwab wrote:
Ioannis Vranos wrote:
>I filed a bug in GCC Bugzilla:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35353

Ahh, sanity from the Gnu. :)

>May anyone explain me how should I apply the suggested solution
"sync_with_stdio (false)", and the fstream suggested solution to the
following failing code?

fstream has nothing to do with it. The Gnu developer just thought you
might have used sync_with_stdio before with file streams. What he means
is that before you do any I/O, you should make a call like this:

std::ios_base::sync_with_stdio(false);

The following program, as far as I can tell, works find on my system.

#include <iostream>
#include <locale>

int main()
{
std::ios_base::sync_with_stdio(false);

std::wcout.imbue(std::locale("el_GR.UTF-8"));

std::wcout<< L"Δοκιμαστικό μήνυμα\n";
std::wcout.flush();
}

Yes it works OK here too. How does desynchronizing with stdio makes the
program work?
Feb 24 '08 #27
Correction:
Ioannis Vranos wrote:
I managed to make it work under the newer C++ facilities, after having a
look in ISO C++ example code:
The following codes work:
#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

locale::global(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}

Although the above works, I think the correct version is the following:
#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

locale::global(locale("greek"));

wcin.imbue(locale());
wcout.imbue(locale());

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
Feb 24 '08 #28
Ioannis Vranos wrote:
Jeff Schwab wrote:
>Ioannis Vranos wrote:
>>I filed a bug in GCC Bugzilla:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35353

Ahh, sanity from the Gnu. :)

>>May anyone explain me how should I apply the suggested solution
"sync_with_stdio (false)", and the fstream suggested solution to the
following failing code?

fstream has nothing to do with it. The Gnu developer just thought you
might have used sync_with_stdio before with file streams. What he
means is that before you do any I/O, you should make a call like this:

std::ios_base::sync_with_stdio(false);

The following program, as far as I can tell, works find on my system.

#include <iostream>
#include <locale>

int main()
{
std::ios_base::sync_with_stdio(false);

std::wcout.imbue(std::locale("el_GR.UTF-8"));

std::wcout<< L"Δοκιμαστικό μήνυμα\n";
std::wcout.flush();
}


Yes it works OK here too. How does desynchronizing with stdio makes the
program work?
No idea. From your other post, the code appears to work without the
sync call.

sync_with_stdio(false) means that the C++ I/O streams no longer have to
coordinate their actions with the C I/O streams: stdin, stdout, and
stderr. The Gnu developer seemed to imply that the C I/O streams had
some fundamental limitation that prevented them to support multi-byte
text. His solution was to "untie" wcout from stdout.
Feb 24 '08 #29
On Sun, 24 Feb 2008 18:15:29 +0200, Ioannis Vranos
<iv*****@nospam.no.spamfreemail.grwrote:
[...]I am only experimenting with the wide character support. Is there
any possibility the "greek" locale to be only a 8-bit one? And if it is,
how can it also display english (latin characters)? By using something
like ASCII extended (0-127 latin characters and 128-255 greek
characters)?
You might want to use this character set:
http://en.wikipedia.org/wiki/ISO/IEC_8859-7

Boris
Feb 24 '08 #30
Jeff Schwab wrote:
>
>>>
The following program, as far as I can tell, works find on my system.

#include <iostream>
#include <locale>

int main()
{
std::ios_base::sync_with_stdio(false);

std::wcout.imbue(std::locale("el_GR.UTF-8"));

std::wcout<< L"Δοκιμαστικό μήνυμα\n";
std::wcout.flush();
}


Yes it works OK here too. How does desynchronizing with stdio makes
the program work?

No idea. From your other post, the code appears to work without the
sync call.

sync_with_stdio(false) means that the C++ I/O streams no longer have to
coordinate their actions with the C I/O streams: stdin, stdout, and
stderr. The Gnu developer seemed to imply that the C I/O streams had
some fundamental limitation that prevented them to support multi-byte
text. His solution was to "untie" wcout from stdout.

The bottom line of this, as far as I can understand, is that my & your
GCC has a bug for standard stream imbue() specialisations when
locale::global() is not specified while the C++ standard I/O is
synchronized with the C subset standard I/O.

When we desync them, the standard stream imbue() locale specialisations
work normally.
Feb 24 '08 #31
Ioannis Vranos wrote:
Jeff Schwab wrote:
>>
>>>>
The following program, as far as I can tell, works find on my system.

#include <iostream>
#include <locale>

int main()
{
std::ios_base::sync_with_stdio(false);

std::wcout.imbue(std::locale("el_GR.UTF-8"));

std::wcout<< L"Δοκιμαστικό μήνυμα\n";
std::wcout.flush();
}
Yes it works OK here too. How does desynchronizing with stdio makes
the program work?

No idea. From your other post, the code appears to work without the
sync call.

sync_with_stdio(false) means that the C++ I/O streams no longer have
to coordinate their actions with the C I/O streams: stdin, stdout,
and stderr. The Gnu developer seemed to imply that the C I/O streams
had some fundamental limitation that prevented them to support
multi-byte text. His solution was to "untie" wcout from stdout.


The bottom line of this, as far as I can understand, is that my & your
GCC has a bug for standard stream imbue() specialisations when
locale::global() is not specified while the C++ standard I/O is
synchronized with the C subset standard I/O.

When we desync them, the standard stream imbue() locale specialisations
work normally.


Actually GNU do not consider it as a bug, but as an
implementation-defined behaviour. As he said:

"Not a bug, given our implementation-defined behavior: the various cin /
wcin, streams are by default synced with stdio (per the standard
requirements) and thus not converting".
Feb 24 '08 #32
Ioannis Vranos wrote:
Ioannis Vranos wrote:
>Jeff Schwab wrote:
>>>
>
The following program, as far as I can tell, works find on my system.
>
#include <iostream>
#include <locale>
>
int main()
{
std::ios_base::sync_with_stdio(false);
>
std::wcout.imbue(std::locale("el_GR.UTF-8"));
>
std::wcout<< L"Δοκιμαστικό μήνυμα\n";
std::wcout.flush();
}
Yes it works OK here too. How does desynchronizing with stdio makes
the program work?

No idea. From your other post, the code appears to work without the
sync call.

sync_with_stdio(false) means that the C++ I/O streams no longer have
to coordinate their actions with the C I/O streams: stdin, stdout,
and stderr. The Gnu developer seemed to imply that the C I/O streams
had some fundamental limitation that prevented them to support
multi-byte text. His solution was to "untie" wcout from stdout.


The bottom line of this, as far as I can understand, is that my & your
GCC has a bug for standard stream imbue() specialisations when
locale::global() is not specified while the C++ standard I/O is
synchronized with the C subset standard I/O.

When we desync them, the standard stream imbue() locale
specialisations work normally.

Actually GNU do not consider it as a bug, but as an
implementation-defined behaviour. As he said:

"Not a bug, given our implementation-defined behavior: the various cin /
wcin, streams are by default synced with stdio (per the standard
requirements) and thus not converting".
That's enough for me, but I rarely need multi-byte characters. Please
post back to c.l.c++ if you com across any other interesting issues
related to languages other than English.
Feb 24 '08 #33
On Feb 25, 2:44*am, Ioannis Vranos <ivra...@nospam.no.spamfreemail.gr>
wrote:
Jeff Schwab wrote:
>The following program, as far as I can tell, works find on my system.
>* * #include <iostream>
* * #include <locale>
>* * int main()
* * {
* * * * std::ios_base::sync_with_stdio(false);
>* * * * std::wcout.imbue(std::locale("el_GR.UTF-8"));
>* * * * std::wcout<< L" \n";
* * * * std::wcout.flush();
* * }
Yes it works OK here too. How does desynchronizing with stdio makes
the program work?
No idea. *From your other post, the code appears to work without the
sync call.
sync_with_stdio(false) means that the C++ I/O streams no longer have to
coordinate their actions with the C I/O streams: *stdin, stdout, and
stderr. *The Gnu developer seemed to imply that the C I/O streams had
some fundamental limitation that prevented them to support multi-byte
text. *His solution was to "untie" wcout from stdout.

The bottom line of this, as far as I can understand, is that my & your
GCC has a bug for standard stream imbue() specialisations when
locale::global() is not specified while the C++ standard I/O is
synchronized with the C subset standard I/O.

When we desync them, the standard stream imbue() locale specialisations
work normally.- Hide quoted text -

- Show quoted text -
may i suggest usage of wcin.imbue(locale("greek")) as well as
wcout.imbue(locale("greek")).Looks like your program worked when
global locale was set. But the program has locale imbued only to wcout
and not to wcin. Setting the global locale sets both wcin and wcout to
that locale. Referring C++ standard I/O streams & locales by Langer/
Kreft, different locales can be imbued to different streams and still
all the streams can be used without crossing each other. Currently i
am handicapped to try this since our environment has only english
locale.

Moreover sync_with_stdio has something to do with performance, again
referring Langer/Kreft.
Feb 25 '08 #34
On Feb 24, 10:52 pm, Ioannis Vranos
<ivra...@nospam.no.spamfreemail.grwrote:

[...]
Actually GNU do not consider it as a bug, but as an
implementation-defined behaviour. As he said:
"Not a bug, given our implementation-defined behavior: the
various cin / wcin, streams are by default synced with stdio
(per the standard requirements) and thus not converting".
That's a wierd statement. wcin inputs wide characters as bytes,
so unless wchar_t is the same size as char, it has to convert
somehow. And I don't see any relationship between the
conversion (which is required to depend on the imbued locale)
and stdio (which is required by the C standard to convert wide
character I/O according to the global locale).

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique oriente objet/
Beratung in objektorientierter Datenverarbeitung
9 place Smard, 78210 St.-Cyr-l'cole, France, +33 (0)1 30 23 00 34

Feb 25 '08 #35
On Feb 25, 5:32 am, kasthurirangan.bal...@gmail.com wrote:

[...]
Moreover sync_with_stdio has something to do with performance,
again referring Langer/Kreft.
sync_with_stdio basically means that mixing stdio and iostream
on the same stream (with cin==stdin, etc.) will work. It's
actual effect is that anytime you do I/O on an iostream, the
streambuf will first call fflush() on stdout, and vice versa.
Or behave as if that were the case.

If the implementation of stdio knows about the implementation of
iostream, and vice versa, this is relatively easy to handle,
without any significant loss of performance. If the two are
independent (usually the case, and definitely the case with g++,
where stdio is normally provided by the platform), stdio isn't
going to sync with the iostream, so iostream basically has to
call fflush before each operation (which shouldn't cost much if
there hasn't actually been any output), and sync after every
operation (which can be very, very expensive in terms of time).

Of course, none of this has anything to do with how the iostream
transcodes its input or output.

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique oriente objet/
Beratung in objektorientierter Datenverarbeitung
9 place Smard, 78210 St.-Cyr-l'cole, France, +33 (0)1 30 23 00 34
Feb 25 '08 #36
ka*******************@gmail.com wrote:
>
may i suggest usage of wcin.imbue(locale("greek")) as well as
wcout.imbue(locale("greek")).Looks like your program worked when
global locale was set. But the program has locale imbued only to wcout
and not to wcin. Setting the global locale sets both wcin and wcout to
that locale. Referring C++ standard I/O streams & locales by Langer/
Kreft, different locales can be imbued to different streams and still
all the streams can be used without crossing each other. Currently i
am handicapped to try this since our environment has only english
locale.

Moreover sync_with_stdio has something to do with performance, again
referring Langer/Kreft.

If you mean this:

#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

// locale::global(locale("greek"));

wcin.imbue(locale("greek"));
wcout.imbue(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
It doesn't work:
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό

[john@localhost src]$

If I uncomment the commented line, it works:

#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

locale::global(locale("greek"));

wcin.imbue(locale("greek"));
wcout.imbue(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$
Feb 25 '08 #37
Ioannis Vranos wrote:
ka*******************@gmail.com wrote:
>>
may i suggest usage of wcin.imbue(locale("greek")) as well as
wcout.imbue(locale("greek")).Looks like your program worked when
global locale was set. But the program has locale imbued only to wcout
and not to wcin. Setting the global locale sets both wcin and wcout to
that locale. Referring C++ standard I/O streams & locales by Langer/
Kreft, different locales can be imbued to different streams and still
all the streams can be used without crossing each other. Currently i
am handicapped to try this since our environment has only english
locale.

Moreover sync_with_stdio has something to do with performance, again
referring Langer/Kreft.


If you mean this:

#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

// locale::global(locale("greek"));

wcin.imbue(locale("greek"));
wcout.imbue(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
It doesn't work:
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό

[john@localhost src]$

If I uncomment the commented line, it works:

#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

locale::global(locale("greek"));

wcin.imbue(locale("greek"));
wcout.imbue(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$


This also works:

#include <iostream>
#include <locale>
#include <string>
int main()
{
using namespace std;

locale::global(locale("en_US"));

wcin.imbue(locale("greek"));
wcout.imbue(locale("greek"));

wstring ws;

wcin>ws;

wcout<< ws<< endl;
}

[john@localhost src]$ ./foobar-cpp
Δοκιμαστικό
Δοκιμαστικό
[john@localhost src]$
In summary, the locale specialisations work only either when we use the
locale::global() statement, or we use the
std::ios_base::sync_with_stdio(false); statement. I have the feeling
this is a bug and not an implementation-defined behaviour.
Feb 25 '08 #38
On Feb 25, 4:53*pm, Ioannis Vranos <ivra...@nospam.no.spamfreemail.gr>
wrote:
Ioannis Vranos wrote:
kasthurirangan.bal...@gmail.com wrote:
may i suggest usage of wcin.imbue(locale("greek")) as well as
wcout.imbue(locale("greek")).Looks like your program worked when
global locale was set. But the program has locale imbued only to wcout
and not to wcin. Setting the global locale sets both wcin and wcout to
that locale. Referring C++ standard I/O streams & locales by Langer/
Kreft, different locales can be imbued to different streams and still
all the streams can be used without crossing each other. Currently i
am handicapped to try this since our environment has only english
locale.
Moreover sync_with_stdio has something to do with performance, again
referring Langer/Kreft.
If you mean this:
#include <iostream>
#include <locale>
#include <string>
int main()
{
* * using namespace std;
* // *locale::global(locale("greek"));
* * wcin.imbue(locale("greek"));
* * wcout.imbue(locale("greek"));
* * wstring ws;
* * wcin>ws;
* * wcout<< ws<< endl;
}
It doesn't work:
[john@localhost src]$ ./foobar-cpp
[john@localhost src]$
If I uncomment the commented line, it works:
#include <iostream>
#include <locale>
#include <string>
int main()
{
* * using namespace std;
* * locale::global(locale("greek"));
* * wcin.imbue(locale("greek"));
* * wcout.imbue(locale("greek"));
* * wstring ws;
* * wcin>ws;
* * wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp


[john@localhost src]$

This also works:

#include <iostream>
#include <locale>
#include <string>

int main()
{
* * *using namespace std;

* * *locale::global(locale("en_US"));

* * *wcin.imbue(locale("greek"));
* * *wcout.imbue(locale("greek"));

* * *wstring ws;

* * *wcin>ws;

* * *wcout<< ws<< endl;

}

[john@localhost src]$ ./foobar-cpp


[john@localhost src]$

In summary, the locale specialisations work only either when we use the
locale::global() statement, or we use the
std::ios_base::sync_with_stdio(false); statement. I have the feeling
this is a bug and not an implementation-defined behaviour.- Hide quoted text -

- Show quoted text -
could you pls try without global locale!!!

i am not sure how the conclusion was arrived - use
std::ios_base::sync_with_stdio(false). Shall update if i come across
something similar.

Thanks,
Balaji.
Feb 25 '08 #39
On Feb 25, 5:22*pm, kasthurirangan.bal...@gmail.com wrote:
On Feb 25, 4:53*pm, Ioannis Vranos <ivra...@nospam.no.spamfreemail.gr>
wrote:


Ioannis Vranos wrote:
kasthurirangan.bal...@gmail.com wrote:
>may i suggest usage of wcin.imbue(locale("greek")) as well as
>wcout.imbue(locale("greek")).Looks like your program worked when
>global locale was set. But the program has locale imbued only to wcout
>and not to wcin. Setting the global locale sets both wcin and wcout to
>that locale. Referring C++ standard I/O streams & locales by Langer/
>Kreft, different locales can be imbued to different streams and still
>all the streams can be used without crossing each other. Currently i
>am handicapped to try this since our environment has only english
>locale.
>Moreover sync_with_stdio has something to do with performance, again
>referring Langer/Kreft.
If you mean this:
#include <iostream>
#include <locale>
#include <string>
int main()
{
* * using namespace std;
* // *locale::global(locale("greek"));
* * wcin.imbue(locale("greek"));
* * wcout.imbue(locale("greek"));
* * wstring ws;
* * wcin>ws;
* * wcout<< ws<< endl;
}
It doesn't work:
[john@localhost src]$ ./foobar-cpp
[john@localhost src]$
If I uncomment the commented line, it works:
#include <iostream>
#include <locale>
#include <string>
int main()
{
* * using namespace std;
* * locale::global(locale("greek"));
* * wcin.imbue(locale("greek"));
* * wcout.imbue(locale("greek"));
* * wstring ws;
* * wcin>ws;
* * wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp


[john@localhost src]$
This also works:
#include <iostream>
#include <locale>
#include <string>
int main()
{
* * *using namespace std;
* * *locale::global(locale("en_US"));
* * *wcin.imbue(locale("greek"));
* * *wcout.imbue(locale("greek"));
* * *wstring ws;
* * *wcin>ws;
* * *wcout<< ws<< endl;
}
[john@localhost src]$ ./foobar-cpp


[john@localhost src]$
In summary, the locale specialisations work only either when we use the
locale::global() statement, or we use the
std::ios_base::sync_with_stdio(false); statement. I have the feeling
this is a bug and not an implementation-defined behaviour.- Hide quoted text -
- Show quoted text -

could you pls try without global locale!!!
the post didn't show your other work - already done for this, atleast
in my browser or maybe i didn't have a proper look at it. hence
suggested this.

It looks wierd. By the way i guess, we need to initialize the locale
first before use(one try could be locale("C") though this should be by
default) - which i do not know if the standard specifies that way or
its the implementation.

Thanks,
Balaji.
>
i am not sure how the conclusion was arrived *- use
std::ios_base::sync_with_stdio(false). Shall update if i come across
something similar.

Thanks,
Balaji.- Hide quoted text -

- Show quoted text -
Feb 25 '08 #40
On 2008-02-24 10:51:32, James Kanze wrote:
>>>Also I have MS Visual C++ 2008 Express installed.
>>Under Linux ! :-)
>Linux::VMWare::Windows::VC++2008 Express.

Thanks. I'll give it a try myself. (Of course, the executables it
generates will also require VMWare to run, but it will allow at least
verifying that my code compiles with VC++ before trying to port it to
Windows.)
How will the generated executables require VMware to run? If you generate
executables in such an environment, they'll run on any of the Windows
platforms supported by your code and compilation, whether under VMware or
not.

Gerhard
Feb 25 '08 #41
On Feb 25, 2:08 pm, Gerhard Fiedler <geli...@gmail.comwrote:
On 2008-02-24 10:51:32, James Kanze wrote:
>>Also I have MS Visual C++ 2008 Express installed.
>Under Linux ! :-)
Linux::VMWare::Windows::VC++2008 Express.
Thanks. I'll give it a try myself. (Of course, the
executables it generates will also require VMWare to run,
but it will allow at least verifying that my code compiles
with VC++ before trying to port it to Windows.)
How will the generated executables require VMware to run?
How will they not? VC++ will certainly still produce a Windows
executable, not a Linux executable.
If you generate executables in such an environment, they'll
run on any of the Windows platforms supported by your code and
compilation, whether under VMware or not.
If I had a Windows platform handy, I'd run VC++ on it. If I
have to run it under VMware, it's because I don't have a Windows
platform available otherwise.

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique oriente objet/
Beratung in objektorientierter Datenverarbeitung
9 place Smard, 78210 St.-Cyr-l'cole, France, +33 (0)1 30 23 00 34
Feb 26 '08 #42
On Feb 25, 1:22 pm, kasthurirangan.bal...@gmail.com wrote:
On Feb 25, 4:53 pm, Ioannis Vranos <ivra...@nospam.no.spamfreemail.gr>
wrote:
[...]
i am not sure how the conclusion was arrived - use
std::ios_base::sync_with_stdio(false).
What does sync_with_stdio have to do with the locales? With the
exception of possible performance issues, and when buffers are
getting flushed (which is unspecified anyway), sync_with_stdio
should have no impact as long as all I/O actually is done with
iostream. If this is not the case, there's a problem with the
implementation.

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique oriente objet/
Beratung in objektorientierter Datenverarbeitung
9 place Smard, 78210 St.-Cyr-l'cole, France, +33 (0)1 30 23 00 34
Feb 26 '08 #43
On 2008-02-26 05:59:44, James Kanze wrote:
>>(Of course, the executables it generates will also require VMWare to
run, but it will allow at least verifying that my code compiles with
VC++ before trying to port it to Windows.)
>How will the generated executables require VMware to run?

How will they not? VC++ will certainly still produce a Windows
executable, not a Linux executable.
Exactly. And that exe requires Windows, not VMware.
If I had a Windows platform handy, I'd run VC++ on it. If I
have to run it under VMware, it's because I don't have a Windows
platform available otherwise.
Right. But that's not a requirement of the generated executable. You can
install Windows in a dual-boot config or send the exe to someone else with
a Windows system or use a different virtualizer or Windows emulator, and
the exe will run just fine without VMware. It doesn't even know it was
compiled under VMware.

Gerhard
Feb 26 '08 #44
Hi!

Ioannis Vranos schrieb:
In summary, the locale specialisations work only either when we use the
locale::global() statement, or we use the
std::ios_base::sync_with_stdio(false); statement. I have the feeling
this is a bug and not an implementation-defined behaviour.
You are using gcc on linux. With msvc8 on windows it seems to be the
other way round: When I set an utf8 converting locale on wcout only, it
works. However if I set it on wcout AND globally the output will be
garbled. Maybe it tries to do the conversion twice.

Note: I tested the utf8 output in an eclipse console which is set to
utf8 encoding.

From my current experience there is no usable wcout implementation that
reliably makes use of the conversion facet. Maybe I will just implement
a std wide (wchar_t) stream that outputs to a narrow stream and uses the
utf8 encoding. I could use that to simply wrap cout and replace wcout.

Frank
Jun 27 '08 #45

This discussion thread is closed

Replies have been disabled for this discussion.

By using this site, you agree to our Privacy Policy and Terms of Use.