Jython 2.1 and Cheetah:Cyrillic templates output mixed encoding

[Mike Grigorov]

I am using cheetah with jython 2.1 and I have problem generating templates
that contain cyrillic text. Some texts are printed OK and others are not.

I searched Jython and cheetah projects mailing lists on sourceforge.net as
well as these groups and tried some other fixes proposed but without success.

I call the template like this:

PythonInterpreter pyiInterp = new PythonInterpreter();
pyiInterp.set("textTool", TextToolkit.getInstance());
String _body = readFile(strTemplateFile.replace('\\', '/'));

//----Encoding:
PyString _pyBody = new PyString(_body);
//----

pyiInterp.set("templBody", _pyBody);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
PrintWriter pwOut = new PrintWriter(bos);
pyiInterp.setOut(pwOut);
pyiInterp.exec("from Cheetah.Template import Template");
pyiInterp.exec("t = Template(source=templBody)");
pyiInterp.exec("t.textTool = textTool");
pyiInterp.exec("print t");

I tried replacing the section marked as //---Encoding:
String isoScript = _body;
try {
//The call to sun.io.Converters.getDefaultEncodingName() returns "Cp1251"
byte[] bytes = _body.getBytes(sun.io.Converters.getDefaultEncodin gName());
isoScript = new String(bytes, "ISO-8859-1");
}
catch (Exception ex) {
System.out.println("CONVERSION ERROR:" + ex);
}
PyString _pyBody = new PyString(isoScript);
This resulted in the problem inverted - the 'good' text became 'bad' and vice versa.

Here are sample lines one of which prints good text and the second - not
good(depending on its position in the template) /I guess text will be garbled
here/ :

${textTool.filler(35, " ")}????????? ????
${textTool.filler(35, " ")}???????????? ?? ??????????

textTool is a java object and this call produces 35 spaces

The encoding of the lines is the same(I checked the int values for the
letters).
I tried:

import sys
sys.setdefaultencoding('cp1251')

It didn't help but it caused compilation of the encodings\cp1251.py module ->
cp1251$py.class so I guess it was taken into account

Please guide me to papers that can help solving this or propose ideas what
might be causing this behaviour.

Thanks for your cooperation
Mike

[Mike Grigorov]

mi****@abv.bg (Mike Grigorov) wrote in message news:<9d**************************@posting.google. com>...

I am using cheetah with jython 2.1 and I have problem generating templates
that contain cyrillic text. Some texts are printed OK and others are not.

The fact is that cheetah had nothing to do with the problem. I studied Jython
code and changed PyString and PyFile classes in the org.python.core package. I
changed all calls to deprecated methods of java.lang.String with their
recommended replacements and the problem seems to be solved. I also changed
references to StringBufferInputStream with ByteArrayInputStream in the "Py" and
"parser" classes of the same package but haven't studied if this helped to solve
the problem.
Thanks for your time
Mike