471,342 Members | 1,687 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,342 software developers and data experts.

encode/decode misunderstanding

Hi, I'm beginning to understand the encode/decode string methods, but I'd
like confirmation that I'm still thinking in the right direction:

I have a file of latin1 encoded text. Let's say I put one line of that file
into a string variable 'tocline', as follows:
tocline = 'Ficha Datos de p\xe9rdida AND acci\xf3n'

import codecs
tocFile = codecs.open('mytoc.htm','wb',encoding='utf8',error s='replace')
tocline = tocline.decode('latin1','replace')
tocFile.write(tocline)
tocFile.close()

What I think is that tocFile is wrapped to insure that anything written to
it is in utf8
I decode the latin1 string into python's internal unicode encoding and that
gets written out as utf8.

Questions:
what exactly is the tocline when it's read in with that \xe9 and \xed in the
string? A latin1 encoded string?
Is my method the right way to write such a line out to a file with utf8
encoding?

If I read in the latin1 file using
codecs.open(filename,encoding='latin1') and write out the utf8 file by
opening with
codecs.open(othername,encoding='utf8'), would I no longer have a problem --
I could just read in latin1 and write out utf8 with no more worries about
encoding?

thanks,
--Tim
Jul 26 '07 #1
3 4379
If I read in the latin1 file using
codecs.open(filename,encoding='latin1') and write out the utf8 file by
opening with
codecs.open(othername,encoding='utf8'), would I no longer have a
problem -- I could just read in latin1 and write out utf8 with no more
worries about encoding?

thanks,
Replying to my own post, I feel so lonely! I guess that silence means I *am*
thinking correctly about the encoding/decoding stuff; I'll keep heading in
this direction unless someone out there sees it differently.....

--Tim
Jul 27 '07 #2
Tim Arnold schrieb:
Hi, I'm beginning to understand the encode/decode string methods, but I'd
like confirmation that I'm still thinking in the right direction:

I have a file of latin1 encoded text. Let's say I put one line of that file
into a string variable 'tocline', as follows:
tocline = 'Ficha Datos de p\xe9rdida AND acci\xf3n'

import codecs
tocFile = codecs.open('mytoc.htm','wb',encoding='utf8',error s='replace')
tocline = tocline.decode('latin1','replace')
tocFile.write(tocline)
tocFile.close()

What I think is that tocFile is wrapped to insure that anything written to
it is in utf8
I decode the latin1 string into python's internal unicode encoding and that
gets written out as utf8.

Questions:
what exactly is the tocline when it's read in with that \xe9 and \xed in the
string? A latin1 encoded string?
Yes. A simple, pure byte-string, that happens to contain bytes which
under the latin1-encoding are "correct".
Is my method the right way to write such a line out to a file with utf8
encoding?
Yes.
If I read in the latin1 file using
codecs.open(filename,encoding='latin1') and write out the utf8 file by
opening with
codecs.open(othername,encoding='utf8'), would I no longer have a problem --
I could just read in latin1 and write out utf8 with no more worries about
encoding?
As long as you don't mix bytestrings and only use unicode-objects, you
should be fine, yes.

Diez
Jul 29 '07 #3
"Diez B. Roggisch" <de***@nospam.web.dewrote in message
news:5h*************@mid.uni-berlin.de...
Tim Arnold schrieb:
>Hi, I'm beginning to understand the encode/decode string methods, but I'd
like confirmation that I'm still thinking in the right direction:

I have a file of latin1 encoded text. Let's say I put one line of that
file into a string variable 'tocline', as follows:
tocline = 'Ficha Datos de p\xe9rdida AND acci\xf3n'

import codecs
tocFile = codecs.open('mytoc.htm','wb',encoding='utf8',error s='replace')
tocline = tocline.decode('latin1','replace')
tocFile.write(tocline)
tocFile.close()

What I think is that tocFile is wrapped to insure that anything written
to it is in utf8
I decode the latin1 string into python's internal unicode encoding and
that gets written out as utf8.

Questions:
what exactly is the tocline when it's read in with that \xe9 and \xed in
the string? A latin1 encoded string?

Yes. A simple, pure byte-string, that happens to contain bytes which under
the latin1-encoding are "correct".
>Is my method the right way to write such a line out to a file with utf8
encoding?

Yes.
>If I read in the latin1 file using
codecs.open(filename,encoding='latin1') and write out the utf8 file by
opening with
codecs.open(othername,encoding='utf8'), would I no longer have a
problem -- I could just read in latin1 and write out utf8 with no more
worries about encoding?

As long as you don't mix bytestrings and only use unicode-objects, you
should be fine, yes.

Diez
wow, I was thinking correctly about encoding! time for a beer!
Diez, thanks very much for confirming my thoughts.

--Tim Arnold
Jul 30 '07 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Damir Hakimov | last post: by
1 post views Thread by AR | last post: by
6 posts views Thread by 7stud | last post: by
15 posts views Thread by glacier | last post: by
1 post views Thread by anonymous | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.