473,322 Members | 1,540 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

utf-8 read/write file

Hi!

I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(*,Đ,Ž,Č,Ć).
How can I read/write from/to file in utf-8 encoding?
I read file with fileinput.input.

thanks
Oct 8 '08 #1
4 4711
On Oct 8, 12:49*pm, Bruno <Br...@hi.t-com.hrwrote:
Hi!

I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).
Can you show us what you have so far?
How can I read/write from/to file in utf-8 encoding?
import codecs
data = codecs.open("my-utf8-file.txt").read()
I read file with fileinput.input.

thanks
Oct 8 '08 #2
Benjamin wrote:
On Oct 8, 12:49 pm, Bruno <Br...@hi.t-com.hrwrote:
>Hi!

I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).

Can you show us what you have so far?
>How can I read/write from/to file in utf-8 encoding?

import codecs
data = codecs.open("my-utf8-file.txt").read()
>I read file with fileinput.input.

thanks
I have tried with codecs, but when i use encoding="utf-8" i get this error on
word : ivot

Traceback (most recent call last):
File "C:\Users\Administrator\Desktop\getcontent.py" , line 43, in <module>
encoding="utf-8").readlines()
File "C:\Python25\Lib\codecs.py", line 626, in readlines
return self.reader.readlines(sizehint)
File "C:\Python25\Lib\codecs.py", line 535, in readlines
data = self.read()
File "C:\Python25\Lib\codecs.py", line 424, in read
newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x9e in position 0:
unexpected code byte
i just need to read from file1.txt, process (its simple text processing) some
words and write them to file2.txt without loss of croatian characters. ()
Oct 8 '08 #3
On Oct 8, 5:55*pm, gigs <g...@hi.t-com.hrwrote:
Benjamin wrote:
On Oct 8, 12:49 pm, Bruno <Br...@hi.t-com.hrwrote:
Hi!
I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).

UnicodeDecodeError: 'utf8' codec can't decode byte 0x9e in position 0:
unexpected code byte
Are you sure you have UTF-8 data? I guess your file is encoded in
CP1250 or CP1252; in both of these charsets 0x9e represents LATIN
SMALL LETTER Z WITH CARON.

Kent
Oct 8 '08 #4
Kent Johnson wrote:
On Oct 8, 5:55 pm, gigs <g...@hi.t-com.hrwrote:
>Benjamin wrote:
>>On Oct 8, 12:49 pm, Bruno <Br...@hi.t-com.hrwrote:
Hi!
I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).
UnicodeDecodeError: 'utf8' codec can't decode byte 0x9e in position 0:
unexpected code byte

Are you sure you have UTF-8 data? I guess your file is encoded in
CP1250 or CP1252; in both of these charsets 0x9e represents LATIN
SMALL LETTER Z WITH CARON.

Kent
This data wasnt in utf-8 probably, today i get another one utf-8 and its working

thanks
Oct 9 '08 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

38
by: Haines Brown | last post by:
I'm having trouble finding the character entity for the French abbreviation for "number" (capital N followed by a small supercript o, period). My references are not listing it. Where would I...
32
by: Wolfgang Draxinger | last post by:
I understand that it is perfectly possible to store UTF-8 strings in a std::string, however doing so can cause some implicaions. E.g. you can't count the amount of characters by length() | size()....
0
by: Tim Northrup | last post by:
Help! We have DB2 V7.2 (fixpak 12) installed on Windows2003 Server, and the latest V7.2 client installed on another system. The DB2CODEPAGE on all systems is set to 1208, and the database was...
3
by: Richard Connamacher | last post by:
I'm new to PostgreSQL, and from the looks of it, it's a great database, and I'll be using more of it in the future. I had a quick question if anyone could clear this up. The documentation for...
23
by: Steven T. Hatton | last post by:
This is one of the first obstacles I encountered when getting started with C++. I found that everybody had their own idea of what a string is. There was std::string, QString, xercesc::XMLString,...
1
by: David Bertoni | last post by:
Hi all, I'm trying to resolve what appears to me an inconsistency in the XML 1.0 recommendation involving entities encoding in UTF-16 and the requirement for a byte order mark. Section 4.3.3...
22
by: Albert Oppenheimer | last post by:
I thought my program had to be caught in a loop, and cancelled it through the task manager. It took about one second in Java, but re-implemented in C, it had already run over one minute. I set...
6
by: archana | last post by:
Hi all, can someone tell me difference between unicode and utf 8 or utf 18 and which one is supporting more character set. whic i should use to support character ucs-2. I want to use ucs-2...
7
by: Jimmy Shaw | last post by:
Hi everybody, Is there any SIMPLE way to convert from UTF-16 to UTF-32? I may be mixed up, but is it possible that all UTF-16 "code points" that are 16 bits long appear just the same in UTF-32,...
35
by: Bjoern Hoehrmann | last post by:
Hi, For a free software project, I had to write a routine that, given a Unicode scalar value U+0000 - U+10FFFF, returns an integer that holds the UTF-8 encoded form of it, for example, U+00F6...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, youll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shllpp 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.