472,371 Members | 1,586 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,371 software developers and data experts.

utf-8 read/write file

Hi!

I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(*,Đ,Ž,Č,Ć).
How can I read/write from/to file in utf-8 encoding?
I read file with fileinput.input.

thanks
Oct 8 '08 #1
4 4603
On Oct 8, 12:49*pm, Bruno <Br...@hi.t-com.hrwrote:
Hi!

I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).
Can you show us what you have so far?
How can I read/write from/to file in utf-8 encoding?
import codecs
data = codecs.open("my-utf8-file.txt").read()
I read file with fileinput.input.

thanks
Oct 8 '08 #2
Benjamin wrote:
On Oct 8, 12:49 pm, Bruno <Br...@hi.t-com.hrwrote:
>Hi!

I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).

Can you show us what you have so far?
>How can I read/write from/to file in utf-8 encoding?

import codecs
data = codecs.open("my-utf8-file.txt").read()
>I read file with fileinput.input.

thanks
I have tried with codecs, but when i use encoding="utf-8" i get this error on
word : ivot

Traceback (most recent call last):
File "C:\Users\Administrator\Desktop\getcontent.py" , line 43, in <module>
encoding="utf-8").readlines()
File "C:\Python25\Lib\codecs.py", line 626, in readlines
return self.reader.readlines(sizehint)
File "C:\Python25\Lib\codecs.py", line 535, in readlines
data = self.read()
File "C:\Python25\Lib\codecs.py", line 424, in read
newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x9e in position 0:
unexpected code byte
i just need to read from file1.txt, process (its simple text processing) some
words and write them to file2.txt without loss of croatian characters. ()
Oct 8 '08 #3
On Oct 8, 5:55*pm, gigs <g...@hi.t-com.hrwrote:
Benjamin wrote:
On Oct 8, 12:49 pm, Bruno <Br...@hi.t-com.hrwrote:
Hi!
I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).

UnicodeDecodeError: 'utf8' codec can't decode byte 0x9e in position 0:
unexpected code byte
Are you sure you have UTF-8 data? I guess your file is encoded in
CP1250 or CP1252; in both of these charsets 0x9e represents LATIN
SMALL LETTER Z WITH CARON.

Kent
Oct 8 '08 #4
Kent Johnson wrote:
On Oct 8, 5:55 pm, gigs <g...@hi.t-com.hrwrote:
>Benjamin wrote:
>>On Oct 8, 12:49 pm, Bruno <Br...@hi.t-com.hrwrote:
Hi!
I have big .txt file which i want to read, process and write to another .txt file.
I have done script for that, but im having problem with croatian characters
(,,,,).
UnicodeDecodeError: 'utf8' codec can't decode byte 0x9e in position 0:
unexpected code byte

Are you sure you have UTF-8 data? I guess your file is encoded in
CP1250 or CP1252; in both of these charsets 0x9e represents LATIN
SMALL LETTER Z WITH CARON.

Kent
This data wasnt in utf-8 probably, today i get another one utf-8 and its working

thanks
Oct 9 '08 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

38
by: Haines Brown | last post by:
I'm having trouble finding the character entity for the French abbreviation for "number" (capital N followed by a small supercript o, period). My references are not listing it. Where would I...
32
by: Wolfgang Draxinger | last post by:
I understand that it is perfectly possible to store UTF-8 strings in a std::string, however doing so can cause some implicaions. E.g. you can't count the amount of characters by length() | size()....
0
by: Tim Northrup | last post by:
Help! We have DB2 V7.2 (fixpak 12) installed on Windows2003 Server, and the latest V7.2 client installed on another system. The DB2CODEPAGE on all systems is set to 1208, and the database was...
3
by: Richard Connamacher | last post by:
I'm new to PostgreSQL, and from the looks of it, it's a great database, and I'll be using more of it in the future. I had a quick question if anyone could clear this up. The documentation for...
23
by: Steven T. Hatton | last post by:
This is one of the first obstacles I encountered when getting started with C++. I found that everybody had their own idea of what a string is. There was std::string, QString, xercesc::XMLString,...
1
by: David Bertoni | last post by:
Hi all, I'm trying to resolve what appears to me an inconsistency in the XML 1.0 recommendation involving entities encoding in UTF-16 and the requirement for a byte order mark. Section 4.3.3...
22
by: Albert Oppenheimer | last post by:
I thought my program had to be caught in a loop, and cancelled it through the task manager. It took about one second in Java, but re-implemented in C, it had already run over one minute. I set...
6
by: archana | last post by:
Hi all, can someone tell me difference between unicode and utf 8 or utf 18 and which one is supporting more character set. whic i should use to support character ucs-2. I want to use ucs-2...
7
by: Jimmy Shaw | last post by:
Hi everybody, Is there any SIMPLE way to convert from UTF-16 to UTF-32? I may be mixed up, but is it possible that all UTF-16 "code points" that are 16 bits long appear just the same in UTF-32,...
35
by: Bjoern Hoehrmann | last post by:
Hi, For a free software project, I had to write a routine that, given a Unicode scalar value U+0000 - U+10FFFF, returns an integer that holds the UTF-8 encoded form of it, for example, U+00F6...
2
by: Kemmylinns12 | last post by:
Blockchain technology has emerged as a transformative force in the business world, offering unprecedented opportunities for innovation and efficiency. While initially associated with cryptocurrencies...
0
by: Naresh1 | last post by:
What is WebLogic Admin Training? WebLogic Admin Training is a specialized program designed to equip individuals with the skills and knowledge required to effectively administer and manage Oracle...
0
by: antdb | last post by:
Ⅰ. Advantage of AntDB: hyper-convergence + streaming processing engine In the overall architecture, a new "hyper-convergence" concept was proposed, which integrated multiple engines and...
0
by: Arjunsri | last post by:
I have a Redshift database that I need to use as an import data source. I have configured the DSN connection using the server, port, database, and credentials and received a successful connection...
1
by: Matthew3360 | last post by:
Hi, I have been trying to connect to a local host using php curl. But I am finding it hard to do this. I am doing the curl get request from my web server and have made sure to enable curl. I get a...
0
by: Carina712 | last post by:
Setting background colors for Excel documents can help to improve the visual appeal of the document and make it easier to read and understand. Background colors can be used to highlight important...
0
BLUEPANDA
by: BLUEPANDA | last post by:
At BluePanda Dev, we're passionate about building high-quality software and sharing our knowledge with the community. That's why we've created a SaaS starter kit that's not only easy to use but also...
1
by: ezappsrUS | last post by:
Hi, I wonder if someone knows where I am going wrong below. I have a continuous form and two labels where only one would be visible depending on the checkbox being checked or not. Below is the...
0
by: jack2019x | last post by:
hello, Is there code or static lib for hook swapchain present? I wanna hook dxgi swapchain present for dx11 and dx9.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.