473,382 Members | 1,377 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,382 software developers and data experts.

convert gb18030 to utf16

i have a bunch of files encoded in GB18030. Is there a way to convert
them to utf16 with python?

Xah
xa*@xahlee.org
http://xahlee.org/PageTwo_dir/more.html

Jul 18 '05 #1
2 4007
Xah Lee <xa*@xahlee.org> wrotE:
i have a bunch of files encoded in GB18030. Is there a way to convert
them to utf16 with python?


You will need CJKCodecs (http://cjkpython.i18n.org/), or Python 2.4,
which has them built in. Then just use them like any other codec. eg.

f= open(path, 'rb')
content= unicode(f.read(), 'gb18030')
f.close()
f= open(path, 'wb')
f.write(content.encode('utf-16'))
f.close()

--
Andrew Clover
mailto:an*@doxdesk.com
http://www.doxdesk.com/

Jul 18 '05 #2
Truely superb!

Thanks!

Xah
xa*@xahlee.org
http://xahlee.org/

an********@doxdesk.com wrote:
Xah Lee <xa*@xahlee.org> wrotE:
i have a bunch of files encoded in GB18030. Is there a way to convert them to utf16 with python?


You will need CJKCodecs (http://cjkpython.i18n.org/), or Python 2.4,
which has them built in. Then just use them like any other codec. eg.

f= open(path, 'rb')
content= unicode(f.read(), 'gb18030')
f.close()
f= open(path, 'wb')
f.write(content.encode('utf-16'))
f.close()

--
Andrew Clover
mailto:an*@doxdesk.com
http://www.doxdesk.com/


Jul 18 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: John Perks and Sarah Mount | last post by:
(My Python uses UTF16 natively; can someone with UTF32 Python let me know if that behaves differently?) >>> import codecs >>> u'\ud800' # part of surrogate pair u'\ud800'...
12
by: Chris Mullins | last post by:
I'm implementing RFC 3491 in .NET, and running into a strange issue. Step 1 of RFC 3491 is performing a set of mappings dicated by tables B.1 and B.2. I'm having trouble with the following...
17
by: Nikolay Petrov | last post by:
How can I convert DOS cyrillic text to Unicode
4
by: Fuzzyman | last post by:
Hello all, I'm handling some text files where I don't (necessarily) know the encoding beforehand. Because I use regular expressions to parse the text I *must* decode UTF16 encoded text...
15
by: Buddy Home | last post by:
Hello, I'm trying to speed up a piece of code that is causing performance issues with our product. The problem is we are using serialization to convert the object to a string, this is costing us...
4
by: R Wood | last post by:
Greetings - A recent Perl experiment hasn't turned out so well, which has piqued my interest in Python. The project is this: take a Vcard file exported from Apple's Addressbook and use a...
1
by: Server Applications | last post by:
Hello I am trying to build a system where I can full-text index documents with UTF8 or UTF16 data using Oracle Text. I am doing the filtering in a third-party component outside the database, so...
19
by: est | last post by:
From python manual str( ) Return a string containing a nicely printable representation of an object. For strings, this returns the string itself. The difference with repr(object) is that...
7
by: firepol | last post by:
Hello there, I am dealing with files encoded in UTF8 and I can't find a way to convert them into ANSI. I've already searched in google for this since a while, and I'm not achieving the result I...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.