473,718 Members | 2,030 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Python / Chinese Encodings


I need to convert Big5 or GB encoded chinese strings to unicode. It would
be also nice to be able to detect the encoding of the original string.
Search with groups.google.c om I found some links to different projects but
they all look not very active. Can somebody give me a short overview of the
status of processing chinese texts with python?

Jul 18 '05 #1
1 2670
"Achim Domma" <do***@procoder s.net> writes:
I need to convert Big5 or GB encoded chinese strings to unicode. It would
be also nice to be able to detect the encoding of the original string.
Search with groups.google.c om I found some links to different projects but
they all look not very active. Can somebody give me a short overview of the
status of processing chinese texts with python?

The very short summary: Use the CJK codecs package; it supports all
encodings you might encounter, and it is actively maintained.

As for detecting the encoding of the original string: Forget it. Tell
your communication partners to always properly declare the encoding.


Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: Achim Domma | last post by:
Hi, I build successfully different COM servers with py2exe. In my setup.py I have something like this: opts = { "py2exe" : { "includes" : , "packages" : , }
by: Anthony Liu | last post by:
The following 4 lines of code parses an XML document very well if the XML document contains only English words. But when I insert one Chinese character into the XML document, then Python starts to complain when it hits the Chinese character, saying that it is an invalid token and thus it is not well-formed. This is the complaint of Python:
by: Peter Otten | last post by:
I took the freedom to forward GvR's mail concerning decorator cosmetics. I think you should know about it. Peter ---------- Forwarded Message ---------- Subject: Strawman decision: @decorator won't change Date: Wednesday 15 September 2004 23:46 From: Guido van Rossum <gvanrossum@gmail.com>
by: Coco | last post by:
Hi! I managed to display chinese character in my web form (.aspx), in certain situation i need to to set the text of the label of my web form in chinese character programatically which is done in my code behind. when running the web form, the character which is entered directly to the web form during design time is displayed correctly, while those set from code behind appear as funny character
by: pabv | last post by:
Hello all, I am having a few issues with encoding to chinese characters and perhaps someone might be able to assist. At the moment I am only able to see chinese characters when displayed as part of a datagrid. When an input textbox is displayed it does not display chinese characters, but rather the unicode characters stored in the mssql 2000 server backend.
by: Petr Prikryl | last post by:
I did observe the problem when using the -U option on Windows 2000. Seems like some infinite recursion in cp1250.py -- see below. I did not try it with earlier versions of Python. Can this be reproduced on your computer? Thanks for your time and experience, pepr
by: Markus Dahlbokum | last post by:
Hello, I'm trying to link python statically with qt and pyqt. I've tried this in several ways but never succeeded. At the moment the final make runs without errors but I get import errors when accessing pyqt. How can I solve this problem? Markus # installing zipimport hook
by: Liang Chen | last post by:
Hope you all had a nice weekend. I have a question that I hope someone can help me out. I want to run a Python program that uses Tkinter for the user interface (GUI). The program allows me to type Chinese characters, but neverthelss is unable to show them up on screen. The follow is some of the error message I received after I logged off the program: "Could not write output: <type "exceptions: UnicodeEncodeError'>, 'ascii' codec can't...
by: Philip Semanchuk | last post by:
On Nov 9, 2008, at 7:00 PM, News123 wrote: Look under the heading "Standard Encodings": http://docs.python.org/library/codecs.html Note that both the page you found (which appears to be a copy of the Python documentation) and the reference I provide say, "Neither the list of aliases nor the list of languages is meant to be exhaustive".
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.