convert database to unicode - PostgreSQL Database

David Lutz

Hello, I want to convert an existing database with
SQL_ASCII encoding to UNICODE encoding.

(postgresql ver 7.3.2)

I thought that it might be as easy as:
pg_dump mydatabase > dump.sql
createdb --encoding=unicod e newdatabase
psql newdatabase < dump.sql

but it wasn't going to be that easy now was it :-(

I'm getting errors such as:
ERROR: copy: line 54, Invalid UNICODE character
sequence found (0xe46765)
lost synchronization with server, resetting connection

Can anyone suggest a way I might be able to do it?
Thanks, David
http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 11 '05 #1

Subscribe Reply

9323

Gianni Mariani

David Lutz wrote:

Hello, I want to convert an existing database with
SQL_ASCII encoding to UNICODE encoding.

(postgresql ver 7.3.2)

I thought that it might be as easy as:
pg_dump mydatabase > dump.sql
createdb --encoding=unicod e newdatabase
psql newdatabase < dump.sql

but it wasn't going to be that easy now was it :-(

I'm getting errors such as:
ERROR: copy: line 54, Invalid UNICODE character
sequence found (0xe46765)
lost synchronization with server, resetting connection

Can anyone suggest a way I might be able to do it?
Thanks, David

You *might* be able to convert dump.sql to unicode.

i.e.

iconv -f iso8859-1 -t utf-8 < dump.sql > dump-utf-8.sql

If all the bytea chars are appropriately escaped, this may do it.

Can't harm in giving it a go !

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 11 '05 #2

David Lutz

--- Gianni Mariani <gi****@mariani .ws> wrote: > David
Lutz wrote:

Hello, I want to convert an existing database with
SQL_ASCII encoding to UNICODE encoding.

(postgresql ver 7.3.2)

I thought that it might be as easy as:
pg_dump mydatabase > dump.sql
createdb --encoding=unicod e newdatabase
psql newdatabase < dump.sql

You *might* be able to convert dump.sql to unicode.

i.e.

iconv -f iso8859-1 -t utf-8 < dump.sql >
dump-utf-8.sql

If all the bytea chars are appropriately escaped,
this may do it.

Can't harm in giving it a go !

Thanks Gianni, yes that appears to have worked.
I didn't know about the iconv command.

Cheers, David

http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Nov 11 '05 #3

Similar topics

7804

Convert a Unicode Database into a non-unicode database

by: Rodger Dodger | last post by:

Hi there. We have an application that can run on a non-unicode or a unicode sql server database. Currently the application is running in a unicode database, as a non-unicode database is less than half the size, I would prefer to have a non-unicode database for demo purposes to be on my laptop, etc etc

Microsoft SQL Server

7773

Read UTF8 (mixed byte) file & convert to Unicode

by: hunterb | last post by:

I have a file which has no BOM and contains mostly single byte chars. There are numerous double byte chars (Japanese) which appear throughout. I need to take the resulting Unicode and store it in a DB and display it onscreen. No matter which way I open the file, convert it to Unicode/leave it as is or what ever, I see all single bytes ok, but double bytes become 2 seperate single bytes. Surely there is an easy way to convert these mixed...

.NET Framework

11742

How to convert binary or hexa to decimal?

by: Adam | last post by:

Hello, I'm trying to decifer the data in the table that stores the data in the binary format. All numbers are placed in varbinary fields. All I know is the MS SQL 2000 database useing collation SQL_Latin1_General_CP1_CI_AS (default). For example the content of the field is: (0xB4F5000000000000) in unicode and defined as varbinary(8).

Microsoft SQL Server

3625

convert string function and built-in conversions

by: culley harrelson | last post by:

It seems to me that these values should be the same: select 'lydia eugenia treviño', convert('lydia eugenia treviño' using ascii_to_utf_8); but they seem to be different. What am I missing? culley ---------------------------(end of broadcast)---------------------------

PostgreSQL Database

28331

Convert Encoding from Shift-JIS to UTF-8

by: DbNetLink | last post by:

I am trying to convert some Japanese text encoded as Shift-JIS/ISO-2022-JP to UTF-8 so I can store all data in my database with a common encoding. My problem is the encoding conversion code works for Japanese characters encoded as "iso-2022-jp" but does not for "shift-jis" What looked straight forward is proving less so, my test code looks like this: <%@ Page Language="C#"%>

C# / C Sharp

1250

how to convert non-unicode tables to unicode tables in ms sql.

by: mohankumar11 | last post by:

hi..i am having ms sql server 2005...now we are in upgrading process... in that database is unicode but the tables are not unicode... the tables are in non-unicode..tell me how to convert the tables from non-unicode to unicode tables... we are failing in this process.. thanks in advance.

MySQL Database

5069

Encoding: how to convert ISO-8559 to Unicode

by: deloford | last post by:

Hi This is going to be a question for anyone who is an expert in C# Text Encoding. My situation is this: I have a Sybase database which is firing back ISO-8559 encoded strings. I am unable to get the db to translate to UTF-8 for non technical reasons. So I have a string coming back with the character œ (ISO value 156). this character appears in .NET as a box character because 156 is not a valid Unicode character value. I have been...

.NET Framework

571

Re: convert unicode characters to visibly similar ascii characters

by: M.-A. Lemburg | last post by:

On 2008-07-01 20:31, Peter Bulychev wrote: You could write a codec which translates Unicode into a ASCII lookalike characters, but AFAIK there is no standard for doing this. I guess the best choice is to use the Unicode code point names as basis. These can be accessed via unicodedata.name(). You can then create a mapping which can be processed by the character map codec.

Python

4882

how to convert entities into unicode

by: neovantage | last post by:

hey geeks, I am using a function which convert unicode to entities. So that i can save values into mysql database into entities. This function really helps me when i display the store entity data into web page n it shows special charactor easily. Here is the function code function charset_decode_utf_8($string) { /* Only do the slow convert if there are 8-bit characters */ /* avoid using 0xA0 (\240) in ereg ranges. RH73 does not...

PHP

9636

What is ONU?

by: marktang | last post by:

ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...

General

10306

Problem With Comparison Operator <=> in G++

by: Oralloy | last post by:

Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...

C / C++

10075

The easy way to turn off automatic updates for Windows 10/11

by: Hystou | last post by:

Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...

Windows Server

7485

Access Europe - Using VBA to create a class based on a table - Wed 1 May

by: isladogs | last post by:

The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...

Microsoft Access / VBA

6727

Couldn’t get equations in html when convert word .docx file to html file in C#.

by: conductexam | last post by:

I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...

C# / C Sharp

5373

Trying to create a lan-to-lan vpn between two differents networks

by: TSSRALBI | last post by:

Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...

Networking - Hardware / Configuration

5504

Windows Forms - .Net 8.0

by: adsilva | last post by:

A Windows Forms form does not have the event Unload, like VB6. What one acts like?

Visual Basic .NET

4037

transfer the data from one system to another through ip address

by: 6302768590 | last post by:

Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

C# / C Sharp

2869

Comprehensive Guide to Website Development in Toronto: Expert Insights from BSMN Consultancy

by: bsmnconsultancy | last post by:

In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

General