471,073 Members | 1,434 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,073 software developers and data experts.

Mass Import/Generate PKs

I have a CSV file with 400,000 lines of email mailing list information that
I need to migrate to a new PostgreSQL database.

Each line has all the info I need except a PK (I usually use an int4 column
for a PK).

If the file were smaller I would probably just use Excel to pop in a PK and
then just load into the table...

Since Excel chokes on files greater than 65k lines, this won't work.

Is there a way to get this done inside psql for instance? Or another route?

Thanks,
Hunter

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 23 '05 #1
5 3677
Hunter Hillegas wrote:
I have a CSV file with 400,000 lines of email mailing list
information that I need to migrate to a new PostgreSQL database.

Each line has all the info I need except a PK (I usually use an int4
column for a PK).


You could import the file into PostgreSQL and add a primary key column
later.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 23 '05 #2
That sounds nice and easy...

So, I would do something like 'ALTER TABLE' to generate the PK column? What
would be the best way to populate it? Is there an area of doco I should be
looking at?

Thanks,
Hunter

From: Peter Eisentraut <pe*****@gmx.net>
Date: Sat, 6 Nov 2004 21:21:25 +0100
To: Hunter Hillegas <li***@lastonepicked.com>, PostgreSQL
<pg***********@postgresql.org>
Subject: Re: [GENERAL] Mass Import/Generate PKs

Hunter Hillegas wrote:
I have a CSV file with 400,000 lines of email mailing list
information that I need to migrate to a new PostgreSQL database.

Each line has all the info I need except a PK (I usually use an int4
column for a PK).


You could import the file into PostgreSQL and add a primary key column
later.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 23 '05 #3
the simplest way to do it seems to be adding a SERIAL column to your
table, and then adding a primary key constraint:

1)insert data into table
2)ALTER TABLE <table> ADD id SERIAL;
3)ALTER TABLE <table> ADD CONSTRAINT <table>_pk PRIMARY KEY (id);

you can check the docs for the SERIAL type:
http://www.postgresql.org/docs/7.4/s...ATATYPE-SERIAL

On Sat, 2004-11-06 at 17:29, Hunter Hillegas wrote:
That sounds nice and easy...

So, I would do something like 'ALTER TABLE' to generate the PK column? What
would be the best way to populate it? Is there an area of doco I should be
looking at?

Thanks,
Hunter

From: Peter Eisentraut <pe*****@gmx.net>
Date: Sat, 6 Nov 2004 21:21:25 +0100
To: Hunter Hillegas <li***@lastonepicked.com>, PostgreSQL
<pg***********@postgresql.org>
Subject: Re: [GENERAL] Mass Import/Generate PKs

Hunter Hillegas wrote:
I have a CSV file with 400,000 lines of email mailing list
information that I need to migrate to a new PostgreSQL database.

Each line has all the info I need except a PK (I usually use an int4
column for a PK).


You could import the file into PostgreSQL and add a primary key column
later.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/




---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (FreeBSD)

iD8DBQBBjT5221dVnhLsBV0RAlGvAJ0aK0iF/6oH53xdMiI8YKqhCQsH6ACeKH5F
SK0L3yjPNatg65thWZyOIck=
=xlVQ
-----END PGP SIGNATURE-----

Nov 23 '05 #4
On Saturday November 6 2004 2:13, Franco Bruno Borghesi wrote:
the simplest way to do it seems to be adding a SERIAL column to your
table, and then adding a primary key constraint:

1)insert data into table
2)ALTER TABLE <table> ADD id SERIAL;
3)ALTER TABLE <table> ADD CONSTRAINT <table>_pk PRIMARY KEY (id);


You may also need to populate the id column with unique values in between
these two steps with something like "

update table set id = nextval('table_id_seq'::text) where id isnull"

I don't think SERIAL does that for you.

Ed
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly

Nov 23 '05 #5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (FreeBSD)

iD8DBQBBjUw/21dVnhLsBV0RAgIsAJ0XYXium3Qs+1SPflb9gRs73Ap6SwCgle O0
+djocSv8cNc9eL/VIIiwQnA=
=NJ/7
-----END PGP SIGNATURE-----

Nov 23 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by Neter Smith | last post: by
reply views Thread by kent.anderson | last post: by
1 post views Thread by kent.anderson | last post: by
reply views Thread by Steven Bolard | last post: by
1 post views Thread by danibecr | last post: by
1 post views Thread by evaisse | last post: by
reply views Thread by leo001 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.