473,320 Members | 1,916 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

DBF to PostgreSQL

Hi all,
I am working on a project to convert the data from dozens of
..dbf files to PostgreSQL Database. I wrote some perl script to employ
dbfdump to extract data from .dbf files which sit on other remote
server to a source text file. (Actually I just simply mount the
directory to local server.) Then I modified the data in the source
text file and output them into a target text file as the structure of
the data is different between .dbf and PostgreSQL database. Then I
just use PostgresSQL SQL command COPY to copy data from the target
text file to PostgreSQL. It works fine!! I have about 40 ~ 50 tables,
each table with about 1 million ~ 2 million records. Everything is
done in 2 hours. I think it is amazing (it used to take upto 8 hours
to finish with simple select and insert statement.) I run the same set
of perl scripts every single day at the morning. It work great most of
the time.

However, I found out that the dbfdump give me the wrong output
once or twice a week! The dbfdump mess up the data of one field of one
single record (usually the date field or numeric field.) Since the
messed up data will violate the format of the copy statement, I could
find out such litte mistake among million records. This problem won't
happen everyday. However, it did happen!~

I am wondering what causes this kind of problem? Is there anyway
to correct or avoid this happen again?? Oh!~ I've forgot to mention
everything is happen on Linux Redhat 8.1 server. I am using Postgres
7.3 and dbfdump 0.220.

Any suggestion will be welcomed!~
Thank You Very Much!~

Best regards,
Harry Yau
Nov 12 '05 #1
0 3593

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Otis Green | last post by:
Vote for or against a new newsgroup proposal. To summarize what you need to do, just send an empty e-mail to postgresql-ballot@netagw.com You will receive a ballot by e-mail. Follow the...
1
by: Mateusz [PEYN] Adamus | last post by:
Hi I'm a developer currently wondering which DB choose for my next project. I looked on the net found ofcourse Oracle but also came up with PostgreSQL. I heard quite few things about it, all...
0
by: Bill J. | last post by:
I have to update a PostgreSQL linked server through MSSQL2K. I first configured the connection with ODBC as follows and I can do queries with no problem: EXEC sp_droplinkedsrvlogin @rmtsrvname...
0
by: Bill J. | last post by:
I have to update a PostgreSQL linked server through MSSQL2K. I first configured the connection with ODBC as follows and I can do queries with no problem: EXEC sp_droplinkedsrvlogin @rmtsrvname...
0
by: ruhunu Gamarala | last post by:
Hi, I get the following exception periodically. does anybody what is the reason for it to throw this exception? I really appriciate if anyone can help me on this. thanks, Chinthaka at...
0
by: greg | last post by:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This is a PGP-signed copy of the checksums for PostgreSQL version 7.4. The latest copy of the checksums for this and other versions, as well...
29
by: Paul Ganainm | last post by:
Hi all, Following up on another thread, here is a comparison between FB and PG from an FB'ers POV. BTW, FB is the love-child of Open-Source-Interbase. BTW, I have tried to be as accurate as I...
13
by: Zlatko Matić | last post by:
Is it possible to use Access as front-end for POstgreSQL and how ? What about Access Projects (.adp) and PostgreSQL ?
1
by: phil campaigne | last post by:
On Mon, 1 Mar 2004, phil campaigne wrote: >> Nigel J. Andrews wrote: >> > > >>> >On Mon, 1 Mar 2004, Phil Campaigne wrote: >>> > >>> >
0
by: Greg Sabino Mullane | last post by:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This is a PGP-signed copy of the checksums for following PostgreSQL versions: 7.4.5 7.4.4 7.3.7
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.