473,466 Members | 1,513 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

COPY error handling

Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.

Cheers!

Bricklen
Nov 23 '05 #1
4 1617
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@yahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.


Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 23 '05 #2
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@yahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.


Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 23 '05 #3
Bruno Wolff III wrote:
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@yahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.

Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

Thanks for the clarification!

Cheers,

Bricklen
Nov 23 '05 #4
Bruno Wolff III wrote:
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@yahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.

Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

Thanks for the clarification!

Cheers,

Bricklen
Nov 23 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: WSeeger | last post by:
When creating a new class, is it encouraged to always include error handling routines within your LET and GET procedures? It's seems that most text books never seem to include much about error...
1
by: Rich | last post by:
Hi, I have a query regarding VC6 and its handling of templated copy constructors. Here goes: Take a look at the following code sample... template<class _Ty, size_t t_uiSize = 10 > class...
6
by: Squirrel | last post by:
I have a command button on a subform to delete a record. The only statement in the subroutine is: DoCmd.RunCommand acCmdDeleteRecord The subform's recordsource is "select * from tblVisit order...
13
by: Thelma Lubkin | last post by:
I use code extensively; I probably overuse it. But I've been using error trapping very sparingly, and now I've been trapped by that. A form that works for me on the system I'm using, apparently...
21
by: Anthony England | last post by:
Everyone knows that global variables get re-set in an mdb when an un-handled error is encountered, but it seems that this also happens when the variable is defined as private at form-level. So...
12
by: Andrew Schepler | last post by:
When compiled with Visual C++ .NET 2003 (only), the program below aborts as though no matching catch clause is present. If the copy constructor of A is made public, it successfully catches the...
3
by: Stefan Johansson | last post by:
Hi all I'am moving from Visual Foxpro and have a question regarding "best practice" error handling in vb .net. In VFP I have always used a "central" error handling object in order to have a...
19
by: Angus | last post by:
I have a socket class CTestClientSocket which I am using to simulate load testing. I create multiple instances of the client like this: for (int i = 0; i < 5; i++) { CTestClientSocket* pTemp...
0
by: subramanian100in | last post by:
Consider the following program: #include <cstdlib> #include <iostream> using namespace std; class Test { public:
9
by: MrDeej | last post by:
Hello guys! We have an SQL server which sometimes makes timeouts and connection errors. And we have an function witch writes and updates data in 2 tables on this server. When the SQL server error...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
0
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
0
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.