473,563 Members | 2,916 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

COPY error handling

Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.

Cheers!

Bricklen
Nov 23 '05 #1
4 1624
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@y ahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.


Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 23 '05 #2
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@y ahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.


Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 23 '05 #3
Bruno Wolff III wrote:
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@y ahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.

Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

Thanks for the clarification!

Cheers,

Bricklen
Nov 23 '05 #4
Bruno Wolff III wrote:
On Fri, Jun 04, 2004 at 14:11:19 +0000,
Bricklen <br**********@y ahoo.comz> wrote:
Hi, I'm not sure if this is the correct group for this question, but
I'll post it hoping that it is.
I'm loading several ~15 million row files into a table using the COPY
command. Apparently one of the rows, about 6 million in, has an invalid
entry. This is causing the COPY command to fail, so my question is this:
Is there any way to skip invalid rows? Or send them to a separate log
file etc to go through later?
I've gone through the docs, but I didn't see anything specific to this.
Any information, links, or hints are greatly appreciated.

Currently there isn't a builtin way to do this. You can pass the data
through a filter script that removes rows that are not in a proper format.

Thanks for the clarification!

Cheers,

Bricklen
Nov 23 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
3258
by: WSeeger | last post by:
When creating a new class, is it encouraged to always include error handling routines within your LET and GET procedures? It's seems that most text books never seem to include much about error handling within classes. Just hoping to hear some programmer's thoughts on error handling.
1
1854
by: Rich | last post by:
Hi, I have a query regarding VC6 and its handling of templated copy constructors. Here goes: Take a look at the following code sample... template<class _Ty, size_t t_uiSize = 10 > class my_template
6
8444
by: Squirrel | last post by:
I have a command button on a subform to delete a record. The only statement in the subroutine is: DoCmd.RunCommand acCmdDeleteRecord The subform's recordsource is "select * from tblVisit order by VisitDt" I'm getting this error message: Errno is 2465. Err.description is "Can't find field '|' referred to in your expression"
13
4447
by: Thelma Lubkin | last post by:
I use code extensively; I probably overuse it. But I've been using error trapping very sparingly, and now I've been trapped by that. A form that works for me on the system I'm using, apparently runs into problems on the system where it will actually be used, and since I used so little error-trapping it dies very ungracefully. I will of...
21
4383
by: Anthony England | last post by:
Everyone knows that global variables get re-set in an mdb when an un-handled error is encountered, but it seems that this also happens when the variable is defined as private at form-level. So if "global variables get re-set" doesn't tell the whole story, then what does? ***please note*** I'm not looking for a solution - I'm looking for a...
12
2663
by: Andrew Schepler | last post by:
When compiled with Visual C++ .NET 2003 (only), the program below aborts as though no matching catch clause is present. If the copy constructor of A is made public, it successfully catches the exception instead. If I step into __CxxThrowException in debug/assembly mode, it looks like the pThrowInfo parameter created by the compiler excluded...
3
2837
by: Stefan Johansson | last post by:
Hi all I'am moving from Visual Foxpro and have a question regarding "best practice" error handling in vb .net. In VFP I have always used a "central" error handling object in order to have a easy and reusable way of handling all errors in a program. The VB 6 coding examples I have seen there has always been error handling code in each...
19
4172
by: Angus | last post by:
I have a socket class CTestClientSocket which I am using to simulate load testing. I create multiple instances of the client like this: for (int i = 0; i < 5; i++) { CTestClientSocket* pTemp = new CTestClientSocket(this, ip, port); pTemp->Connect(); m_collClients.push_back(pTemp);
0
1004
by: subramanian100in | last post by:
Consider the following program: #include <cstdlib> #include <iostream> using namespace std; class Test { public:
9
3284
by: MrDeej | last post by:
Hello guys! We have an SQL server which sometimes makes timeouts and connection errors. And we have an function witch writes and updates data in 2 tables on this server. When the SQL server error appears it, in 99%, of the cases, works if we just press the play button in VBA debug. Therefor we have maked an error handling which just tryes...
0
7664
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
1
7638
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7948
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
1
5484
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5213
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3642
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3626
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1198
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
923
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.