473,326 Members | 2,124 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

pg_restore with Fc fails with [archiver] out of memory error

Created a pg_dump with Fc (custom format compression) that resulted in a
300+MB file. Now trying to pg_restore this thing fails with either an
out of memory error (as in the subject line) on FreeBSD 4.10-STABLE or
a:
pg_restore in malloc(): error: allocation failed
Abort (core dumped)
error on FreeBSD 5.2.1-P9 (RELEASE)

In both cases I have increased the max data segment size to 1.5GB and
the max stack size to 768M or so.

So obviously we have some error message when malloc'ing for the AR
structure. Any ideas here? Is it trying to stuff the 300MB file into
memory and uncompress and sort it there? Has anyone found a workaround
for this? (I have found similar questions and zero answers doing google
searches and mailing list searches). Is there any way of uncompressing
this file *other* than using pg_restore?

Sven

p.s. Although I will check the archives, I would appreciate a CC on any
responses.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 23 '05 #1
3 4894


Tom Lane wrote:
Sven Willenberger <sv**@dmv.com> writes:
pg_restore in malloc(): error: allocation failed
Abort (core dumped)

A stack trace from that core file would be useful in figuring out what
the problem is... also, what PG version is this?

regards, tom lane


psql -V
psql (PostgreSQL) 7.4.3
contains support for command-line editing

gdb:
Reading symbols from /usr/local/lib/libpq.so.3...done.
Loaded symbols for /usr/local/lib/libpq.so.3
Reading symbols from /usr/local/lib/libintl.so.6...done.
Loaded symbols for /usr/local/lib/libintl.so.6
Reading symbols from /usr/local/lib/libssl.so.3...done.
Loaded symbols for /usr/local/lib/libssl.so.3
Reading symbols from /usr/local/lib/libcrypto.so.3...done.
Loaded symbols for /usr/local/lib/libcrypto.so.3
Reading symbols from /lib/libz.so.2...done.
Loaded symbols for /lib/libz.so.2
Reading symbols from /lib/libreadline.so.4...done.
Loaded symbols for /lib/libreadline.so.4
Reading symbols from /lib/libcrypt.so.2...done.
Loaded symbols for /lib/libcrypt.so.2
Reading symbols from /lib/libm.so.2...done.
Loaded symbols for /lib/libm.so.2
Reading symbols from /lib/libutil.so.4...done.
Loaded symbols for /lib/libutil.so.4
Reading symbols from /lib/libc.so.5...done.
Loaded symbols for /lib/libc.so.5
Reading symbols from /usr/local/lib/libiconv.so.3...done.
Loaded symbols for /usr/local/lib/libiconv.so.3
Reading symbols from /lib/libncurses.so.5...done.
Loaded symbols for /lib/libncurses.so.5
Reading symbols from /libexec/ld-elf.so.1...done.
Loaded symbols for /libexec/ld-elf.so.1
#0 0x282a3dcf in kill () from /lib/libc.so.5
(gdb) bt
#0 0x282a3dcf in kill () from /lib/libc.so.5
#1 0x28298878 in raise () from /lib/libc.so.5
#2 0x28310f82 in abort () from /lib/libc.so.5
#3 0x2830f6fe in tcflow () from /lib/libc.so.5
#4 0x2830ff9b in tcflow () from /lib/libc.so.5
#5 0x28310356 in malloc () from /lib/libc.so.5
#6 0x0804c9a1 in ReadStr (AH=0x805e000) at pg_backup_archiver.c:1530
#7 0x0804d2d8 in ReadToc (AH=0x805e000) at pg_backup_archiver.c:1878
#8 0x0804fa94 in InitArchiveFmt_Custom (AH=0x805e000) at
pg_backup_custom.c:196
#9 0x0804cf7b in _allocAH (FileSpec=0xbfbfed19 "./callingdata",
fmt=archCustom, compression=0, mode=archModeWrite)
at pg_backup_archiver.c:1743
#10 0x0804a628 in OpenArchive (FileSpec=0x0, fmt=archUnknown) at
pg_backup_archiver.c:103
#11 0x0804a22a in main (argc=5, argv=0xbfbfebe8) at pg_restore.c:333
#12 0x08049b52 in _start ()

Frame 6 - 11:
(gdb) up 6
#6 0x0804c9a1 in ReadStr (AH=0x805e000) at pg_backup_archiver.c:1530
1530 pg_backup_archiver.c: No such file or directory.
in pg_backup_archiver.c
(gdb) up
#7 0x0804d2d8 in ReadToc (AH=0x805e000) at pg_backup_archiver.c:1878
1878 in pg_backup_archiver.c
(gdb) up
#8 0x0804fa94 in InitArchiveFmt_Custom (AH=0x805e000) at
pg_backup_custom.c:196
196 pg_backup_custom.c: No such file or directory.
in pg_backup_custom.c
(gdb) up
#9 0x0804cf7b in _allocAH (FileSpec=0xbfbfed19 "./callingdata",
fmt=archCustom, compression=0, mode=archModeWrite)
at pg_backup_archiver.c:1743
1743 pg_backup_archiver.c: No such file or directory.
in pg_backup_archiver.c
(gdb) up
#10 0x0804a628 in OpenArchive (FileSpec=0x0, fmt=archUnknown) at
pg_backup_archiver.c:103
103 in pg_backup_archiver.c
(gdb) up
#11 0x0804a22a in main (argc=5, argv=0xbfbfebe8) at pg_restore.c:333
333 pg_restore.c: No such file or directory.
in pg_restore.c

Let me know if you want any more details, a disassembled frame, what
have you.
Thanks,
Sven

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org

Nov 23 '05 #2
Sven Willenberger <sv**@dmv.com> writes:
pg_restore in malloc(): error: allocation failed
Abort (core dumped)


A stack trace from that core file would be useful in figuring out what
the problem is... also, what PG version is this?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org

Nov 23 '05 #3
On Tue, 2004-08-10 at 19:28 -0400, Sven Willenberger wrote:

Tom Lane wrote:
Sven Willenberger <sv**@dmv.com> writes:
pg_restore in malloc(): error: allocation failed
Abort (core dumped)

A stack trace from that core file would be useful in figuring out what
the problem is... also, what PG version is this?

regards, tom lane


psql -V
psql (PostgreSQL) 7.4.3
contains support for command-line editing

<snip>
Frame 6 - 11:
(gdb) up 6
#6 0x0804c9a1 in ReadStr (AH=0x805e000) at pg_backup_archiver.c:1530
1530 pg_backup_archiver.c: No such file or directory.
in pg_backup_archiver.c
(gdb) up
#7 0x0804d2d8 in ReadToc (AH=0x805e000) at pg_backup_archiver.c:1878
1878 in pg_backup_archiver.c
(gdb) up
#8 0x0804fa94 in InitArchiveFmt_Custom (AH=0x805e000) at
pg_backup_custom.c:196
196 pg_backup_custom.c: No such file or directory.
in pg_backup_custom.c
(gdb) up
#9 0x0804cf7b in _allocAH (FileSpec=0xbfbfed19 "./callingdata",
fmt=archCustom, compression=0, mode=archModeWrite)
at pg_backup_archiver.c:1743
1743 pg_backup_archiver.c: No such file or directory.
in pg_backup_archiver.c
(gdb) up
#10 0x0804a628 in OpenArchive (FileSpec=0x0, fmt=archUnknown) at
pg_backup_archiver.c:103
103 in pg_backup_archiver.c
(gdb) up
#11 0x0804a22a in main (argc=5, argv=0xbfbfebe8) at pg_restore.c:333
333 pg_restore.c: No such file or directory.
in pg_restore.c

Let me know if you want any more details, a disassembled frame, what
have you.


For future searches,etc, the problem was a corrupted dump file due to
ftp issues when transferring the file. Thanks, Tom, for helping diagnose
this.

Sven
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 23 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Sam | last post by:
I'm having trouble restoring databases that have to lo type installed in /contrib/lo. The dump seems to work just fine, I get no errors when I execute the following command #pg_dump -Fc -o -b...
0
by: andy morrow | last post by:
hi, fairly new to postgres admin stuff...... i have a production machine that is running postgresql 7.1.3 also, there's a test machine which already had 7.0.3, and which i newly installed...
0
by: Joshua D. Drake | last post by:
Alright, we are testing pg_restore while restoring a 7GB database. This database has about 6GB of large objects and about 1Gb of textual data. The problem has been verified on PostgreSQL 7.3.2,...
0
by: Aienthiwan | last post by:
Hello all, I'm running a debian server and recently updated my version of PostgreSQL to "unstable", that being v7.4.1. I had no trouble at all getting everything up to date and going. But my...
1
by: nednieuws | charles | last post by:
What does this error mean: pg_restore: creating TABLE author pg_restore: creating SEQUENCE author_id pg_restore: could not execute query: ERROR: parser: parse error at or near "BY" at...
6
by: Mike Charnoky | last post by:
Hello, I am currently using PostgreSQL v7.3.4 on a RedHat 8.0 system (2.4.23 kernel) using the ext3 filesystem. I am experiencing problems when performing a pg_restore using a file which is...
1
by: Roopali Sharma | last post by:
Hi, I am trying to restore data from a production server (using Postgresql 7.3.x) to a test server using the latest release 7.4.1. This is what I did: 1) dropdb -U postgres testdb 2) createdb...
1
by: Plant Thomas | last post by:
Hi, I have a problem restoring a database from Postgresql 7.3 ( backup created with 'pg_dump -Ft -b > test.tar' ) to a new database on PostgreSQL 7.4.3 on Cygwin. I get the following error: ...
7
by: Tim Penhey | last post by:
Maybe it's just me, but I can't seem to get pg_restore to restore a database... I am running 8.0 beta 2 (using the dev3 installer) on Windows XP. I created a very simple database with one...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.