473,320 Members | 1,867 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Another Bizzare Crash...But no Recovery. SOS!!

Hello Gurus,
At about 11:00 am this morning, tried to connect to an instance of DB2 UDB v8.
2 fixpak 14 on a Red Hat Linux environment. The message returned say DB2
alias not found. Alarmed, we researched extensively and found that all data
and files in the /sqllib is gone...vamosed!!
Called system folks for help and we are still trying to recover as I write.
Question: (1) Anyone faced this kind of situation before and how did they
recover? Please share as much detail as you have to resolve similar problem.

(2) How can I b egin to research what happened, what agent or processes wiped
out the SQLLIB library?
(3) How can I prevent this type of event from happening again?

If you need any information, please post and I will see if I can still get
the information for you...please this is a serious matter as I am being
"beaten up" with ugly stares and decidedly subdued hostility by my users!!

Okonita

Aug 21 '07 #1
7 1838
Okonita wrote:
Hello Gurus,
At about 11:00 am this morning, tried to connect to an instance of DB2 UDB v8.
2 fixpak 14 on a Red Hat Linux environment. The message returned say DB2
alias not found. Alarmed, we researched extensively and found that all data
and files in the /sqllib is gone...vamosed!!
Called system folks for help and we are still trying to recover as I write.
Question: (1) Anyone faced this kind of situation before and how did they
recover? Please share as much detail as you have to resolve similar problem.

(2) How can I b egin to research what happened, what agent or processes wiped
out the SQLLIB library?
(3) How can I prevent this type of event from happening again?
I'm no expert in file systems, so I keep my mouth shut on possible
technical causes,
but could it be human error or worse.. sabotage?
We had a really nasty PMR like this years ago. DB2 files magically
disappeared every few days. In the end a keystroke-tracker was installed
and the problem was tracked to a disgruntled employee who didn't like
the switch to DB2. Quite embarassing and rather criminal....
--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab
Aug 22 '07 #2
Okonita wrote:
Hello Gurus,
At about 11:00 am this morning, tried to connect to an instance of DB2 UDB
v8. 2 fixpak 14 on a Red Hat Linux environment. The message returned say
DB2 alias not found. Alarmed, we researched extensively and found that all
data and files in the /sqllib is gone...vamosed!!
Called system folks for help and we are still trying to recover as I
write. Question:
(1) Anyone faced this kind of situation before and how
did they recover? Please share as much detail as you have to resolve
similar problem.
One way to recover would be to run "db2iupdt" for this instance. That will
reconstruct most files in the sqllib/ directory. What will be messing is
the database/DCS directory and other config files like db2cli.ini.

--
Knut Stolze
DB2 z/OS Utilities Development
IBM Germany
Aug 22 '07 #3
aj
I had a new, overzealous sysadmin notice that the home dir of the DB2
instance owner was much larger than all the rest, and more disk was
needed there - so he moved everything (including ./sqllib) someplace else.

The engine was not pleased. I got a call on vacation in Denver at
7:30AM in my hotel room. ah.....the life of a DBA...

As I recall, we moved everything back, restarted, and I think everything
was fine...

aj

PS - was ./sqllib possibly a symlink to somewhere else, and the symlink
got clobbered? Was it an NFS mount point?
Okonita wrote:
Hello Gurus,
At about 11:00 am this morning, tried to connect to an instance of DB2 UDB v8.
2 fixpak 14 on a Red Hat Linux environment. The message returned say DB2
alias not found. Alarmed, we researched extensively and found that all data
and files in the /sqllib is gone...vamosed!!
Called system folks for help and we are still trying to recover as I write.
Question: (1) Anyone faced this kind of situation before and how did they
recover? Please share as much detail as you have to resolve similar problem.

(2) How can I b egin to research what happened, what agent or processes wiped
out the SQLLIB library?
(3) How can I prevent this type of event from happening again?

If you need any information, please post and I will see if I can still get
the information for you...please this is a serious matter as I am being
"beaten up" with ugly stares and decidedly subdued hostility by my users!!

Okonita
Aug 22 '07 #4
Knut Stolze wrote:
>Hello Gurus,
At about 11:00 am this morning, tried to connect to an instance of DB2 UDB
[quoted text clipped - 6 lines]
>did they recover? Please share as much detail as you have to resolve
similar problem.

One way to recover would be to run "db2iupdt" for this instance. That will
reconstruct most files in the sqllib/ directory. What will be messing is
the database/DCS directory and other config files like db2cli.ini.
Knutz, thanks for chipping in on my problem. I appreciate this.
...And if you are not able to invoke the CLP to run db2iupdt? We are not able
to do that either for this instance and can't use the command center because
it is not available also.

Assuming I can run db2iupdt, how then can I recover the database DCSand other
config files like db2cli.ini?

--
Message posted via http://www.dbmonster.com

Aug 23 '07 #5
aj wrote:
>I had a new, overzealous sysadmin notice that the home dir of the DB2
instance owner was much larger than all the rest, and more disk was
needed there - so he moved everything (including ./sqllib) someplace else.

The engine was not pleased. I got a call on vacation in Denver at
7:30AM in my hotel room. ah.....the life of a DBA...

As I recall, we moved everything back, restarted, and I think everything
was fine...

aj

PS - was ./sqllib possibly a symlink to somewhere else, and the symlink
got clobbered? Was it an NFS mount point?
>Hello Gurus,
At about 11:00 am this morning, tried to connect to an instance of DB2 UDB v8.
[quoted text clipped - 14 lines]
>>
Okonita
aj, my thanks for chipping in goes to you also. I am new in this environment
and don't know all the subtleties of our Red Hat linux environment. What I
can do is look into answering your questions in the morning as I am not able
to access our production system (Linux) from home in the state that it is now.
...And if the ./sqllib is a symlink to somewhere else or it is a NFS mount
point, what nugget of technical wisdom can you share with me?
As you know, I am much oblidged for your discussion...

okonita

--
Message posted via DBMonster.com
http://www.dbmonster.com/Uwe/Forums....m-db2/200708/1

Aug 23 '07 #6
Okonita via DBMonster.com wrote:
..And if you are not able to invoke the CLP to run db2iupdt? We are not
able to do that either for this instance and can't use the command center
because it is not available also.
You have to run "db2iupdt" as root/Administrator from a regular command
shell. On Linux, you will find this tool under /opt/IBM/db2/V9.1/instance/
Assuming I can run db2iupdt, how then can I recover the database DCSand
other config files like db2cli.ini?
You will have to catalog all databases again. Once the base sqllib/
directory is reconstructed with "db2iupdt", just do:

$ db2 catalog db dcsand

(You may need additional parameters if the database doesn't reside in the
default database path.)

As for the config files, I don't know if there is any support to recover
those. If not, you have a backup, I hope?

--
Knut Stolze
DB2 z/OS Utilities Development
IBM Germany
Aug 23 '07 #7
aj
Okonita via DBMonster.com wrote:
aj, my thanks for chipping in goes to you also. I am new in this
environment
and don't know all the subtleties of our Red Hat linux environment. What I
can do is look into answering your questions in the morning as I am not able
to access our production system (Linux) from home in the state that it is now.
..And if the ./sqllib is a symlink to somewhere else or it is a NFS mount
point, what nugget of technical wisdom can you share with me?
If this was indeed the case, you'll need to figure out where that
somewhere else is, and reestablish the symlink that points at it.
Try a
find . -name sqllib -xdev
from the root dir.

I have seen some *very* weird stuff occur w/ NFS in Red Hat. Assuming
NFS, perhaps NFS file mounts were unmounted somehow? Look in
/etc/fstab. This is less likely than a symlink..

Good luck.

cheers
aj
Aug 23 '07 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Mads Jørgensen | last post by:
Greetings all. I have a bit of a problem here, a database i'm administering was somehow corrupted, and i'm unable to recover it in any way. Is there any way at all to recover a corrupt InnoDB...
10
by: xixi | last post by:
i have db2 udb v8.1 on windows 64 bit 2003 server, after db2 server start , i found this in the db2diag.log, is this error? 2004-05-05-15.28.30.780000 Instance:DB2 Node:000...
5
by: NG | last post by:
Hi, We are having DB2-V7.2 DB on AIX 5.2 machine. Recently we upgraded our system to fixpack 13 and activated activate AIX asynchronous IO function. Our DB is going to crash recovery with...
1
by: Michel Esber | last post by:
Hello, Linux V8 FP12 with a 64 bit instance. My DBA called me and said our instance crashed and recovery would never end. $ db2 list utilities show detail ID ...
2
by: Racerx | last post by:
Hi All : I use db2 8.1 fixpack 3 on AIX. I recieved the following message in the diaglog ====================================================== ADM7513W Database manager has started. ...
3
by: John Wright | last post by:
I want to create an autosave and crash recovery module for my program. I imagine I would have to use a timer control on the form to call the autosave functionality but I need a starting point. I...
0
by: John Nagle | last post by:
Can't really blame BeautifulSoup for this, but our crawler hit a page ("http://clagnut.com/privacy/") with an out of range character escape: 𔃷 in this text: If you provide a name, email...
4
by: Will Honea | last post by:
Running Express-C 9.5 on Suse and I made the gross error of deciding to look at Vista. Making a long story short, Vista managed to trash the partitions on the drive containing Linux. I have...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.