Raw Device Wiggle Room?

[tech.witch]

I'm doing a postmortem from an outage at my workplace that looks too
similar to an outage we had last fall to not be related. Both database
outages had the following characteristics:

1) VERY large, frequently-accessed & updated tablespace defined on a
raw device of about 348GIG, with the indexes defined on a separate raw
device of a SUN machine.
2) Something happened to the device that made it start logging hardware
and I/O errors, which in turn marked the tablespace as bad.
3) There were no errors in the db2diag.log about the tablespace filling
up. The database just crashed one day and came back up with a mangled
tablespace.

While I've got the server ops dude checking out the hardware, I'm
curious about the device. 2 hardware failures under seemingly the same
set of conditions...I'm wondering............Is it possible to define a
tablespace on a raw device with a size parameter that is acceptable to
DB2 (therefore letting it create the tablespace successfully), but
somehow causes an issue with the OS later on? Meaning, if I had a
device that was 300GIG, could I define a tablespace on it for 300GIG,
or should I leave a little wiggle room?

[Matt Emmerton]

<te********@gmail.com> wrote in message
news:11*********************@g44g2000cwa.googlegro ups.com...

I'm doing a postmortem from an outage at my workplace that looks too
similar to an outage we had last fall to not be related. Both database
outages had the following characteristics:

1) VERY large, frequently-accessed & updated tablespace defined on a
raw device of about 348GIG, with the indexes defined on a separate raw
device of a SUN machine.
2) Something happened to the device that made it start logging hardware
and I/O errors, which in turn marked the tablespace as bad.
3) There were no errors in the db2diag.log about the tablespace filling
up. The database just crashed one day and came back up with a mangled
tablespace.

While I've got the server ops dude checking out the hardware, I'm
curious about the device. 2 hardware failures under seemingly the same
set of conditions...I'm wondering............Is it possible to define a
tablespace on a raw device with a size parameter that is acceptable to
DB2 (therefore letting it create the tablespace successfully), but
somehow causes an issue with the OS later on? Meaning, if I had a
device that was 300GIG, could I define a tablespace on it for 300GIG,
or should I leave a little wiggle room?

When you create a tablespace on a raw device, DB2 will check to see if it
can access the "end" of the device, presumably to protect against this kind
of problem. So as long as DB2 does not complain when creating the
tablespace, you will be fine.

Granted, both the OS and DB2 have some overhead, so a 300GB physical device
may only have (300GB - xx KB) in usuable space, and once a DB2 container is
created on that device, there will only be (300GB - xx KB - yy KB) available
to use for pages.

--
Matt Emmerton

[TechWitch]

That makes sense to me. The nature of a DMS container is to claim all
the space you have defined for it up front and let DB2 manage the
space. Still, it's very strange that two of our servers failed in the
same way on two different physical machines. I guess those poor old
RAID 0 array's just gave out from all the activity.

[Mark Yudkin]

RAID 0? That's just "marketing" for ***NO*** RAID. In simple language,
you're asking for disaster, and you apparently got what you asked for.

"TechWitch" <te********@gmail.com> wrote in message
news:11**********************@g47g2000cwa.googlegr oups.com...

That makes sense to me. The nature of a DMS container is to claim all
the space you have defined for it up front and let DB2 manage the
space. Still, it's very strange that two of our servers failed in the
same way on two different physical machines. I guess those poor old
RAID 0 array's just gave out from all the activity.

[TechWitch]

Well, unfortunately, I did not have any say in the matter. I'm just
the one who they run screaming to when it fails. < shakes head > This
has happened to them twice now over the past year...you think they'd
learn their lesson by now. I can't use words to describe how
FRUSTRATING it is when you try to explain to people that DBMS software
does NOT cause hardware failures and don't believe you. It's the other
way around!!! ( sorry had to vent there. )

FORTUNATELY, we were able to recover some summary tables on a different
device. That satisfied management for the time being.

< sigh >

TW

[Phil Sherman]

I sure hope your backups and logs are on a different physical device
from the tablespaces. Your management needs to become educated in RAID
terminology and architecture so they can properly assess the business
consequences of using disk configurations optimized for speed instead of
reliability.

Phil Sherman
TechWitch wrote:

That makes sense to me. The nature of a DMS container is to claim all
the space you have defined for it up front and let DB2 manage the
space. Still, it's very strange that two of our servers failed in the
same way on two different physical machines. I guess those poor old
RAID 0 array's just gave out from all the activity.