473,836 Members | 1,495 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

The old raw devices chestnut.

Note the cross-posting - but no flame wars please.

This question was prompted by a thread on the a postgres mailing list
during which someone (Gregory Williamson) claimed

<quote>
raw devices, at least on Solaris, are about 10 times as fast as cooked
file systems for Informix.
<quote>

This made me think about the old arguments, and I wondered about the
current state of thinking. Some of my knowledge will be a bit out of
date.

Oracle: (my main experience)
At various times Oracle have claimed (talking to consultants, not
marketers) that raw devices are 5-20% faster than filesystems. This may
vary on the current state of the oracle code and/or the filesystem being
compared against. Veritas seem to agree by producing QuickIO for Oracle,
claiming "performanc e of raw with the management of filesystem".

I have never been sufficiently convinced to implement a major system
with raw.

Sybase: (some experience)
Sybase claim filesystems are faster, because of OS buffering, but unsafe
for the same reason. They only ever suggest filesystem for tempdb. They
don't seem to have heard of fsync()[1]

DB2:
No idea

Informix:
No idea beyond the claim which started this off.

What is the latest thinking, both in terms of vendor claims and
practical experience?

[1] or whatever system call forces write-through caching
--
Jim Smith
Because of their persistent net abuse, I ignore mail from
these domains (among others) .yahoo.com .hotmail.com .kr .cn .tw
For an explanation see <http://www.jimsmith.de mon.co.uk/spam>
Nov 12 '05
42 4700
> From what I remember of Oracle's "basic" backup tool (and I haven't
had much to do with it since 8.1.6) you have to put tablespaces
(equivalent to our dbspaces) into "backup mode" before you can back
them up which means they can't be written to. It's also typical to do
all this one tablespace at a time. If my memory is serving me
correctly (and I'm quite open to being told it isn't) this is a vastly
more primitive mechanism than either ontape or onbar.

Has the basic backup tool moved on since then, and does RMAN avoid any
such limitations?


RMAN does not have this limitation. You do not have to put any
tablespaces in backup mode for RMAN to perform a hot backup.
HTH,
Brian

--
=============== =============== =============== =============== =======

Brian Peasland
dba@remove_spam .peasland.com

Remove the "remove_spa m." from the email address to email me.
"I can give it to you cheap, quick, and good. Now pick two out of
the three"
Nov 12 '05 #31
Brian Peasland wrote:
From what I remember of Oracle's "basic" backup tool (and I haven't
had much to do with it since 8.1.6) you have to put tablespaces
(equivalent to our dbspaces) into "backup mode" before you can back
them up which means they can't be written to. It's also typical to do
all this one tablespace at a time. If my memory is serving me
correctly (and I'm quite open to being told it isn't) this is a vastly
more primitive mechanism than either ontape or onbar.

Has the basic backup tool moved on since then, and does RMAN avoid any
such limitations?

RMAN does not have this limitation. You do not have to put any
tablespaces in backup mode for RMAN to perform a hot backup.
HTH,
Brian


And in fact, pretty much nothing in Andy's memory of Oracle 8.1.6 is
actually correct. RMAN was available with 8.1.6, tablespaces did not
need to be placed in backup mode for a hot backup, and even if you did
do it this was, they could still be written to. And I don't think that
RMAN could ever be described as more primitive than ontape, and
definately not onbar, especially if you do a comparison of capabilites
over the timeline.

And, just to be clear, the comment from Andrew Hamm is also incorrect
ok - I've heard that Oracle has a "variety" of backup procedures, some of
them home-cooked, some of them tools you have to pay for


RMAN is not a chargeable option with Oracle. It provided as part of the
base product, just as onbar is.

Nov 12 '05 #32
"Andy Kent" <an************ ******@virgin.n et> wrote in message
news:ac******** *************** **@posting.goog le.com...
I wonder whether it makes the same difference on RAID.


Shouldn't do, because with RAID, and cacheing, the OS has no idea physically
where, or even actually whether, a write has been effected.
Nov 12 '05 #33

"Data Goob" <da******@hotma il.com> wrote in message
news:tk******** ***********@fe1 1.usenetserver. com...
Andrew, The real question is, how will you backup, much less restore
raw device databases? Are you prepared to deal with the
inflexibility it presents? Have you ever run Informix or
other vendor database through a complete backup and restore,
testing all the options with raw-device dbspaces? Most DBAs
I've run into have never had to restore a database at any
time in their career much less even test it. Is that amazing
or what!
Any of Oracle, Informix or DB2 will cope with this equally as well as if on
"cooked" disk. Informix will even let you restore a dbspace held on cooked
chunks to be restored to a raw disk (or vice versa). The others may do too
although I think there is more fannying around needed.

Finally you describe EMC Timefinder: provided you block the database
(onmode -b) for the few seconds the Timefinder split takes, you can then
spool the BCV copy out to tape and use this as the basis for an external
restore.
Consider too, clustering of systems and disks, and the
administrative challenges associated with that. Add raw
devices to multitudes of servers and it becomes more risky ...


If you don't know what you're doing .... :-)
Nov 12 '05 #34
Andy Kent wrote:
From what I remember of Oracle's "basic" backup tool (and I haven't
had much to do with it since 8.1.6) you have to put tablespaces
(equivalent to our dbspaces) into "backup mode" before you can back
them up which means they can't be written to.

That was never true, actually. Yes, if you were performing an O/S-based
backup, you had to say 'alter tablespace X begin backup', and '...end
backup' when you'd done. But in between those two statements, the
tablespace remains completely and utterly open for business, and is read
from and written to perfectly normally. It would be a sad backup
mechanism indeed if it suddenly rendered half your database read only!
It's also typical to do
all this one tablespace at a time. If my memory is serving me
correctly (and I'm quite open to being told it isn't) this is a vastly
more primitive mechanism than either ontape or onbar.
It is *sensible* to do one tablespace at a time, because by saying
'begin backup', you cause the first modification that takes place to a
block of data to write the entire block into what we would call the redo
logs (I think informix would call them the physical logs). Normally, the
same piece of DML would only write a few bytes of redo. Suddenly, in
backup mode, it is writing thousands of bytes. So to avoid your entire
database suddenly swamping the redo logs, yes... the sensible approach
is to do it one tablespace at a time.
Has the basic backup tool moved on since then,
The basic 'begin backup... copy... end backup' mechanism hasn't changed
*at all*.
and does RMAN avoid any
such limitations?
One of RMAN's *great* claims to fame is that, because it is an Oracle
tool that understands the concept of an Oracle block (whereas the O/S is
dumb, and only understands indiviudal O/S blocks), it "knows" when an
Oracle block is in flux. Instead of therefore writing the entire thing
to redo to make sure that there's a consistent image of the block, it
simply bides its time, and waits until the Oracle block is *not* in flux
before copying it. Therefore, there's no 'begin backup' or 'end backup'
command, no massive amounts of redo generated, and no need to do one
tablespace at a time.

And the command to make it happen used to be

run {
allocate channel d1 type disk;
backup database;
}

....but is now....

backup database;

Regards
HJR


(Sorry to drift off-topic but it's always useful to know what else is
out there. Hell, I might need a job with that stuff some day.)

Andy
"Howard J. Rogers" <hj*@dizwell.co m> wrote in message news:<40******* *************** *@news.optusnet .com.au>...
Andrew Hamm wrote:

[snip]

ok - I've heard that Oracle has a "variety" of backup procedures, some of
them home-cooked, some of them tools you have to pay for. In that case, you
would need suitable tools to manage the objects. With Informix, it's shipped
with one (now two) tools that perform complete, live, point-in-time archives
along with continuous storage of logical logs. Any timestamps etc that need
touching are self-contained within the engine spaces, so it's completely
unnecessar y and almost certainly destructive to do anything with informix
spaces apart from via the utilities provided.


[snip]

For many aeons, it has been this on Oracle. Shell-scripted backups,
with commands provided by the database to make the O/S-produced backup
recoverable and usable. But since version 8.0 (and we've had 8.1.5,
8.1.6, 8.1.7, 9.0.1, 9.2, and 10g since then), Oracle has provided RMAN
-for free- which does cleverer and safer backups than any shell script
could manage. It's a command-line tool for the scripting addicts, but
has a GUI front-end for those so inclined. Works very nicely.

The emphasis is very much on using RMAN these days (when first released
it was a bit rough round the edges!), and O/S-based backups are
gradually becoming a thing of the past, or at least not too well looked
upon, generally (though it's nice to have the choice).

Point is, given the context of this discussion, RMAN works just as well
with raw devices as it does with file systems, and will happily backup a
raw-based database onto a file system, or vice versa.

Raw isn't the utterly inflexible nightmare in Oracle it's sometimes made
out to be.

Regards
HJR

Nov 12 '05 #35
"Andrew Hamm" <ah***@mail.com > wrote in message news:<c5******* *****@ID-79573.news.uni-berlin.de>...
Mark Bole wrote:
I haven't worked with an Oracle raw device since version 7.3 seven
years ago, and would never go back. The administrative overhead is
just too much of a headache.
[Posting from comp.databases. informix]

I've never understood the "administra tive overhead" argument against raw
spaces. Can you elicidate on the administration you think needs to be
applied? In my experience, the only admin overhead is making sure that a
new, naive sysop doesn't try to turn a raw space into a filesystem - I saved
an engine by SECONDS once, by looking over the shoulder of said sysop as I
walked past....


Funnily enough, that very reason is the one I've considered most
important for years:
http://groups.google.com/groups?selm...&output=gplain
( the "other hand" line has been superceded in more recent versions ).

I'd add that if you are that close to saturation where a small
percentage will make such a difference, you probably need some good
tuning and/or more hardware.

Apart from that, I find that admin is slightly simpler simply because you
don't need to run a mkfs style of command ;-)

I have heard that Solaris (?) is in the habit of re-creating /dev at boot
time, and I know that DG-UX used to do the same thing. Therefore some sort
of tool is needed to make sure the raw devs exist after a boot. Is this what
you mean?


jg
--
@home.com is bogus.
Please ignore this link. http://www.securityfocus.com/columnists/224
Nov 12 '05 #36
Mark Townsend wrote:

And, just to be clear, the comment from Andrew Hamm is also incorrect
ok - I've heard that Oracle has a "variety" of backup procedures,
some of them home-cooked, some of them tools you have to pay for


RMAN is not a chargeable option with Oracle. It provided as part of
the base product, just as onbar is.


yup - good info from Howard on this one, and a bit of history where this
opinion was based.
Nov 12 '05 #37
Joel Garry wrote:

Funnily enough, that very reason is the one I've considered most
important for years:
http://groups.google.com/groups?selm...&output=gplain ( the "other hand" line has been superceded in more recent versions ).
He-heh - 1996? that would be around the time my engine was under threat.
However, dangerous people can do damage to nearly everything. I've also seen
LOST engines from people either deleting or moving cooked files. You can
run, but you cannot hide.
I'd add that if you are that close to saturation where a small
percentage will make such a difference, you probably need some good
tuning and/or more hardware.


Wellllllll, no. NIMM (not in my mileage)

a 20% improvement is good - why not? It also improves throughput - 20% would
be Just About noticable too. But more importantly, it's during periods of
heavy sequentials that it kicks in. An overall benefit of 20% will probably
give you bursts of much bigger improvements. Like the (Informix) situations
I've mentioned such as creating spaces, but also things like light scans,
checkpoints (a major bugbear for some people) and of course big sequential
reads and builds.

Further (with Informix, once again) the engine can use KAIO, and with the
architecture of Informix, this can lead to further significant improvements.
It all adds up. Why do you think F1 now make their pedals out of carbon
fibre? And they *still* drill 'em out for extra lightness. An Informix
engine using raw with KAIO and a decent layout of spaces on the disk can
feel very spiffy indeed even compared to one that merely drops KAIO and raw.

Some UNIX platforms with Big Hairy storage boxes do provide device drivers
etc that support KAIO by either faking the raw device or just providing the
feature. What you do with that hardware depends on the machinery of course.
Nov 12 '05 #38
"Andrew Hamm" <ah***@mail.com > wrote in message
news:c5******** ****@ID-79573.news.uni-berlin.de...
Further (with Informix, once again) the engine can use KAIO, and with the
architecture of Informix, this can lead to further significant improvements. It all adds up. Why do you think F1 now make their pedals out of carbon
fibre? And they *still* drill 'em out for extra lightness. An Informix
engine using raw with KAIO and a decent layout of spaces on the disk can
feel very spiffy indeed even compared to one that merely drops KAIO and

raw.

A dangerous analogy, the formula 1 one. An F1 Car is made to last a
weekend - used to be a race - and to be rebuilt after that. This is not a
desirable* thing in a database. Robustness and reliability are important as
well as performance - probably more so.
--
Niall Litchfield
Oracle DBA
Audit Commission UK
http://www.niall.litchfield.dial.pipex.com
*************** *************** ***********
Please include version and platform
and SQL where applicable
It makes life easier and increases the
likelihood of a good answer
*************** *************** ************

*actually it is desirable in one situation - when running a TPC or similar
benchmark. Again you engineer the product to last pretty much for the life
of the performance test and sacrifice everything else for speed.
Nov 12 '05 #39
> From what I remember of Oracle's "basic" backup tool (and I haven't
had much to do with it since 8.1.6) you have to put tablespaces
(equivalent to our dbspaces) into "backup mode" before you can back
them up which means they can't be written to. It's also typical to do


This is a myth.

Backup mode doesn't stall any writes to the tablespaces - you can perfectly
do all of the read/write operations on a tablespace in backup mode just as
on a tablesace not in backup mode. Oracle just may do some additional redo
logging if necessary to guarantee the consistency of the database.

This has worked such way since version 7, maybe even before...

Tanel.
Nov 12 '05 #40

This thread has been closed and replies have been disabled. Please start a new discussion.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.