473,854 Members | 1,834 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

crash during file writing, how to recover ?

Hi
I'm writing a commercial program which must be reliable. It has to do
some basic reading and writing to and from files on the hard disk,
and also to a floppy.
I have foreseen a potential problem. The program may crash
unexpectedly while writing to the file. If so, my program should
detect this during startup, and then (during startup) probably delete the
data added to the file and redo the writing operation.

Are file writing operations atomic ? ie when you write to a file,
will it either do it succesfully, OR say half fail (eg write a few letters
and not finish), OR not commit any changes to the file if a crash at
this point occurs?

My next question is how is this handled in commercial programming? I
plan on writing a flag (say, a simple char) to another file (this
would signal that a file write is about to begin), and then
removing this char after the file writing operation is completed.
Then on startup i just check the flags. if flag hasn't been removed a
crash occurred, so have to open file and get rid of any garbage.

Has anyone done anything similar b4? if so how did you handle this
crash scenario. My application could totally stuff up if i don't
handle this right.

by the way, i'm using the java language and api. this might effect
how files are written to, so i thought i should mention this.
MANY THANKS
Joseph

Jul 17 '05
48 8531
Nick Landsberg wrote:

There are two situations which could
have cause the problem, one of which is
*probably* not IBM's fault.


I don't know the details. But multiple people knowing
more details than me have told, that there is no doubt it
was IBM's fault. If I understood it correctly, it happened
as some IBM technical personel was doing some maintainance
on the system. And it also sounded like some software bug
in the database must have been part of the reason.

Later IBM actually payed part of the 10 million dollars of
damages. Though I think the exact amount IBM payed in this
case is kept secret.

--
Kasper Dupont -- der bruger for meget tid paa usenet.
For sending spam use ab***@mk.lir.dk and ka*****@mk.lir. dk
I'd rather be a hammer than a nail.
Jul 17 '05 #41
On Tue, 04 May 2004 02:01:06 GMT, CBFalconer <cb********@yah oo.com>
wrote:
Nick Landsberg wrote:

Were you that person, Chuck? :)


Unfortunatel y, no.


Hm, I thought everyone knew it was Richard Pryor ;^D

Darko
Jul 17 '05 #42
Roedy Green wrote:
If you really wanted to be safe, you have three teams running it and
three programming teams working to the same spec.


Indeed. Failing to do this brought down the EMS coordination system in
Berlin during the millenium change. IIRC it was a bug indirectly related to
the Y2K problem. They had a mirror. Which exactly replicated the bug.
Then they switched back to their old system. Which couldn't handle the
load of emergencies generated by 4 Million people partying...
Jul 17 '05 #43
[ egregious cross-posting elided ]

Joseph <ka****@bigpond .com> wrote:
I'm thankful to everyone for sharing their opinion. I read
everybody's post and learned allot. It turns out that my algorithm
will be simple, maybe something like this:

Make a copy of existing data_file in the same directory
operate on this new copy (data_file2) to add data
close data_file2 when done
delete data_file (original)
rename data_file2 to data_file

on startup the sw will be able to detect if a crash occurred and
respond appropriately. The code on startup should be pretty
straight forward

By the way, the platform is java on linux (probably red hat)


Since you're on Linux, you can eliminate the fourth step.
File.renameTo() fails on Windows if the destination exists, but it's
fine on Unix. That way, you don't need any special startup logic at
all; rename() on Unix between files in the same directory (actually,
filesystem) is guaranteed to be atomic. However, if your code needs
to run on Windows, you need what you described above.

BTW, I asked once on a Microsoft group whether MoveFile() is
guaranteed to be atomic on Windows between names on the same
filesystem, like rename() in POSIX. Unfortunately, I didn't get
anything definitive, just a lot of mumbling that amounted to
"probably, as long as you're on NTFS." Any Windows gurus here that
know if there is such a guarantee (or that it is specifically *not*
guaranteed)?

-- Lucas
Jul 17 '05 #44
CBFalconer wrote:
In the US we round 0.50..0 of anything up to 1.0, which produces
a bias of some size. If the perpetrator had been satisfied with
this he might still be collecting. Rounding down also produces
a bias, while round to even is (normally) unbiased.


Isn't "Banker's Rounding" rounding to even (or similar)?

--
|_ CJSonnack <Ch***@Sonnack. com> _____________| How's my programming? |
|_ http://www.Sonnack.com/ _______________ ____| Call: 1-800-DEV-NULL |
|______________ _______________ _______________ _|_____________ __________|
Jul 17 '05 #45
Hi
So just to confirm, if the OS crashes while doing a rename file or delete
operation, the files being renamed or deleted can't be corrupted right (ie
these operations are atomic) ? Just want to be 100 percent sure.

And, if my temp file is corrupted, i wont be able to write to it. However,
it's possible to detect corrupted files using java api fuctions. I'll be
able to detect any corrupted file and simply delete it.

I know i'm asking alot of questions which have pretty much been answered,
but I'd like to feel more sure about all this.

thanks

Joseph
Joseph wrote:

Hi all

I'm thankful to everyone for sharing their opinion. I read everybody's
post
and learned allot. It turns out that my algorithm will be simple, maybe
something like this:
Make a copy of existing data_file in the same directory

operate on this new copy (data_file2) to add data

close data_file2 when done

delete data_file (original)

rename data_file2 to data_file


on startup the sw will be able to detect if a crash occurred and respond
appropriately. The code on startup should be pretty straight forward

By the way, the platform is java on linux (probably red hat)

Joseph


Jul 17 '05 #46
Liz

"Joseph" <ka****@bigpond .com> wrote in message
news:qi******** **********@news-server.bigpond. net.au...
Hi
So just to confirm, if the OS crashes while doing a rename file or delete
operation, the files being renamed or deleted can't be corrupted right (ie
these operations are atomic) ? Just want to be 100 percent sure.
I have not read all the preceding emails but I was
just reading a java book last night and it says to
write the file to a temp file first then do a copy
as a single operation. If the crash occurs:
1. writing temp -> the old official file is still good
2. during the copy -> the temp is still good
you probably have to figure out 1 or 2 manually during
the crash recovery

And, if my temp file is corrupted, i wont be able to write to it. However, it's possible to detect corrupted files using java api fuctions. I'll be
able to detect any corrupted file and simply delete it.

I know i'm asking alot of questions which have pretty much been answered,
but I'd like to feel more sure about all this.

thanks

Joseph
Joseph wrote:

Hi all

I'm thankful to everyone for sharing their opinion. I read everybody's
post
and learned allot. It turns out that my algorithm will be simple, maybe
something like this:
Make a copy of existing data_file in the same directory

operate on this new copy (data_file2) to add data

close data_file2 when done

delete data_file (original)

rename data_file2 to data_file


on startup the sw will be able to detect if a crash occurred and respond
appropriately. The code on startup should be pretty straight forward

By the way, the platform is java on linux (probably red hat)

Joseph

Jul 17 '05 #47


could you explain this a little more . thanks
Joseph
I have not read all the preceding emails but I was
just reading a java book last night and it says to
write the file to a temp file first then do a copy
as a single operation. If the crash occurs:
1. writing temp -> the old official file is still good
2. during the copy -> the temp is still good
you probably have to figure out 1 or 2 manually during
the crash recovery

And, if my temp file is corrupted, i wont be able to write to it.

However,
it's possible to detect corrupted files using java api fuctions. I'll be
able to detect any corrupted file and simply delete it.

I know i'm asking alot of questions which have pretty much been answered,
but I'd like to feel more sure about all this.

thanks

Joseph
Joseph wrote:
>
> Hi all
>
> I'm thankful to everyone for sharing their opinion. I read everybody's
> post
> and learned allot. It turns out that my algorithm will be simple,
> maybe something like this:
>
>
> Make a copy of existing data_file in the same directory
>
> operate on this new copy (data_file2) to add data
>
> close data_file2 when done
>
> delete data_file (original)
>
> rename data_file2 to data_file
>
>
>
>
> on startup the sw will be able to detect if a crash occurred and
> respond
> appropriately. The code on startup should be pretty straight forward
>
> By the way, the platform is java on linux (probably red hat)
>
> Joseph


Jul 17 '05 #48
that'll work....

- perry

Joseph wrote:
Hi
So just to confirm, if the OS crashes while doing a rename file or delete
operation, the files being renamed or deleted can't be corrupted right (ie
these operations are atomic) ? Just want to be 100 percent sure.

And, if my temp file is corrupted, i wont be able to write to it. However,
it's possible to detect corrupted files using java api fuctions. I'll be
able to detect any corrupted file and simply delete it.

I know i'm asking alot of questions which have pretty much been answered,
but I'd like to feel more sure about all this.

thanks

Joseph
Joseph wrote:

Hi all

I'm thankful to everyone for sharing their opinion. I read everybody's
post
and learned allot. It turns out that my algorithm will be simple, maybe
something like this:
Make a copy of existing data_file in the same directory

operate on this new copy (data_file2) to add data

close data_file2 when done

delete data_file (original)

rename data_file2 to data_file


on startup the sw will be able to detect if a crash occurred and respond
appropriately . The code on startup should be pretty straight forward

By the way, the platform is java on linux (probably red hat)

Joseph



Jul 17 '05 #49

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

110
10666
by: alf | last post by:
Hi, is it possible that due to OS crash or mysql itself crash or some e.g. SCSI failure to lose all the data stored in the table (let's say million of 1KB rows). In other words what is the worst case scenario for MyISAM backend? Also is it possible to not to lose data but get them corrupted?
0
9903
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9752
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
11041
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10692
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10373
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9525
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7921
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7083
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
3
3193
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.