473,320 Members | 1,910 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

HDD Burn In with Python

Hi Again,

I have Macs, Windows and Linux PCs. I'd like to have a generic HDD
burn in program to test new drives on all platforms. I know Python is
portable enough to handle the platform differences, but would it be a
good language for this type of application? The process would involve
lots of reads and writes all over the drives for an extended period of
time (several hours). Anybody else done something similar to this? Any
suggestions?

Thanks !!!
Jul 18 '05 #1
9 2551
I dont know of any application that currently does this.

I think for applications that perform a lot of I/O, Python
might not be a good choice. The file I/O in Python is written
in optimized C, but still you would be better off writing
a C program for this, that is portable. For example, if you
are targeting only Unix/Linux platforms there wont be much
difficulty writing a portable C app optimized, say for the
gcc compiler.

Since you have Windows also mentioned, I recommend Perl.
Perl is a bit faster than Python in system related tasks and
it is portable also. Python, though my favorite language comes
only second here. And of course, forget about Jython ;-)

If you do write your app in Python, perform benchmarks to
time the Python app itself. A good way to do this is by
using the 'Profile' class (of 'profile' module) by its 'calibrate'
method. This will make sure that the timings are done accurately.
This will be the major difference between a C app and a Python app,
since the C system calls are fast enough that their times can be
ignored.

It will be a tough job to get it portable and just right
on all platforms. So let me know if you do come up with an app
like that in Python.

-Anand

ho********@hotmail.com (hokiegal99) wrote in message news:<93**************************@posting.google. com>...
Hi Again,

I have Macs, Windows and Linux PCs. I'd like to have a generic HDD
burn in program to test new drives on all platforms. I know Python is
portable enough to handle the platform differences, but would it be a
good language for this type of application? The process would involve
lots of reads and writes all over the drives for an extended period of
time (several hours). Anybody else done something similar to this? Any
suggestions?

Thanks !!!

Jul 18 '05 #2
Anand Pillai wrote:
Perl is a bit faster than Python in system related tasks and
it is portable also.
[...]
If you do write your app in Python, perform benchmarks to
time the Python app itself.


Hokiegal99 didn't ask for a disk benchmark tool that has to
be high-performant, he (she?) wanted a harddisk IO stress tool.
Just letting your harddisk work hard for a long time doesn't
require optimized C code. Python is perfectly suitable for that.

The only problem is, that for a true harddisk stress test, you
will want to access sectors all over the disk. I'm not aware of
any mechanism to do that with portable Python...
(although you could format the disk in one huge partition and
then write random files to it until the disk is full).

--Irmen

Jul 18 '05 #3
Anand Pillai wrote:
I dont know of any application that currently does this.

I think for applications that perform a lot of I/O, Python
might not be a good choice. The file I/O in Python is written
in optimized C, but still you would be better off writing
a C program for this, that is portable. For example, if you
are targeting only Unix/Linux platforms there wont be much
difficulty writing a portable C app optimized, say for the
gcc compiler.

Since you have Windows also mentioned, I recommend Perl.
Perl is a bit faster than Python in system related tasks and
it is portable also. Python, though my favorite language comes
only second here. And of course, forget about Jython ;-)
Speed isn't a huge concern for this test. Heck, it will take many hours
or possibly a day or two depending on the size of the drive. I like Perl
as well, but it's too convoluted. I sometimes look back at one month old
Perl scripts and wonder what the heck they do. To me, that's scary. I
have a close friend who heads up a large NOC at a big US university and
they have banned Perl for this very reason. I only use Perl when there
is no other option, and I think Python is a valid choice for this task.
If you do write your app in Python, perform benchmarks to
time the Python app itself. A good way to do this is by
using the 'Profile' class (of 'profile' module) by its 'calibrate'
method. This will make sure that the timings are done accurately.
This will be the major difference between a C app and a Python app,
since the C system calls are fast enough that their times can be
ignored.


This is a good tip. Thanks, I'll try it.


Jul 18 '05 #4
Irmen de Jong wrote:
Just letting your harddisk work hard for a long time doesn't
require optimized C code. Python is perfectly suitable for that.
Yes, I agree! Now, if I only knew how to conduct a proper stress test.
Read, write, verify reads & writes... would that be sufficient?

The only problem is, that for a true harddisk stress test, you
will want to access sectors all over the disk. I'm not aware of
any mechanism to do that with portable Python...
(although you could format the disk in one huge partition and
then write random files to it until the disk is full).


That's an idea. But what approach might one take if there were already
files and folders on the volume that should not be destroyed? Some type
of non-destructive test?

Jul 18 '05 #5

"hokieghal99" <ho********@hotmail.com> wrote in message
news:bs**********@solaris.cc.vt.edu...
Irmen de Jong wrote:

(although you could format the disk in one huge partition and
then write random files to it until the disk is full).


That's an idea. But what approach might one take if there were already
files and folders on the volume that should not be destroyed? Some type
of non-destructive test?


You can start start with the root directory and use os.path.walk to read
everything already the disk.

You could also get the assembly coded SpinRite5 (<$100) which will
systematically read and test every block, including boot sectors, on the
disk (moving each block of data temporarily to another place while write
testing). It also fixes bad formatting and has twice saved a disk for me
that ScanDisk could not.

Terry J. Reedy
Jul 18 '05 #6
Terry Reedy wrote:
You could also get the assembly coded SpinRite5 (<$100) which will
systematically read and test every block, including boot sectors, on the
disk (moving each block of data temporarily to another place while write
testing). It also fixes bad formatting and has twice saved a disk for me
that ScanDisk could not.

Terry J. Reedy


I've used SpinRite before. An excellent piece of software, but it has a
few short falls:

Only works on x86.
Only works on fat or fat32 filesyatems.
Very, Very slow because of its throughness.
Has trouble with bigish (> 60GB) HDDs.

If you use Windows 98 with a fat32 filesystem on a smallish IDE based
HDD... it's a great program, otherwise it's not very useful.

Jul 18 '05 #7
ho********@hotmail.com (hokiegal99) wrote...
I have Macs, Windows and Linux PCs. I'd like to have a generic HDD
burn in program to test new drives on all platforms. I know Python is
portable enough to handle the platform differences, but would it be a
good language for this type of application? The process would involve
lots of reads and writes all over the drives for an extended period of
time (several hours). Anybody else done something similar to this? Any
suggestions?


Properly burn-in testing a hard drive via lots of seeks, reads, and
writes etc. is something I wouldn't have thought Python was
particularly applicable to. The problems I can see here are:

- Python can't provide uniform low-level access to your disks. You can
use the os module, or write your own uniform disk module, which would
possibly be more effort than it was worth.

- Using the os module to create a massive file, then doing seeks,
reads, and writes around in it is not my idea of a good solution to
the problem. You are also then including the underlying operating
system's filesystem layer which will almost certainly be doing all
manner of caching and additional journalling which you can't control.

- Once you've solved the problem of uniform low level access in a
'disk' module, you also have the problem of ensuring that data
actually hits the disk. The disk has buffers, and the os has buffers.
If your targets are only *NIX/x86 and OS/X, then you are at least
confined to a unix-style environment, which makes getting low-level
disk access easy and the only problem is working out device file
names. I'm not sure how fsync() works on file descriptors open on
device files, but at a guess, I think you're going to have to try
harder to make sure the data actually hits the disk (ioctls?).

I don't think the problem is even confined to making sure writes hit
the disk. You may have problems with the operating system caching
reads too.

Are the disks IDE-based? 99.9%+ of all modern drives manufactured have
SMART capability, which has a lot of disk tests built into it,
although I don't think it has the ability to do thorough write tests.

On top of that, if you are expecting to use the burn-in tests for
benchmarking too, the results you'll get will probably not represent
the true performance of the drive in a platform-neutral way.
HTH,
David.
Jul 18 '05 #8
Irmen de Jong fed this fish to the penguins on Tuesday 30 December 2003
03:11 am:
then write random files to it until the disk is full).
Probably using multiple threads to do overlapping I/O operations, and
maybe insert some reads into the mess too?
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Bestiaria Home Page: http://www.beastie.dm.net/ <
Home Page: http://www.dm.net/~wulfraed/ <


Jul 18 '05 #9
David M. Wilson:
If your targets are only *NIX/x86 and OS/X, then you are at least
confined to a unix-style environment, which makes getting low-level
disk access easy and the only problem is working out device file
names.


On Windows NT or XP the drives have names like \\.\PhysicalDrive0 so the
following Python code can do some reading from a particular position on the
first physical disk:

f = open("\\\\.\\PhysicalDrive0", "rb")
f.seek(1800*1024*1024)
for i in range(8000000):
x = f.read(1024)

Neil
Jul 18 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Bernie | last post by:
Hi, I'm very new to .NET but have been using VB since version 5. I need a way to write a backup file to CD without user intervention. A command line batch file would be easiest but I also want...
5
by: Sobhan.Vezzu | last post by:
Hi All, In our project we need to burn CD/DVD according to the size/availability. For this we chose www.binarymagics.com software and we are having many issues with this. Have people tried any...
2
by: Elioth | last post by:
Hi, I want to know, how to burn CD from VB 2005 ?. First burn data CD and then Audio CD. Thanks for all help. Elioth Collazo
6
by: SStory | last post by:
I need to be able to burn a CD from VB.NET. The OS is XP so I don't care how it does it, but not having to go through the wizard of course--and free. I have searched and found XPBurn, but...
1
by: =?Utf-8?B?ZGF2aWQ=?= | last post by:
when trying to burn a track from burn list a pop up says insert blank disc.when i do this the disc is recognized on the lhs of screen in d drive but not on rhs of screen where it says insert blank...
6
by: Academia | last post by:
I have a computer at home with a (slow) dialup connection and a computer at school with a fast connection. I want to go there (it is not close) and down load Vb2008 express, burn it into a CD,...
4
by: Jerry Spence1 | last post by:
I am creating a project involving video files and I need to be able to offer the user the ability to burn the result to DVD. I don't need any fancy titles, transitions or anything. I know there...
2
by: Jim Rutledge | last post by:
well I went and bought some software that said it would make DVDs playable on any machine . Needless to say it didn't do the job and I am looking into developing some software in C# that will ....
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.