By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
432,247 Members | 871 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 432,247 IT Pros & Developers. It's quick & easy.

Scanning hard drive

P: n/a
Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey
Oct 21 '07 #1
Share this Question
Share on Google+
23 Replies


P: n/a
It depends entirely on how you are doing the writing code! 13 or 4
mins sounds big - how much data?

Oct 21 '07 #2

P: n/a
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.
How much, what are your expectations? The running time will depend on the
number of folders and files and the amount of info you are storing in the
list.
So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.
This is normal, the "fileinfo" was cached in the filesystem cache with the
last run, this highly reduces the Filesystem IO rate to get at the file
information.
I am wondering if I can improve on this even more??
My guess is that your system starts paging because you are exhausting the
Free RAM space with the data in your list. So, you need to start looking at
your memory consumption, and change your design, for instance by serializing
the List as a File on disk per directory you have scanned.
Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?
No, this has nothing to do with the language, really.
Willy.

Oct 21 '07 #3

P: n/a
It depends entirely on how you are doing the writing code! 13 or 4
mins sounds big - how much data?
Ignore; I misread the question

Oct 21 '07 #4

P: n/a
Interesting, I rebooted and ran it.

This time approx 10 mins to scan 56GB.

It seems you are write that the fileinfo was being cached.

I checked mem usage in task manager and it say 110MB
for the app after it ran.

Which is not a problem if you 2GB like me or even 1GB really.

So I don't see why it would have paging issues, do you think??

So you don't think I can improve on this??????

Malcolm

"Willy Denoyette [MVP]" <wi*************@telenet.bewrote in message
news:OH**************@TK2MSFTNGP03.phx.gbl...
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
>Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

How much, what are your expectations? The running time will depend on the
number of folders and files and the amount of info you are storing in the
list.
>So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

This is normal, the "fileinfo" was cached in the filesystem cache with the
last run, this highly reduces the Filesystem IO rate to get at the file
information.
>I am wondering if I can improve on this even more??
My guess is that your system starts paging because you are exhausting the
Free RAM space with the data in your list. So, you need to start looking
at your memory consumption, and change your design, for instance by
serializing the List as a File on disk per directory you have scanned.
>Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

No, this has nothing to do with the language, really.
Willy.

Oct 21 '07 #5

P: n/a
Rotsey wrote:
Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.
That's not because you've used a struct. It's because the data is cached
by the operating system. Restart your computer and try it again.

The problem is the speed that you can get the data from the disk. There
isn't a lot you can do about that (except buy a faster hard disk, or
delete data, or make sure your OS is caching the stuff)

Alun Harford
Oct 21 '07 #6

P: n/a
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ua*************@TK2MSFTNGP06.phx.gbl...
Interesting, I rebooted and ran it.

This time approx 10 mins to scan 56GB.

It seems you are write that the fileinfo was being cached.

I checked mem usage in task manager and it say 110MB
for the app after it ran.
You need to look while it runs, not after it ran, and you need to use
perfmon to watch the working set size and the private bytes.
Which is not a problem if you 2GB like me or even 1GB really.

So I don't see why it would have paging issues, do you think??
This shouldn't be an issue, unless you have a very large number of files
stored on disk and you are storing a lot of file properties in the list.
Most likely is that you are running this on a slow disk and/or system. Most
info must be retrieved from disk, this involves Disk IO , but 10 minutes
to get at this info is really slow, even 4 minutes for a second run is quite
slow.

That said, you need to tell us how many files you have on disk, what data
you are storing in the list, what OS and system HW you are running this on,
and (preferably) post a complete sample that illustrates the issue.

Willy.

Oct 21 '07 #7

P: n/a
What do you scan? I do a directory and File scan in my 100GB (64GB of data)
hard drive and took 24 seconds. (Scanned every directory and every file in
the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey

Oct 21 '07 #8

P: n/a
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1, ref
double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@microsof t.com...
What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
>Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey


Oct 22 '07 #9

P: n/a
Approx. 13 seconds on a 250GB disk with 30,000+ directories and 1,000,000+
files.

What is DriveData? (You omitted to post it).

What do you do with the content of DriveData when the call to
ScanFolder(...) returns?

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1, ref
double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@microsof t.com...
>What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
>>Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey


Oct 22 '07 #10

P: n/a
Sorry about that , but it just a class with the generic list and a string
drive property

All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the problem.

Any chance of seeing your code???

public class DriveData

{

private List<PieFileInfomList = new List<PieFileInfo>();

public List<PieFileInfoDataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

public string Drive

{

get { return mDrive; }

set { mDrive = value; }

}

}

"Stephany Young" <noone@localhostwrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl...
Approx. 13 seconds on a 250GB disk with 30,000+ directories and 1,000,000+
files.

What is DriveData? (You omitted to post it).

What do you do with the content of DriveData when the call to
ScanFolder(...) returns?

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
>OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@microso ft.com...
>>What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey


Oct 22 '07 #11

P: n/a
It was YOUR code!
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:%2****************@TK2MSFTNGP06.phx.gbl...
Sorry about that , but it just a class with the generic list and a string
drive property

All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the problem.

Any chance of seeing your code???

public class DriveData

{

private List<PieFileInfomList = new List<PieFileInfo>();

public List<PieFileInfoDataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

public string Drive

{

get { return mDrive; }

set { mDrive = value; }

}

}

"Stephany Young" <noone@localhostwrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl...
>Approx. 13 seconds on a 250GB disk with 30,000+ directories and
1,000,000+ files.

What is DriveData? (You omitted to post it).

What do you do with the content of DriveData when the call to
ScanFolder(...) returns?

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
>>OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@micros oft.com...
What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and
every file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
Hi,
>
I am writing an app that scans hard drives and logs info
about every fine on the drive.
>
The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.
>
I wanted it to be quicker.
>
So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.
>
I am wondering if I can improve on this even more??
>
Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?
>
rotsey
>
>

Oct 22 '07 #12

P: n/a
Sorry Stephen what do you mean?

"Stephany Young" <noone@localhostwrote in message
news:%2***************@TK2MSFTNGP04.phx.gbl...
It was YOUR code!
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:%2****************@TK2MSFTNGP06.phx.gbl...
>Sorry about that , but it just a class with the generic list and a string
drive property

All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the problem.

Any chance of seeing your code???

public class DriveData

{

private List<PieFileInfomList = new List<PieFileInfo>();

public List<PieFileInfoDataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

public string Drive

{

get { return mDrive; }

set { mDrive = value; }

}

}

"Stephany Young" <noone@localhostwrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl...
>>Approx. 13 seconds on a 250GB disk with 30,000+ directories and
1,000,000+ files.

What is DriveData? (You omitted to post it).

What do you do with the content of DriveData when the call to
ScanFolder(...) returns?

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@micro soft.com...
What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and
every file in the drive for info).
>
PD: for reference 170,187 files and 21,899 folders.
>
>
Regards,
>
Bela Istok
>
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl.. .
>Hi,
>>
>I am writing an app that scans hard drives and logs info
>about every fine on the drive.
>>
>The first iteration of my code used a class and a generic list
>to store the data and rhis took 13min on my 60 GB drive.
>>
>I wanted it to be quicker.
>>
>So the second time I changed the class to a struct and still
>with the generic list got the time down to 4 min.
>>
>I am wondering if I can improve on this even more??
>>
>Anyone know if it would be possible to improve this
>using C# or would it need to be wrritten in assembler or the like?
>>
>rotsey
>>
>>
>


Oct 22 '07 #13

P: n/a
It's Stephany!

You asked 'Any chance of seeing your code???'

I replied 'It was your code (that I ran)'.

The DriveData class did not add any significant overhead.

I used 10 for Age1Days and 20 for Age2Days.

Here are my results for my main dev machine (Vista).

Drive size1 size2 size3 objects elapsed time
objects per second
-----------------------------------------------------------------------------------------------
C:\ 7,406,454,214 1,166,404,747 45,787,333,348 167,342 1:57.0308662
1429.8962780812093
D:\ 395,375,683 582,398,194 31,460,898,868 59,563 0:11.8571418
5023.3859900368232
E:\ 168 0 87,079,722,334 36,356 0:04.6710676
7783.2313966083470
-----------------------------------------------------------------------------------------------
7,801,830,065 1,748,802,941 164,328,154,550 263,461 2:13.5590756
1972.6177260244529

Grand Total: 173,878,787,556 (161.937240 GB)

C:\ drive is the system drive and has the paging file so those 2 aspects go
someway to explaining the slower rate.

Quite acceptable!
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:e6**************@TK2MSFTNGP04.phx.gbl...
Sorry Stephen what do you mean?

"Stephany Young" <noone@localhostwrote in message
news:%2***************@TK2MSFTNGP04.phx.gbl...
>It was YOUR code!
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:%2****************@TK2MSFTNGP06.phx.gbl...
>>Sorry about that , but it just a class with the generic list and a
string drive property

All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the problem.

Any chance of seeing your code???

public class DriveData

{

private List<PieFileInfomList = new List<PieFileInfo>();

public List<PieFileInfoDataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

public string Drive

{

get { return mDrive; }

set { mDrive = value; }

}

}

"Stephany Young" <noone@localhostwrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl.. .
Approx. 13 seconds on a 250GB disk with 30,000+ directories and
1,000,000+ files.

What is DriveData? (You omitted to post it).

What do you do with the content of DriveData when the call to
ScanFolder(...) returns?

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.
>
Can you see any issues here?
>
Bela, would you be able to post your code that scans the drive.
>
I am scanning 56GB in about 10 minutes.
public struct PieFileInfo
>
{
>
public int RowID;
>
public PieInfoType Type;
>
public string Name;
>
public string FullName;
>
public double Filesize;
>
public DateTime LastAccess;
>
public DateTime Created;
>
public string Extension;
>
public DateTime LastWrite;
>
public int ParentFolderRowID;
>
public string Drive;
>
public double FileAge1;
>
public double FileAge2;
>
public double FileAge3;
>
public bool IsRoot;
>
}
>
public class ScanDrive
>
{
>
private DriveData mList = new DriveData();
>
public DriveData DataList
>
{
>
get { return mList; }
>
set { mList = value; }
>
}
>
private string mDrive;
>
private int mNewFolderID = 0;
>
public ScanDrive(string drivefolder)
>
{
>
mDrive = drivefolder;
>
mList.Drive = drivefolder;
>
}
>
public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)
>
{
>
DirectoryInfo objDir = new DirectoryInfo(folder);
>
int newFolderID;
>
double size1 = 0;
>
double size2 = 0;
>
double size3 = 0;
>
System.DateTime fileDate;
>
try
>
{
>
newFolderID = WriteRow(objDir, folderID);
>
if (newFolderID == 0)
>
{
>
return false;
>
}
>
foreach (DirectoryInfo dir in objDir.GetDirectories())
>
{
>
if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))
>
{
>
return true;
>
}
>
size1 += psize1;
>
size2 += psize2;
>
size3 += psize3;
>
}
>
foreach (FileInfo fle in objDir.GetFiles())
>
{
>
WriteRow(fle, newFolderID);
>
fileDate = GetAgeDate(fle);
>
if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age1Days * -1))
>
size1 += fle.Length;
>
else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age2Days * -1))
>
size2 += fle.Length;
>
else
>
size3 += fle.Length;
>
}
>
UpdateFolderRow(newFolderID, size1, size2, size3);
>
psize1 = size1;
>
psize2 = size2;
>
psize3 = size3;
>
return true;
>
}
>
catch (Exception)
>
{
>
return true;
>
}
>
>
>
}
>
private DateTime GetAgeDate(FileInfo f)
>
{
>
if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age1Days * -1))
>
return f.CreationTime;
>
else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age2Days * -1))
>
return f.LastWriteTime;
>
else
>
return f.LastAccessTime;
>
}
>
private int WriteRow(DirectoryInfo dir, int folderID)
>
{
>
PieFileInfo info = new PieFileInfo();
>
if (mNewFolderID == 0)
>
info.IsRoot = true;
>
mNewFolderID++;
>
info.RowID = mNewFolderID;
>
info.Type = PieInfoType.Folder;
>
info.FullName = dir.FullName;
>
info.Name = dir.Name;
>
info.LastAccess = dir.LastAccessTime;
>
info.LastWrite = dir.LastWriteTime;
>
info.Created = dir.CreationTime;
>
info.ParentFolderRowID = folderID;
>
info.Drive = mDrive;
>
mList.DataList.Add(info);
>
return mNewFolderID;
>
}
>
private int WriteRow(FileInfo file, int folderID)
>
{
>
PieFileInfo info = new PieFileInfo();
>
info.Type = PieInfoType.File;
>
info.Name = file.Name;
>
info.FullName = file.FullName;
>
info.Filesize = file.Length;
>
info.Extension = file.Extension;
>
info.LastAccess = file.LastAccessTime;
>
info.LastWrite = file.LastWriteTime;
>
info.Created = file.CreationTime;
>
info.ParentFolderRowID = folderID;
>
info.Drive = mDrive;
>
if (file.LastAccessTime >= DateTime.Today.AddDays(-30))
>
info.FileAge1 = file.Length;
>
else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))
>
info.FileAge2 = file.Length;
>
else
>
info.FileAge3 = file.Length;
>
mList.DataList.Add(info);
>
return folderID;
>
}
>
private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)
>
{
>
int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)
>
{
>
if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)
>
return true;
>
else
>
return false;
>
});
>
PieFileInfo info = mList.DataList[index];
>
info.FileAge1 += size1;
>
info.FileAge2 += size2;
>
info.FileAge3 += size3;
>
info.Filesize += size1 + size2 + size3;
>
mList.DataList[index] = info;
>
}
>
}
>
>
>
"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@micr osoft.com...
>What do you scan? I do a directory and File scan in my 100GB (64GB of
>data) hard drive and took 24 seconds. (Scanned every directory and
>every file in the drive for info).
>>
>PD: for reference 170,187 files and 21,899 folders.
>>
>>
>Regards,
>>
>Bela Istok
>>
>"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
>news:er**************@TK2MSFTNGP05.phx.gbl. ..
>>Hi,
>>>
>>I am writing an app that scans hard drives and logs info
>>about every fine on the drive.
>>>
>>The first iteration of my code used a class and a generic list
>>to store the data and rhis took 13min on my 60 GB drive.
>>>
>>I wanted it to be quicker.
>>>
>>So the second time I changed the class to a struct and still
>>with the generic list got the time down to 4 min.
>>>
>>I am wondering if I can improve on this even more??
>>>
>>Anyone know if it would be possible to improve this
>>using C# or would it need to be wrritten in assembler or the like?
>>>
>>rotsey
>>>
>>>
>>
>
>

Oct 22 '07 #14

P: n/a
Oh i see you mean you ran my code on your machine.

surely it can't be that fast

I have a 7200RPM 60GB drive in 1.86GHz Pentium M Laptop

What is your specs?
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:e6**************@TK2MSFTNGP04.phx.gbl...
Sorry Stephen what do you mean?

"Stephany Young" <noone@localhostwrote in message
news:%2***************@TK2MSFTNGP04.phx.gbl...
>It was YOUR code!
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:%2****************@TK2MSFTNGP06.phx.gbl...
>>Sorry about that , but it just a class with the generic list and a
string drive property

All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the problem.

Any chance of seeing your code???

public class DriveData

{

private List<PieFileInfomList = new List<PieFileInfo>();

public List<PieFileInfoDataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

public string Drive

{

get { return mDrive; }

set { mDrive = value; }

}

}

"Stephany Young" <noone@localhostwrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl.. .
Approx. 13 seconds on a 250GB disk with 30,000+ directories and
1,000,000+ files.

What is DriveData? (You omitted to post it).

What do you do with the content of DriveData when the call to
ScanFolder(...) returns?

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.
>
Can you see any issues here?
>
Bela, would you be able to post your code that scans the drive.
>
I am scanning 56GB in about 10 minutes.
public struct PieFileInfo
>
{
>
public int RowID;
>
public PieInfoType Type;
>
public string Name;
>
public string FullName;
>
public double Filesize;
>
public DateTime LastAccess;
>
public DateTime Created;
>
public string Extension;
>
public DateTime LastWrite;
>
public int ParentFolderRowID;
>
public string Drive;
>
public double FileAge1;
>
public double FileAge2;
>
public double FileAge3;
>
public bool IsRoot;
>
}
>
public class ScanDrive
>
{
>
private DriveData mList = new DriveData();
>
public DriveData DataList
>
{
>
get { return mList; }
>
set { mList = value; }
>
}
>
private string mDrive;
>
private int mNewFolderID = 0;
>
public ScanDrive(string drivefolder)
>
{
>
mDrive = drivefolder;
>
mList.Drive = drivefolder;
>
}
>
public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)
>
{
>
DirectoryInfo objDir = new DirectoryInfo(folder);
>
int newFolderID;
>
double size1 = 0;
>
double size2 = 0;
>
double size3 = 0;
>
System.DateTime fileDate;
>
try
>
{
>
newFolderID = WriteRow(objDir, folderID);
>
if (newFolderID == 0)
>
{
>
return false;
>
}
>
foreach (DirectoryInfo dir in objDir.GetDirectories())
>
{
>
if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))
>
{
>
return true;
>
}
>
size1 += psize1;
>
size2 += psize2;
>
size3 += psize3;
>
}
>
foreach (FileInfo fle in objDir.GetFiles())
>
{
>
WriteRow(fle, newFolderID);
>
fileDate = GetAgeDate(fle);
>
if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age1Days * -1))
>
size1 += fle.Length;
>
else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age2Days * -1))
>
size2 += fle.Length;
>
else
>
size3 += fle.Length;
>
}
>
UpdateFolderRow(newFolderID, size1, size2, size3);
>
psize1 = size1;
>
psize2 = size2;
>
psize3 = size3;
>
return true;
>
}
>
catch (Exception)
>
{
>
return true;
>
}
>
>
>
}
>
private DateTime GetAgeDate(FileInfo f)
>
{
>
if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age1Days * -1))
>
return f.CreationTime;
>
else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age2Days * -1))
>
return f.LastWriteTime;
>
else
>
return f.LastAccessTime;
>
}
>
private int WriteRow(DirectoryInfo dir, int folderID)
>
{
>
PieFileInfo info = new PieFileInfo();
>
if (mNewFolderID == 0)
>
info.IsRoot = true;
>
mNewFolderID++;
>
info.RowID = mNewFolderID;
>
info.Type = PieInfoType.Folder;
>
info.FullName = dir.FullName;
>
info.Name = dir.Name;
>
info.LastAccess = dir.LastAccessTime;
>
info.LastWrite = dir.LastWriteTime;
>
info.Created = dir.CreationTime;
>
info.ParentFolderRowID = folderID;
>
info.Drive = mDrive;
>
mList.DataList.Add(info);
>
return mNewFolderID;
>
}
>
private int WriteRow(FileInfo file, int folderID)
>
{
>
PieFileInfo info = new PieFileInfo();
>
info.Type = PieInfoType.File;
>
info.Name = file.Name;
>
info.FullName = file.FullName;
>
info.Filesize = file.Length;
>
info.Extension = file.Extension;
>
info.LastAccess = file.LastAccessTime;
>
info.LastWrite = file.LastWriteTime;
>
info.Created = file.CreationTime;
>
info.ParentFolderRowID = folderID;
>
info.Drive = mDrive;
>
if (file.LastAccessTime >= DateTime.Today.AddDays(-30))
>
info.FileAge1 = file.Length;
>
else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))
>
info.FileAge2 = file.Length;
>
else
>
info.FileAge3 = file.Length;
>
mList.DataList.Add(info);
>
return folderID;
>
}
>
private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)
>
{
>
int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)
>
{
>
if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)
>
return true;
>
else
>
return false;
>
});
>
PieFileInfo info = mList.DataList[index];
>
info.FileAge1 += size1;
>
info.FileAge2 += size2;
>
info.FileAge3 += size3;
>
info.Filesize += size1 + size2 + size3;
>
mList.DataList[index] = info;
>
}
>
}
>
>
>
"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@micr osoft.com...
>What do you scan? I do a directory and File scan in my 100GB (64GB of
>data) hard drive and took 24 seconds. (Scanned every directory and
>every file in the drive for info).
>>
>PD: for reference 170,187 files and 21,899 folders.
>>
>>
>Regards,
>>
>Bela Istok
>>
>"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
>news:er**************@TK2MSFTNGP05.phx.gbl. ..
>>Hi,
>>>
>>I am writing an app that scans hard drives and logs info
>>about every fine on the drive.
>>>
>>The first iteration of my code used a class and a generic list
>>to store the data and rhis took 13min on my 60 GB drive.
>>>
>>I wanted it to be quicker.
>>>
>>So the second time I changed the class to a struct and still
>>with the generic list got the time down to 4 min.
>>>
>>I am wondering if I can improve on this even more??
>>>
>>Anyone know if it would be possible to improve this
>>using C# or would it need to be wrritten in assembler or the like?
>>>
>>rotsey
>>>
>>>
>>
>
>


Oct 22 '07 #15

P: n/a
The earlier run was on a super grunty server with really fast scsi drives,
truckloads of ram and multiple dual-core perocessors.
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:OS**************@TK2MSFTNGP04.phx.gbl...
Oh i see you mean you ran my code on your machine.

surely it can't be that fast

I have a 7200RPM 60GB drive in 1.86GHz Pentium M Laptop

What is your specs?
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:e6**************@TK2MSFTNGP04.phx.gbl...
>Sorry Stephen what do you mean?

"Stephany Young" <noone@localhostwrote in message
news:%2***************@TK2MSFTNGP04.phx.gbl...
>>It was YOUR code!
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:%2****************@TK2MSFTNGP06.phx.gbl.. .
Sorry about that , but it just a class with the generic list and a
string drive property

All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the
problem.

Any chance of seeing your code???

public class DriveData

{

private List<PieFileInfomList = new List<PieFileInfo>();

public List<PieFileInfoDataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

public string Drive

{

get { return mDrive; }

set { mDrive = value; }

}

}

"Stephany Young" <noone@localhostwrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl. ..
Approx. 13 seconds on a 250GB disk with 30,000+ directories and
1,000,000+ files.
>
What is DriveData? (You omitted to post it).
>
What do you do with the content of DriveData when the call to
ScanFolder(...) returns?
>
>
>
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl.. .
>OK. Here is the class the scans the drive.
>>
>Can you see any issues here?
>>
>Bela, would you be able to post your code that scans the drive.
>>
>I am scanning 56GB in about 10 minutes.
>public struct PieFileInfo
>>
>{
>>
>public int RowID;
>>
>public PieInfoType Type;
>>
>public string Name;
>>
>public string FullName;
>>
>public double Filesize;
>>
>public DateTime LastAccess;
>>
>public DateTime Created;
>>
>public string Extension;
>>
>public DateTime LastWrite;
>>
>public int ParentFolderRowID;
>>
>public string Drive;
>>
>public double FileAge1;
>>
>public double FileAge2;
>>
>public double FileAge3;
>>
>public bool IsRoot;
>>
>}
>>
>public class ScanDrive
>>
>{
>>
>private DriveData mList = new DriveData();
>>
>public DriveData DataList
>>
>{
>>
>get { return mList; }
>>
>set { mList = value; }
>>
>}
>>
>private string mDrive;
>>
>private int mNewFolderID = 0;
>>
>public ScanDrive(string drivefolder)
>>
>{
>>
>mDrive = drivefolder;
>>
>mList.Drive = drivefolder;
>>
>}
>>
>public bool ScanFolder(string folder, int folderID, ref double
>psize1, ref double psize2, ref double psize3)
>>
>{
>>
>DirectoryInfo objDir = new DirectoryInfo(folder);
>>
>int newFolderID;
>>
>double size1 = 0;
>>
>double size2 = 0;
>>
>double size3 = 0;
>>
>System.DateTime fileDate;
>>
>try
>>
>{
>>
>newFolderID = WriteRow(objDir, folderID);
>>
>if (newFolderID == 0)
>>
>{
>>
>return false;
>>
>}
>>
>foreach (DirectoryInfo dir in objDir.GetDirectories())
>>
>{
>>
>if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
>psize3))
>>
>{
>>
>return true;
>>
>}
>>
>size1 += psize1;
>>
>size2 += psize2;
>>
>size3 += psize3;
>>
>}
>>
>foreach (FileInfo fle in objDir.GetFiles())
>>
>{
>>
>WriteRow(fle, newFolderID);
>>
>fileDate = GetAgeDate(fle);
>>
>if (fileDate >=
>DateTime.Today.AddDays(Settings.Instance.Sett ingsData.Age1Days * -1))
>>
>size1 += fle.Length;
>>
>else if (fileDate >=
>DateTime.Today.AddDays(Settings.Instance.Sett ingsData.Age2Days * -1))
>>
>size2 += fle.Length;
>>
>else
>>
>size3 += fle.Length;
>>
>}
>>
>UpdateFolderRow(newFolderID, size1, size2, size3);
>>
>psize1 = size1;
>>
>psize2 = size2;
>>
>psize3 = size3;
>>
>return true;
>>
>}
>>
>catch (Exception)
>>
>{
>>
>return true;
>>
>}
>>
>>
>>
>}
>>
>private DateTime GetAgeDate(FileInfo f)
>>
>{
>>
>if (f.CreationTime >=
>DateTime.Today.AddDays(Settings.Instance.Sett ingsData.Age1Days * -1))
>>
>return f.CreationTime;
>>
>else if (f.LastWriteTime >=
>DateTime.Today.AddDays(Settings.Instance.Sett ingsData.Age2Days * -1))
>>
>return f.LastWriteTime;
>>
>else
>>
>return f.LastAccessTime;
>>
>}
>>
>private int WriteRow(DirectoryInfo dir, int folderID)
>>
>{
>>
>PieFileInfo info = new PieFileInfo();
>>
>if (mNewFolderID == 0)
>>
>info.IsRoot = true;
>>
>mNewFolderID++;
>>
>info.RowID = mNewFolderID;
>>
>info.Type = PieInfoType.Folder;
>>
>info.FullName = dir.FullName;
>>
>info.Name = dir.Name;
>>
>info.LastAccess = dir.LastAccessTime;
>>
>info.LastWrite = dir.LastWriteTime;
>>
>info.Created = dir.CreationTime;
>>
>info.ParentFolderRowID = folderID;
>>
>info.Drive = mDrive;
>>
>mList.DataList.Add(info);
>>
>return mNewFolderID;
>>
>}
>>
>private int WriteRow(FileInfo file, int folderID)
>>
>{
>>
>PieFileInfo info = new PieFileInfo();
>>
>info.Type = PieInfoType.File;
>>
>info.Name = file.Name;
>>
>info.FullName = file.FullName;
>>
>info.Filesize = file.Length;
>>
>info.Extension = file.Extension;
>>
>info.LastAccess = file.LastAccessTime;
>>
>info.LastWrite = file.LastWriteTime;
>>
>info.Created = file.CreationTime;
>>
>info.ParentFolderRowID = folderID;
>>
>info.Drive = mDrive;
>>
>if (file.LastAccessTime >= DateTime.Today.AddDays(-30))
>>
>info.FileAge1 = file.Length;
>>
>else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))
>>
>info.FileAge2 = file.Length;
>>
>else
>>
>info.FileAge3 = file.Length;
>>
>mList.DataList.Add(info);
>>
>return folderID;
>>
>}
>>
>private void UpdateFolderRow(int folderID, double size1, double
>size2, double size3)
>>
>{
>>
>int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)
>>
>{
>>
>if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)
>>
>return true;
>>
>else
>>
>return false;
>>
>});
>>
>PieFileInfo info = mList.DataList[index];
>>
>info.FileAge1 += size1;
>>
>info.FileAge2 += size2;
>>
>info.FileAge3 += size3;
>>
>info.Filesize += size1 + size2 + size3;
>>
>mList.DataList[index] = info;
>>
>}
>>
>}
>>
>>
>>
>"Bela Istok" <be****@hotmail.comwrote in message
>news:ED**********************************@mic rosoft.com...
>>What do you scan? I do a directory and File scan in my 100GB (64GB
>>of data) hard drive and took 24 seconds. (Scanned every directory
>>and every file in the drive for info).
>>>
>>PD: for reference 170,187 files and 21,899 folders.
>>>
>>>
>>Regards,
>>>
>>Bela Istok
>>>
>>"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
>>news:er**************@TK2MSFTNGP05.phx.gbl.. .
>>>Hi,
>>>>
>>>I am writing an app that scans hard drives and logs info
>>>about every fine on the drive.
>>>>
>>>The first iteration of my code used a class and a generic list
>>>to store the data and rhis took 13min on my 60 GB drive.
>>>>
>>>I wanted it to be quicker.
>>>>
>>>So the second time I changed the class to a struct and still
>>>with the generic list got the time down to 4 min.
>>>>
>>>I am wondering if I can improve on this even more??
>>>>
>>>Anyone know if it would be possible to improve this
>>>using C# or would it need to be wrritten in assembler or the like?
>>>>
>>>rotsey
>>>>
>>>>
>>>
>>
>>
>


Oct 22 '07 #16

P: n/a
Sorry Stephany!!!

I see well that explains 2 min approx for C Drive.

I am trying to gauge what would be the case
for my app on the typical setup either business or
domestic.

Thanks for that.
"Stephany Young" <noone@localhostwrote in message
news:u2**************@TK2MSFTNGP02.phx.gbl...
The earlier run was on a super grunty server with really fast scsi drives,
truckloads of ram and multiple dual-core perocessors.
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:OS**************@TK2MSFTNGP04.phx.gbl...
>Oh i see you mean you ran my code on your machine.

surely it can't be that fast

I have a 7200RPM 60GB drive in 1.86GHz Pentium M Laptop

What is your specs?
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:e6**************@TK2MSFTNGP04.phx.gbl...
>>Sorry Stephen what do you mean?

"Stephany Young" <noone@localhostwrote in message
news:%2***************@TK2MSFTNGP04.phx.gbl...
It was YOUR code!
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:%2****************@TK2MSFTNGP06.phx.gbl. ..
Sorry about that , but it just a class with the generic list and a
string drive property
>
All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the
problem.
>
Any chance of seeing your code???
>
public class DriveData
>
{
>
private List<PieFileInfomList = new List<PieFileInfo>();
>
public List<PieFileInfoDataList
>
{
>
get { return mList; }
>
set { mList = value; }
>
}
>
private string mDrive;
>
public string Drive
>
{
>
get { return mDrive; }
>
set { mDrive = value; }
>
}
>
}
>
"Stephany Young" <noone@localhostwrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl.. .
>Approx. 13 seconds on a 250GB disk with 30,000+ directories and
>1,000,000+ files.
>>
>What is DriveData? (You omitted to post it).
>>
>What do you do with the content of DriveData when the call to
>ScanFolder(...) returns?
>>
>>
>>
>"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
>news:ei**************@TK2MSFTNGP04.phx.gbl. ..
>>OK. Here is the class the scans the drive.
>>>
>>Can you see any issues here?
>>>
>>Bela, would you be able to post your code that scans the drive.
>>>
>>I am scanning 56GB in about 10 minutes.
>>public struct PieFileInfo
>>>
>>{
>>>
>>public int RowID;
>>>
>>public PieInfoType Type;
>>>
>>public string Name;
>>>
>>public string FullName;
>>>
>>public double Filesize;
>>>
>>public DateTime LastAccess;
>>>
>>public DateTime Created;
>>>
>>public string Extension;
>>>
>>public DateTime LastWrite;
>>>
>>public int ParentFolderRowID;
>>>
>>public string Drive;
>>>
>>public double FileAge1;
>>>
>>public double FileAge2;
>>>
>>public double FileAge3;
>>>
>>public bool IsRoot;
>>>
>>}
>>>
>>public class ScanDrive
>>>
>>{
>>>
>>private DriveData mList = new DriveData();
>>>
>>public DriveData DataList
>>>
>>{
>>>
>>get { return mList; }
>>>
>>set { mList = value; }
>>>
>>}
>>>
>>private string mDrive;
>>>
>>private int mNewFolderID = 0;
>>>
>>public ScanDrive(string drivefolder)
>>>
>>{
>>>
>>mDrive = drivefolder;
>>>
>>mList.Drive = drivefolder;
>>>
>>}
>>>
>>public bool ScanFolder(string folder, int folderID, ref double
>>psize1, ref double psize2, ref double psize3)
>>>
>>{
>>>
>>DirectoryInfo objDir = new DirectoryInfo(folder);
>>>
>>int newFolderID;
>>>
>>double size1 = 0;
>>>
>>double size2 = 0;
>>>
>>double size3 = 0;
>>>
>>System.DateTime fileDate;
>>>
>>try
>>>
>>{
>>>
>>newFolderID = WriteRow(objDir, folderID);
>>>
>>if (newFolderID == 0)
>>>
>>{
>>>
>>return false;
>>>
>>}
>>>
>>foreach (DirectoryInfo dir in objDir.GetDirectories())
>>>
>>{
>>>
>>if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
>>psize3))
>>>
>>{
>>>
>>return true;
>>>
>>}
>>>
>>size1 += psize1;
>>>
>>size2 += psize2;
>>>
>>size3 += psize3;
>>>
>>}
>>>
>>foreach (FileInfo fle in objDir.GetFiles())
>>>
>>{
>>>
>>WriteRow(fle, newFolderID);
>>>
>>fileDate = GetAgeDate(fle);
>>>
>>if (fileDate >=
>>DateTime.Today.AddDays(Settings.Instance.Set tingsData.Age1Days
>>* -1))
>>>
>>size1 += fle.Length;
>>>
>>else if (fileDate >=
>>DateTime.Today.AddDays(Settings.Instance.Set tingsData.Age2Days
>>* -1))
>>>
>>size2 += fle.Length;
>>>
>>else
>>>
>>size3 += fle.Length;
>>>
>>}
>>>
>>UpdateFolderRow(newFolderID, size1, size2, size3);
>>>
>>psize1 = size1;
>>>
>>psize2 = size2;
>>>
>>psize3 = size3;
>>>
>>return true;
>>>
>>}
>>>
>>catch (Exception)
>>>
>>{
>>>
>>return true;
>>>
>>}
>>>
>>>
>>>
>>}
>>>
>>private DateTime GetAgeDate(FileInfo f)
>>>
>>{
>>>
>>if (f.CreationTime >=
>>DateTime.Today.AddDays(Settings.Instance.Set tingsData.Age1Days
>>* -1))
>>>
>>return f.CreationTime;
>>>
>>else if (f.LastWriteTime >=
>>DateTime.Today.AddDays(Settings.Instance.Set tingsData.Age2Days
>>* -1))
>>>
>>return f.LastWriteTime;
>>>
>>else
>>>
>>return f.LastAccessTime;
>>>
>>}
>>>
>>private int WriteRow(DirectoryInfo dir, int folderID)
>>>
>>{
>>>
>>PieFileInfo info = new PieFileInfo();
>>>
>>if (mNewFolderID == 0)
>>>
>>info.IsRoot = true;
>>>
>>mNewFolderID++;
>>>
>>info.RowID = mNewFolderID;
>>>
>>info.Type = PieInfoType.Folder;
>>>
>>info.FullName = dir.FullName;
>>>
>>info.Name = dir.Name;
>>>
>>info.LastAccess = dir.LastAccessTime;
>>>
>>info.LastWrite = dir.LastWriteTime;
>>>
>>info.Created = dir.CreationTime;
>>>
>>info.ParentFolderRowID = folderID;
>>>
>>info.Drive = mDrive;
>>>
>>mList.DataList.Add(info);
>>>
>>return mNewFolderID;
>>>
>>}
>>>
>>private int WriteRow(FileInfo file, int folderID)
>>>
>>{
>>>
>>PieFileInfo info = new PieFileInfo();
>>>
>>info.Type = PieInfoType.File;
>>>
>>info.Name = file.Name;
>>>
>>info.FullName = file.FullName;
>>>
>>info.Filesize = file.Length;
>>>
>>info.Extension = file.Extension;
>>>
>>info.LastAccess = file.LastAccessTime;
>>>
>>info.LastWrite = file.LastWriteTime;
>>>
>>info.Created = file.CreationTime;
>>>
>>info.ParentFolderRowID = folderID;
>>>
>>info.Drive = mDrive;
>>>
>>if (file.LastAccessTime >= DateTime.Today.AddDays(-30))
>>>
>>info.FileAge1 = file.Length;
>>>
>>else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))
>>>
>>info.FileAge2 = file.Length;
>>>
>>else
>>>
>>info.FileAge3 = file.Length;
>>>
>>mList.DataList.Add(info);
>>>
>>return folderID;
>>>
>>}
>>>
>>private void UpdateFolderRow(int folderID, double size1, double
>>size2, double size3)
>>>
>>{
>>>
>>int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)
>>>
>>{
>>>
>>if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)
>>>
>>return true;
>>>
>>else
>>>
>>return false;
>>>
>>});
>>>
>>PieFileInfo info = mList.DataList[index];
>>>
>>info.FileAge1 += size1;
>>>
>>info.FileAge2 += size2;
>>>
>>info.FileAge3 += size3;
>>>
>>info.Filesize += size1 + size2 + size3;
>>>
>>mList.DataList[index] = info;
>>>
>>}
>>>
>>}
>>>
>>>
>>>
>>"Bela Istok" <be****@hotmail.comwrote in message
>>news:ED**********************************@mi crosoft.com...
>>>What do you scan? I do a directory and File scan in my 100GB (64GB
>>>of data) hard drive and took 24 seconds. (Scanned every directory
>>>and every file in the drive for info).
>>>>
>>>PD: for reference 170,187 files and 21,899 folders.
>>>>
>>>>
>>>Regards,
>>>>
>>>Bela Istok
>>>>
>>>"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in
>>>message news:er**************@TK2MSFTNGP05.phx.gbl...
>>>>Hi,
>>>>>
>>>>I am writing an app that scans hard drives and logs info
>>>>about every fine on the drive.
>>>>>
>>>>The first iteration of my code used a class and a generic list
>>>>to store the data and rhis took 13min on my 60 GB drive.
>>>>>
>>>>I wanted it to be quicker.
>>>>>
>>>>So the second time I changed the class to a struct and still
>>>>with the generic list got the time down to 4 min.
>>>>>
>>>>I am wondering if I can improve on this even more??
>>>>>
>>>>Anyone know if it would be possible to improve this
>>>>using C# or would it need to be wrritten in assembler or the like?
>>>>>
>>>>rotsey
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>


Oct 22 '07 #17

P: n/a
The first I see in you code is the use of Doubles, you don't need dobles you
only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1, ref
double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@microsof t.com...
>What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
>>Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey


Oct 22 '07 #18

P: n/a
Liz

Where is PieInfoType defined? do you have a driver stub which launches this
thing?
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1, ref
double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsD ata.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@microsof t.com...
>What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
>>Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey



Oct 22 '07 #19

P: n/a
public enum PieInfoType
{
Folder,
File
}

Regards,

Bela Istok
"Liz" <li*@tiredofspam.comwrote in message
news:um**************@TK2MSFTNGP06.phx.gbl...
>
Where is PieInfoType defined? do you have a driver stub which launches
this thing?
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
>OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@microso ft.com...
>>What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey


Oct 22 '07 #20

P: n/a
I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you have
multi processor or multi core machine, with only 1 core you don't win much).
I will try to post the code later when have time to clean it a bit.

Regards,

Bela Istok
"Bela Istok" <be****@hotmail.comwrote in message
news:1B**********************************@microsof t.com...
The first I see in you code is the use of Doubles, you don't need dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
>OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Settings Data.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@microso ft.com...
>>What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey

Oct 22 '07 #21

P: n/a
Last run with code:
Using delegated function
Start time:10/22/2007 5:13:19 PM
End time:10/22/2007 5:16:58 PM
Total time:00:03:39.6027394
Size1: 769462539; Size2: 2074779247; Size3:52272046924; File Number:169593;
Directory Number:21854;

Code:
Attached.

Regard,

Bela Istok

"Bela Istok" <be****@hotmail.comwrote in message
news:FA**********************************@microsof t.com...
>I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you have
multi processor or multi core machine, with only 1 core you don't win
much). I will try to post the code later when have time to clean it a bit.

Regards,

Bela Istok
"Bela Istok" <be****@hotmail.comwrote in message
news:1B**********************************@microsof t.com...
>The first I see in you code is the use of Doubles, you don't need dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
>>OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Setting sData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@micros oft.com...
What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and
every file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.
Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl...
Hi,
>
I am writing an app that scans hard drives and logs info
about every fine on the drive.
>
The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.
>
I wanted it to be quicker.
>
So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.
>
I am wondering if I can improve on this even more??
>
Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?
>
rotsey
>
>
Oct 22 '07 #22

P: n/a
ok thanks Bela,

I will run it.

When you ran this code, the file system info is cached
so the second time your un it it is much faster because of this.

Were these stats from the first run????

"Bela Istok" <be****@hotmail.comwrote in message
news:6F**********************************@microsof t.com...
Last run with code:
Using delegated function
Start time:10/22/2007 5:13:19 PM
End time:10/22/2007 5:16:58 PM
Total time:00:03:39.6027394
Size1: 769462539; Size2: 2074779247; Size3:52272046924; File
Number:169593;
Directory Number:21854;

Code:
Attached.

Regard,

Bela Istok

"Bela Istok" <be****@hotmail.comwrote in message
news:FA**********************************@microsof t.com...
>>I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you
have
multi processor or multi core machine, with only 1 core you don't win
much). I will try to post the code later when have time to clean it a bit.

Regards,

Bela Istok
"Bela Istok" <be****@hotmail.comwrote in message
news:1B**********************************@microso ft.com...
>>The first I see in you code is the use of Doubles, you don't need dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}

}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Settin gsData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}

"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@micro soft.com...
What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and
every file in the drive for info).
>
PD: for reference 170,187 files and 21,899 folders.
>
>
Regards,
>
Bela Istok
>
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:er**************@TK2MSFTNGP05.phx.gbl.. .
>Hi,
>>
>I am writing an app that scans hard drives and logs info
>about every fine on the drive.
>>
>The first iteration of my code used a class and a generic list
>to store the data and rhis took 13min on my 60 GB drive.
>>
>I wanted it to be quicker.
>>
>So the second time I changed the class to a struct and still
>with the generic list got the time down to 4 min.
>>
>I am wondering if I can improve on this even more??
>>
>Anyone know if it would be possible to improve this
>using C# or would it need to be wrritten in assembler or the like?
>>
>rotsey
>>
>>
>

Oct 23 '07 #23

P: n/a
The stats for the first run was the sended the other times takes < 30 sec.

Regards,

Bela Istok

"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:%2****************@TK2MSFTNGP05.phx.gbl...
ok thanks Bela,

I will run it.

When you ran this code, the file system info is cached
so the second time your un it it is much faster because of this.

Were these stats from the first run????

"Bela Istok" <be****@hotmail.comwrote in message
news:6F**********************************@microsof t.com...
>Last run with code:
Using delegated function
Start time:10/22/2007 5:13:19 PM
End time:10/22/2007 5:16:58 PM
Total time:00:03:39.6027394
Size1: 769462539; Size2: 2074779247; Size3:52272046924; File
Number:169593;
Directory Number:21854;

Code:
Attached.

Regard,

Bela Istok

"Bela Istok" <be****@hotmail.comwrote in message
news:FA**********************************@microso ft.com...
>>>I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you
have
multi processor or multi core machine, with only 1 core you don't win
much). I will try to post the code later when have time to clean it a
bit.

Regards,

Bela Istok
"Bela Istok" <be****@hotmail.comwrote in message
news:1B**********************************@micros oft.com...
The first I see in you code is the use of Doubles, you don't need
dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
news:ei**************@TK2MSFTNGP04.phx.gbl...
OK. Here is the class the scans the drive.
>
Can you see any issues here?
>
Bela, would you be able to post your code that scans the drive.
>
I am scanning 56GB in about 10 minutes.
public struct PieFileInfo
>
{
>
public int RowID;
>
public PieInfoType Type;
>
public string Name;
>
public string FullName;
>
public double Filesize;
>
public DateTime LastAccess;
>
public DateTime Created;
>
public string Extension;
>
public DateTime LastWrite;
>
public int ParentFolderRowID;
>
public string Drive;
>
public double FileAge1;
>
public double FileAge2;
>
public double FileAge3;
>
public bool IsRoot;
>
}
>
public class ScanDrive
>
{
>
private DriveData mList = new DriveData();
>
public DriveData DataList
>
{
>
get { return mList; }
>
set { mList = value; }
>
}
>
private string mDrive;
>
private int mNewFolderID = 0;
>
public ScanDrive(string drivefolder)
>
{
>
mDrive = drivefolder;
>
mList.Drive = drivefolder;
>
}
>
public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)
>
{
>
DirectoryInfo objDir = new DirectoryInfo(folder);
>
int newFolderID;
>
double size1 = 0;
>
double size2 = 0;
>
double size3 = 0;
>
System.DateTime fileDate;
>
try
>
{
>
newFolderID = WriteRow(objDir, folderID);
>
if (newFolderID == 0)
>
{
>
return false;
>
}
>
foreach (DirectoryInfo dir in objDir.GetDirectories())
>
{
>
if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))
>
{
>
return true;
>
}
>
size1 += psize1;
>
size2 += psize2;
>
size3 += psize3;
>
}
>
foreach (FileInfo fle in objDir.GetFiles())
>
{
>
WriteRow(fle, newFolderID);
>
fileDate = GetAgeDate(fle);
>
if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age1Days * -1))
>
size1 += fle.Length;
>
else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age2Days * -1))
>
size2 += fle.Length;
>
else
>
size3 += fle.Length;
>
}
>
UpdateFolderRow(newFolderID, size1, size2, size3);
>
psize1 = size1;
>
psize2 = size2;
>
psize3 = size3;
>
return true;
>
}
>
catch (Exception)
>
{
>
return true;
>
}
>
>
>
}
>
private DateTime GetAgeDate(FileInfo f)
>
{
>
if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age1Days * -1))
>
return f.CreationTime;
>
else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.Setti ngsData.Age2Days * -1))
>
return f.LastWriteTime;
>
else
>
return f.LastAccessTime;
>
}
>
private int WriteRow(DirectoryInfo dir, int folderID)
>
{
>
PieFileInfo info = new PieFileInfo();
>
if (mNewFolderID == 0)
>
info.IsRoot = true;
>
mNewFolderID++;
>
info.RowID = mNewFolderID;
>
info.Type = PieInfoType.Folder;
>
info.FullName = dir.FullName;
>
info.Name = dir.Name;
>
info.LastAccess = dir.LastAccessTime;
>
info.LastWrite = dir.LastWriteTime;
>
info.Created = dir.CreationTime;
>
info.ParentFolderRowID = folderID;
>
info.Drive = mDrive;
>
mList.DataList.Add(info);
>
return mNewFolderID;
>
}
>
private int WriteRow(FileInfo file, int folderID)
>
{
>
PieFileInfo info = new PieFileInfo();
>
info.Type = PieInfoType.File;
>
info.Name = file.Name;
>
info.FullName = file.FullName;
>
info.Filesize = file.Length;
>
info.Extension = file.Extension;
>
info.LastAccess = file.LastAccessTime;
>
info.LastWrite = file.LastWriteTime;
>
info.Created = file.CreationTime;
>
info.ParentFolderRowID = folderID;
>
info.Drive = mDrive;
>
if (file.LastAccessTime >= DateTime.Today.AddDays(-30))
>
info.FileAge1 = file.Length;
>
else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))
>
info.FileAge2 = file.Length;
>
else
>
info.FileAge3 = file.Length;
>
mList.DataList.Add(info);
>
return folderID;
>
}
>
private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)
>
{
>
int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)
>
{
>
if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)
>
return true;
>
else
>
return false;
>
});
>
PieFileInfo info = mList.DataList[index];
>
info.FileAge1 += size1;
>
info.FileAge2 += size2;
>
info.FileAge3 += size3;
>
info.Filesize += size1 + size2 + size3;
>
mList.DataList[index] = info;
>
}
>
}
>
>
>
"Bela Istok" <be****@hotmail.comwrote in message
news:ED**********************************@micr osoft.com...
>What do you scan? I do a directory and File scan in my 100GB (64GB of
>data) hard drive and took 24 seconds. (Scanned every directory and
>every file in the drive for info).
>>
>PD: for reference 170,187 files and 21,899 folders.
>>
>>
>Regards,
>>
>Bela Istok
>>
>"Rotsey" <ma***********@RemoveThis.optusnet.com.auwrote in message
>news:er**************@TK2MSFTNGP05.phx.gbl. ..
>>Hi,
>>>
>>I am writing an app that scans hard drives and logs info
>>about every fine on the drive.
>>>
>>The first iteration of my code used a class and a generic list
>>to store the data and rhis took 13min on my 60 GB drive.
>>>
>>I wanted it to be quicker.
>>>
>>So the second time I changed the class to a struct and still
>>with the generic list got the time down to 4 min.
>>>
>>I am wondering if I can improve on this even more??
>>>
>>Anyone know if it would be possible to improve this
>>using C# or would it need to be wrritten in assembler or the like?
>>>
>>rotsey
>>>
>>>
>>
>
>


Oct 24 '07 #24

This discussion thread is closed

Replies have been disabled for this discussion.