Read the whole hard disk

[giannakis]

Hi, I want to read the whole hard disk by using c/c++. I don't want to access only the "visible" files, but also the deleted files (the survived files).
Is it possible? How can I achieve it?

I'm using windows environment.

Many thanks

[sicarie]

What do you want to do with them? Did you look into the Win32 API? What have you looked at so far?

[giannakis]

What do you want to do with them? Did you look into the Win32 API? What have you looked at so far?

I'm involved in a forensic analysis and I need to develop a kind of a forensic tool. I don't want to use Win32 API. Instead, I want to use c/c++ libraries ( I want to develop a similar tool for unix and windows environment).

Any suggestion?

Thanks

[RRick]

When you say C/C++ libraries do you mean the libraries just supplied by the compiler or do you include the system level C libraries (there are very few if any system C++ libraries).

The system C libraries give you the best low level access to the file system. Which ones you pick are going to be based on your application and system. Linux and windows don't see eye to eye on this subject.

For linux, a good place to start is dirent.h. This will allow you to read the directories and the files found in them. This will work with "normal" files.

For hidden files, Linux starts the name with a '.'. For deleted files, linux doesn't keep any of that information. Once a file is deleted, its gone.

[oler1s]

I don’t want to use Win32 API. Instead, I want to use c/c++ libraries ( I want to develop a similar tool for unix and windows environment).

Ok, but that’s not an option. And uh, there is no language known as C/C++. I know it seems a bit picky to complain about that, but there’s a good reason for saying so. They are separate languages, with C++ having significant additions to its libraries. You might as well as say I want to use C++/Java. Well, pick a language already and tell us what you are using.

C++ libraries aren’t designed to be an all encompassing platform. Instead, they provide a small core and everything else is offloaded to 3rd party libraries. You will need 3rd party function for what you have to do.

The idea in recovering deleted files can be broken down into the following essence. When you delete a file, effectively the OS sets a mark in the filesystem that says the storage area for the entire file is deleted or is free to use. It’s like taking a paper and stamping the word “shredded” on a corner of it. That’s great, but the paper can still be read. And that’s precisely what file recovery programs do. They walk through the file system, reading files despite the marks.

Such programs are very low system level programs. Concepts like “creating a file”, “opening” it, and “deleting” it are actually quite high level. The OS has to manage the hardware, the filesystem, and do a number of operations to manage whatever is considered to be “files”. You’re not just dipping into OS specific code. You’re jumping into very, very low level OS code.

I get the feeling you aren’t aware of the enormity of the task you want to undertake. For one thing, give me a brief description of the various filesystems you can encounter under windows and *NIX environments. Do you understand how they work? What does a deletion operation under those filesystems mean?

[giannakis]

Ok, but that’s not an option. And uh, there is no language known as C/C++. I know it seems a bit picky to complain about that, but there’s a good reason for saying so. They are separate languages, with C++ having significant additions to its libraries. You might as well as say I want to use C++/Java. Well, pick a language already and tell us what you are using.

C++ libraries aren’t designed to be an all encompassing platform. Instead, they provide a small core and everything else is offloaded to 3rd party libraries. You will need 3rd party function for what you have to do.

The idea in recovering deleted files can be broken down into the following essence. When you delete a file, effectively the OS sets a mark in the filesystem that says the storage area for the entire file is deleted or is free to use. It’s like taking a paper and stamping the word “shredded” on a corner of it. That’s great, but the paper can still be read. And that’s precisely what file recovery programs do. They walk through the file system, reading files despite the marks.

Such programs are very low system level programs. Concepts like “creating a file”, “opening” it, and “deleting” it are actually quite high level. The OS has to manage the hardware, the filesystem, and do a number of operations to manage whatever is considered to be “files”. You’re not just dipping into OS specific code. You’re jumping into very, very low level OS code.

I get the feeling you aren’t aware of the enormity of the task you want to undertake. For one thing, give me a brief description of the various filesystems you can encounter under windows and *NIX environments. Do you understand how they work? What does a deletion operation under those filesystems mean?

Look at the hard disk as a single file. No matter what is the content of the hard disk (therefore, it doesn't matter what is/are the file system(s)). How can I read the content of the hard disk? I assume that the source code should not be more than 40-50 lines of code.

Meaning of C/C++: I want to use C or C++. I guess I have to use C, which offers more low level capabilities than C++.

[giannakis]

Look at the hard disk as a single file. No matter what is the content of the hard disk (therefore, it doesn't matter what is/are the file system(s)). How can I read the content of the hard disk? I assume that the source code should not be more than 40-50 lines of code.

Meaning of C/C++: I want to use C or C++. I guess I have to use C, which offers more low level capabilities than C++.

I made a mistake. We should look the hard disk not as a single file, but as a set of sectors.

[sicarie]

I made a mistake. We should look the hard disk not as a single file, but as a set of sectors.

And as oler1s asked, do you know how each file system handles those sectors?

[arunmib]

As all have mentioned there is a lot you need to do before doing programming.

In Linux, I think you have deleted files recovery option if the file system is ext2 under midnight root. Get a hang of that code and may be you can use that as a starter.

[giannakis]

And as oler1s asked, do you know how each file system handles those sectors?

I don't need to know the structure of each file system right now. I have books which describe their format, or how they handle the files. I will start worry about the File Systems after I will be able to access the content (data without a meaning) of the hard disk. Now, I'm looking for a way to access the whole hard disk (sector by sector, byte by byte).

[giannakis]

As all have mentioned there is a lot you need to do before doing programming.

In Linux, I think you have deleted files recovery option if the file system is ext2 under midnight root. Get a hang of that code and may be you can use that as a starter.

I need to develop this tool for Windows 2003 and later I need to develop the same tool (with the same functionalities) for Linux...but thank you very much for the direction.

[RRick]

My guess is that your project is loosely based on windows, since you want to find various types of files (i.e hidden, deleted, and who knows). Somewhere in your design, you realize you'll have to go pretty deep to get the information you want, possibly down to the sector level

What people here are trying to tell you is that there is a BIG distance between the two ends described here. You can spend years trying to fill the gaps between these two extremes. You are going to need to do a fair amount of requirements and design before you go off coding your solution.

Even before you start the design, I would look at some of the available products. For windows, there are some really good programs that do this already and the price is cheap. (That said, there are thousands of programs that do this poorly). That might give you an idea of the scope you want to tackle.

[Cucumber]

Never get discouraged when you want to learn something.
A couple of hints:

You might want to take a look into how the FAT works.
And I think there should be a couple of software interruptions to read hard disk sectors directly.

[arunmib]

I need to develop this tool for Windows 2003 and later I need to develop the same tool (with the same functionalities) for Linux...but thank you very much for the direction.

There is one more reason I wanted you take a look at the code. I assume it will give you a fair idea about the amount of work you need to put in. You had mentioned in one of your posts that you need to read the entire contents of hard drive in 40 odd lines of code. You will get to know whether the target is atleast possible or not.