Low Level File I/0

[emaghero]

Greetings and Salutations,

I am currently doing some file I/0 on data that I have and I do not wish to use fstream. I have data stored in its untranslated binary format and I want to read it into another program. I have been using _open and _write to export the data but I have been having problems implementing _read.

I will give a simple example of what I want to do.

The following code creates a file and stores it in some location. The file is in binary format and is open for reading and writing.

    int data;
    int file=_open("g:\\testfile.rob",_O_BINARY|_O_WRONLY|_O_CREAT|_O_TRUNC,_S_IREAD|_S_IWRITE);
 
    if(file==-1){
        perror("Failed to open file\n");
    }
    else{
        cout<<"file is open\n";
    }
 
    char *header="Test File";
 
    data=_write(file,header,(unsigned int)(strlen(header)));
 
    double arr[20];
 
    srand((unsigned)time(NULL));//Seed the random numbers
 
    //Create an array of random numbers
    for(int i=0;i<20;i++){
        arr[i]=rand()%100;//Create an array of random numbers
    }
 
    //Send these random numbers to the output file
    double val;
    for(int i=0;i<20;i++){
        val=arr[i];
        data=_write(file,(void *)&val,sizeof(double));
    }
 
    _close(file);
 

This works fine once the following have been included

#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <io.h>
#include <stdio.h>
 

I have been trying to read the data back from the file using the

_read("file desciptor", "buffer", "Number of bytes");
 

function. This function is supposed to take a file, similar to the one used above, read in Number of bytes data to the buffer and the return the number of bytes that were read.

Here is what I have attempted to do.

int data;
    int file=_open("g:\\testfile.rob",_O_BINARY|_O_WRONLY|_O_CREAT|_O_TRUNC,_S_IREAD|_S_IWRITE);
 
    if(file==-1){
        perror("Failed to open file\n");
    }
    else{
        cout<<"file is open\n";
    }
 
    char *header="Test File";
 
    data=_write(file,header,(unsigned int)(strlen(header)));
 
    double arr[20];
 
    srand((unsigned)time(NULL));//Seed the random numbers
 
    //Create an array of random numbers
    for(int i=0;i<20;i++){
        arr[i]=rand()%100;//Create an array of random numbers
    }
 
    //Send these random numbers to the output file
    double val;
    for(int i=0;i<20;i++){
        val=arr[i];
        data=_write(file,(void *)&val,sizeof(double));
    }
 
    char BUF[1024];
 
    int testvar;
 
    unsigned int num=300;
 
    testvar=_read(file,(void *)&BUF,num);
 
    cout<<testvar<<endl;
 
    cout<<BUF<<endl;
 
   _close(file);
 

Upon output the value of testvar is -1, which indicates an error, and the output of BUF is gibberish.

Can anybody suggest a possible remedy.

Any help will be greatly appreciated.

[emaghero]

I have found out why I was getting a return of -1.

Basically I should not have tried to read from the fiel while it was still open.

The following code will produce error free output.

int data;
    int file=_open("g:\\testfile.rob",_O_BINARY|_O_WRONLY|_O_CREAT|_O_TRUNC,_S_IREAD|_S_IWRITE);
 
    if(file==-1){
        perror("Failed to open file\n");
    }
    else{
        cout<<"file is open\n";
    }
 
    char *header="Test File";
 
    data=_write(file,header,(unsigned int)(strlen(header)));
 
    double arr[20];
 
    srand((unsigned)time(NULL));
 
    //Create an array of random numbers
    for(int i=0;i<20;i++){
        arr[i]=rand()%100;
    }
    //Send these random numbers to the output file
    double val;
    for(int i=0;i<20;i++){
        val=arr[i];
        data=_write(file,(void *)&val,sizeof(double));
    }
 
    _close(file); //Note the closing of the file here and the opening of a new file below
 
    int newfile=_open("g:\\testfile.rob",_S_IREAD);
 
    char BUF[1024];
 
    int testvar;
 
    unsigned int num=600;
 
    testvar=_read(newfile,(void *)&BUF,num);
 
    cout<<testvar<<endl;
 
    cout<<BUF<<endl;
 
    _close(newfile);
 

The output from this code is 169, the number of bytes read from the file, and "Test Header", the head that was assigned at the start.

Can anybody tell me how to extract the numbers that are stored in the file.
This being fairly important for a numerical application.

[xoinki]

hi,
you can use ascii range for detection..
read everything character by character then check whether it is within the range if it is then it is a number else ignore and continue till EOF.
Regards
Xoinki

[RRick]

The ascii technique is not going to work because the numbers are stored as binary. No ascii conversion was made when the numbers were written to the file.

Instead, you're going to have to treat the buffer as a pointer to doubles. If the index where the double data begins is dInd, then the following will allow you to view buffer as doubles.

double * dPtr = (double *) & buffer[ dInd];
cout << dPtr[0] << etc.
 

Reading and writing binary data this way is very platform dependent. There are issues like big/little endian byte ordering. Also, the format and precision of the double can change between systems and even compilers.

Becareful how you use this.

[emaghero]

The ascii technique is not going to work because the numbers are stored as binary. No ascii conversion was made when the numbers were written to the file.

Instead, you're going to have to treat the buffer as a pointer to doubles. If the index where the double data begins is dInd, then the following will allow you to view buffer as doubles.

double * dPtr = (double *) & buffer[ dInd];
cout << dPtr[0] << etc.
 

Reading and writing binary data this way is very platform dependent. There are issues like big/little endian byte ordering. Also, the format and precision of the double can change between systems and even compilers.

Becareful how you use this.

Hey RRick, thanks very much. That worked quite well. I was thinking of something similar but I couldn't get the syntax right.

With regard to your warning: What is "big/ little endian byte ordering"?

I don't plan on changing compilers so that won't be an issue.

[RRick]

Endian has to do with the byte ordering with ints, longs, doubles, etc. I'll let you google that subject because I always get is backwards. A diagram really helps.

It appears you are doing this for speed and size, but be careful with the low level reads and writes. They are not necessarily buffered which can be expensive when you read/write small chunks of data. The C and C++ routines do support buffering and also support binary data.

I use the low level routines when I want to do a single read or write. I had one job that needed to read and write images. Since I knew how big the largest image was, I could do all my calculations in memory and then write it out with one write. Same thing for reading.