Char* operation

[Airslash]

Sorry,

couldn't come up with a better title, but here's what I'm trying to do:

I'm making a small console application that reads .pak files (they're from a game). I'm building it in steps, so I don't make errors.

Right now i'm trying to obtain the amount of files stored inside such a .pak file. This is normally done by reading the first 2 bytes from the .pak files, reversing them and then interpreting them as a header.
I'm able to read the first 2 bytes by using an fstream object (ifstream) and read 2 bytes into a char* buffer.

Now I have as question, how can I reverse the 2 bytes, and then interpret it as an integer/short. ?

I tried switching the bytes of place inside the char* (treated as array) but didn't give me the result I wanted.


example :

24 00 should be read as 00 24, and resulting in 36 as integer.

[JosAH]

The two bytes are stored big endian in your file and you want them to be in little
endian order. Read those two bytes in a short int and apply the ntohs() macro
to it like this:

unsigned short s= /* read the first two bytes */
s= ntohs(s);
 

kind regards,

Jos

[Airslash]

Ok, I managed to get a bit further.
Here's the code I'm currently using, it's very rough just to get the 2 bytes at the moment, nothing else:

int main() {
    // Create a new reference to a streamreader
    ifstream* mystream = new ifstream("C:\\Netts\\Florensia\\Data\\Map.pak");
    // Read the first 2 bytes to check the amount of files present
    char* buffer = new char[2];
    mystream->read(buffer, 2);
    // Cast to an short
    unsigned short amount = *buffer;
    // reverse the byte order
    amount = ntohs(amount);
    // print out the result
    cout << "Amount of files in Map.pak : " << amount << endl;
}
 

when I compile, the ntohs() command gets highlighted with the following error:

C:/Users/Arne/workspace/Florensia/Debug/../main.cpp:22: undefined reference to `_ntohs@4'
 

any idea what this means?

[Airslash]

never mind found it :)

had to link to ws2_32.lib

but now I get as output : Amount of files in Map.pak : 9216

but it should be : Amount of files in Map.pak : 36.

is the conversion going wrong then?

[donbock]

The purpose of ntohs is to convert an unsigned short encoded according to "TCP/IP network byte order" to host encoding. Network byte order is most-significant byte first (also called big-endian). If host encoding matches network byte order then ... either ntohs returns the unchanged input or it performs a spurious byte swap -- check your documentation.

However, your example stated that the sequence 0x24, 0x00 should be interpreted as 0x0024 = 36. This is least-significant byte first (also called little-endian).

It is a lot less confusing if you forget about these endian terms and concentrate on your problem domain:

/* Convert pak-encoded 16-bit value into unsigned short */
const char *paktohs(const char *pointer, unsigned short *value)
{
    unsigned short lowbyte, highbyte;
    lowbyte = *pointer++;
    highbyte = *pointer++;
    *value = (highbyte << 8) | lowbyte;
    return pointer;
}

[Airslash]

Ok I think I got it, but I cannot count on it that low endian always fits, because sometimes the second set of bytes isn't always 00. i have pak files that contain over 10k files, so sometimes it can 24 1C, and then it doesn't count anymore I suppose.


Bit new to this, I know c+++ general programming but these tidbits make it confusing sometimes.

[JosAH]

Bit new to this, I know c+++ general programming but these tidbits make it confusing sometimes.

As always the venom is in the nitty gritty details; stick to my reply #2 and you'll
be done, i.e. read those two bytes into a short, apply that ntohs macro to the
short and voila; just a few lines of code, no more no less.

kind regards,

Jos

[donbock]

Ok I think I got it, but I cannot count on it that low endian always fits, because sometimes the second set of bytes isn't always 00. i have pak files that contain over 10k files, so sometimes it can 24 1C, and then it doesn't count anymore I suppose.

It doesn't matter whether the second byte is 0. The point is that your pak file expresses 16-bit values low-byte-first. That is, the byte sequence 0x24, 0x1C corresponds to 16-bit value 0x1C24 (which equals 7204).

Assembling the bytes into the unsigned short yourself as I suggested in reply #5 means that you don't have to know whether your processor is big-endian or little-endian. That code will work on either kind of system.

[Airslash]

aah ok, now it makes sense :)
I've copied your code and I'm trying to pour it into several functions now.
Maybe when it's finished i'll post the entire code online to share my .pâk reader :)

[Airslash]

Ok I've been trying to pour your code into a class object so I can already partiall read the 2 bytes from the entire pak file

header

#ifndef PAKFILE_H_
#define PAKFILE_H_
#include <iostream>
#include <fstream>
using namespace std;
 
class PakFile {
    public:
        PakFile(char* const filename);
        ~PakFile();
 
        char* const getFilename() const;
        unsigned short* const getAmountOfFiles() const;
 
        void setFilename(char* const filename);
 
        void load();
 
    private:
        unsigned short* amount_of_files;
        char* file_name;
 
        void readAmountOfFiles(ifstream* fileStream);
};
 
#endif /* PAKFILE_H_ */
 

cpp file implementation

#include "../pakfile.h"
#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;
 
PakFile::PakFile(char* const filename) {
    this->setFilename(filename);
    this->amount_of_files = new unsigned short();
}
 
PakFile::~PakFile() {
    delete this->file_name;
    delete this->amount_of_files;
}
 
char* const PakFile::getFilename() const {
    return this->file_name;
}
 
void PakFile::setFilename(char* const filename) {
    this->file_name = filename;
}
 
void PakFile::readAmountOfFiles(ifstream* fileStream) {
    // Set the pointer of the stream to the beginning of the file.
    fileStream->seekg(0, ios::beg);
    // Create a buffer to read the 2 chars into
    char* buffer = new char[2];
    // Read the 2 bytes into the buffer
    fileStream->read(buffer, 2);
    // Reverse the 2 bytes
    // And convert them to an unsigned short.
    unsigned short lowbyte, highbyte;
    lowbyte = *buffer++;
    highbyte = *buffer++;
    *this->amount_of_files = (highbyte << 8) | lowbyte;
}
 
void PakFile::load() {
    // Create a new stream to read through the file.
    ifstream* fileStream = new ifstream(this->getFilename());
    // Read the amount of files first.
    this->readAmountOfFiles(fileStream);
}
 
unsigned short* const PakFile::getAmountOfFiles() const {
    return this->amount_of_files;
}
 

main.cpp

#include <cstdlib>
#include <iostream>
#include <fstream>
#include "pakfile.h"
 
using namespace std;
 
int main() {
    // create a new pakfile
    PakFile* file = new PakFile("C:\\Netss\\Florensia\\Data\\Map.pak");
    // Load the information
    file->load();
    // Print the information
    cout << "Amount of files in " << file->getFilename()  << " : " << *file->getAmountOfFiles() << endl;
    // delete the pointer
    delete file;
}
 

the output is this : Amount of files in C:\Netss\Florensia\Data\Map.pak : 65480
and not 36 (bytes in the file are 24 00, so should be read as 0x0024)

have I done the part of your code wrong ?

[donbock]

I don't see an obvious problem with your code. Let's print out the values of lowbyte, highbyte, and amount_of_files in hexadecimal to see what's going on.

[newb16]

the output is this : Amount of files in C:\Netss\Florensia\Data\Map.pak : 65480
and not 36 (bytes in the file are 24 00, so should be read as 0x0024)

have I done the part of your code wrong ?

this->file_name - it is not necessary to prepend "this->" as it is executing inside member function.
Second, why amount_of_files is pointer to integer? There is no need to do it, plain int will be enough. Next, did you try to debug ( debugprint ) it and check values of lowbyte and highbyte right after read operation - were they corrupted right there or later? Were the data actually read from the stream?
And your buffer[2] is leaking - better allocate 2-byte array locally like
char buffer[2],
, accessing it like buffer[0] and buffer[1], and there will be no need to delete it.
And you should not delete file_name in your class, as you do not allocate it, but only copy pointer value. You should not

char * c = "abcde";
delete c;

[Airslash]

added following code in my read method:

cout << "lowbyte: " << lowbyte << endl;
    cout << "highbyte: " << highbyte << endl;
 

result:

lowbyte: 65480
highbyte: 36
Amount of files in C:\Netss\Florensia\Data\Map.pak : 65480

[donbock]

result:
lowbyte: 65480
highbyte: 36
Amount of files in C:\Netss\Florensia\Data\Map.pak : 65480

lowbyte: 65480 = 0xFFC8
highbyte: 36 = 0x24
Looks like there's been sign-extension in lowbyte. Declaring buffer as an array of unsigned char should fix that (or you could mask the lowbyte and highbyte values with 0xFF). Even if we manually remove the sign-extension, the resulting combined value of 0x24C8 (9416) is not what you expect. Maybe you should dump the first several bytes of the file and compare that to what's in your buffer array. Maybe the file format is not what you expect.

[Airslash]

the file format matches, I have the technical details right next to me.
got them not so long ago:

- 2 bytes reversed : amount of files
- 2 zero padding bytes
---- repeating area ----
- 196 bytes : filename
- 2 zero padding bytes
- 2 bytes : start location
- 2 zero padding bytes
- 2 bytes : file length
---- end of area ----
- file contents

I even opened the files with Ultraedit in Binary view so I can see the raw output in hex values

I'll see about declaring the buffer, what it gives, stay tuned :)

[Airslash]

Problem solved......


the issue was a typo in the patch, so nothing got loaded and it was just reading random stuff from memory.