Endianness Converter

[MrPickle]

I am trying to make a function to read in a value, then convert it to little endian.

I wrote this:

template<typename T>
void Read(T& value, std::ifstream& in) {
    std::vector<unsigned char> bytes(sizeof(T));
    in.read(reinterpret_cast<char*>(&bytes[0]), sizeof(T));
    value = 0;
 
    for(int i = 0; i < sizeof(T); i++) {
        value |= (bytes[i] << i * 8);
    }
}

But this is only working for int and the compiler won't accept trying to give it a float or double.

Should I scrap the template and just overload it?

[weaknessforcats]

You can't use the |= operator with double. It only works for integer types.

You may have to use an explicit specialization for double.

[MrPickle]

I just tried it with a unsigned short & short, it doesn't work correctly for them either? No matter what the value it's meant to be reading in, it output 7543 :3

[JosAH]

Does a good old C union help? Something like this:

union enian {
   unsigned char b[sizeof(double)];
   unsigned short s;
   unsigned int i;
   unsigned long l;
   float f;
   double d;
} buffer;
 

Read n bytes in the buffer member, reverse n bytes and fetch one of the other members.

kind regards,

Jos

[MrPickle]

I've never really used unions before.

[JosAH]

@MrPickle
Well, what's keeping you?

kind regards,

Jos

[MrPickle]

@JosAH
I'm not to sure what they are exactly, a reduced class?

[JosAH]

@MrPickle
Yep, a very reduced class, even a reduced struct. All the members of a union start at the same place. So in the example above the buffer member takes the same memory as the double d or the int i. So, e.g., when I change the first four bytes of the buffer I directly change the bytes that make up the int i etc.

The buffer char member is just another 'view' of the int i or double d etc. which is exactly what you want although you may not realize that yet ;-)

kind regards,

Jos

[MrPickle]

So what that was doing was moving everything along, 4 bytes in memory? How does that change the order of the variables in memory?

[vmpstr]

What JosAH was saying is that you can have

union myunion
{
    int i;
    char bytes[4];
} hello;
 

then, you can do something like hello.i = 123;
That assigns the number 123 into the memory occupied by hello.i
However, because you have a union, this memory is also occupied by hello.bytes, which is a character array.

In turn, this means that you can manipulate the elements of hello.bytes array (like say, reverse their order), which will also change hello.i

You won't "change the order of the variables", but by reversing the bytes, you will... well, reverse the bytes.

[MrPickle]

OH, I see. So whatever happens to bytes, also happens to i?

Can I read directly into bytes then? eg;
in.read(&hello.bytes[0], sizeof(int));

Ahh, yes. I get it, thanks for all the help.

[vmpstr]

@MrPickle
correct

@MrPickle
sure, why not

@MrPickle
all the credit goes to JosAH :D

[MrPickle]

A couple questions, will this method work for std::string. I'm not sure it will, will I just have to read it in char by char until char == '\0'.

Have I done this right, because it's giving me linker errors when I try called Read<int>... or Read<unsigned short>...

template <typename T> union Endian {
    unsigned char bytes[sizeof(T)];
    T value;
};
 
template<typename T> void Read(T& value, std::ifstream& in) {
    Endian<T> e;
    in.read((&e.bytes[0]), sizeof(T));
 
    for(int i = 0; i < sizeof(T); i++) {
        bytes[i] = bytes[sizeof(T) - i]; //Not sure here, might cause access violation?
    }
 
    value = e.value;
}

[Savage]

bytes[i] = bytes[sizeof(T) - i]; //Not sure here, might cause access violation?
 
//It might.You mean:
 
 bytes[i] = bytes[sizeof(T) -1- i];
 

That's the only error I see(it would compile with this error).
So which are the errors?
Can you post them?

[JosAH]

@Savage
Better change the loop boundary condition as well because as it is now you haven't done anything except for juggling those bytes back and forth ;-)

kind regards,

Jos

[MrPickle]

I've fixed the linker bug now. There was an error in the header that it wasn't reporting.

The function I wrote doesn't work though, I don't really get what you was trying to say Jos

[weaknessforcats]

It looks like your template is not in the same file as main().

Templates are not like functions. They are patterns the compiler copies and substitutes the type you specify on your code. That means the pattern has to be in the same file as the call.

[MrPickle]

Ok, thanks. I've fixed it now.

I don't know why it's now working though.

It outputs this:

bytes[0] = }
bytes[1] =
bytes[2] =
bytes[3] =
value = 0

It should equal 123

[JosAH]

Change your loop to this:

for(int i= 0, j= sizeof(T); i < --j; i++) {
   char   t= e.bytes[i]; 
   e.bytes[i]= e.bytes[j];
   e.bytes[j]= t;
}
 

kind regards,

Jos

[MrPickle]

Yey, it's working. Thanks everybody.

template <typename T> union Buffer {
    unsigned char bytes[sizeof(T)];
    T value;
};
 
template<typename T> void Read(T& value, std::ifstream& in) {
    Buffer<T> e;
    in.read(reinterpret_cast<char*>(&e.bytes[0]), sizeof(T));
 
    char t;
    int j;
    for(int i = 0; i < sizeof(T); i++) {
        std::cout << i << " ";
        j = (sizeof(T)-1)-i;
        t = e.bytes[i];
        e.bytes[i] = e.bytes[j];
        e.bytes[j] = t;
    }
    std::cout << "\n";
 
    value = e.value;
}

[vmpstr]

You want to be careful in your "swap the bytes" loop:
Suppose sizeof(T) is 4, then you have
1. swap byte 0 and byte 3
2. swap byte 1 and byte 2
3. swap byte 2 and byte 1
4. swap byte 3 and byte 0

In effect, step 3 cancels step 2 and step 4 cancels step 1 - so the bytes aren't swapped.

You might want to iterate only until sizeof(T)/2

[MrPickle]

I'm not sure, but should I check if the machine is little endian? Otherwise it's going to make it big endian, right?

[vmpstr]

If it's little endian, it's going to make it big endian. If it's big endian, it's going to make it little endian.

This just converts it from one to another, it's up to you to figure out how you want to use it...

[JosAH]

The OP might want to look at the htons() and htonl() macros. They convert two and four byte integers to 'network byte' order, i.e. big endian order. Those macros don't do anything if the host machine happens to be a big endian machine.

kind regards,

Jos

[MrPickle]

Alright, thanks for your help.