C to Java socket input question

[Richard]

Level: Java newbie, C experienced
Platform: Linux and Win32, Intel

Another programmer and I are working on a small project together.
He's writing a server process in Java that accepts input from
processes I've written over a TCP connection. My processes are all
written in C; his are all done in Java. He's new to Java, and I've
never really used it.

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

We're starting to spin our wheels about how to present input to his
server, since there's no native 32-bit unsigned int in Java (so far
as I understand it). He's game, but is dealing with learning Java,
and we're at the point where we have to get past this issue and on to
other parts of what we're building. I'm not getting a very clear
picture of what the Java-side problem is, and I'm at the point where
I'll take on the obligation of learning what the problem is, and
doing any necessary data conversion, if that's what it takes to get
past this.

Any references to, or discussion of, this kind of IO problem would be
greatly appreciated.

--
For email, put NOT SPAM in Subject or I'll probably miss it.
<><

[David Zimmerman]

Richard wrote:

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

Treat it as a 4 byte array. InetAddress.getByAddress() wants a byte[]
anyway.

Or put it in a signed 32 bit integer and never do arithmetic with it,
the bits are the same regardless of how they're interpretted.

Either way, mind the endienness of the data. Best idea here would be to
get the C porogram to put everything in network order, the same as Java's.

[Richard]

dz***@hiddenmind.com wrote...

Richard wrote:

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.
Treat it as a 4 byte array. InetAddress.getByAddress() wants a byte[]
anyway.

Or put it in a signed 32 bit integer and never do arithmetic with it,
the bits are the same regardless of how they're interpretted.

Sorry for being brain-dead, but will he be able to do arithmetic on
it if it's in a 4-byte array? As in: unsigned char fbarray[4] ; ?
Either way, mind the endienness of the data. Best idea here would be to
get the C porogram to put everything in network order, the same as Java's.

Already been bit by that one. :)

--
For email, put NOT SPAM in Subject or I'll probably miss it.
<><

[Thomas Gagné]

Don't send binary data. Send it in text. Both C and Java understand text and
you don't have to worry about network byte ordering or whether your CPUs are
big or little-endian.

Or you could use some middleware to make your job easier.
--
..tom
remove dashes in email for replies
open source middleware at http://isectd.sourceforge.net

[Richard]

jo*@joncruz.org wrote...

Richard wrote:

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses)
Ummm.... Maybe. Or maybe not. :-D

32-bit is *one* form of an IP address. Another is 128-bit addresses in IPv6.

It's best that you account for both.

<Sound of hand slapping forehead>

Yes, that's true.
see java.net.InetAddress, java.net.Inet4Address and
java.net.Inet6Address for more.

along with the occasional fixed-
length char array.
Fixed length char array? Or string? Logically which do you have?

String of fixed length.
And what encoding? If you want to avoid many-to-many complexity
nightmares, it's probably best to use UTF-8.
Oui. Of course.
Hmmm..... so maybe fixed might not be as robust.

We're starting to spin our wheels about how to present input to his
server, since there's no native 32-bit unsigned int in Java (so far
> as I understand it).

But an inet address is not really a 32-bit unsigned int.

??? Unless you're referring to IPV6,

in_addr_t => uint32_t (<netinet/in.h>) => "unsigned 32-bit int"
So you can just
use Java's int, which is 32-bit signed. No problems.

Any references to, or discussion of, this kind of IO problem would be
greatly appreciated.

Hmmmm.... you could search these newsgroups in Google or some such for
any posts where I've touched on this (I do when it comes up).

a few off hand:
Message-ID: <3D**************@joncruz.org>
Message-ID: <3F**************@joncruz.org>
Message-ID: <pa***************************@virgin.NO_SPAM.ne t>
Message-ID: <3D**************@joncruz.org>
Message-ID: <3E**************@joncruz.org>
Message-ID: <3F**************@joncruz.org>

Thank you.

--
For email, put NOT SPAM in Subject or I'll probably miss it.
<><

[Roedy Green]

On Wed, 16 Jul 2003 12:53:55 GMT, David Zimmerman
<dz***@hiddenmind.com> wrote or quoted :

Either way, mind the endienness of the data. Best idea here would be to
get the C porogram to put everything in network order, the same as Java's.

An alternative is to use LEDataInputStream on the Java end to read
little endian data or use the nio classes.

see http://mindprod.com/jgloss/endian.html

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.

[Sudsy]

Richard wrote:

Level: Java newbie, C experienced
Platform: Linux and Win32, Intel

Another programmer and I are working on a small project together.
He's writing a server process in Java that accepts input from
processes I've written over a TCP connection. My processes are all
written in C; his are all done in Java. He's new to Java, and I've
never really used it.

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

We're starting to spin our wheels about how to present input to his
server, since there's no native 32-bit unsigned int in Java (so far
as I understand it). He's game, but is dealing with learning Java,
and we're at the point where we have to get past this issue and on to
other parts of what we're building. I'm not getting a very clear
picture of what the Java-side problem is, and I'm at the point where
I'll take on the obligation of learning what the problem is, and
doing any necessary data conversion, if that's what it takes to get
past this.

Any references to, or discussion of, this kind of IO problem would be
greatly appreciated.

Why not just read as a stream of bytes and use
InetAddress.getByAddress( byte[] )
to convert into a format which can be conveniently
used by various Java methods?
So while "there's no native 32-bit unsigned in in Java"
there IS this wonderful class called InetAddress which
encapsulates IP addresses.
Drop me a line if you need some sample code.

[Steve Horsley]

On Wed, 16 Jul 2003 11:15:47 +0000, Richard wrote:

Level: Java newbie, C experienced
Platform: Linux and Win32, Intel

Another programmer and I are working on a small project together. He's
writing a server process in Java that accepts input from processes I've
written over a TCP connection. My processes are all written in C; his
are all done in Java. He's new to Java, and I've never really used it.

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed- length
char array.

We're starting to spin our wheels about how to present input to his
server, since there's no native 32-bit unsigned int in Java (so far as I
understand it). He's game, but is dealing with learning Java, and we're
at the point where we have to get past this issue and on to other parts
of what we're building. I'm not getting a very clear picture of what
the Java-side problem is, and I'm at the point where I'll take on the
obligation of learning what the problem is, and doing any necessary data
conversion, if that's what it takes to get past this.

Any references to, or discussion of, this kind of IO problem would be
greatly appreciated.

Well, you have already had some good responses, but let me add my own
opinion anyway...

Start by defining the protocol between the machines. Do this by defining
the byte stream in terms of bytes, not any higher structure, since TCP
does not inherently support any higher structure. If you MUST send larger
structures such as IP addresses (yes I know you must), define how they
will be sent byte-by-byte. E.g. an IPv4 address goes as 4 bytes, in
the following order: 1.2.3.4 is sent as 0x01, 0x02, 0x03, 0x04.
This is how the RFCs define their protocols.

Now each of you can go about assembling and disassembling these messages
in the way that seems most natural to you and your language. This may well
turn out to be that the C code mainly handles addresses as unsigned ints
but the java end handles them as byte[4]. You can ignore each other's
implementation and concentrate on meeting the network spec.

By the way, NEVER send structures from C like this:
socket.write(myStruct, sizeof myStruct);
This can introduce hidden padding, and does not specify which order things
will be sent in. Always specify the byte stream format, and work to that.

Steve

[Jordan Zimmerman]

Send it as a String. You have a problem sitting there in C as well.
Different machine types have different long int encodings. It's much safer
to send a String.

--
Jordan Zimmerman
http://www.jordanzimmerman.com

"Richard" <rh***@hotmail.com> wrote in message
news:MP************************@news.verizon.net.. .

[Richard]

bi*********@hotmail.com wrote...

Richard wrote:

Level: Java newbie, C experienced
Platform: Linux and Win32, Intel

Another programmer and I are working on a small project together.
He's writing a server process in Java that accepts input from
processes I've written over a TCP connection. My processes are all
written in C; his are all done in Java. He's new to Java, and I've
never really used it.

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

We're starting to spin our wheels about how to present input to his
server, since there's no native 32-bit unsigned int in Java (so far
as I understand it). He's game, but is dealing with learning Java,
and we're at the point where we have to get past this issue and on to
other parts of what we're building. I'm not getting a very clear
picture of what the Java-side problem is, and I'm at the point where
I'll take on the obligation of learning what the problem is, and
doing any necessary data conversion, if that's what it takes to get
past this.

Any references to, or discussion of, this kind of IO problem would be
greatly appreciated.

Why not just read as a stream of bytes and use
InetAddress.getByAddress( byte[] )
to convert into a format which can be conveniently
used by various Java methods?
So while "there's no native 32-bit unsigned in in Java"
there IS this wonderful class called InetAddress which
encapsulates IP addresses.
Drop me a line if you need some sample code.

Actually, IP addresses were just an example of the various unsigned
ints I have to pass the server. I should have been clearer.

Thanks to the guidance I've picked up here, we're already well past
our sticking point. Thanks to you and everyone who helped.

--
For email, put NOT SPAM in Subject or I'll probably miss it.
<><

[Richard]

st************@virgin.NO_SPAM.net wrote...

On Wed, 16 Jul 2003 11:15:47 +0000, Richard wrote:

Level: Java newbie, C experienced
Platform: Linux and Win32, Intel

Another programmer and I are working on a small project together. He's
writing a server process in Java that accepts input from processes I've
written over a TCP connection. My processes are all written in C; his
are all done in Java. He's new to Java, and I've never really used it.

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed- length
char array.

We're starting to spin our wheels about how to present input to his
server, since there's no native 32-bit unsigned int in Java (so far as I
understand it). He's game, but is dealing with learning Java, and we're
at the point where we have to get past this issue and on to other parts
of what we're building. I'm not getting a very clear picture of what
the Java-side problem is, and I'm at the point where I'll take on the
obligation of learning what the problem is, and doing any necessary data
conversion, if that's what it takes to get past this.

Any references to, or discussion of, this kind of IO problem would be
greatly appreciated.

Well, you have already had some good responses, but let me add my own
opinion anyway...

Start by defining the protocol between the machines. Do this by defining
the byte stream in terms of bytes, not any higher structure, since TCP
does not inherently support any higher structure.

Actually, we already had that; it's definitely the first place to
start. Turning the protocol spec into a working protocol was what
was hanging us up.

--
For email, put NOT SPAM in Subject or I'll probably miss it.
<><

[Roedy Green]

On Wed, 16 Jul 2003 15:24:27 -0700, "Jordan Zimmerman"
<jo*****@altura.com> wrote or quoted :

Send it as a String. You have a problem sitting there in C as well.
Different machine types have different long int encodings. It's much safer
to send a String.

You have endian problems but I think everything has settled on 2-twos
complements for longs.

Strings have their own set of problems.

1. are they printable, 7bit, 8bit, 16 bit? If 16 bit, you have endian
issues again! PHHTT.

2. what encoding is being used? See
http://mindprod.com/jgloss/encoding.html

3. how are they terminated? Do they have length bytes, implied
lengths, or null terminators or some other separator/terminator?

Printable strings in a CSV file with 7-bit ascii are tractable, but
the others, you need to be just as on your toes as ever.

see http://mindprod.com/jgloss/products.html#CSV
--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.

[Thomas G. Marshall]

In news:3F**************@hiddenmind.com,
David Zimmerman <dz***@hiddenmind.com> spoke unto us:

Richard wrote:

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

Treat it as a 4 byte array. InetAddress.getByAddress() wants a byte[]
anyway.

Or put it in a signed 32 bit integer and never do arithmetic with it,
the bits are the same regardless of how they're interpretted.

Most of the time even arithmetic is fine. You just get overflow effects.

For example, in 32 bits alone, adding a FFFFFFFF to a FFFFFFFF yields
FFFFFFFE regardless of whether you see it as -1 + -1 or not.

Then again, you can always use long's (in java these are 64 bits).

[David Zimmerman]

Thomas G. Marshall wrote:

In news:3F**************@hiddenmind.com,
David Zimmerman <dz***@hiddenmind.com> spoke unto us:

Richard wrote:

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.
Or put it in a signed 32 bit integer and never do arithmetic with it,
the bits are the same regardless of how they're interpretted.

Most of the time even arithmetic is fine. You just get overflow effects.

For example, in 32 bits alone, adding a FFFFFFFF to a FFFFFFFF yields
FFFFFFFE regardless of whether you see it as -1 + -1 or not.

Everything except the residual/modulo operator ('%') should work fine.
Then again, you can always use long's (in java these are 64 bits).

For some reason this always leaves a bad taste for me.

[David Zimmerman]

Richard wrote:

dz***@hiddenmind.com wrote...

Richard wrote:

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

Treat it as a 4 byte array. InetAddress.getByAddress() wants a byte[]
anyway.

Sorry for being brain-dead, but will he be able to do arithmetic on
it if it's in a 4-byte array? As in: unsigned char fbarray[4] ; ?

No, but why do you want to do arithmetic on IP addresses?

(Actually, he can do arithmetic on byte arrays, but it's tedious as all
get out.)

[Jon A. Cruz]

Richard wrote:

jo*@joncruz.org wrote...

But an inet address is not really a 32-bit unsigned int.

??? Unless you're referring to IPV6,

in_addr_t => uint32_t (<netinet/in.h>) => "unsigned 32-bit int"

No, you're being a little confused by how your platform decides to deal
with it.

First hint is that uint32_t lives in stdint.h, which is a relatively new
creature. C99, IIRC.

However, instead of just looking to how it's implemented on one platform
you're looking at, we should look to the official standard.

RFC-791 Internet Protocol
"Addresses are fixed length of four octets..."

Four separate parts, totalling 32 bits. That seems quite clear.
However... if some might be mislead by the "(32 bits)" that follows.
However, when the RFC's say "octets", then they are clearly talking
about what we commonly know as bytes.

And as to the contention that addresses are unsigned, well, not once in
the entire RFC does the word "unsigned" appear.

Just 32 bits. However you want to treat it. Or 4 octets.

In fact, inet addresses in Java are accessed as byte arrays, not ints or
longs.

[Jon A. Cruz]

Jordan Zimmerman wrote:

Send it as a String. You have a problem sitting there in C as well.
Different machine types have different long int encodings. It's much safer
to send a String.

Actually, it's not.

Just follow the practices of the ancients and use octects in specified
order. For some reason TCP/IP hasn't caused the end of the world yet. :-)
Oh, and if you need to deal with floats, just use IEEE754 32 and 64 bit
in some explicit order.

[Richard]

jo*@joncruz.org wrote...

Richard wrote:

jo*@joncruz.org wrote...

But an inet address is not really a 32-bit unsigned int.
??? Unless you're referring to IPV6,

in_addr_t => uint32_t (<netinet/in.h>) => "unsigned 32-bit int"

No, you're being a little confused by how your platform decides to deal
with it.

Good point, except that how any given platform "decides to deal with
it" is what the problem often boils down to.
First hint is that uint32_t lives in stdint.h, which is a relatively new
creature. C99, IIRC.
You wouldn't be confusing when and where a typedef exists with the
existence and use of unsigned 32-bit integer values, would you? (You
seem to have missed the transform from 'uint32_t' to 'unsigned 32-bit
int' I think.)
[....] In fact, inet addresses in Java are accessed as byte arrays, not ints or
longs.

Hence my sending 4 bytes, in NBO of course, over to that Java
process: To make it easy for the Java programmer wrt how Java
"decides to deal with it."

Thanks for your points, though. It's very useful to be reminded of
first principles.

--
For email, put NOT SPAM in Subject or I'll probably miss it.
<><

[Thomas G. Marshall]

In news:3F**************@joncruz.org,
Jon A. Cruz <jo*@joncruz.org> spoke unto us:

Richard wrote:

My input is basically a stream of 32-bit unsigned integers (e.g., the
> low-level form of IP addresses)

Ummm.... Maybe. Or maybe not. :-D

32-bit is *one* form of an IP address. Another is 128-bit addresses
in IPv6.

It's best that you account for both.

I always wondered if they intended to have IP addresses for all machines in
the visible universe with that 128 bit thing. But I'm all for it.

What I'm not sure of is just how backward compatible it would be, and if we
are facing a y2k scenario of converting over to it.

[Richard]

dz***@hiddenmind.com wrote...

Richard wrote:

dz***@hiddenmind.com wrote...

Richard wrote:My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.Treat it as a 4 byte array. InetAddress.getByAddress() wants a byte[]
anyway.

Sorry for being brain-dead, but will he be able to do arithmetic on
it if it's in a 4-byte array? As in: unsigned char fbarray[4] ; ?

No, but why do you want to do arithmetic on IP addresses?

The ints aren't exclusively IP addresses. See above; I'm passing 32-
bit integers (the "e.g. the low level form of IP addresses" means
"for example, ... IP addresses").
(Actually, he can do arithmetic on byte arrays, but it's tedious as all
get out.)

--
For email, put NOT SPAM in Subject or I'll probably miss it.
<><

[Mark McIntyre]

On Thu, 17 Jul 2003 04:21:41 GMT, in comp.lang.c , David Zimmerman
<dz***@hiddenmind.com> wrote:

No, but why do you want to do arithmetic on IP addresses?

iteration over IP blocks for instance.

--
Mark McIntyre
CLC FAQ <http://www.eskimo.com/~scs/C-faq/top.html>
CLC readme: <http://www.angelfire.com/ms3/bchambless0/welcome_to_clc.html>
----== Posted via Newsfeed.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeed.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= 19 East/West-Coast Specialized Servers - Total Privacy via Encryption =---

[Jon A. Cruz]

Thomas G. Marshall wrote:

I always wondered if they intended to have IP addresses for all machines in
the visible universe with that 128 bit thing. But I'm all for it.

Ummm... yup. They do.

:-)
http://www.ipnsig.org/home.htm

[Nigel Wade]

>

Jon A. Cruz wrote:

Text is for people, computers can deal with more.

Thomas Gagné wrote: If the question had come from a computer I might have suggested it use
something more. But since people are involved text is still preferable.
Will
it be a machine debugging the code?
Of course it will, how would you intend to execute the code?
Will it be a machine translating the
network-ordered bytes into integers?
Of course it will, or did you think they intended to do it by hand?
Will it be a machine snooping
network packets wondering why the message aren't being
assembled/disassembled properly?
I rather think so, how do you intend to snoop the values otherwise?
Are you able to read packets without one?

You're right, machines are capable of more so humans must depend on them
to do more, like translate scalar values into text so we can read them, or
format them into XML so we can both read and understand their format by
reading them.
So use it for that. When you want to look at the values as text, use the
machine to convert them for you. But when the data is being sent
machine-to-machine text is inappropriate unless it's actually text that is
being transferred. The extra network overhead of sending text instead of
binary is totally unnecessary, as is the CPU overhead of converting every
number to and from text at either end.

I presume you don't send much data over networks.

Not using text is a good approach for job security.

Sorry, but that is rubbish.
--
Nigel Wade, System Administrator, Space Plasma Physics Group,
University of Leicester, Leicester, LE1 7RH, UK
E-mail : nm*@ion.le.ac.uk
Phone : [phone], Fax : [phone]

[Tukla Ratte]

On Thu, 17 Jul 2003 04:19:33 GMT, David Zimmerman
<dz***@hiddenmind.com> wrote:

Thomas G. Marshall wrote:

< snip >

Then again, you can always use long's (in java these are 64 bits).

For some reason this always leaves a bad taste for me.

Maybe because it doubles your bandwidth requirements?

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

--
Tukla, Squeaker of Chew Toys
Official Mascot of Alt.Atheism

[David Zimmerman]

Tukla Ratte wrote:

On Thu, 17 Jul 2003 04:19:33 GMT, David Zimmerman
<dz***@hiddenmind.com> wrote:

Thomas G. Marshall wrote:

< snip >

Then again, you can always use long's (in java these are 64 bits).

For some reason this always leaves a bad taste for me.

Maybe because it doubles your bandwidth requirements?

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

Why not just pass the bytes? Why complicate your life and increase the
demand on bandwidth?

[Jon A. Cruz]

Tukla Ratte wrote:

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

Network byte order.

explicit sizes.

stdint.h helps now.

floats in IEE754

XML is probably a poor choice for that.

(I've been using XML for quite some time. I even use it to make source
code. Abused, it's bad)

[xarax]

tu*********@tukla.net (Tukla Ratte) wrote in message news:<3f***************@news.cis.dfn.de>...

On Thu, 17 Jul 2003 04:19:33 GMT, David Zimmerman
<dz***@hiddenmind.com> wrote:

Thomas G. Marshall wrote:

< snip >

Then again, you can always use long's (in java these are 64 bits).

For some reason this always leaves a bad taste for me.

Maybe because it doubles your bandwidth requirements?

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

I use a custom program generator that reads an XML specification
file that describes the various message packet types. The program
generator then spits out Java classes and C source files for each
message packet type.

Each message packet has a fixed length header that includes:

a. A 4-byte magic number for verifying that the byte stream
consists of message packets that I expect.

b. A 4-byte transaction ID number.

c. A 4-byte version number (another constant for sanity checking).

d. A 2-byte message type number that includes a "reply flag"
that indicates whether this is an original message, or a reply
to an earlier message (with the same transaction ID).

e. A 2-byte message length that must correlate to the known
message length for the message type (another sanity check).

If the header portion doesn't match expected values (i.e.,
the sanity checks fail), then the TCP/IP socket is closed
(causing a reset condition on the other side).

If the message packet also has payload data fields, then
those fields follow the header.

The generated Java code has a class for each message type.
The class knows how to read/write a message packet using
the java.nio.ByteBuffer class, which has built-in support
for Big Endian vs. Little Endian byte ordering. (My application
uses Big Endian.) The java class has getter/setter methods
for each field in the message packet.

The generated C code has header files that define a C struct
for mapping the message packet, and source files for each
message packet. The source file for a message packet has
functions for getting/setting each field in the buffer.

Neither the Java or C application code refer directly
to the packet fields, but rather use the getter/setter
routines. This extra level of indirection makes the
application more readable and easier to maintain.

I also have GUI front-end for specifying the message
packets, which generates the XML specification file and
invokes the program generator. I then compile the sources
and rebuild the application. Very simple and reliable.

This design removes any knowledge of the "other side of
the TCP/IP pipeline". Each side doesn't know whether it's
talking to a Java or a C application on the other side,
and it doesn't need to know.

[Steve Horsley]

On Fri, 18 Jul 2003 21:50:41 +0000, Tukla Ratte wrote:

On Thu, 17 Jul 2003 04:19:33 GMT, David Zimmerman <dz***@hiddenmind.com>
wrote:

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

Have you considered TCP/IP? It provides a byte stream. Bytes in one end,
same bytes out the other end - no loss and no re-ordering. This is
probably exactly what you need.

Steve

[Jos A. Horsmeier]

"Richard" <rh***@hotmail.com> wrote in message news:MP************************@news.verizon.net.. .

Level: Java newbie, C experienced
Platform: Linux and Win32, Intel

Another programmer and I are working on a small project together.
He's writing a server process in Java that accepts input from
processes I've written over a TCP connection. My processes are all
written in C; his are all done in Java. He's new to Java, and I've
never really used it.

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

I'm hopping in a bit late, but nevertheless. I've implemented something
similar and, the lazy bones that I am, I just wrapped a DataInputStream
and DataOutputStream around the raw SocketStream (this is the Java side
of the story). Those two streams take care of about anything.

If the other side happens to the 'C side' running on some box, all the
other side has to do is use the htonl(), ntohl() macros and their compadres
to send/receive binary integer data. Binary byte arrays can be passed
around without conversion.

Strings can be passed around using a little trick -- Java characters take
up 16 bits (the Unicode stuff) while, normally the C side uses char[]s
to store their strings. (wide characters are a different cup of tea).
This is how Java Strings can be written to the socket:

void write(String s) {
byte[] buf= s.getBytes("ISO-8859-1");
write(buf);
}

where the 'write(buf)' call simply writes a bunch of bytes. The other
side (the C side) simply reads the chars and stores them somewhere.
Sending strings (char[]s) from the C side to the Java side is easy
too. Simply read a bunch of bytes in a Java byte[] buf and create
a new Java String as follows:

String s= new String(buf, "ISO-8859-1");

Double values are easy to send/receive too; simple reverse the byte
order on the C side before sending and after receiving and simply
read/write the double on the Java side.

Of course you still have to design a little protocol, say, sending
the length of the next datum first (using those htonl() thingies
on the C side and using the simple DataOutputStream object on the
Java side), but that is left as an exercide for the reader.

kind regards,

Jos

[Thomas Gagné]

Jon A. Cruz wrote:

Tukla Ratte wrote:

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

Network byte order.

explicit sizes.

And what sizes should those be? And what happens when something exceeds the
limits of an integer? Should the interface be rewritten?

stdint.h helps now.

floats in IEE754
What if more precision is required and/or demanded? Why not fractions or some
other expression that can be arbitrarily precise?

XML is probably a poor choice for that.
XML or any parsable text-based information would be more flexible than scalar
values. Admittedly they would use more bandwidth but I haven't detected any
bandwidth requirements--just bandwidth anxieties. Perhaps everyone would like
to omit the century from our dates?

Bandwidth increases every year--more programmers should take advantage of
it--unless of course you're sending spam.

(I've been using XML for quite some time. I even use it to make source
code. Abused, it's bad)

--
..tom
remove dashes in email for replies
opensource middleware at <http://isectd.sourceforge.net>

[Jon A. Cruz]

Jos A. Horsmeier wrote:

I'm hopping in a bit late, but nevertheless. I've implemented something
similar and, the lazy bones that I am, I just wrapped a DataInputStream
and DataOutputStream around the raw SocketStream (this is the Java side
of the story). Those two streams take care of about anything.
Bingo!

That's probably much of why they are as they are.

If the other side happens to the 'C side' running on some box, all the
other side has to do is use the htonl(), ntohl() macros and their compadres
to send/receive binary integer data. Binary byte arrays can be passed
around without conversion.
Generally, I dislike those macros. First, 'cause macros are bad. :-)
But more because there are some usage problems. On a little-endian
machine, they flip bytes. If one calls something twice, you undo the fix
and break it again. Also, you have to play magic anyway to get the
initial value.

I prefer clean code that just uses bit math to shift things. No macros
needed, and works on any platform regardless of primitive size. Since
you have to play a little code to get the size issue compensated for,
you can easily fix byte order at the same time.

Strings can be passed around using a little trick -- Java characters take
up 16 bits (the Unicode stuff) while, normally the C side uses char[]s
to store their strings. (wide characters are a different cup of tea).
This is how Java Strings can be written to the socket:

void write(String s) {
byte[] buf= s.getBytes("ISO-8859-1");
write(buf);
}
That's probably not good. Mainly because Latin1 (AKA ISO-8859-1) will
loose lots of characters. On a MS Windows platform you will lose 32
places in the visible character range. On a Mac, you'll use 50%.
It's better to use UTF-8 for your wire format.

String s= new String(buf, "ISO-8859-1");
Here too.

Double values are easy to send/receive too; simple reverse the byte
order on the C side before sending and after receiving and simply
read/write the double on the Java side.

Not quite.

Do that on my Mac G3 and it will be scrambled. :-)

Seriously, though, this is a little tricky. So I think it's a good idea
to add some asserts in the C code.
Assert that sizeof(float) == 32 and that sizeof(double) == 64. Then you
might want to add a simple one to be sure that your platform uses
IEE754. If any of these are not true, then you'll need some platform
specific C code in that case (of course, since IEEE754 became standard
and widespread, that's much easier)

And you want to be sure not to just "reverse" the bytes. Instead you
want to "send them in network byte order". IIRC, this is one of the few
places where I cast a pointer and use htnl.