473,593 Members | 2,907 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Sending both binary data and strings over the same stream

21 New Member

I've bumbed into a slight problem now, and I just don't seem to know how to fix it. What I want to do is the following:
Send over a socket:
1. Number of files to be send (not as an integer, just as a string)
then for each file to be send:
2. Length of Filename (again as a string)
3. Filename
4. File as binary data.

I grabbed my Core Java book and figured it'd be easily doable using a bufferd data input/output stream, As i can cast it to a buffered data stream for the binary data in the file, and to the data stream for sending/receiving strings (is that even a valid way of thinking ?)

The whole 'plan' just started crumbling down when I tried to implement it and it said that the DataInputStream method is deprecated. And I can't use a Reader as I must be able to receive binary data...

Now, I think it'd be a grave mistake (please do correct me if I'm wrong) to hand the same input/output stream over to a reader, and hand construct a bufferedInput/OutputStream with the same input/output stream as is being used by the reader. And on second thought, the same problem probably occurs with Wrapping the stream in as a DataInputStream (new BufferedInputSt ream ( socket.getInput stream())) so i'll just scratch that plan too...

So... I'm kinda stuck here... How can I send both binary data and strings over the same socket ? While not making it that complicated that it's really difficult to construct a correctly working C++ counterpart using the same protocol that can communicate with the java implementation.

Thanks in advance.

Feb 8 '09 #1
20 9396
3,112 Recognized Expert Specialist
Hi Tom!

I checked the DataInputStream API and it seems that the readLine() method is depreciated, but not the whole class.

Also I think, it shouldn't be a problem to reuse a stream in the way you suggested.

Oh, and for sending Strings, you could also use an ObjectOutputStr eam / ObjectInputStre am (although that might be problematic when trying to create a C++ counterpart, I really don't know that).

Feb 8 '09 #2
21 New Member

Thanks for the reply :).

What I'm not sure about with the reusing of a stream is both the closing of the stream, and the buffered data.
If data enters on the original stream, is it copied to, for example both the buffer for the BufferedInputSt ream and to the buffer of a BufferedReader, or is it only present in one of the buffers ?
The latter case would be quite problematic as I would never know where my data is at. If it's the former then I should know exactly how many bytes to skip in each stream as I read from the other Stream, but that shouldn't be to hard.

Another thing I was thinking about is, how about if I take a pure bytestream and just cast it char's for example for all the things I know are chars (for example until the result of a cast of 2 bytes results in a newline) and then interpret the next part as it is supposed to be.
But tbh, I could do that in c++ But i have no idea whatsoever on doing this in java.

Do you have any idea's about that ?

Thanks in advance,
Feb 9 '09 #3
13,262 MVP
If you just send the binary file you should be able to get both its name and length after sending the file, right?
Feb 9 '09 #4
21 New Member
But, I can't send the java file object since we have to 'interface' with c++. Or isn't that what you meant ?
The thing is also, we want it to be possible to for example 10 files after one another without having to openup new connections.

So far what I'm thinking that might work is just sending bytes over and casting everything to chars, but I think it'll be a bit inefficient. But I think it might work...

So, just using a raw BufferedOutput/InputStream and getting the bytes from the strings, and then on the receiving end just cast every 1 or 2 bytes (according to java standard) as a character (if that's possible, sigh, c++ is so much easier :P) and seperate based on new lines that way.

When the protocol then says that the next part should be interpreted as a binary file, e.g. after reading file lenght using the above described method, I don't interpret any of the bytes and just write them to a file, until the whole file has been received. Then I start casting again for the next filename length.

Would that be a doable approach ?

Thanks in advance
Feb 9 '09 #5
11,448 Recognized Expert MVP
No need to over-complicate things: when Strings are written/read by a stream those Strings are encoded/decoded. ASCII Strings (each char <= 0x7f) encode to a single byte in UTF-8. UTF-8 decoding (on the C++ side) isn't much trouble either; I suggest a simple protocol:

0,1: file name length in high endian format
2 ... n: file name UTF-8 encoded
n+1 ... n+4: length of the file in high endian format
n+5 ...: binary content of the file

The C++ end shouldn't have any trouble with this data format. All you need is a simple OutputStream on the Java sending side. The String class itself can take care of the encoding (UTF-8)

kind regards,

Feb 9 '09 #6
21 New Member
Oh, the standard encoded size of a char is 1 byte ? I thought it was 2 bytes... If it's 1 byte that indeed simplifies the matter a bit. And I didn't know the String class itself took care of the encoding.
Thanks a lot Jos, I'll try and get it fixed that way. I'll post here if another corpse jumps out of the closet on my line of thought.

Thanks a lot already !

Feb 10 '09 #7
11,448 Recognized Expert MVP
The UTF-8 encoding scheme encodes the bytes 0x00 - 0x7f to the same range: 0x00 - 0x7f. All the ASCII characters happen to be in that range, so they get encoded to themselves.

Internally a char takes up two bytes in Java; all chars do. When you write them to an OutputStream they are encoded because streams write bytes, not chars.

kind regards,

Feb 10 '09 #8
21 New Member

As it seems the people working on the C++ counterpart are not sending the lengths of the filenames and the lengths of the files are being send as integers (4 bytes long).
Now, keeping in mind that characters are encoded in UTF-8 by default i've thought up of the following 'draft' implementation:

Both the fReader and fWriter are actually DataInput/OutputStreams.

Expand|Select|Wrap|Line Numbers
  1. /*
  2.      * (non-Javadoc)
  3.      * 
  4.      * @see firefile.shared.net.ISocket#readString()
  5.      */
  6.     public String readString() throws IOException {
  7.         int length = this.fReader.readInt();
  9.         System.out.print("Lenght received:");
  10.         System.out.println(length);
  12.         final char[] ch = new char[length];
  13.         for (int i = 0; i < length; i++) {
  14.             final int tmp = this.fReader.read();
  15.             if (tmp == -1) {
  16.                 throw new IOException("End of stream prematurely ended.");
  17.             } else {
  18.                 ch[i] = (char) tmp;
  19.             }
  20.         }
  21.         final String in = new String(ch);
  23.         System.out.print("Readstring returned: ");
  24.         System.out.println(in);
  26.         return in;
  27.     }
  29.     /*
  30.      * (non-Javadoc)
  31.      * 
  32.      * @see firefile.shared.net.ISocket#sendString(java.lang.String)
  33.      */
  34.     public void sendString(final String msg) throws IOException {
  35.         this.fWriter.writeInt(msg.length());
  36.         this.fWriter.flush();        
  37.         this.fWriter.writeBytes(msg);
  38.         this.flush();
  39.     }
So, I'm thinking this will do what I want it to do.
Feb 10 '09 #9
21 New Member

I just wanted to let you know that I've found it :). The 'final' version (except for the debug output) is much like the above. I've used DataInputStream s and DataOutputStrea ms to be able to easily send and receive integers.
To receive characters, I just read the bytes one by one until all have been read ( as advertised by the length ) and cast them to characters which works nicely.
For the binary data I also read bytes one at a time but I just immediately write them using a BufferedOutputS tream to a file. So all problems have been solved, and it's working perfectly with the c++ counterparts.

Following code is the implementation for receiving a single file. (with debug output though)

Expand|Select|Wrap|Line Numbers
  1. public void receiveFile(String uri) throws NetworkException, FileException {
  2.         System.out.println("receivefile() start");
  3.         try {            
  4.             //    length of filename + filename
  5.             final int fileNameLength = in.readInt();
  6.             System.out.print("Filename length: ");
  7.             System.out.println(fileNameLength);
  8.             final char[] ch = new char[fileNameLength];
  9.             for (int i = 0; i < fileNameLength; i++) {
  10.                 final int tmp = in.read();
  11.                 if (tmp == -1) {
  12.                     throw new IOException("End of stream prematurely ended.");
  13.                 } else {
  14.                     ch[i] = (char) tmp;
  15.                 }
  16.             }
  17.             String fileName = new String(ch);
  18.             fileName = resolveNameCollisions(uri, fileName);
  19.             System.out.print("Filename - after collision resolving: ");
  20.             System.out.println(fileName);
  21.             BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(uri+pathSeparator+fileName)); 
  23.             // length of file + file
  24.             final int fileSize = in.readInt();
  25.             for(int i = 0; i < fileSize; ++i){
  26.                 final byte tmp = in.readByte();
  27.                 bos.write(tmp);
  28.             }
  29.             bos.flush();
  30.             bos.close();
  31.             System.out.println("recieveFile() - end");
  32.         } catch (IOException e) {
  33.             throw new NetworkException(e);
  34.         }        
  35.     }
I'd like to thank you a lot for all the help ! You've really enlighted me on the whole character encoding problem which was somewhat the largest 'black hole' for me, so thanks for shedding some light on that.
And thanks to r035198x and Nepomuk too !

greets !
Feb 10 '09 #10

Sign in to post your reply or Sign up for a free account.

Similar topics

by: coder_1024 | last post by:
I'm trying to send a packet of binary data to a UDP server. If I send a text string, it works fine. If I attempt to send binary data, it sends a UDP packet with 0 bytes of data (just the headers). I can see this because I'm running Ethereal and watching the packets. I'm defining the packets as shown below: $text_msg = "Hello, world\r\n"; $binary_msg = chr(0x01).chr(0x02).chr(0x03).chr(0x00).chr(0xA0); $binary_msg_size = 5;
by: Patient Guy | last post by:
Has anyone written code that successfully manipulates binary file data using Javascript? It might---and in the case of doing I/O, will---make use of browser- specific functions (ActiveX/COM with Internet Explorer, XPCOM/XPConnect with Mozilla/Firefox). I am writing client-side code that will generate binary data for producing a GIF file through an OBJECT element. (The GIF is an image of a line and points on a two-axis plot.)
by: Robert McNally | last post by:
Hello, I'm currently learning c# and have been trying to write a simple program with sockets. The problem is i'm trying to send an email with an attachment, (which is the program itself) by using base64. When i run it it sends the email ok, but and attchment doesn't work i get an error when i run it. The attchment seems to be bigger (20k) then the orginal (16k) i've included the source which i have been using. Can someone tell me what...
by: thorley | last post by:
Greetings, since there was no reponse to my previous post about an existing FastCGI server in python, I've taken to writing my own. (which of course I'll share--*if* there's something to share ;) My problem now, is that I need to send certain binary data over a socket. That is, I want to make some bytes, and stuff them in a TCP packet, send them down the pipe, and then listen for a response. socket.send, as best I can tell, will only...
by: ... | last post by:
Hi I need to send a chr(255) to a serial port. When I send it, through comm.write (chr(255)) it sends a chr(63) ... in Hex, I write chr(&FF) and it actually sends chr(&3F) ... why does this happen, and how can I send it right ? I'm using vb.net 2005 express with framework 2.0 Thanks for an answear ...
by: Miro | last post by:
VB 2003 at the end of the code, this works great. bytCommand = Encoding.ASCII.GetBytes("testing hello send text") udpClient.Send(bytCommand, bytCommand.Length) and this recieves it Dim strReturnData As String = _ System.Text.Encoding.ASCII.GetString(receiveBytes)
by: BuddyWork | last post by:
Hello, Could someone please explain why the Socket.Send is slow to send to the same process it sending from. Eg. Process1 calls Socket.Send which sends to the same IP address and port, the receiver is running within Process1. If I move the receiver into Process2 then its fast. Please can someone explain.
by: David Hirschfield | last post by:
I have a pair of programs which trade python data back and forth by pickling up lists of objects on one side (using pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket connection to the receiver, who unpickles the data and uses it. So far this has been working fine, but I now need a way of separating multiple chunks of pickled binary data in the stream being sent back and forth. Questions:
by: =?Utf-8?B?enRSb24=?= | last post by:
Hi all, I recently came across something really strange and after a couple of days of debugging, I finally nailed the cause of it. However, I have absolutely no idea what I am doing wrong or is it just a bug in binary serialization. The following is a simple example of the code:
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.