Hello,
I am using HadoopStreaming using a BinaryInputStre am. What this
basically does is send a stream of bytes ( the java type is : private
byte[] bytes) to my python program.
I have done a test like this,
while 1:
x=sys.stdin.rea d(100)
if x:
print x
else:
break
Now, the incoming data is binary(though mine is actually merely ascii
text) but the output is not what is expected. I expect for e.g
all/86000/114.310.151.209 .60370-121.110.5.176.1 13\n62485.9718
118.010.241.12 60370 128.210.5.176
However i get a 1 before all and a 4 just after \n and before the 6.
My question is : how do i read binary data(Java's byte stream) from
stdin?
Or is this actually what i'm getting?
Thanks
Sapsi 6 3162
I should also mention that for some reason there are several binay
values popping in between for some reason. This behavior (for the
inputr stream) is not expected
Now, the incoming data is binary(though mine is actually merely ascii
text) but the output is not what is expected. I expect for e.g
all/86000/114.310.151.209 .60370-121.110.5.176.1 13\n62485.9718
118.010.241.12 60370 128.210.5.176
However i get a 1 before all and a 4 just after \n and before the 6.
My question is : how do i read binary data(Java's byte stream) from
stdin?
Or is this actually what i'm getting?
Thanks
Sapsi
On Sun, 18 May 2008 22:11:33 -0700, sapsi wrote:
I am using HadoopStreaming using a BinaryInputStre am. What this
basically does is send a stream of bytes ( the java type is : private
byte[] bytes) to my python program.
I have done a test like this,
while 1:
x=sys.stdin.rea d(100)
if x:
print x
else:
break
Now, the incoming data is binary(though mine is actually merely ascii
text) but the output is not what is expected. I expect for e.g
all/86000/114.310.151.209 .60370-121.110.5.176.1 13\n62485.9718
118.010.241.12 60370 128.210.5.176
However i get a 1 before all and a 4 just after \n and before the 6.
My question is : how do i read binary data(Java's byte stream) from
stdin?
Or is this actually what i'm getting?
If there's extra data in `x` then it was sent to stdin. Maybe there's
some extra information like string length, Java type information, or
checksums encoded in that data!?
Ciao,
Marc 'BlackJack' Rintsch
Yes, that could be the case. Browsing through hadoop's source, i see
stdin in the above code is reading from piped Java DataOutputStrea m.
I read of a libray on the net Javadata.py that reads this but it has
disappeared.
What is involved in reading from a Dataoutputstrea m?
Thank you
Sapsi
On Mon, 19 May 2008 00:14:25 -0700, sapsi wrote:
Yes, that could be the case. Browsing through hadoop's source, i see
stdin in the above code is reading from piped Java DataOutputStrea m.
I read of a libray on the net Javadata.py that reads this but it has
disappeared.
What is involved in reading from a Dataoutputstrea m?
According to the Java docs of `DataInput` and `DataOutput` it is quite
simple. Most methods just seem to write the necessary bytes for the
primitive types except `writeUTF()` which prefixes the string data with
length information.
So if it is not Strings you are writing then "hadoop" seems to throw in
some information into the stream.
Ciao,
Marc 'BlackJack' Rintsch
On 19 May, 06:11, sapsi <saptarshi.g... @gmail.comwrote :
Hello,
I am using HadoopStreaming using a BinaryInputStre am. What this
basically does is send a stream of bytes ( the java type is : private
byte[] bytes) to my python program.
I have done a test like this,
while 1:
x=sys.stdin.rea d(100)
if x:
print x
else:
break
Now, the incoming data is binary(though mine is actually merely ascii
text) but the output is not what is expected. I expect for e.g
all/86000/114.310.151.209 .60370-121.110.5.176.1 13\n62485.9718
118.010.241.12 60370 128.210.5.176
However i get a 1 before all and a 4 just after \n and before the 6.
My question is : how do i read binary data(Java's byte stream) from
stdin?
Or is this actually what i'm getting?
Thanks
Sapsi
In the past I've sent binary data to a java applet reading
DataInputStream using xdrlib from the standard library. I'd expect
that it would work in the reverse direction so I suggest you have a
look at that.
Giles
sapsi wrote:
I should also mention that for some reason there are several binay
values popping in between for some reason. This behavior (for the
inputr stream) is not expected
>Now, the incoming data is binary(though mine is actually merely ascii text) but the output is not what is expected. I expect for e.g
all/86000/114.310.151.209 .60370-121.110.5.176.1 13\n62485.9718 118.010.241. 12 60370 128.210.5.176
However i get a 1 before all and a 4 just after \n and before the 6.
My question is : how do i read binary data(Java's byte stream) from stdin? Or is this actually what i'm getting?
Consider changing "print x" to "print repr(x)" ... this would mean that
you have a better chance of understanding what the extra or unexpected
popping-in bytes are. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: RobertMaas |
last post by:
After many years of using LISP, I'm taking a class in Java and finding
the two roughly comparable in some ways and very different in other
ways. Each has a decent size library of useful utilities as a standard
portable part of the core language, the LISP package, and the java.lang
package, respectively. Both have big integers, although only LISP has
rationals as far as I can tell. Because CL supports keyword arguments,
it has a wider range...
|
by: Richard |
last post by:
Level: Java newbie, C experienced
Platform: Linux and Win32, Intel
Another programmer and I are working on a small project together.
He's writing a server process in Java that accepts input from
processes I've written over a TCP connection. My processes are all
written in C; his are all done in Java. He's new to Java, and I've
never really used it.
My input is basically a stream of 32-bit unsigned integers (e.g., the
|
by: Fernando Arbeiza |
last post by:
Hi:
I need some clarification about a code like this:
printf("%s", "a string with NO trailing newline");
scanf("%d", &i);
Regarding if a fflush() of the standard output is needed or not. I think
the relevant portions of the standard are:
|
by: Nick |
last post by:
I have found a class that compresses and uncompresses data but need
some help with how to use part of it below is the deflate method which
compresses the string that I pass in, this works OK. At the end of
this message is the inflate method this is where I get stuck I know
that I need a byte array but because I am decompressing a string I
have no idea of how big the byte array will need to be in the end (the
inflate and deflate methods...
|
by: Vitaly |
last post by:
// Open input file and create the BinaryReader.
br = new BinaryReader(new FileStream("Test.dat",
FileMode.Open,
FileAccess.Read));
// Read binary data.
d = br.ReadDouble();
A question is how to do the same but read from standard input stream … like
Console.In.
| |
by: junk5 |
last post by:
Hi
I need to read raw 16 bit data from a file, where the first byte is the
most significant byte of the first data value and the second byte is
the least significant byte of the first data value (the next pair or
bytes and the next etc. specify the second and third values).
For example, if I had a file containing
0xff22
|
by: Marc Gravell |
last post by:
It might just be my tired eyes, but I can't see what is wrong in the
following:
I have a byte array filled with random data (with a fixed seed to make
reproducable); I then compress and decompress this using the 2.0 GZip
compression classes - however, as you see, the last byte is getting dropped.
Any ideas? Almost certainly me having a blond moment ;-(
Marc
|
by: Sir Psycho |
last post by:
Hi,
For some reason, when i step over this code, it returns the full byte
stream im expecting from the server, however when I let it run with no
intervention, it only seems to grab a small chunk on the stream.
What am I doing wrong? There is more code than this, but this is the
problem code.
|
by: radhikams |
last post by:
hi
I have written a code for writing the data of a column from database into a file
and again reading that file and displaying.
Im writing into an .html file....Now the problem is im getting the output but that output is followed by an error.....
This is the output
-----------------------------
A large number of processes are available, and can be applied to an input or a group of inputs. These processes can be chained...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |