473,320 Members | 2,006 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Dynamic Parsing of a stream of binary data

Hello everyone and happy holidays!

I have a situation where I want to parse on the fly a stream of binary data.
I am developing an httpmodule for asp.net for file uploading.

I do not whant to use the built-in control because it's inadequate.

In theory I would use a que and feed data as it comes in and then have
a parser that extracts the data from the que, parse them and do whatever.

The problem here is that I am dealing with raw binary data that sometimes
could
be hunderds of megabytes.

Should I use the built-in que class and store the data byte by byte?
How would someone go on creating a first-in first-out buffer of bytes with
managed code?
Michael.
Dec 29 '05 #1
2 1706
One idea might be to wrap the stream in a BufferedStream. That way you
would not need to cache the data in a queue as you could read it directly
from the BS. That way you are not bound by just bytes or fixed length
strings. You could provide the BS to the consuming code and read as many
bytes as necessary for that particular operation.
"Michael Kolias" <mi***@alspaconsulting.com> wrote in message
news:aK******************************@speakeasy.ne t...
Hello everyone and happy holidays!

I have a situation where I want to parse on the fly a stream of binary
data.
I am developing an httpmodule for asp.net for file uploading.

I do not whant to use the built-in control because it's inadequate.

In theory I would use a que and feed data as it comes in and then have
a parser that extracts the data from the que, parse them and do whatever.

The problem here is that I am dealing with raw binary data that sometimes
could
be hunderds of megabytes.

Should I use the built-in que class and store the data byte by byte?
How would someone go on creating a first-in first-out buffer of bytes with
managed code?
Michael.

Dec 29 '05 #2
Your problem is a lot deeper than you think, as the binary stream contains a
whole bunch of data that forms part of RFC 1867 for uploading multipart form
data from a web browser. So you have to parse the stream before you can
even begin to pick out the part you need containing the file for any other
submitted form data, and if your boundaries are wrong your file will be
corrupted along with ther rest of your http submission.

Take a look at what others have done as a starting point to get an idea of
what it involves.
http://www.asp.net/ControlGallery/Co...791&tabindex=0

Me- I would buy one of the existing ones, for what they cost its not worth
the work these days.

--
Regards

John Timney
Microsoft MVP

"Michael Kolias" <mi***@alspaconsulting.com> wrote in message
news:aK******************************@speakeasy.ne t...
Hello everyone and happy holidays!

I have a situation where I want to parse on the fly a stream of binary
data.
I am developing an httpmodule for asp.net for file uploading.

I do not whant to use the built-in control because it's inadequate.

In theory I would use a que and feed data as it comes in and then have
a parser that extracts the data from the que, parse them and do whatever.

The problem here is that I am dealing with raw binary data that sometimes
could
be hunderds of megabytes.

Should I use the built-in que class and store the data byte by byte?
How would someone go on creating a first-in first-out buffer of bytes with
managed code?
Michael.

Dec 29 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Hans Nieser | last post by:
Hi, I'm pretty new to C#, and recently I have been experimenting with sockets. However, I can't find a nice way (I've been fiddling with for-loops to no avail) to parse the data that comes in...
9
by: Hemang Shah | last post by:
Hello fellow Coders! ok, I"m trying to write a very simple application in C#. (Yes its my first program) What I want to do is : 1) Open a binary file 2) Search this file for a particular...
4
by: Pedro Leite | last post by:
Good Afternoon. the code below is properly retreiving binary data from a database and saving it. but instead of saving at client machine is saving at the server machine. what is wrong with my...
2
by: Cuong.Tong | last post by:
Greeting, Can anyone give me some hints about parsing the mulitpart/form-data stream? I have a form looks something like this <form action="process.dll> <input type=file...
3
by: Cuong.Tong | last post by:
Greeting, I am writing my own web server and having some problme parsing the the mulitpart/form-data stream that is sent from the browsers. I have a form looks something like this <form...
3
by: toton | last post by:
Hi, I have some ascii files, which are having some formatted text. I want to read some section only from the total file. For that what I am doing is indexing the sections (denoted by .START in...
13
by: Chris Carlen | last post by:
Hi: Having completed enough serial driver code for a TMS320F2812 microcontroller to talk to a terminal, I am now trying different approaches to command interpretation. I have a very simple...
4
by: Tom | last post by:
I am trying to update another developers code, and am stuck in a cludge. It works like this: A user uploads a file from the web page. Our code reads from a NetworkStream to a Filestream, and...
20
by: tomPee | last post by:
Hi, I've bumbed into a slight problem now, and I just don't seem to know how to fix it. What I want to do is the following: Send over a socket: 1. Number of files to be send (not as an integer,...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.