473,895 Members | 3,071 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Binary data handling ?

Hello ppl,

I'm having difficulties to accomplish some simple chores with binary data.
I'm having a string (?) which I received via an HTTP transactions which is a
binary file.
Problem is the webserver I'm communicating with injected a \x0D before every
\x0A,
and I need to remove those 0x0D characters from my buffer before saving it
to disk.

any ideas ?

I tried the following without any success:
string.replace( "%c%c" % (13,10), "%c" % (10))
string.replace( "\x0d\x0a", "\0x0a")

thx
~B
Jul 18 '05 #1
5 7577
Bill Loren wrote:

Hello ppl,

I'm having difficulties to accomplish some simple chores with binary data.
I'm having a string (?) which I received via an HTTP transactions which is a
binary file.
Problem is the webserver I'm communicating with injected a \x0D before every
\x0A,
and I need to remove those 0x0D characters from my buffer before saving it
to disk.
This sounds wrong. I don't think a properly configured web server should
be transmitting unencoded binary files with newline conversion.
any ideas ?

I tried the following without any success:
string.replace( "%c%c" % (13,10), "%c" % (10))
string.replace( "\x0d\x0a", "\0x0a")


Strings are immutable. Are you expecting the above to change the string
(which doesn't happen) or to return a new string with the changes made?
Assign the result of the replace() call to a new variable and it should
work. (Except in the latter example you should have \x0a, not \0x0a.)

-Peter
Jul 18 '05 #2
"Bill Loren" <lo*****@bezeqi nt.net> wrote in message news:<ma******* *************** ************@py thon.org>...
Hello ppl,

I'm having difficulties to accomplish some simple chores with binary data.
I'm having a string (?) which I received via an HTTP transactions which is a
binary file.
Problem is the webserver I'm communicating with injected a \x0D before every
\x0A,
and I need to remove those 0x0D characters from my buffer before saving it
to disk.

any ideas ?

I tried the following without any success:
string.replace( "%c%c" % (13,10), "%c" % (10))
string.replace( "\x0d\x0a", "\0x0a")

thx
~B

# A short function to generate a string with 0x0D's
def make_bin(): .... bytes=[]
.... for i in range(20):
.... bytes.append(ch r(i))
.... # every 5th char is a 0x0D
.... if not i%5:
.... bytes.append(ch r(0x0D))
.... return ''.join(bytes)
.... binary=make_bin ()
binary '\x00\r\x01\x02 \x03\x04\x05\r\ x06\x07\x08\t\n \r\x0b\x0c\r\x0 e\x0f\r\x10\x11 \x12\x13' # split the string at the 0x0D's and join it again
binary_without_ x0D=''.join(bin ary.split(chr(0 x0D)))
#et voila
binary_without_ x0D '\x00\x01\x02\x 03\x04\x05\x06\ x07\x08\t\n\x0b \x0c\x0e\x0f\x1 0\x11\x12\x13'


Regards
Peter
Jul 18 '05 #3
Bill Loren wrote:

about the code problem, I did fix that bug you mentioned, but it still
doesn't work.
the code is:
data = data.replace(.. .the two options I mentioned before...)
but alas... no replacement...


Since the following clearly works, you must be confused about what
is in the string called "data" prior to the replacement:

C:\>python22
Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
Type "help", "copyright" , "credits" or "license" for more information.
s = 'test 1\r\ntest 2\rtest 3\ntest4\r\n\r\ ntest5'
s 'test 1\r\ntest 2\rtest 3\ntest4\r\n\r\ ntest5' t = s.replace('\r\n ', '\n')
u = s.replace('\x0d \x0a', '\x0a')
v = s.replace('%c%c ' % (13, 10), '%c' % (10))
t 'test 1\ntest 2\rtest 3\ntest4\n\ntes t5' u 'test 1\ntest 2\rtest 3\ntest4\n\ntes t5' v

'test 1\ntest 2\rtest 3\ntest4\n\ntes t5'

(Please post to the newsgroup/mailing list instead of mailing directly,
so that others can benefit from or participate in the discussion.)

-Peter

Jul 18 '05 #4
"Bill Loren" <lo*****@bezeqi nt.net> wrote:

I'm having difficulties to accomplish some simple chores with binary data.
I'm having a string (?) which I received via an HTTP transactions which is a
binary file.
Problem is the webserver I'm communicating with injected a \x0D before every
\x0A,
and I need to remove those 0x0D characters from my buffer before saving it
to disk.

any ideas ?


I'll bet you real money that the problem is not in the web server. I'd
wager that the string is correct when you receive it, but that you are
writing it to file like this:
file('out.txt', 'w').write(down load)

On a Windows system, that'll turn all the LFs into CR-LFs. Use this
instead:
file('out.txt', 'wb').write(dow nload)
--
- Tim Roberts, ti**@probo.com
Providenza & Boekelheide, Inc.
Jul 18 '05 #5
Indeed !

thanks !!!

if I use the 'wb' will it work on a unix system, too ?

~B
----- Original Message -----
From: "Tim Roberts" <ti**@probo.com >
Newsgroups: comp.lang.pytho n
To: <py*********@py thon.org>
Sent: Saturday, August 30, 2003 7:11 AM
Subject: Re: Binary data handling ?

"Bill Loren" <lo*****@bezeqi nt.net> wrote:

I'm having difficulties to accomplish some simple chores with binary data.I'm having a string (?) which I received via an HTTP transactions which is abinary file.
Problem is the webserver I'm communicating with injected a \x0D before every\x0A,
and I need to remove those 0x0D characters from my buffer before saving itto disk.

any ideas ?


I'll bet you real money that the problem is not in the web server. I'd
wager that the string is correct when you receive it, but that you are
writing it to file like this:
file('out.txt', 'w').write(down load)

On a Windows system, that'll turn all the LFs into CR-LFs. Use this
instead:
file('out.txt', 'wb').write(dow nload)
--
- Tim Roberts, ti**@probo.com
Providenza & Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list

Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2093
by: David List | last post by:
I am wondering what I miss to be able to handle binary data from the mysql client. I have ensured that the user has file_priv set to 'Y' and that max_allowed_packet is larger that the binary lumps handled. Here is my version: Ver 4.1.5-gamma-log for pc-linux on i686 (Source distribution) Right now I'm trying this to insert binary data: mysql> update table set col=load_file("/path/file") where id=1;
8
2123
by: Bruce Lee | last post by:
Hi I have the following 2 questions: 1.How to write binary data to a text file in C? I have a number like 10001010110001 say, I need to put it in the text file so that I can see the corresponding characters when I open the file. 2. How to extract bits from a byte. I think it is using bit operations, but for example, how would I extract the 2nd most significant bit in a byte?
2
1413
by: Feng | last post by:
Hi, I need a VB.Net function that reads in a stream of binary data coming in from a legacy data source. The data are actually hex numbers in binary format but the problem is that I don't know how handle it in my VB program. All I need is to be able to transform those bytes into hex, such as 'oxF9', but right now all I can see are un-readable binary data. What is the best way to do this kind of work? Thanks
4
4047
by: Holger Marzen | last post by:
Hi all, AFAIK it is possible for columns to be very large, up to about 2 GB. Are there any hints or experiences about storing binary data (jpg-images, pdf-documents) in PostgrreSQL with or without the complicated lo-stuff? Of course it's in many cases a good approach to store those files simply in the file system but there's always a risk of running out of sync (filesystem and tables), e.g. by deleting files and not deleting the table...
6
2741
by: | last post by:
Hi all, is there a better way to stream binary data stored in a table in sql 2005 to a browser in .net 2.0? Or is the code same as in .net 1.1? We noticed that in certain heavy load scenarios, every now and then the client would timeout and have to re-initiate the request... TIA!
0
2367
by: Wescotte | last post by:
I'm abit confused on how to work with binary data with an ODBC connection (My database is DB2 btw) Say I have a table like CREATE TABLE EJWLIB.BLOBTEST ( ID NUMERIC(5) NOT NULL, FILENAME VARCHAR(128) NOT NULL, BINARY BLOB(2M) ) Now I (I assume this is the correct method) insert data in the
3
7006
by: stockblaster | last post by:
Hello all.. Is it possible to convert a DataTable (i create the DataTable from a CSV file) into binary data and save it into an sql 2005 table (into binary field). After that I want to have the ability to add a row to the beginning of the to the binary data..
0
1251
by: perlprod | last post by:
Hi, I want to generate a data file that is to be read by a 'C' program. But I got in dilemma after checking the file generated by the following program. $buffer = ""; $path = "C:/Documents and Settings"; $bfile = "$path/binary.dat";
9
3249
by: lokeshrajoria | last post by:
hello everybody, i need some help in binary file handling in perl. can anybody give me some information about binary file. actully i am reading data from some text file and extracting some usefull information from there and want store in my own binary file with .vbf extension ( not like .dat file.) i am putting code here for reading text file and extract some information from there but how can i store that information in my own binary file. ...
0
9833
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10841
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10931
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10469
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
8025
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7175
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5860
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4684
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3293
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.