473,230 Members | 4,199 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,230 software developers and data experts.

stripping the first byte from a binary file

rvr
Would someone mind showing me how to strip the first byte from a
binary file? For some reason I can't figure this out from the binary
file editing examples I've read. Thanks.

~rvr

Jul 10 '07 #1
11 10473
rvr wrote:
Would someone mind showing me how to strip the first byte from a
binary file? For some reason I can't figure this out from the binary
file editing examples I've read. Thanks.
Do you mean something like this?

f = open('test.dat', 'rb')
f.read(1) # read 1st byte and ignore it
rest = f.read() # read rest

or

data = f.read()
data = data[1:] # skip 1st byte

?

--
Jeremy Sanders
http://www.jeremysanders.net/
Jul 10 '07 #2
rvr
On Jul 10, 6:37 pm, Jeremy Sanders <jeremy
+complangpyt...@jeremysanders.netwrote:
rvr wrote:
Would someone mind showing me how to strip the first byte from a
binary file? For some reason I can't figure this out from the binary
file editing examples I've read. Thanks.

Do you mean something like this?

f = open('test.dat', 'rb')
f.read(1) # read 1st byte and ignore it
rest = f.read() # read rest

or

data = f.read()
data = data[1:] # skip 1st byte
Is there a way to edit the file in place? The best I seem to be able
to do is to use your second solution to read the file into the string,
then re-open the file for writing and put the whole thing back (minus
the first byte). Thanks.

~rvr

Jul 11 '07 #3
On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
Is there a way to edit the file in place? The best I seem to be able to
do is to use your second solution to read the file into the string, then
re-open the file for writing and put the whole thing back (minus the
first byte). Thanks.
I don't believe that any of the popular operating systems in common use
(Windows, Linux, Mac, *BSD) have any such functionality.

For safety, you are best off copying the file (minus the first byte) to a
temporary file, then renaming the copy over the original. That way if
your process dies midway through copying the file, you don't lose data.

Renaming the file is atomic under Linux and (probably) Mac, so it is as
safe as possible. Even under Windows, which isn't atomic, it has a
smaller margin for disaster than over-writing the file in place.
--
Steven.
Jul 11 '07 #4
rvr
On Jul 11, 1:28 pm, Steven D'Aprano
<ste...@REMOVE.THIS.cybersource.com.auwrote:
On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
Is there a way to edit the file in place? The best I seem to be able to
do is to use your second solution to read the file into the string, then
re-open the file for writing and put the whole thing back (minus the
first byte). Thanks.

I don't believe that any of the popular operating systems in common use
(Windows, Linux, Mac, *BSD) have any such functionality.

For safety, you are best off copying the file (minus the first byte) to a
temporary file, then renaming the copy over the original. That way if
your process dies midway through copying the file, you don't lose data.

Renaming the file is atomic under Linux and (probably) Mac, so it is as
safe as possible. Even under Windows, which isn't atomic, it has a
smaller margin for disaster than over-writing the file in place.
Thanks for your response. While searching for solution, I found this:

http://mail.python.org/pipermail/pyt...er/116519.html

Quoting from it:

"""
Replace 2 bytes in place beginning at offset 100 (101st byte):

f = open('text_input', 'r+b')
f.seek(100)
f.write(chr(123) + chr(0x80))
f.seek(0,2)
f.close()
"""

Can I use the seek() and write() methods in a similar way to remove
the first byte? For whatever reason I can't seem to make it work
myself. Thanks again.

~rvr

Jul 11 '07 #5
rvr wrote:
On Jul 11, 1:28 pm, Steven D'Aprano
<ste...@REMOVE.THIS.cybersource.com.auwrote:
>On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
>>Is there a way to edit the file in place? The best I seem to be able to
do is to use your second solution to read the file into the string, then
re-open the file for writing and put the whole thing back (minus the
first byte). Thanks.
I don't believe that any of the popular operating systems in common use
(Windows, Linux, Mac, *BSD) have any such functionality.

For safety, you are best off copying the file (minus the first byte) to a
temporary file, then renaming the copy over the original. That way if
your process dies midway through copying the file, you don't lose data.

Renaming the file is atomic under Linux and (probably) Mac, so it is as
safe as possible. Even under Windows, which isn't atomic, it has a
smaller margin for disaster than over-writing the file in place.

Thanks for your response. While searching for solution, I found this:

http://mail.python.org/pipermail/pyt...er/116519.html

Quoting from it:

"""
Replace 2 bytes in place beginning at offset 100 (101st byte):

f = open('text_input', 'r+b')
f.seek(100)
f.write(chr(123) + chr(0x80))
f.seek(0,2)
f.close()
"""

Can I use the seek() and write() methods in a similar way to remove
the first byte? For whatever reason I can't seem to make it work
myself. Thanks again.
Funny. I just happened to read ESR's "how to ask questions the smart way" and
your posts match quite a few of the examples. :)

No, you can't. Steven's solution is what I'd go for.

Stefan
Jul 11 '07 #6
On Jul 11, 1:25 pm, Stefan Behnel <stefan.behnel-n05...@web.dewrote:
rvr wrote:
On Jul 11, 1:28 pm, Steven D'Aprano
<ste...@REMOVE.THIS.cybersource.com.auwrote:
On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
Is there a way to edit the file in place? The best I seem to be able to
do is to use your second solution to read the file into the string, then
re-open the file for writing and put the whole thing back (minus the
first byte). Thanks.
I don't believe that any of the popular operating systems in common use
(Windows, Linux, Mac, *BSD) have any such functionality.
For safety, you are best off copying the file (minus the first byte) to a
temporary file, then renaming the copy over the original. That way if
your process dies midway through copying the file, you don't lose data.
Renaming the file is atomic under Linux and (probably) Mac, so it is as
safe as possible. Even under Windows, which isn't atomic, it has a
smaller margin for disaster than over-writing the file in place.
Thanks for your response. While searching for solution, I found this:
http://mail.python.org/pipermail/pyt...er/116519.html
Quoting from it:
"""
Replace 2 bytes in place beginning at offset 100 (101st byte):
f = open('text_input', 'r+b')
f.seek(100)
f.write(chr(123) + chr(0x80))
f.seek(0,2)
f.close()
"""
Can I use the seek() and write() methods in a similar way to remove
the first byte? For whatever reason I can't seem to make it work
myself. Thanks again.

Funny. I just happened to read ESR's "how to ask questions the smart way" and
your posts match quite a few of the examples. :)

No, you can't. Steven's solution is what I'd go for.

Stefan
Forgive my newbie ignorance, but I am wondering why the other method
would not work? I mean it may not be very safe,
but I guess it may perform a lot better, than having to read the whole
file just to cut out the first byte.

TIA,

../alex
--
..w( the_mindstorm )p.

Jul 11 '07 #7
>
Forgive my newbie ignorance, but I am wondering why the other method
would not work? I mean it may not be very safe,
but I guess it may perform a lot better, than having to read the whole
file just to cut out the first byte.
Because seeking is not moving? Shifting data bytewise isn't something that
is supported by the underlying OS filesystems, and thus not supported. But
replacing bytes with others is. Which seek is for.

Diez
Jul 11 '07 #8
Alex Popescu wrote:
Forgive my newbie ignorance, but I am wondering why the other method
would not work? I mean it may not be very safe,
but I guess it may perform a lot better, than having to read the whole
file just to cut out the first byte.
Why would you expect that? It *might* perform better if there was a system
call for removing bytes from inside a file, as that could reduce the
intermediate space requirements to the size of a hard disk sector rather than
the remaining size of the file (note that the time consumption would not be
reduced significantly, if the remaining file has to be copied around to fill
the sectors). But since that is a rather rare use case that most people would
prefer being handled in a safe rather than space-optimal way, I don't see the
need for such a function.

Stefan
Jul 11 '07 #9
On Jul 11, 4:15 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Forgive my newbie ignorance, but I am wondering why the other method
would not work? I mean it may not be very safe,
but I guess it may perform a lot better, than having to read the whole
file just to cut out the first byte.

Because seeking is not moving? Shifting data bytewise isn't something that
is supported by the underlying OS filesystems, and thus not supported. But
replacing bytes with others is. Which seek is for.

Diez
As far as I know seek is just about positioning and nothing else.
So, in fact the problem boils down to os support for deleting a bytes.

bests,

../alex
--
..w( the_mindstorm )p.
Jul 11 '07 #10
Alex Popescu wrote:
On Jul 11, 4:15 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Forgive my newbie ignorance, but I am wondering why the other method
would not work? I mean it may not be very safe,
but I guess it may perform a lot better, than having to read the whole
file just to cut out the first byte.

Because seeking is not moving? Shifting data bytewise isn't something
that is supported by the underlying OS filesystems, and thus not
supported. But replacing bytes with others is. Which seek is for.

Diez

As far as I know seek is just about positioning and nothing else.
It is.
So, in fact the problem boils down to os support for deleting a bytes.
Which there isn't, as I and several others pointed out. And "replacing" is
not removing.

Diez
Jul 11 '07 #11
On Jul 11, 7:45 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Alex Popescu wrote:
On Jul 11, 4:15 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Forgive my newbie ignorance, but I am wondering why the other method
would not work? I mean it may not be very safe,
but I guess it may perform a lot better, than having to read the whole
file just to cut out the first byte.
Because seeking is not moving? Shifting data bytewise isn't something
that is supported by the underlying OS filesystems, and thus not
supported. But replacing bytes with others is. Which seek is for.
Diez
As far as I know seek is just about positioning and nothing else.

It is.
So, in fact the problem boils down to os support for deleting a bytes.

Which there isn't, as I and several others pointed out. And "replacing" is
not removing.

Diez
I do agree with your last statement (and it was my mistake to consider
replacing a possible way to remove). Just wanted to
clarify that seek is only for positioning.

bests,

../alex
--
..w( the_mindstorm )p.
Jul 11 '07 #12

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Albert Tu | last post by:
Hi, I am learning and pretty new to Python and I hope your guys can give me a quick start. I have an about 1G-byte binary file from a flat panel x-ray detector; I know at the beggining there...
10
by: J. Campbell | last post by:
OK...I'm in the process of learning C++. In my old (non-portable) programming days, I made use of binary files a lot...not worrying about endian issues. I'm starting to understand why C++ makes...
9
by: Arnold | last post by:
I need to read a binary file and store it into a buffer in memory (system has large amount of RAM, 2GB+) then pass it to a function. The function accepts input as 32 bit unsigned longs (DWORD). I...
17
by: Arnold | last post by:
Is using fseek and ftell a reliable method of getting the file size on a binary file? I thought I remember reading somewhere it wasn't... If not what would be the "right" and portable method to...
2
by: DBC User | last post by:
Hi Sharpies, I have a C program I am converting it into C#. Everything is fine except this process creates a 6K byte binary file. This file initially filled with 6K null and then start...
12
by: Adam J. Schaff | last post by:
I am writing a quick program to edit a binary file that contains file paths (amongst other things). If I look at the files in notepad, they look like: ...
7
by: John Dann | last post by:
I'm trying to read some binary data from a file created by another program. I know the binary file format but can't change or control the format. The binary data is organised such that it should...
2
by: Mad Scientist Jr | last post by:
i'm trying to read a file byte by byte (and later alter the data and write it to a 2nd file byte by byte) and running into a problem where it seems to keep reading the same byte over and over again...
2
by: Jack | last post by:
Hi I am having a little trouble trying to read a binary file, I would like to write an ascii to Metastock converter in python but am not having a lot of success. The file formats are ...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.