473,320 Members | 1,794 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Reading unformatted big-endian files

Hello NG,

that may sound a silly question, but I didn't find anything really
clear about the issue of reading unformatted big endian files with
Python. What I was doing till now, was using Fortran to read those
files and compile this Fortran extension using F2PY. Now that it seems
that no possible combinations of Fortran/C compilers actually *work*
with Python 2.4 on Windows XP, I was trying to translate the Fortran
subroutine to Python. Basically, what I do (in Fortran, I hope to
explain the code clearly) is:

! Declare an integer
integer number

! Declare a 4-chars character
character*4 keytype

! Declare a 8-chars character
character*8 keyword

! feof is not very important here
logical feof
feof = .false.

! Open the file as unformatted big-endian
open(unit = , file = filename, form = 'UNFORMATTED', convert = 'BIG_ENDIAN')

! loop until you find a particular keyword
! here "end=10" means that if the routine finds the EOF, it should go to
! the label "10 continue". "err=8" means that, if an error occours in
reading the file,
! it should go to the label "8 continue" and continue reading the file

do while(.not.feof)

! Read the 3 variables keyword, number and keytype
read(1, end=10, err=8) keyword, number, keytype

! If the keyword is 'DIMENS', break the loop and go to the end
if (keyword == 'DIMENS') then
read(1, end=10, err=8) dimens
goto 10
endif
8 continue
enddo

10 continue

! Close the file
close(1)
Well, does anyone have some suggestion about which kind of
material/tutorial on similar things I should read? How can I deal in
Python with variables that must be 8-chars or 4-chars in order to read
correctly the file? Am I missing something else?

Thank you very much for every suggestion.

Andrea.

--
"Imagination Is The Only Weapon In The War Against Reality."

http://xoomer.virgilio.it/infinity77/
Aug 11 '06 #1
1 5580
Andrea Gavana wrote:
"err=8" means that, if an error occours in
reading the file,
it should go to the label "8 continue" and continue reading the file
Silently ignoring errors when reading a file doesn't sound like a good
idea to me at all, especially if different records have different
formats.
>
Well, does anyone have some suggestion about which kind of
material/tutorial on similar things I should read? How can I deal in
Python with variables that must be 8-chars or 4-chars in order to read
correctly the file?
(a) read the docs on the struct module
(b) eyeball this rough untested translation:
8<---
def filereader(filename):
import struct
f = open(fname, 'rb') # 'rb' is read binary, very similar to C
stdio
fmt = '>8si4s'
# Assuming unformatted means binary,
# and integer means integer*4, which is signed.
# Also assuming that the 3-variable records are fixed-length.
fmtsz = struct.calcsize(fmt)
while True:
buff = f.read(fmtsz)
if not buff: # EOF
break
keyword, number, keytype = struct.unpack(fmt)
keyword = keyword.rstrip() # remove trailing spaces
keytype = keytype.rstrip()
if keyword == 'DIMENS':
# 'dimens' is neither declared nor initialised in the
FORTRAN
# so I'm just guessing here ...
buff2 = f.read(4)
dimens = struct.unpack('>i', buff2)
break
print keyword, number, keytype # or whatever
# reached end of file (dimens *NOT* defined),
# or gave up (dimens should have a value)
f.close() # not absolutely necessary especially when only reading

if __name__ == "__main__":
import sys
filereader(sys.argv[1])
8<---

If this doesn't work, and it's not obvious how to fix it, it might be a
good idea when you ask again if you were to supply a
FORTRAN-independent layout of the file, and/or a dump of a short test
file that includes the DIMENS/dimens caper -- you can get such a dump
readily with the *x od command or failing that, use Python:

#>>>repr(open('thetestfile', 'rb').read(100)) # yes, I said *short*

HTH,
John

Aug 11 '06 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: David M. Wilson | last post by:
Hello! Is there a simple way of reading from a file object up to a specific byte value? I would like to do this without reading one character at a time, or reading in chunks and holding a...
19
by: Lionel B | last post by:
Greetings, I need to read (unformatted text) from stdin up to EOF into a char buffer; of course I cannot allocate my buffer until I know how much text is available, and I do not know how much...
8
by: Yeow | last post by:
hello, i was trying to use the fread function on SunOS and ran into some trouble. i made a simple test as follows: i'm trying to read in a binary file (generated from a fortran code) that...
6
by: KevinD | last post by:
assumption: I am new to C and old to COBOL I have been reading a lot (self teaching) but something is not sinking in with respect to reading a simple file - one record at a time. Using C, I am...
3
by: Nick | last post by:
I have found a class that compresses and uncompresses data but need some help with how to use part of it below is the deflate method which compresses the string that I pass in, this works OK. At...
2
by: Matt McGonigle | last post by:
Hi all, Please help me out with this. Perhaps it is a dumb question, but I can't seem to make it work. I am doing a file conversion using an unformatted binary file for input and outputting to...
0
by: Eric | last post by:
Visual C++ 2005 Express MVP's and experience programmer's only please!... I need to get the number of lines in a textbox so I can insert them into a listview. The text comes from my database...
3
by: Bob | last post by:
I had an app in VB6 that I'm "upgrading" <GGGG> to Vb.net 2005. In Vb6 I had rtf text in a field in a Sql server 2000 field and it would show up as formatted RTF in my richtextbox. In vs.net that...
7
by: cmfvulcanius | last post by:
I am using a script with a single file containing all data in multiple sections. Each section begins with "#VS:CMD:command:START" and ends with "#VS:CMD:command:STOP". There is a blank line in...
3
by: Willy Stevens | last post by:
Hello, In my application I have to read sometimes quite big chunk of binary data. I have a buffer which default size is 32000 bytes. But how could I read binary data that exceeds 32000 bytes?...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.