473,320 Members | 1,856 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Making a copy (not reference) of a file handle,or starting stdin over at line 0

I wrote a script which will convert a tab-delimited file to a
fixed-width file, or a fixed-width file into a tab-delimited. It reads
a config file which defines the field lengths, and uses it to convert
either way.

Here's an example of the config file:

1:6,7:1,8:9,17:15,32:10

This converts a fixed-width file to a tab-delimited where the first
field is the first six characters of the file, the second is the
seventh, etc. Conversely, it converts a tab-delimited file to a file
where the first six characters are the first tab field, right-padded
with spaces, and so on.

What I want to do is look at the file and decide whether to run the
function to convert the file to tab or FW. Here is what works
(mostly):

x = inputFile.readline().split("\t")
inputFile.seek(0)

if len(x) 1:
toFW(inputFile)
else:
toTab(inputFile)
The problem is that my file accepts the input file via stdin (pipe) or
as an argument to the script. If I send the filename as an argument,
everything works perfectly.

If I pipe the input file into the script, it is unable to seek() it. I
tried making a copy of inputFile and doing a readline() from it, but
being a reference, it makes no difference.

How can I check a line (or two) from my input file (or stdin stream)
and still be able to process all the records with my function?

Thanks,
Shawn
Aug 17 '07 #1
1 2051
Shawn Milochik wrote:
How can I check a line (or two) from my input file (or stdin stream)
and still be able to process all the records with my function?
One way:

from itertools import chain
firstline = instream.next()
head = [firstline]

# loop over entire file
for line in chain(head, instream):
process(line)
You can of course read more than one line as long as you append it to the
head list. Here's an alternative:

from itertools import tee
a, b = tee(instream)

for line in a:
# determine file format,
# break when done

# this is crucial for memory efficiency
# but may have no effect in implementations
# other than CPython
del a

# loop over entire file
for line in b:
# process line
Peter

Aug 17 '07 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Mark Adams | last post by:
I am a relative newbie to MySQL. I had a Postfix+Courier+MySQL mail server running for several months. It took me a week or so to get it up and running in September. Now, I did a clean upgrade to...
1
by: Oz | last post by:
This is long. Bear with me, as I will really go through all the convoluted stuff that shows there is a problem with streams (at least when used to redirect stdout). The basic idea is that my...
0
by: Joshua Ginsberg | last post by:
Howdy -- I have a class that has an attribute that is a dictionary that contains an object that has a kword argument that is a lambda. Confused yet? Simplified example: import copy class...
2
by: Kevin R | last post by:
I'm trying to get asp.net 1.1 running on my home PC. When I try creating a new ASP.NET Web Application in 'Visual Studio .NET 2003' I get the following error: "Visual Studio .NET has detected...
0
by: ezra epstein | last post by:
I'm struggling using COPY FROM. COPY ... FROM STDIN expects stdin to be whatever file is being processed, so: $ cat data_file.txt | psql -f load_script.sql MyDB Does not do what one...
9
by: Adi | last post by:
Hello eveyone, I wanna ask a very simple question here (as it was quite disturbing me for a long time.) My problem is to read a file line by line. I've tried following implementations but still...
4
by: Emin | last post by:
Dear experts, I got some unexpected behavior in getattr and copy.deepcopy (see transcript below). I'm not sure if this is actually a bug in copy.deepcopy or if I'm doing something too magical...
3
by: maheshkadam | last post by:
Hi friends I am new to perl so please guide me. I have one application which created backup log file every day.But it appends that file so you can see logs for different day in one file only. ...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.