473,287 Members | 1,426 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Safely dealing with arbitrary file objects

I have found myself writing functions rather like these:

def openfile(filename):
if filename == '-':
# convention for shell scripts in Unix-land is to use
# '-' for stdin/stdout for reading/writing.
outfile = sys.stdout
if filename == '2-':
outfile = sys.stderr
else:
outfile = file(filename, 'w')
return outfile

def closefile(fileobj):
# don't close standard file objects, or their replacements
if not fileobj in (sys.stdout, sys.stderr, sys.stdin,
sys.__stdout__, sys.__stderr__, sys.__stdin__):
fileobj.close()
def processfile(filename):
outfile = openfile(filename)
try:
# lots of processing here, which may raise exceptions
var = "stuff happens"
outfile.write(var)
finally:
closefile(outfile)

A question:

I know I'm being paranoid about not closing files I shouldn't close, but
am I being excessively paranoid, or not paranoid enough?

Suggestions for improvements welcome; while I'm happy to read suggestions
using the new with statement, I can't yet rely on having Python 2.5 or
better so I have to stick to try...finally.
--
Steven.
Oct 14 '07 #1
2 1190
Steven D'Aprano wrote:
I have found myself writing functions rather like these:

def openfile(filename):
if filename == '-':
# convention for shell scripts in Unix-land is to use
# '-' for stdin/stdout for reading/writing.
outfile = sys.stdout
if filename == '2-':
outfile = sys.stderr
else:
outfile = file(filename, 'w')
return outfile

def closefile(fileobj):
# don't close standard file objects, or their replacements
if not fileobj in (sys.stdout, sys.stderr, sys.stdin,
sys.__stdout__, sys.__stderr__, sys.__stdin__):
fileobj.close()
def processfile(filename):
outfile = openfile(filename)
try:
# lots of processing here, which may raise exceptions
var = "stuff happens"
outfile.write(var)
finally:
closefile(outfile)

A question:

I know I'm being paranoid about not closing files I shouldn't close, but
am I being excessively paranoid, or not paranoid enough?

Suggestions for improvements welcome; while I'm happy to read suggestions
using the new with statement, I can't yet rely on having Python 2.5 or
better so I have to stick to try...finally.
How about a wrapper that passes along all but 'close', so you don't
have to worry about closing something you means to leave open.
Something like:

class DontCloseOutput(file):
'''subclass file only to allow isinstance checks to work out'''
def __init__(self, referent):
self._actual = referent
self.closed = False

def close(self):
self.closed = True

def write(self, data):
if self.closed:
raise IOError('Tried to write closed pseudo-file')
return self._actual.write(data)
def writeopen(filename):
if filename == '-':
# convention for shell scripts in Unix-land is to use
# '-' for stdin/stdout for reading/writing.
return DontCloseOutput(sys.stdout)
if filename == '2-':
return DontCloseOutput(sys.stderr)
else:
return open(filename, 'w') # BTW, open is the preferred way
# file was for a short time.

You of course can do similar things (probably forwarding more message)
with readopen.

-Scott David Daniels
Sc***********@Acm.Org
Oct 14 '07 #2
On Oct 14, 5:01 am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:
I have found myself writing functions rather like these:

def openfile(filename):
if filename == '-':
# convention for shell scripts in Unix-land is to use
# '-' for stdin/stdout for reading/writing.
outfile = sys.stdout
if filename == '2-':
outfile = sys.stderr
else:
outfile = file(filename, 'w')
return outfile

def closefile(fileobj):
# don't close standard file objects, or their replacements
if not fileobj in (sys.stdout, sys.stderr, sys.stdin,
sys.__stdout__, sys.__stderr__, sys.__stdin__):
fileobj.close()

def processfile(filename):
outfile = openfile(filename)
try:
# lots of processing here, which may raise exceptions
var = "stuff happens"
outfile.write(var)
finally:
closefile(outfile)

A question:

I know I'm being paranoid about not closing files I shouldn't close, but
am I being excessively paranoid, or not paranoid enough?

Suggestions for improvements welcome; while I'm happy to read suggestions
using the new with statement, I can't yet rely on having Python 2.5 or
better so I have to stick to try...finally.
With with:

import sys

class leave_open(object):
def __init__(self, obj):
self.obj = obj
def __enter__(self):
return self.obj
def __exit__(self, *exc_info):
pass

def open_file(name, special={'-':sys.stdout, '2-':sys.stderr}):
if name in special:
return leave_open(special[name])
else:
return open(name, 'w')

def process_file(filename):
with open_file(filename) as outfile:
outfile.write("stuff happens")
Without with:

import sys

def open_file(name, special={'-':sys.stdout, '2-':sys.stderr}):
if name in special:
return False, special[name]
return True, open(name, 'w')

def process_file(filename):
should_close, outfile = open_file(filename)
try:
outfile.write('stuff happens')
finally:
if should_close:
outfile.close()

--
Paul Hankin

Oct 14 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Bret Pehrson | last post by:
I'm working on a (non-trivial) project which includes various & numerous objects, some in my direct source control, some not. I'm finding the need to somehow associate one object with another,...
1
by: Patrick | last post by:
Hi, This post is the 'sequel' ;) of the "Data Oriented vs Object Oriented Design" post, but it can be read and treated apart from that one. I will just quote the beginning of my previous message...
4
by: Russell Warren | last post by:
I've got a case where I want to convert binary blocks of data (various ctypes objects) to base64 strings. The conversion calls in the base64 module expect strings as input, so right now I'm...
27
by: pkirk25 | last post by:
Assume an array of structs that is having rows added at random. By the time it reaches your function, you have no idea if it has a few hundred over over 10000 rows. When your function recieves...
17
by: Christopher Benson-Manica | last post by:
Some recent posts got me thinking about how one might have dealt with simplistic malloc() implementations which might return NULL for a 64K request but might accept two 32K requests or four 16K...
28
by: walterbyrd | last post by:
Python seems to have a log of ways to do collections of arbitrary objects: lists, tuples, dictionaries. But what if I want a collection of non-arbitrary objects? A list of records, or something...
3
by: sophie_newbie | last post by:
Hi, I want to store python text strings that characters like "é" "Č" in a mysql varchar text field. Now my problem is that mysql does not seem to accept these characters. I'm wondering if there...
4
by: arnaudk | last post by:
I am trying to come up with a class design to deal with asynchronous data to be stored and analyzed over multiple time frames and could really use some design input. This is a rather long question...
16
by: Fett | last post by:
I am creating a program that requires some data that must be kept up to date. What I plan is to put this data up on a web-site then have the program periodically pull the data off the web-site. ...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.