feature request / max size limit on StringIO

[Clark C. Evans]

Hello. I've not been able to use cStringIO since I have the need to
ensure that the memory buffers created are bounded within a resonable
limit set by specifications. No, this code does not properly belong
in my application as the modules that use "files" should not have
to care about any resource limitations that may be imposed.

class LimitedBuffer(StringIO):
def __init__(self, buffer = None, maxsize = 5 * 1000 * 1000):
StringIO.__init__(self,buffer)
self.cursize = 0
self.maxsize = maxsize
def write(self,str):
self.cursize += len(str)
if self.cursize > self.maxsize:
raise IOError("allocated buffer space exceeded")
return StringIO.write(self,str)

Kind Regards,

Clark Evans

[Bengt Richter]

On Tue, 25 Oct 2005 14:15:02 -0400, "Clark C. Evans" <cc*@clarkevans.com> wrote:

Hello. I've not been able to use cStringIO since I have the need to
ensure that the memory buffers created are bounded within a resonable
limit set by specifications. No, this code does not properly belong
in my application as the modules that use "files" should not have
to care about any resource limitations that may be imposed.

class LimitedBuffer(StringIO):
def __init__(self, buffer = None, maxsize = 5 * 1000 * 1000):
StringIO.__init__(self,buffer)
self.cursize = 0
self.maxsize = maxsize
def write(self,str):
self.cursize += len(str)
if self.cursize > self.maxsize:
raise IOError("allocated buffer space exceeded")
return StringIO.write(self,str)

You might want to use StringIO's own knowledge of its writing position (self.pos or self.tell())
and then you wouldn't need cursize, nor to worry about whether seek has been used to
reposition, maybe writing the same section of file over and over for some reason.

I'm not sure whether seek beyond the end actually causes allocation, but I don't think so.
So you should be ok just checking self.pos+len(strarg)> self.maxsize.
There's also some interesting things going on with self.buf and self.buflist attributes, which
probably is not doing as simple a job of buffering as you might think. Maybe you'll want to wrap
a byte array or an mmap instance to store your info, depending on what you are doing?

Regards,
Bengt Richter