Best way of finding terminal width/height?

[Joel Hedlund]

Hi all!

I use python for writing terminal applications and I have been bothered
by how hard it seems to be to determine the terminal size. What is the
best way of doing this?

At the end I've included a code snippet from Chuck Blake 'ls' app in
python. It seems to do the job just fine on my comp, but regrettably,
I'm not sassy enough to wrap my head around the fine grain details on
this one. How cross-platform is this? Is there a more pythonic way of
doing this? Say something like:

from ingenious_module import terminal_info
cols, rows = terminal_info.size()

Thanks for your time (and thanks Chuck for sharing your code!)
/Joel Hedlund
IFM Bioinformatics
Linköping University

Chuck Blake's terminal_size code snippet:
(from http://pdos.csail.mit.edu/~cblake/cls/cls.py).

def ioctl_GWINSZ(fd): #### TABULATION FUNCTIONS
try: ### Discover terminal width
import fcntl, termios, struct, os
cr = struct.unpack('hh',
fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234'))
except:
return None
return cr

def terminal_size():
### decide on *some* terminal size
# try open fds
cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
if not cr:
# ...then ctty
try:
fd = os.open(os.ctermid(), os.O_RDONLY)
cr = ioctl_GWINSZ(fd)
os.close(fd)
except:
pass
if not cr:
# env vars or finally defaults
try:
cr = (env['LINES'], env['COLUMNS'])
except:
cr = (25, 80)
# reverse rows, cols
return int(cr[1]), int(cr[0])

[Grant Edwards]

On 2006-02-05, Joel Hedlund <jo**********@gmail.com> wrote:

I use python for writing terminal applications and I have been
bothered by how hard it seems to be to determine the terminal
size. What is the best way of doing this?
The way used in the code you found. Alternatively, yould use
ncurses, but that's massive overkill if all you want is the
terminal window size.
At the end I've included a code snippet from Chuck Blake 'ls'
app in python. It seems to do the job just fine on my comp,
but regrettably, I'm not sassy enough to wrap my head around
the fine grain details on this one.
Which details? We'd be happy to explain the code. Not that
you need to understand the details to use the code.
How cross-platform is this?
It will work on any Unix system you're likely to encounter. I
know it works on BSD, Solaris, and Linux. I no longer yave
acces to a v7 system, but I'm pretty sure it would have worked
on that as well -- except the one I used didn't have a large
enough address space or disk drive to run Python.

I don't know if it will work on MS Windows or not.
Is there a more pythonic way of doing this?
What's unpythonic about the example you found? It looks
extremely pythonic to me. The use of the "or" operator when
making ioctl calls and the try/except block when opening a file
is classically pythonic.
Say something like:

from ingenious_module import terminal_info
cols, rows = terminal_info.size()

If that's what you want, put the code you found in to a file
named "ingenious_module.py". I don't see the point of of
layering the namespace like that, though. Just call it
terminal_info.py and do

import terminal_info
cols,rows = terminal_info.terminal_size()

--
Grant Edwards grante Yow! Oh, FISH sticks,
at CHEEZ WHIZ, GIN fizz,
visi.com SHOW BIZ!!

[Joel Hedlund]

> Which details? We'd be happy to explain the code. Not that

you need to understand the details to use the code.
OK, why '1234' in here, and what's termios.TIOCGWINSZ, and how should I
have known this was the way too do it?
fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234')

Am I interpreting C structs here, and if so - why is python giving me C
structs? And what's 'hh' anyway?
struct.unpack('hh', ... )

Why 0, 1 and 2?
cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
I don't know if it will work on MS Windows or not.
Linux and unix are my main concerns, but it would be neat to know if it
would work on Win/Mac.

What OS:es set the COLS/ROWS env vars? What OS:es can leverage the
termios module? I have a hunch that we have 100% overlap there, and then
this solution leaves me at square one (arbitrary choice of 80*25) for
the others OS:es (or am I wrong?). How do Win/Mac people do this?
What's unpythonic about the example you found?

Maybe I did bit of poor wording there, but In my experience python
generally has a high level of abstraction, which provides linguistically
appealing (as in "in english") solutions to almost any problem. Like for
example how os.path.isfile(s) tells me if my string s corresponds to a
file. I guess that's what I mean really. I sort of expected to find
something like my terminal_size() example in the built-in modules. I
didn't expect to have to do that struct fcntl ioctl boogey to solve this
relatively simple (?) problem.

Thanks for your help!
/Joel

[Grant Edwards]

On 2006-02-05, Joel Hedlund <jo**********@gmail.com> wrote:

Which details? We'd be happy to explain the code. Not that
you need to understand the details to use the code.
OK, why '1234' in here, and what's termios.TIOCGWINSZ,

Second question first: TIOCGWINSZ is just a platform-defined
magic number that you pass to ioctl() to tell it that you want
to get the terminal window size:

Terminal IO Control Get WINdow SiZe
T IO C G WIN S Z
and how should I have known this was the way too do it?
Because if you're writing low level screen management stuff for
Unix you know that sort of stuff. BTW: don't forget to attach
a handler to the window-size-change signal (SIGWINCH) so that
you know when your terminal changes sizes (which means you need
to call the function to get the current size again and
re-layout your screen).
fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234')
Now the '1234' question. This bit _is_ a bit ugly. ioctl()
expects a third parameter which is input data in the case of
"write-like" ioctl() calls. If the ioctl() is a "read-like"
call, the third argument is merely a string that specifies how
many data bytes the ioctl call is expected to return. In this
case the TIOGWINSZ ioctl will return 4 bytes of data (two
16-bit integers), so we pass it a 4 byte string to tell the
ioctl module how much memory to allocate for the value to be
returned.

It's explained in more detail at

http://docs.python.org/lib/module-fcntl.html
Am I interpreting C structs here,
Yes. Which I can see might be considered "un-pythonic", but
it's a common enough paradigm that the "struct" module is
included in the standard library for just that task.
and if so - why is python giving me C structs?
Because ioctl() is Unix's junk-drawer. It's full of
miscellaneous crap that didn't fit anywhere else (and often was
an after-thought). The python module would require a _lot_ of
very nasty system-specific knowledge and logic to implicitly
figure out the types of passed and returned data. Since it's
not used that much, nobody has put the work required to make
Python's ioctl module omniscient. Some of the more common
stuff probably has been provided in a more abstract manner.
And what's 'hh' anyway? struct.unpack('hh', ... )
It's specifying the types of the "fields" within the struct.
'h' means a short (16-bit) integer. The struct returned from
the ioctl() is going to contain two 16-bit integers. The full
description of struct formats is at

http://docs.python.org/lib/module-struct.html

Why 0, 1 and 2?
cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
0 is the file descriptor for standard input, 1 is standard
output, and 2 is standard error. If your program has a
terminal, one of them may be connected to it.

I don't know if it will work on MS Windows or not.

Linux and unix are my main concerns, but it would be neat to know if it
would work on Win/Mac.

Homey don't do Windows, so you'll have to ask somebody else that.
What OS:es set the COLS/ROWS env vars?
None. Some shells set those environment variables. They may
be wrong if the terminal has changed size since your program
was started.
What OS:es can leverage the termios module?
I don't know what you mean by "leverage the termios module".
I have a hunch that we have 100% overlap there,
No comprende.
and then this solution leaves me at square one (arbitrary
choice of 80*25) for the others OS:es (or am I wrong?).
Dunno. I've been using Unix for 25+ years. I tried Windows a
few times and though it sucked rather badly. Now that Cygwin
is around it sucks slightly less (but only slightly).
How do Win/Mac people do this?
AFAICT, Win/Mac people don't.

They just don't write console programs. Unless you're talking
about Mac OS X, and that's just BSD Unix, so they do it using
ioctl().

What's unpythonic about the example you found?

Maybe I did bit of poor wording there, but In my experience python
generally has a high level of abstraction, which provides linguistically
appealing (as in "in english") solutions to almost any problem.

For popular and common tasks that's true. For stuff nobody
ever does, you usually fall back to the underlying platform
mechanism.
Like for example how os.path.isfile(s) tells me if my string s
corresponds to a file. I guess that's what I mean really. I
sort of expected to find something like my terminal_size()
example in the built-in modules.
I would guess that the real answer is that so few people have
ever wanted it that nobody has ever created a module and
submitted it. Feel free... ;)
I didn't expect to have to do that struct fcntl ioctl boogey
to solve this relatively simple (?) problem.

Getting terminal size may be a simple problem, but it's only a
small part of a complex and difficult task (managing
"full-screen" terminal-mode apps). People who write
terminal-mode apps in python generally do one of two things:

1) Just write a normal Unix-like text processing "filter".
Those programs don't generally care about the terminal size
since they may not be connected to a terminal at all.

2) Use a terminal management library like ncurses+panel or
slang+newt. That way you get a much higher level more
abstract API.

--
Grant Edwards grante Yow! Don't SANFORIZE me!!
at
visi.com

[Joel Hedlund]

Thank you for a very quick, informative and concise response.

BTW: don't forget to attach a handler to the window-size-change
signal (SIGWINCH) so that you know when your terminal changes sizes
Do you mean something like this?

import signal, os
# terminal_info contains the example from my first post
from terminal_info import get_terminal_size

TERMINAL_SIZE = get_terminal_size()

def update_terminal_size(signum, frame):
global TERMINAL_SIZE
TERMINAL_SIZE = get_terminal_size()

signal.signal(signal.SIGWINCH, update_terminal_size)

while True:
# Do lots of IO (fishing for exceptions...)
open('/a/large/file').read()
print TERMINAL_SIZE

The docs for the signal module (http://docs.python.org/lib/module-signal.html) say that
"""
When a signal arrives during an I/O operation, it is possible that the I/O operation raises an exception after the signal handler returns. This is dependent on the underlying Unix system's semantics regarding interrupted system calls.

"""
So this is what I'm trying to provoke in the final while loop. In this case I get no exceptions (hooray!). However, if I replace open('/a/large/file').read() with raw_input() I get EOFError (no errmsg), and even worse, if I replace it with sys.stdin.read() or even print open('/a/large/file').read() I get IOError: [Errno 4] Interrupted system call.

I do lots of IO in my work, and primarily with gigantic text files (welcome to bioinformatics :-). Protecting my code from this sort of error (i believe) will be quite hard, and probably won't look pretty. Or am I missing something?

Note though that these realtime updates aren't essential to me at the moment. Basically all I need is to find out for each run how much space I have so I can text wrap command line help text (as in myprog --help) as user friendly as possible.

I run Ubuntu 5.10 btw, but I try to stay as cross-platform as I can. My priorities are Ubuntu, other linuxes, other unixes, Win, Mac.
Homey don't do Windows, so you'll have to ask somebody else that.
:-)
I don't know what you mean by "leverage the termios module".

I have a hunch that we have 100% overlap there,
No comprende.

I mean: I believe that for those environments where $COLS and $ROWS are set
then python will probably have access to the termios module as well, and for those environments that don't have $COLS and $ROWS set then python probably will not have access to the termios module either. So, in the latter case I'm back to square one, which is arbitrary guesswork.
1) Just write a normal Unix-like text processing "filter".
Yes that's what I normally do. But I also like to hook them up with a --help option that shows usage and options and such, and I like that warm fuzzy feeling you get from seeing really readable user friendly help... :-)
I would guess that the real answer is that so few people have
ever wanted it that nobody has ever created a module and
submitted it. Feel free... ;)

I just might. I've got some stuff that people may benefit from (or possibly hate, I don't know ;-). If I ever sum up the courage to publish it, would it be a good idea to post the modules to py********@python.org, or is there some better route?

Thanks again for your time!
/Joel

[Grant Edwards]

On 2006-02-06, Joel Hedlund <jo**********@gmail.com> wrote:

Thank you for a very quick, informative and concise response.

BTW: don't forget to attach a handler to the window-size-change
signal (SIGWINCH) so that you know when your terminal changes sizes
Do you mean something like this?

import signal, os
# terminal_info contains the example from my first post
from terminal_info import get_terminal_size

TERMINAL_SIZE = get_terminal_size()

A stylistic point: In almost all Python programs, uppercase
identifiers are almost always reserved for constants (as in C).
The language itself doesn't care, of course.
def update_terminal_size(signum, frame):
global TERMINAL_SIZE
TERMINAL_SIZE = get_terminal_size()

signal.signal(signal.SIGWINCH, update_terminal_size)

while True:
# Do lots of IO (fishing for exceptions...)
open('/a/large/file').read()
print TERMINAL_SIZE
Hmm. There things you're not allowed to do in a signal handler.
I don't know if the TIOCGWINSZ ioctl() is one of them or not.
If it's not allowed, then what one usually does is set a flag
in the signal handler then check the flag in your main loop and
do the ioctl() there.
The docs for the signal module
(http://docs.python.org/lib/module-signal.html) say that
"""
When a signal arrives during an I/O operation, it is possible
that the I/O operation raises an exception after the signal
handler returns. This is dependent on the underlying Unix
system's semantics regarding interrupted system calls.

"""
So this is what I'm trying to provoke in the final while loop.
In this case I get no exceptions (hooray!).
I wouldn't be surprised if Python catches that exception and
restarts the I/O operation.
However, if I replace open('/a/large/file').read() with
raw_input() I get EOFError (no errmsg), and even worse, if I
replace it with sys.stdin.read() or even print
open('/a/large/file').read() I get IOError: [Errno 4]
Interrupted system call.
Yup, that's the exception. Standard practice is to catch it and
retry the I/O operation.
I do lots of IO in my work, and primarily with gigantic text
files (welcome to bioinformatics :-). Protecting my code from
this sort of error (i believe) will be quite hard, and
probably won't look pretty. Or am I missing something?
You might want to try just setting a flag in the signal handler
to see if that prevents the I/O operations on stdin/stdout from
being interrupted.
Note though that these realtime updates aren't essential to me
at the moment. Basically all I need is to find out for each
run how much space I have so I can text wrap command line help
text (as in myprog --help) as user friendly as possible.
It sounds like getting the terminal width once at the start of
the program is probably good enough. That's what a lot of
text-mode programs that aren't doing a log of fancy full-screen
stuff do.
I mean: I believe that for those environments where $COLS and
$ROWS are set then python will probably have access to the
termios module as well, and for those environments that don't
have $COLS and $ROWS set then python probably will not have
access to the termios module either. So, in the latter case
I'm back to square one, which is arbitrary guesswork.
Yes, that's probably true. In that case, most people assume
80x24.
I just might. I've got some stuff that people may benefit from
(or possibly hate, I don't know ;-). If I ever sum up the
courage to publish it, would it be a good idea to post the
modules to py********@python.org, or is there some better
route?

I think what most people do initially is to put the module
somewhere publically accessible and post an announcement to
c.l.p and c.l.p.a. If you feel particularly ambitious you
could create a project for it on sourceforge or somewhere like
that. I don't know what the procedure is for nominating a
module for inclusion in the standard library.

--
Grant Edwards grante Yow! YOW!! Now I
at understand advanced
visi.com MICROBIOLOGY and th' new
TAX REFORM laws!!

[Jorgen Grahn]

On Mon, 06 Feb 2006 15:17:43 -0000, Grant Edwards <gr****@visi.com> wrote:

On 2006-02-06, Joel Hedlund <jo**********@gmail.com> wrote:

[finding the terminal dimensions]

access to the termios module either. So, in the latter case
I'm back to square one, which is arbitrary guesswork.

Yes, that's probably true. In that case, most people assume
80x24.

And, I might add, as a Unix user that is what I expect programmers to do --
stick to 80x24 or 80 x any-height-they-need. If it's not a full-screen
program (which would use curses anyway), I don't expect them to work harder
than that.

/Jorgen

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.dyndns.org> R'lyeh wgah'nagl fhtagn!

[Joel Hedlund]

> You might want to try just setting a flag in the signal handler

to see if that prevents the I/O operations on stdin/stdout from
being interrupted.
Tried this:

<source>

import signal, os, sys
from terminal_info import get_terminal_size

terminal_size = get_terminal_size()

_bTerminalSizeChanged = False

def report_terminal_size_change(signum, frame):
global _bTerminalSizeChanged
_bTerminalSizeChanged = True

def update_terminal_size():
global _bTerminalSizeChanged, terminal_size
terminal_size = get_terminal_size()
_bTerminalSizeChanged = False

signal.signal(signal.SIGWINCH, report_terminal_size_change)

while True:
# Do lots of IO (I'm trying to provoke exceptions with signal)
open('/a/large/file').read()
#raw_input()
#sys.stdin.read()
#print open('/a/large/file').read()

if _bTerminalSizeChanged:
update_terminal_size()
print terminal_size

</source>

As before, the only IO case above that doesn't throw exceptions is the uncommented one.
Yup, that's the exception. Standard practice is to catch it and
retry the I/O operation.

Hmm... I guess it's not that easy to "retry" IO operations on pipes and streams (stdin/stdout in this case)... And I tend to lean pretty heavily on those since I usually write UNIX style text "filters".

So in case I haven't missed something fundamental I guess my best option is to accept defeat (of sorts :-) and be happy with picking a terminal width at program startup.

But anyway, it's been really interesting trying this out.

Thank you Grant (och Jorgen) for all help and tips!
/Joel

[Grant Edwards]

On 2006-02-07, Joel Hedlund <jo**********@gmail.com> wrote:

As before, the only IO case above that doesn't throw
exceptions is the uncommented one.

Yup, that's the exception. Standard practice is to catch it and
retry the I/O operation.
Hmm... I guess it's not that easy to "retry" IO operations on
pipes and streams (stdin/stdout in this case)...

You just call the failed read() or write() again. Unless
there's some way that the read/write partially succeeded and
you don't have any way to know how many bytes were
read/written, If that's the case then Python's "file" object
read and write would appear to be broken by design.
And I tend to lean pretty heavily on those since I usually
write UNIX style text "filters".

So in case I haven't missed something fundamental I guess my
best option is to accept defeat (of sorts :-) and be happy
with picking a terminal width at program startup.

But anyway, it's been really interesting trying this out.

Thank you Grant (och Jorgen) for all help and tips!

--
Grant Edwards grante Yow! BRILL CREAM is
at CREAM O' WHEAT in another
visi.com DIMENSION...

[Joel Hedlund]

> You just call the failed read() or write() again. Unless

there's some way that the read/write partially succeeded and
you don't have any way to know how many bytes were
read/written, If that's the case then Python's "file" object
read and write would appear to be broken by design.

Wow... I tried to set up an example that would fail, and it didn't. It seems the test only fails if I use the keyboard to cram stuff into stdin, and not if stdin is a regular pipe.

Could this perhaps be some kind of misbehavior on behalf of my terminal emulator (GNOME Terminal 2.12.0 in Ubuntulinux 5.10)?

Example follows:

<winch.py>

import signal, os, sys
from terminal_info import get_terminal_size

terminal_size = get_terminal_size()

_bTerminalSizeChanged = False

def report_terminal_size_change(signum, frame):
global _bTerminalSizeChanged
_bTerminalSizeChanged = True

def update_terminal_size():
global _bTerminalSizeChanged, terminal_size
terminal_size = get_terminal_size()
_bTerminalSizeChanged = False

signal.signal(signal.SIGWINCH, report_terminal_size_change)

# Retry IO operations until successful.
io_successful = False
while not io_successful:
try:
s = sys.stdin.read()
io_successful = True
except IOError:
pass

io_successful = False
while not io_successful:
try:
sys.stdout.write(s)
io_successful = True
except IOError:
pass

</winch.py>

Then run the prog and pipe a large chunk of text into stdin, and redirect stdout to a file:

$ cat /a/large/text/file | python winch.py > copy.of.the.large.file

Now, what happens for me is exactly what I wanted. I can resize the window as much as I like, and a diff

$ diff /a/large/text/file copy.of.the.large.file

comes up empty. A perfectly good copy.

However, if I do this instead (try to use keyboard to push stuff into stdin):

$ python winch.py > copy.of.the.large.file

I expect python not to return until I press Ctrl-D on my keyboard, but it does return as soon as I enter more than one line of text and then resize the window (one unterminated line is ok).

Example text to type in:
moo moo
cow cow

As soon as I have typed in something that includes a newline charater through the keyboard and try to resize the terminal, sys.stdin.read() will return whatever I put in no far and no exception raised.

Weird. Could it in fact my terminal that's screwing things up for me?

/Joel

[Grant Edwards]

On 2006-02-07, Joel Hedlund <jo**********@gmail.com> wrote:

You just call the failed read() or write() again. Unless
there's some way that the read/write partially succeeded and
you don't have any way to know how many bytes were
read/written, If that's the case then Python's "file" object
read and write would appear to be broken by design.
Wow... I tried to set up an example that would fail, and it
didn't. It seems the test only fails if I use the keyboard to
cram stuff into stdin, and not if stdin is a regular pipe.

That's what I'd expect. Resizing the terminal should have no
effect on read() calls that are pending on other things.
Could this perhaps be some kind of misbehavior on behalf of my
terminal emulator (GNOME Terminal 2.12.0 in Ubuntulinux 5.10)?
Nope. Resign the terminal will abort pending I/O operations on
that terminal. It won't terminal I/O operations pending on
other devices/files.
Then run the prog and pipe a large chunk of text into stdin, and redirect stdout to a file:

$ cat /a/large/text/file | python winch.py > copy.of.the.large.file

Now, what happens for me is exactly what I wanted. I can
resize the window as much as I like, and a diff
Sure.
$ diff /a/large/text/file copy.of.the.large.file

comes up empty. A perfectly good copy.

However, if I do this instead (try to use keyboard to push stuff into stdin):

$ python winch.py > copy.of.the.large.file

I expect python not to return until I press Ctrl-D on my
keyboard,
sys.stdin.read() will return when there's an EOF or when the
underyling read() call is aborted by a signal.
but it does return as soon as I enter more than one
line of text and then resize the window (one unterminated line
is ok).

Example text to type in:
moo moo
cow cow

As soon as I have typed in something that includes a newline
charater through the keyboard and try to resize the terminal,
sys.stdin.read() will return whatever I put in no far and no
exception raised.
Yup. That does indeed appear to be the way it works. :)
Weird. Could it in fact my terminal that's screwing things up
for me?

No.

Try this out:

----------------------------------------------------------------------
#!/usr/bin/python
import signal, os, sys

_bTerminalSizeChanged = False

def report_terminal_size_change(signum, frame):
global _bTerminalSizeChanged
_bTerminalSizeChanged = True

signal.signal(signal.SIGWINCH, report_terminal_size_change)

while True:
try:
s = sys.stdin.read()
if not s:
break
sys.stdout.write(s)
except IOError:
sys.stderr.write("IOError\n")
if _bTerminalSizeChanged:
sys.stderr.write("SIGWINCH recevied\n")
_bTerminalSizeChanged = False
----------------------------------------------------------------------

In that example, I handle IOError on write with the same
exception handler as the one for read. That may not be exactly
what you want to do, but it does demonstrate what
window-resizing does.

When the window is resized, the SIGWINCH handler will be
called. A pending read() may abort with an IOError, or it may
just return some buffered data.

--
Grant Edwards grante Yow! Send your questions
at to "ASK ZIPPY", Box 40474,
visi.com San Francisco, CA 94140,
USA

[Joel Hedlund]

sys.stdin.read() will return when ... the
underyling read() call is aborted by a signal.
Not "return", really? Won't it just pass an exception? I thought that was what I was catching with the "except IOError" part there? I assumed that sys.stdin.read() would only return a value properly at EOF?

It looks to me as if sys.stderr.read() really gets an EOF at the final linebreak fed into the terminal prior to window size change, because the final unterminated line shows up on my shell prompt. Like so:

$ python winch.py
moo moo
cow cowmoo moo
$ cow cow

In this example I type moo moo[ENTER]cow cow on my keyboard and then resize the window.

Now that EOF has to come from somewhere (since there's no IOError or other exception, or the program wouldn't terminate nicely with nothing on stderr) and I'd like to point the blame at the terminal. Or is there something really fishy inside sys.stdin.read() or signal that puts EOFs into streams?

But anyway, as long as this behavior only shows up on interactive operation, the user will likely spot it anyway and can react to it. So I think this code would be pretty safe to use. What do you think?
Resign the terminal will abort pending I/O operations on
that terminal. It won't terminal I/O operations pending on
other devices/files.
What do you mean by "abort"? I can accept that "aborting" may lead to raising of IOError (which we can catch and retry), but not to arbitrary insertion of EOFs into streams (which we cannot distinguish from the real deal coming from the user).

Also: try your example and enter moo moo[ENTER]cow cow[RESIZE][ENTER][RESIZE]

Thanks again for your help.
/Joel

but it does return as soon as I enter more than one
line of text and then resize the window (one unterminated line
is ok).

Example text to type in:
moo moo
cow cow

As soon as I have typed in something that includes a newline
charater through the keyboard and try to resize the terminal,
sys.stdin.read() will return whatever I put in no far and no
exception raised.

Yup. That does indeed appear to be the way it works. :)

Weird. Could it in fact my terminal that's screwing things up
for me?

No.

Try this out:

----------------------------------------------------------------------
#!/usr/bin/python
import signal, os, sys

_bTerminalSizeChanged = False

def report_terminal_size_change(signum, frame):
global _bTerminalSizeChanged
_bTerminalSizeChanged = True

signal.signal(signal.SIGWINCH, report_terminal_size_change)

while True:
try:
s = sys.stdin.read()
if not s:
break
sys.stdout.write(s)
except IOError:
sys.stderr.write("IOError\n")
if _bTerminalSizeChanged:
sys.stderr.write("SIGWINCH recevied\n")
_bTerminalSizeChanged = False
----------------------------------------------------------------------

In that example, I handle IOError on write with the same
exception handler as the one for read. That may not be exactly
what you want to do, but it does demonstrate what
window-resizing does.

When the window is resized, the SIGWINCH handler will be
called. A pending read() may abort with an IOError, or it may
just return some buffered data.

[Grant Edwards]

On 2006-02-08, Joel Hedlund <jo**********@gmail.com> wrote:

sys.stdin.read() will return when ... the
underyling read() call is aborted by a signal.
Not "return", really?

Yes, really. On a SIGWINCH, the read() will _either_ return
currently buffered data or thrown an IOError exception. You
need to handle either one.
Won't it just pass an exception?
That seems to be the behavior if there is no buffered data to
return.
I thought that was what I was catching with the "except
IOError" part there? I assumed that sys.stdin.read() would
only return a value properly at EOF?
That assumption seems to be wrong. sys.stdin.read() also
returns w/o an exception if there's buffered data and a
SIGWINCH occurs. Whether that is "proper" or not, I don't
know, but that's what it does.

Hmm, here's what the library reference says about file object's
read() method:

read([size])

Read at most size bytes from the file (less if the read
hits EOF before obtaining size bytes). If the size argument
is negative or omitted, read all data until EOF is reached.
The bytes are returned as a string object. An empty string
is returned when EOF is encountered immediately. (For
certain files, like ttys, it makes sense to continue
reading after an EOF is hit.) Note that this method may
call the underlying C function fread() more than once in an
effort to acquire as close to size bytes as possible. Also
note that when in non-blocking mode, less data than what
was requested may be returned, even if no size parameter
was given.

There appear to be a couple problems with this description:

1) It says that read() in blocking mode without a size
parameter it will read until EOF. This is not what happens
when reading a terminal that receives SIGWINCH, so you're
right: read() it isn't working as described.

2) It also says that it makes sense to continue to read a tty
after you get an EOF. That's not true. Once you get an
EOF on a tty, there's no point in reading it any more:
you'll continue to get an EOF forever.
It looks to me as if sys.stderr.read() really gets an EOF at
the final linebreak fed into the terminal prior to window size
change, because the final unterminated line shows up on my
shell prompt.
Python's read() (which seems to call fread()) didn't get an EOF
from fread(), it got an error from fread() because the C read()
call that was made by fread() returned an error because it was
interrupted by a signal.

In Python, EOF is when read() returns '' (the empty string).
Like so:

$ python winch.py
moo moo
cow cowmoo moo
$ cow cow

In this example I type moo moo[ENTER]cow cow on my keyboard
and then resize the window.

Now that EOF has to come from somewhere
There was no EOF. read() returned because of the signal, not
because of an EOF.
(since there's no
IOError or other exception, or the program wouldn't terminate
nicely with nothing on stderr) and I'd like to point the blame
at the terminal. Or is there something really fishy inside
sys.stdin.read() or signal that puts EOFs into streams?

But anyway, as long as this behavior only shows up on
interactive operation, the user will likely spot it anyway and
can react to it. So I think this code would be pretty safe to
use. What do you think?

Resign the terminal will abort pending I/O operations on
that terminal. It won't terminal I/O operations pending on
other devices/files.

What do you mean by "abort"? I can accept that "aborting" may
lead to raising of IOError (which we can catch and retry), but
not to arbitrary insertion of EOFs into streams (which we
cannot distinguish from the real deal coming from the user).

It didn't insert an EOF, it just caused read() to return
"prematurely". You should call read() again until it receives
a _real_ EOF and returns ''. Take another look at my example:

#!/usr/bin/python
import signal, os, sys

_bTerminalSizeChanged = False

def report_terminal_size_change(signum, frame):
global _bTerminalSizeChanged
_bTerminalSizeChanged = True

signal.signal(signal.SIGWINCH, report_terminal_size_change)

while True:
try:
s = sys.stdin.read()
if not s:
sys.stderr.write("EOF\n")
break
sys.stdout.write(s)
except IOError:
sys.stderr.write("IOError\n")
if _bTerminalSizeChanged:
sys.stderr.write("SIGWINCH\n")
_bTerminalSizeChanged = False

--
Grant Edwards grante Yow! Yow! It's a hole
at all the way to downtown
visi.com Burbank!

[Joel Hedlund]

> It didn't insert an EOF, it just caused read() to return

"prematurely". You should call read() again until it receives
a _real_ EOF and returns ''.
Copy that. Point taken.
There appear to be a couple problems with this description:

1) It says that read() in blocking mode without a size
parameter it will read until EOF. This is not what happens
when reading a terminal that receives SIGWINCH, so you're
right: read() it isn't working as described.

2) It also says that it makes sense to continue to read a tty
after you get an EOF. That's not true. Once you get an
EOF on a tty, there's no point in reading it any more:
you'll continue to get an EOF forever.

Should I post a bug about this?

/Joel

[Joel Hedlund]

I realised there was funny indentation mishap in my previus email. My mail prog indented the following line

signal.signal(signal.SIGWINCH, report_terminal_size_change)

while in fact in IDLE it's unindented, which makes a lot more sense:

signal.signal(signal.SIGWINCH, report_terminal_size_change)

Sorry about that.

/Joel

Joel Hedlund wrote:

You might want to try just setting a flag in the signal handler
to see if that prevents the I/O operations on stdin/stdout from
being interrupted.

Tried this:

<source>

import signal, os, sys
from terminal_info import get_terminal_size

terminal_size = get_terminal_size()

_bTerminalSizeChanged = False

def report_terminal_size_change(signum, frame):
global _bTerminalSizeChanged
_bTerminalSizeChanged = True

def update_terminal_size():
global _bTerminalSizeChanged, terminal_size
terminal_size = get_terminal_size()
_bTerminalSizeChanged = False
signal.signal(signal.SIGWINCH, report_terminal_size_change)

while True:
# Do lots of IO (I'm trying to provoke exceptions with signal)
open('/a/large/file').read()
#raw_input()
#sys.stdin.read()
#print open('/a/large/file').read()
if _bTerminalSizeChanged:
update_terminal_size()
print terminal_size

</source>

As before, the only IO case above that doesn't throw exceptions is the
uncommented one.

Yup, that's the exception. Standard practice is to catch it and
retry the I/O operation.

Hmm... I guess it's not that easy to "retry" IO operations on pipes and
streams (stdin/stdout in this case)... And I tend to lean pretty heavily
on those since I usually write UNIX style text "filters".

So in case I haven't missed something fundamental I guess my best option
is to accept defeat (of sorts :-) and be happy with picking a terminal
width at program startup.

But anyway, it's been really interesting trying this out.
Thank you Grant (och Jorgen) for all help and tips!
/Joel

[Grant Edwards]

On 2006-02-09, Joel Hedlund <jo**********@gmail.com> wrote:

It didn't insert an EOF, it just caused read() to return
"prematurely". You should call read() again until it receives
a _real_ EOF and returns ''.
Copy that. Point taken.

There appear to be a couple problems with this description:

1) It says that read() in blocking mode without a size
parameter it will read until EOF. This is not what happens
when reading a terminal that receives SIGWINCH, so you're
right: read() it isn't working as described.

2) It also says that it makes sense to continue to read a tty
after you get an EOF. That's not true. Once you get an
EOF on a tty, there's no point in reading it any more:
you'll continue to get an EOF forever.

I should qualify comment 2) with the disclaimer that that's how
tty devices on Unix/Linux work -- I don't know about windows.
Should I post a bug about this?

Sure. I was going to do that yesterday, but I realized that I
didn't know how/where to do it. I assume there's a link
somewhere at www.python.org, but I haven't had a chance to look
yet.

--
Grant Edwards grante Yow! I wish I was on a
at Cincinnati street corner
visi.com holding a clean dog!

[Donn Cave]

In article <11*************@corp.supernews.com>,
Grant Edwards <gr****@visi.com> wrote:

2) It also says that it makes sense to continue to read a tty
after you get an EOF. That's not true. Once you get an
EOF on a tty, there's no point in reading it any more:
you'll continue to get an EOF forever.

They were probably thinking of the way the UNIX tty
driver delivers an EOF on <ctrl>D, after which of
course you can continue to read data from the same tty.

Donn Cave, do**@u.washington.edu

[Joel Hedlund]

> Sure. I was going to do that yesterday, but I realized that I

didn't know how/where to do it. I assume there's a link
somewhere at www.python.org, but I haven't had a chance to look
yet.

It's already reported to the bug tracker:

http://www.python.org/sf/210599

Apparently, this has been around since 1.5.2. It's in fact also in the feature request PEP right now:

http://www.python.org/peps/pep-0042.html

and it's listed as a "Big" project under the "Standard library" heading. I assume we're going to have to live with this for a while yet...

Take care everyone!
/Joel

[Grant Edwards]

On 2006-02-15, Joel Hedlund <jo**********@gmail.com> wrote:

Sure. I was going to do that yesterday, but I realized that I
didn't know how/where to do it. I assume there's a link
somewhere at www.python.org, but I haven't had a chance to look
yet.

It's already reported to the bug tracker:

http://www.python.org/sf/210599

Apparently, this has been around since 1.5.2. It's in fact
also in the feature request PEP right now:

http://www.python.org/peps/pep-0042.html

and it's listed as a "Big" project under the "Standard
library" heading. I assume we're going to have to live with
this for a while yet...

Oh.

I've got no complaints about the way things work, I was just
going to suggest changing the documentation to correctly
describe the way things work. Too mundane a solution?

It seems a bit odd that we've known the documentation is wrong
for 6 years and nobody's bothered to fix it. Or is the
documentation describing how we wish things would work
regardless of whether they work that way or not?

--
Grant Edwards grante Yow! It's today's SPECIAL!
at
visi.com

[Kent Johnson]

Grant Edwards wrote:

I've got no complaints about the way things work, I was just
going to suggest changing the documentation to correctly
describe the way things work. Too mundane a solution?

Submit a patch, it's quick and easy and fun:
http://docs.python.org/about.html

Kent

[Grant Edwards]

On 2006-02-15, Kent Johnson <ke**@kentsjohnson.com> wrote:

Grant Edwards wrote:

I've got no complaints about the way things work, I was just
going to suggest changing the documentation to correctly
describe the way things work. Too mundane a solution?

Submit a patch, it's quick and easy and fun:
http://docs.python.org/about.html

Done.

--
Grant Edwards grante Yow! We just joined the
at civil hair patrol!
visi.com