Re: Script using generators produces different results when invokedas a CGI

[Gabriel Genellina]

En Mon, 05 May 2008 00:31:45 -0300, Barclay, Ken <Ke*********@sc hwab.comescribi ó:

I attended David Beazley's awe-inspiring tutorial on the use of
generators in systems programming:
http://www.dabeaz.com/generators/
<BLOCKED::htt p://www.dabeaz.com/generators/>
I used his approach to write a web tool that can display search results
from different log files. But the resulting script produced fewer
results when invoked as CGI than it did when run from the command line,
and I can't figure out why.

Problem: For small sets of files this works great. But when I had 19Meg
worth of log files in a test directory, the script would return the
correct number of matching lines (288) only when it was invoked directly
from the command line. When invoked from a CGI script, it returns 220
lines instead (written to the "tempfile", below.) I don't know where
that limit is coming from. If more logs are added to the test directory,
the result is always the same 220 lines.
I'm using Python 2.5.1 on Red Hat Linux 3.2.3-47. Below is the whole
script I was testing with. It's using hard-coded values in place of ones
I'll be getting from an HTML form (generated with HTMLgen) presented to
the user.
There are no exceptions or errors of any kind. Any pointers on what
might be happening here would be welcome!

No entry in the error log, on the web server? (Apache?)
Perhaps you're hitting some internal limit of the cgi process, either memory or cpu time per request or temp file size...
Are you sure the script runs to completion? Output a message at the end, to be sure.

--
Gabriel Genellina

[Nick Craig-Wood]

Gabriel Genellina <ga*******@yaho o.com.arwrote:

En Mon, 05 May 2008 00:31:45 -0300, Barclay, Ken <Ke*********@sc hwab.comescribi ó:

I attended David Beazley's awe-inspiring tutorial on the use of
generators in systems programming:
http://www.dabeaz.com/generators/
<BLOCKED::htt p://www.dabeaz.com/generators/>
I used his approach to write a web tool that can display search results
from different log files. But the resulting script produced fewer
results when invoked as CGI than it did when run from the command line,
and I can't figure out why.

Problem: For small sets of files this works great. But when I had 19Meg
worth of log files in a test directory, the script would return the
correct number of matching lines (288) only when it was invoked directly
from the command line. When invoked from a CGI script, it returns 220
lines instead

No entry in the error log, on the web server? (Apache?)
Perhaps you're hitting some internal limit of the cgi process, either memory or cpu time per request or temp file size...
Are you sure the script runs to completion? Output a message at the
end, to be sure.

Check the ownership of all the files too. Remember that the web
server (and hence your cgi) will likely run as nobody or www-data.
You are unlikely to be logging in as one of those users.
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick