inspect.findsource problem with llinecache

[Rafe]

Hi,

I think I have discovered two bugs with the inspect module and I would
like to know if anyone can spot any traps in my workaround.

I needed a function which takes a function or method and returns the
code inside it (it also adjusts the indent because I have to paste the
to a text string without indents, which is parsed later...long story).
At first this seemed trivial but then I started getting an error on
line 510 of inspect.py:
504 if iscode(object):
505 if not hasattr(object, 'co_firstlineno'):
506 raise IOError('could not find function definition')
507 lnum = object.co_firstlineno - 1
508 pat = re.compile(r'^(\s*def\s)|(.*(?<!\w)lambda(:|\s))|^
(\s*@)')
509 while lnum 0:
510 if pat.match(lines[lnum]): break
511 lnum = lnum - 1
512 return lines, lnum

I finally figured out that there was a caching problem. The function I
passed was changed, but the code lines (strings) retrieved by
linecache.getlines() (on lines 464 and 466) didn't update with the new
module contents. The resulting error I was getting occurred when real
module had less lines than the cached module (or the other way around,
whatever.)

To get around this, I invoke linecache.clearcache(). Here is the
function (minus doc strings)...

INDENT_SPACES = 4

def get_fn_contents(fn, remove_indents=1):
# Clear the cache so inspect.getsourcelines doesn't try to
# check an older version of the function's module.
linecache.clearcache()
source_lines, n = inspect.getsourcelines(fn)

# Skip the first line which contains the function definition.
# Only want the code inside the function is needed.
fn_contents = source_lines[1:]

# Remove indents
new_indent_lines = [remove_indent(line, remove_indents) for line
in fn_contents]

return "".join(new_indent_lines)
def remove_indent(in_str, num_indent=1):
s = in_str
for i in range(num_indent):
if s[:INDENT_SPACES] == " ": # Whitespace indents
s = s[INDENT_SPACES:]
elif s[:1] == "\t": # Tab characters indents
s = s[1:]

return s

[END CODE]

The second issue is that the last line in the passed function's module
seems to be ignored. So, if the last line of the module is also the
last line of the function, the function is returned one line too
short.

I welcome comments on the bugs or optimization pointers for my code. I
am still quite new to Python. My primary question though is: will
linecache.clearcache() cause me any problems?

Thanks,

- Rafe

[Rafe]

On Nov 12, 2:22*pm, Rafe <[e-mail]:

Hi,

I think I have discovered two bugs with the inspect module and I would
like to know if anyone can spot any traps in my workaround.

I needed a function which takes a function or method and returns the
code inside it (it also adjusts the indent because I have to paste the
to a text string without indents, which is parsed later...long story).
At first this seemed trivial but then I started getting an error on
line 510 of inspect.py:
504 * *if iscode(object):
505 * * * *if not hasattr(object, 'co_firstlineno'):
506 * * * * * *raise IOError('could not find function definition')
507 * * * *lnum = object.co_firstlineno - 1
508 * * * *pat = re.compile(r'^(\s*def\s)|(.*(?<!\w)lambda(:|\s))|^
(\s*@)')
509 * * * *while lnum 0:
510 * * * * * *if pat.match(lines[lnum]): break
511 * * * * * *lnum = lnum - 1
512 * * * *return lines, lnum

I finally figured out that there was a caching problem. The function I
passed was changed, but the code lines (strings) retrieved by
linecache.getlines() (on lines 464 and 466) didn't update with the new
module contents. The resulting error I was getting occurred when real
module had less lines than the cached module (or the other way around,
whatever.)

To get around this, I invoke linecache.clearcache(). Here is the
function (minus doc strings)...

INDENT_SPACES = 4

def get_fn_contents(fn, remove_indents=1):
* * # Clear the cache so inspect.getsourcelines doesn't try to
* * # check an older version of the function's module.
* * linecache.clearcache()
* * source_lines, n = inspect.getsourcelines(fn)

* * # Skip the first line which contains the function definition.
* * # Only want the code inside the function is needed.
* * fn_contents = source_lines[1:]

* * # Remove indents
* * new_indent_lines = [remove_indent(line, remove_indents) for line
in fn_contents]

* * return "".join(new_indent_lines)

def remove_indent(in_str, num_indent=1):
* * s = in_str
* * for i in range(num_indent):
* * * * if s[:INDENT_SPACES] == " * *": * # Whitespace indents
* * * * * * s = s[INDENT_SPACES:]
* * * * elif s[:1] == "\t": * # Tab characters indents
* * * * * * *s = s[1:]

* * return s

[END CODE]

The second issue is that the last line in the passed function's module
seems to be ignored. So, if the last line of the module is also the
last line of the function, the function is returned one line too
short.

I welcome comments on the bugs or optimization pointers for my code. I
am still quite new to Python. My primary question though is: will
linecache.clearcache() cause me any problems?

Thanks,

- Rafe

I forgot to add that while inspect uses the cached module to get the
text, the line number used to find the block of source lines is
retrieved from the passed object. So, if I pass a function which
reports that it starts on line 50, but in the cache it starts on line
40 an error isn't raised but the lines of code returned are wrong. The
error only occurs when the line number is higher than the number of
lines in the cached module.

- Rafe

[Gabriel Genellina]

En Wed, 12 Nov 2008 05:22:55 -0200, Rafe <ra*******@gmail.comescribió:

I think I have discovered two bugs with the inspect module and I would
like to know if anyone can spot any traps in my workaround.

They look like real bugs - please report them at http://bugs.python.org
else this will be forgotten.

I welcome comments on the bugs or optimization pointers for my code. I
am still quite new to Python. My primary question though is: will
linecache.clearcache() cause me any problems?

I don't think so, apart from performance degradation (but good performance
with bad results isn't good at all!)

--
Gabriel Genellina

[Rafe]

On Nov 15, 1:29*pm, "Gabriel Genellina" <[e-mail]>
wrote:

En Wed, 12 Nov 2008 05:22:55 -0200,Rafe<[e-mail]ó:

I think I have discovered two bugs with the inspect module and I would
like to know if anyone can spot any traps in my workaround.

They look like real bugs - please report them athttp://bugs.python.org*
else this will be forgotten.

I welcome comments on the bugs or optimization pointers for my code. I
am still quite new to Python. My primary question though is: will
linecache.clearcache() cause me any problems?

I don't think so, apart from performance degradation (but good performance *
with bad results isn't good at all!)

--
Gabriel Genellina

Thank you for replying Gabriel. I'll report it now.

- Rafe