"Compile time" checking?

[Qopit]

Hi there,

I'm pretty new to Python and am trying to figure out how to get "will
this code compile?"-like code checking. To me this is a pretty basic
language/environment requirement, especially when working with large
projects. It is *much* better to catch errors at "compile-time" rather
than at run-time.

One thing I've "found" is the PyChecker module (conveniently embedded
in SPE), but it doesn't seem to do that great of a job. For example,
the following simple program checks out perfectly as far as PyChecker
is concerned:

#----
def tester(a,b,c):
print "bogus test function",a,b,c
tester(1,2,3) #this runs fine
tester(1,2) #this obviously causes a run-time TypeError exception
#----

It seems to me that this should be an obvious catch for PyChecker. I
suppose you could argue that you don't want PyChecker to bark at you
any time an exception would be raised since you may intentionally be
causing exceptions, but this one seems a pretty simple and obvious one
to catch.

My questions are:
- Am I missing something with my tester example?
- Are there other code-checking options other than PyChecker?

Any other comments appreciated (aside from things like "just right good
code that doesn't have bugs like that" :) ).

Thanks!

[Benjamin Niemann]

Qopit wrote:

[snip]

My questions are:
- Am I missing something with my tester example?
- Are there other code-checking options other than PyChecker?

Try pylint

--
Benjamin Niemann
Email: pink at odahoda dot de
WWW: http://www.odahoda.de/

[phil hunt]

On 10 Aug 2005 08:53:15 -0700, Qopit <ru************@gmail.com> wrote:

Hi there,

I'm pretty new to Python and am trying to figure out how to get "will
this code compile?"-like code checking.

Why not just find out, by trying to compile it? :-)
--
Email: zen19725 at zen dot co dot uk

On 10 Aug 2005 08:53:15 -0700
Qopit wrote:

def tester(a,b,c):
print "bogus test function",a,b,c
tester(1,2,3) #this runs fine
tester(1,2) #this obviously causes a run-time TypeError exception

/tmp% cat >a.py
def tester(a,b,c):
print "bogus test function",a,b,c
tester(1,2,3) #this runs fine
tester(1,2) #this obviously causes a run-time TypeError exception
/tmp% pychecker a.py
Processing a...
bogus test function 1 2 3
Caught exception importing module a:
File "/usr/lib/site-python/pychecker/checker.py", line 587, in setupMainCode()
module = imp.load_module(self.moduleName, file, filename, smt)
File "a.py", line 4
tester(1,2) #this obviously causes a run-time TypeError exception
TypeError: tester() takes exactly 3 arguments (2 given)

Warnings...

a:1: NOT PROCESSED UNABLE TO IMPORT
/tmp% pychecker -V
0.8.14

--
jk

[Qopit]

> Why not just find out, by trying to compile it? :-)

This will likely certify me as a python newbie, but... how do you mean?
How do you compile a .py file?

If you mean to .pyc by doing an import on it, that may work fine for
the simple example I typed up earlier, but that is easy to bypass by
slapping the offending line in a function. The sample below also
passes PyChecker with not even a warning:

#----
def tester(a,b,c):
print "bogus test function",a,b,c

def try1():
tester(1,2,3)
def try2():
tester(1,2) #still no error here
#----

Do you mean something different?

Also - thanks for the pylint comment... haven't tried it yet. It would
be nice to have the capability in an IDE like SPE, though.

[Qopit]

How embarassing... thanks, jk. I grabbed a copy of pychecker v0.8.14
directly (not the one in SPE) and it catches it exactly as you showed.
Now I wonder why the SPE one doesn't catch it (and why it is sooo
comparatively slow)!

Now I'm running into another snag when checking some other code I have.
Pychecker gets hung up on raw_input... it actually executes code
rather than just checking it, it seems. For example, the snippet below
hangs pychecker::

#---
while 1:
x = raw_input("meh:")
#---

Curious.

I'm going to look into some of the code checking capabilities (if
present) of Komodo and Wing. Anyone familiar enough with their ability
to comment?

[Marc 'BlackJack' Rintsch]

In <11**********************@z14g2000cwz.googlegroups .com>, Qopit wrote:

Hi there,

I'm pretty new to Python and am trying to figure out how to get "will
this code compile?"-like code checking. To me this is a pretty basic
language/environment requirement, especially when working with large
projects. It is *much* better to catch errors at "compile-time" rather
than at run-time.
Might sound harsh, but then python ist the wrong language for you.
One thing I've "found" is the PyChecker module (conveniently embedded
in SPE), but it doesn't seem to do that great of a job. For example,
the following simple program checks out perfectly as far as PyChecker
is concerned:

#----
def tester(a,b,c):
print "bogus test function",a,b,c
tester(1,2,3) #this runs fine
tester(1,2) #this obviously causes a run-time TypeError exception
#----

It seems to me that this should be an obvious catch for PyChecker.
It's just obviuos because you know that the first call to `tester()`
doesn't change the name binding for `tester` to a callable object that
also accepts just two parameters. You, as a human, can see easily the
error here, but a program has to follow all possible control flows to be
sure about that. And that's most of the time very complex and sometimes
impossible. It's not enough to look just at the signature of the function.

Simple (?) test::

def tester(a, b, c):
global tester
print "bogus test function", a, b, c
def tester(a, b):
print "other test function", a, b

tester(1, 2, 3) # This runs fine.
tester(1, 2) # This too.
Any other comments appreciated (aside from things like "just right good
code that doesn't have bugs like that" :) ).

Compile it by running it and write unit tests.

Ciao,
Marc 'BlackJack' Rintsch

[Marc 'BlackJack' Rintsch]

In <11**********************@g47g2000cwa.googlegroups .com>, Qopit wrote:

Now I'm running into another snag when checking some other code I have.
Pychecker gets hung up on raw_input... it actually executes code
rather than just checking it, it seems. For example, the snippet below
hangs pychecker::

#---
while 1:
x = raw_input("meh:")
#---

Curious.

AFAIK `pylint` tries to avoid running the code.

A common idiom to protect code from being executed, if the module is just
imported opposed to executed as main module, is to write::

def foo():
# some code

if __name__ == '__main__':
foo()

`__name__` is set to the module name if the module will be imported but to
'__main__' if the module is directly executed.

Ciao,
Marc 'BlackJack' Rintsch

[Qopit]

> def tester(a, b, c):

global tester
print "bogus test function", a, b, c
def tester(a, b):
print "other test function", a, b

tester(1, 2, 3) # This runs fine.
tester(1, 2) # This too.
Interesting example. In that case, pychecker does spit out a warning
since it has trouble deciphering the redefinition. I have no problem
whatsoever with a compiler/code-checker getting confused in such an
oddball situation. As you say, it is difficult for an automated
process to follow such flows. A warning is fine here (as I got with
the "proper" pychecker on my initial example - it did easily catch what
I thought should have been, and was, obvious).

With your example, I was curious how pychecker would deal with it if
you altered the flow a bit so that all calls would/should make sense in
what seems to me to be logical locals order, and tried this:

#---
def tester(a, b, c):
global tester
print "bogus test function", a, b, c
def tester(a, b):
print "other test function", a, b
tester(1, 2) #no pychecker complaint since local
tester(1, 2, 3) # pychecker complains here (?)
#---

I'm a bit confused why pychecker complained where it did - I don't get
how it got the 2 arg version at that point, but I actually don't really
care that much due to the weirdness level of this code. A compiler (or
code-checker) warning on this code is perfectly acceptable to me. I'm
a big fan of Python's ability to easily rebind everything in sight, but
this particular usage seems like a strange abuse I wouldn't expect a
code-checker to be able to figure out. I'll just avoid writing
confusing code like that... it's not only confusing to a program, but
to a human as well! Dynamically massacring a function definition (as
opposed to just rebinding a new implementation) like that seems odd to
me.
Compile it by running it and write unit tests.

.... sure, that works, I'm just used to the integrated tools I've had
available to me for the last 15 years to help me write more robust code
waaaay faster than having to unit test a zillion blocks of code when
you change a single definition somewhere.

PyChecker seems like it may fit the bill right now... just need to try
it some more and figure out how to get around that weird raw_input
thing. The basis for my first post was a jerk what-the-heck reaction
to the fact that it seemed that pychecker didn't get the simple arg
count mismatch error, but jk showed that that was wrong and I just have
to sort out something with SPE.

Cheers,
Russ

[Qopit]

> if __name__ == '__main__':

Yep - that does it... should have thought of that. Thanks.

This works fine for pychecker with no hangage:
#---
if __name__ == "__main__":
while 1:
x = raw_input("meh:")
#---

[phil hunt]

On 10 Aug 2005 12:01:01 -0700, Qopit <ru************@gmail.com> wrote:

Why not just find out, by trying to compile it? :-)
This will likely certify me as a python newbie, but... how do you mean?
How do you compile a .py file?

At the command prompt:

$ python yourfile.py

This compiles it, then runs it.
If you mean to .pyc by doing an import on it,
Indeed so.
that may work fine for
the simple example I typed up earlier, but that is easy to bypass by
slapping the offending line in a function. The sample below also
passes PyChecker with not even a warning:

#----
def tester(a,b,c):
print "bogus test function",a,b,c

def try1():
tester(1,2,3)
def try2():
tester(1,2) #still no error here
#----

Do you mean something different?

I've never used PyChecker myself, so can't comment on it.

I've not personally had problems with the wrong number of argumnets
to a function call -- they get caught at run-time and are easy
enough to fix -- but I do sometimes get errors because a varialbe is
the wrong time, e.g. a string when it should be an int.

One problem I once encountered was wit this and I waasn't picking it
up because my debugging code looked like this:

if debug: print "v=%s" % (v,)

Which of course prints the same output whether v is '2' or 2.

For this reason I tend to debug print statements like this now:

if debug: print "v=%s" % (v,)
--
Email: zen19725 at zen dot co dot uk

[Qopit]

if debug: print "v=%s" % (v,)

Not that important, but I assume the first one was supposed to be:

if debug: print "v=", s

right?

[Grant Edwards]

On 2005-08-11, Qopit <ru************@gmail.com> wrote:

if debug: print "v=%s" % (v,)

Not that important, but I assume the first one was supposed to be:

if debug: print "v=", s

right?

http://docs.python.org/tut/node9.htm...00000000000000

--
Grant Edwards grante Yow! .. Now KEN and BARBIE
at are PERMANENTLY ADDICTED to
visi.com MIND-ALTERING DRUGS...

[Bengt Richter]

On Wed, 10 Aug 2005 20:39:03 +0100, ze******@zen.co.uk (phil hunt) wrote:
[...]

I've not personally had problems with the wrong number of argumnets
to a function call -- they get caught at run-time and are easy
enough to fix -- but I do sometimes get errors because a varialbe is
the wrong time, e.g. a string when it should be an int.

One problem I once encountered was wit this and I waasn't picking it
up because my debugging code looked like this:

if debug: print "v=%s" % (v,)

Which of course prints the same output whether v is '2' or 2.

For this reason I tend to debug print statements like this now:

if debug: print "v=%s" % (v,)

I usually prefer %r over %s for debug prints

if debug: print "v=%r" % (v,)

since it represents (repr's ;-) the v thing better

print 'v=%r' % 2 v=2 print 'v=%r' % '2'

v='2'

Regards,
Bengt Richter

[Fabio Zadrozny]

Just as a note... Pylint is integrated within pydev (http://pydev.sf.net)

Cheers,

Fabio

Qopit wrote:

Why not just find out, by trying to compile it? :-)

This will likely certify me as a python newbie, but... how do you mean?
How do you compile a .py file?

If you mean to .pyc by doing an import on it, that may work fine for
the simple example I typed up earlier, but that is easy to bypass by
slapping the offending line in a function. The sample below also
passes PyChecker with not even a warning:

#----
def tester(a,b,c):
print "bogus test function",a,b,c

def try1():
tester(1,2,3)
def try2():
tester(1,2) #still no error here
#----

Do you mean something different?

Also - thanks for the pylint comment... haven't tried it yet. It would
be nice to have the capability in an IDE like SPE, though.

--
Fabio Zadrozny
------------------------------------------------------
Software Developer

ESSS - Engineering Simulation and Scientific Software
www.esss.com.br

PyDev - Python Development Enviroment for Eclipse
pydev.sf.net
pydev.blogspot.com

[phil hunt]

On 10 Aug 2005 18:32:54 -0700, Qopit <ru************@gmail.com> wrote:

if debug: print "v=%s" % (v,)

Not that important, but I assume the first one was supposed to be:

if debug: print "v=", s

right?

No, I'm trying to print (v) not (s).
--
Email: zen19725 at zen dot co dot uk

[phil hunt]

On Thu, 11 Aug 2005 02:35:40 GMT, Bengt Richter <bo**@oz.net> wrote:

On Wed, 10 Aug 2005 20:39:03 +0100, ze******@zen.co.uk (phil hunt) wrote:
[...]

I've not personally had problems with the wrong number of argumnets
to a function call -- they get caught at run-time and are easy
enough to fix -- but I do sometimes get errors because a varialbe is
the wrong time, e.g. a string when it should be an int.

One problem I once encountered was wit this and I waasn't picking it
up because my debugging code looked like this:

if debug: print "v=%s" % (v,)

Which of course prints the same output whether v is '2' or 2.

For this reason I tend to debug print statements like this now:

if debug: print "v=%s" % (v,)

I usually prefer %r over %s for debug prints

if debug: print "v=%r" % (v,)

Arrghh!! Yes, that's what I meant to write.
--
Email: zen19725 at zen dot co dot uk

[Marc 'BlackJack' Rintsch]

In <11**********************@g43g2000cwa.googlegroups .com>, Qopit wrote:

I'm
a big fan of Python's ability to easily rebind everything in sight, but
this particular usage seems like a strange abuse I wouldn't expect a
code-checker to be able to figure out. I'll just avoid writing
confusing code like that... it's not only confusing to a program, but
to a human as well! Dynamically massacring a function definition (as
opposed to just rebinding a new implementation) like that seems odd to
me.

Well it's a contrived example. But taking a function or method and wrap
it with some other function or method isn't that uncommon. For methods
there is even syntactic sugar: decorator syntax.

Here's a more useful dynamic function wrapping::

def debug_trace(func):
def wrapper(*args, **kwargs):
print 'Calling %s' % func.__name__
print 'args: %r' % (args,)
print 'kwargs: %r' % kwargs
func(*args, **kwargs)
return wrapper

def test(a, b, c):
print 'just a test', a, b, b

test = debug_trace(test)

test(1, 2, 3)
test(1, 2)

Here it's quite clear to a human reader that the wrapping doesn't change
the number of arguments. But it's harder to let a program figure this out.

Ciao,
Marc 'BlackJack' Rintsch

[spe.stani.be]

Hi,

If you find something like that, please report it to the bug tracker of
SPE with an easy example. Also mention that PyChecker is slow, I might
have another look at it.

Probably I need to update the version, as SPE ships with the 0.8.13
version. I don't think it's possible to get it already in the next
release as now SVN is frozen for new features, as I probably will
release sunday the new SPE. The screenshots are already updated.

BTW, maybe a time for a poll: which checker is better? PyChecker or
PyLint?

Stani

http://pythonide.stani.be (mind the new url!)
http://pythonide.stani.be/screenshots
http://developer.berlios.de/bugs/?group_id=4161 (SPE bugtracker)

[Steve Jorgensen]

Since Python does not use manifest typing, there's not much you can do about
this, but typeless languages like this are great if you're using a process
that finds the errors the compiler would otherwise find. I'm referring, of
course, to Test Driven Development (TDD).

If you do TDD, you won't miss compile-time checking much. In fact, the extra
kruft that manifest typing requires is an annoying burden when doing TDD, so
Python is a breath of fresh air in this regard.

On 10 Aug 2005 08:53:15 -0700, "Qopit" <ru************@gmail.com> wrote:

Hi there,

I'm pretty new to Python and am trying to figure out how to get "will
this code compile?"-like code checking. To me this is a pretty basic
language/environment requirement, especially when working with large
projects. It is *much* better to catch errors at "compile-time" rather
than at run-time.

One thing I've "found" is the PyChecker module (conveniently embedded
in SPE), but it doesn't seem to do that great of a job. For example,
the following simple program checks out perfectly as far as PyChecker
is concerned:

#----
def tester(a,b,c):
print "bogus test function",a,b,c
tester(1,2,3) #this runs fine
tester(1,2) #this obviously causes a run-time TypeError exception
#----

It seems to me that this should be an obvious catch for PyChecker. I
suppose you could argue that you don't want PyChecker to bark at you
any time an exception would be raised since you may intentionally be
causing exceptions, but this one seems a pretty simple and obvious one
to catch.

My questions are:
- Am I missing something with my tester example?
- Are there other code-checking options other than PyChecker?

Any other comments appreciated (aside from things like "just right good
code that doesn't have bugs like that" :) ).

Thanks!

On Fri, 12 Aug 2005 22:25:07 -0700
Steve Jorgensen wrote:

Since Python does not use manifest typing, there's not much you can do about
this, but typeless languages like this are great if you're using a process
that finds the errors the compiler would otherwise find. I'm referring, of
course, to Test Driven Development (TDD).

If you do TDD, you won't miss compile-time checking much. In fact, the extra
kruft that manifest typing requires is an annoying burden when doing TDD, so
Python is a breath of fresh air in this regard.
What test should one implement to catch that kind of errors like in OP
example?
On 10 Aug 2005 08:53:15 -0700, "Qopit" <ru************@gmail.com> wrote:

#----
def tester(a,b,c):
print "bogus test function",a,b,c
tester(1,2,3) #this runs fine
tester(1,2) #this obviously causes a run-time TypeError exception
#----

--
jk

[Steve Jorgensen]

On Sat, 13 Aug 2005 11:04:55 +0400, <en**********@ospaz.ru> wrote:

On Fri, 12 Aug 2005 22:25:07 -0700
Steve Jorgensen wrote:

Since Python does not use manifest typing, there's not much you can do about
this, but typeless languages like this are great if you're using a process
that finds the errors the compiler would otherwise find. I'm referring, of
course, to Test Driven Development (TDD).

If you do TDD, you won't miss compile-time checking much. In fact, the extra
kruft that manifest typing requires is an annoying burden when doing TDD, so
Python is a breath of fresh air in this regard.

What test should one implement to catch that kind of errors like in OP
example?

On 10 Aug 2005 08:53:15 -0700, "Qopit" <ru************@gmail.com> wrote:

>#----
>def tester(a,b,c):
> print "bogus test function",a,b,c
>tester(1,2,3) #this runs fine
>tester(1,2) #this obviously causes a run-time TypeError exception
>#----

None - other tests you would write would fail if other code in your system is
trying to call a procedure with the wrong number of parameters.

[Benjamin Niemann]

en**********@ospaz.ru wrote:

On Fri, 12 Aug 2005 22:25:07 -0700
Steve Jorgensen wrote:

Since Python does not use manifest typing, there's not much you can do
about this, but typeless languages like this are great if you're using a
process
that finds the errors the compiler would otherwise find. I'm referring,
of course, to Test Driven Development (TDD).

If you do TDD, you won't miss compile-time checking much. In fact, the
extra kruft that manifest typing requires is an annoying burden when
doing TDD, so Python is a breath of fresh air in this regard.

What test should one implement to catch that kind of errors like in OP
example?

A 'good' testsuite should run each line of the code at least once. There's
more to do for a good code coverage, but that's the most basic requirement.
So the faulty line will be executed by the testsuite and the runtime
checking will raise an exception.

Static typing is a very rudimentary type of code checking that is performed
by the compiler with 100% code coverage. Such test won't tell you, if your
code is actually doing what it is supposed to do (or anything sensible at
all) and are mostly useless. If you think that code quality is important,
you'll have to perform much more and more sophisticated tests. Such a
testsuite is usually (I can't think of cases where it is not) a superset of
static code checking - all type bugs are found be the testsuite even in the
absence if static type checking.

So you won't loose anything, if you drop static typing.

--
Benjamin Niemann
Email: pink at odahoda dot de
WWW: http://www.odahoda.de/