473,837 Members | 1,541 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Automatic debugging of copy by reference errors?

Is there a module that allows me to find errors that occur due to copy
by reference? I am looking for something like the following:
>>import mydebug
mydebug.check copybyreference = True
a=2
b=[a]
a=4
Traceback (most recent call last):
File "<stdin>", line 1, in ?
CopyByReference Error: Variable b refers to variable a, so please do not
change variable a.

Does such a module exist?
Would it be possible to code such a module?
Would it be possible to add the functionality to array-copying in
numpy?
What would be the extra cost in terms of memory and CPU power?

I realize that this module would disable some useful features of the
language. On the other hand it could be helpful for new python users.

Niels

Dec 9 '06
16 1190
Niels L Ellegaard wrote:
Is there a module that allows me to find errors that occur due to copy
by reference? I am looking for something like the following:
I don't have a solution for you, but I certainly appreciate your
concern. The copy by reference semantics of Python give it great
efficiency but are also its achille's heel for tough-to-find bugs.

I once wrote a simple "vector" class in Python for doing basic vector
operations such as adding and subtracting vectors and multiplying them
by a scalar. (I realize that good solutions exist for that problem, but
I was more or less just experimenting.) The constructor took a list as
an argument to initialize the vector. I later discovered that a
particularly nasty bug was due to the fact that my constructor "copied"
the initializing list by reference. When I made the constructor
actually make its own copy using "deepcopy," the bug was fixed. But
deecopy is less efficient than copy by reference, of course.

So a fundamental question in Python, it seems to me, is when to take
the performance hit and use "copy" or "deepcopy."

If a debugger could tell you how many references exist to an object,
that would be helpful. For all I know, maybe some of them already do. I
confess I don't use debuggers as much as I should.

Dec 11 '06 #11
Russ wrote:
If a debugger could tell you how many references exist to an object,
that would be helpful.
import sys
sys.getrefcount (a)

But I doubt it would be very helpful.

Carl Banks

Dec 11 '06 #12

Niels L Ellegaard wrote:
Marc 'BlackJack' Rintsch wrote:
In <11************ **********@n67g 2000cwd.googleg roups.com>, Niels L
Ellegaard wrote:
I have been using scipy for some time now, but in the beginning I made
a few mistakes with copying by reference.
But "copying by reference" is the way Python works. Python never copies
objects unless you explicitly ask for it. So what you want is a warning
for *every* assignment.

Maybe I am on the wrong track here, but just to clarify myself:

I wanted a each object to know whether or not it was being referred to
by a living object, and I wanted to warn the user whenever he tried to
change an object that was being refered to by a living object.
This really wouldn't work, trust us. Objects do not know who
references them, and are not notified when bound to a symbol or added
to a container. However, I do think you're right about one thing: it
would be nice to have a tool that can catch errors of this sort, even
if it's imperfect (as it must be).

ISTM the big catch for Fortran programmers is when a mutable container
is referenced from multiple places; thus a change via one reference
will confusingly show up via the other one. (This, of course, is
something that happens all the time in Python, but it's not something
people writing glue for numerical code need to do a lot.) Instead of
the interpreter doing it automatically, it might be better (read:
possible) to use an explicit check. Here is a simple, untested
example:
class DuplicateRefere nce(Exception):
pass

def _visit_containe r(var,cset):
if id(var) in cset:
raise DuplicateRefere nce("container referenced twice")
cset.add(id(var ))

def assert_no_dupli cate_container_ refs(datalist,c set=None):
if cset is None:
cset = set()
for var in data:
if isinstance(var, (list,set)):
_visit_containe r(var,cset)
assert_no_dupli cate_container_ refs(var,cset)
elif isinstance(var, dict):
_visit_containe r(var,cset)
assert_no_dupli cate_container_ refs(var.iterva lues(),cset)
Then, at important points along the way, call this function with data
you provide yourself. For example, we might modify your example as
follows:

import os
output=[]
firstlines =[0,0]
for filename in os.listdir('.') :
try:
firstlines[0] = open(filename," r").readline s()[0]
firstlines[1] = open(filename," r").readline s()[1]
output.append(( filename,firstl ines))
except (IOError, IndexError):
# please don't use bare except unless reraising exception
continue
assert_no_dupli cate_container_ refs([output,firstlin es])
print output
We passed the function output and firstlines because those are the two
names you defined yourself. It'll check to see if any containers are
referenced more than once. And it turns out there are--firstlines is
referenced many times. It'll raise DuplicateRefere nce on you.

There's plenty of room for improvement in the recipe, for sure. It'll
catch real basic stuff, but it doesn't account for all containers, and
doesn't show you the loci of the duplicate references.

Now, to be honest, the biggest benefit of this check is it gives
newbies a chance to learn about references some way other than the hard
way. It's not meant to catch a common mistake, so much as a
potentially very confusing one. (It's really not a common mistake for
programmers who are aware of references.) Once the lesson is learned,
this check should be put aside, as it unnecessarily restricts
legitimate use of referencing.
Carl Banks

Dec 11 '06 #13
Russ wrote:
The copy by reference semantics of Python give it great
efficiency but are also its achille's heel for tough-to-find bugs.
You need to stop using the term "copy by reference",
because it's meaningless. Just remember that assignment
in Python is always reference assignment. If you want
something copied, you need to be explicit about it.
I later discovered that a
particularly nasty bug was due to the fact that my constructor "copied"
the initializing list by reference.
The reason you made that mistake was that you were
using the wrong mental model for how assignment works
in Python -- probably one that you brought over from
some other language.

When you become more familiar with Python, you won't
make mistakes like that anywhere near as often. And
if you do, you'll be better at recognising the
symptoms, so the cause won't be hard to track down.
So a fundamental question in Python, it seems to me, is when to take
the performance hit and use "copy" or "deepcopy."
Again, this is something you'll find easier when
you've had more experience with Python. Generally,
you only need to copy something when you want an
independent object that you can manipulate without
affecting anything else, although that probably
doesn't sound very helpful.

In your vector example, it depends on whether you
want your vectors to be mutable or immutable. It
sounds like you were treating them as mutable, i.e.
able to be modified in-place. In that case, each
vector obviously needs to be a new object with the
initial values copied into it.

The alternative would be to treat your vectors as
immutable, i.e. once created you never change their
contents, and any operation, such as adding two
vectors, produces a new vector holding the result.
In that case, two vectors could happily share a
reference to a list of values (as long as there is
nothing else that might modify the contents of the
list).

--
Greg
Dec 11 '06 #14

Carl Banks wrote:
Niels L Ellegaard wrote:
Marc 'BlackJack' Rintsch wrote:
In <11************ **********@n67g 2000cwd.googleg roups.com>, Niels L
Ellegaard wrote:
I have been using scipy for some time now, but in the beginning I made
a few mistakes with copying by reference.
But "copying by reference" is the way Python works. Python never copies
objects unless you explicitly ask for it. So what you want is a warning
for *every* assignment.
Maybe I am on the wrong track here, but just to clarify myself:

I wanted a each object to know whether or not it was being referred to
by a living object, and I wanted to warn the user whenever he tried to
change an object that was being refered to by a living object.

This really wouldn't work, trust us. Objects do not know who
references them, and are not notified when bound to a symbol or added
to a container. However, I do think you're right about one thing: it
would be nice to have a tool that can catch errors of this sort, even
if it's imperfect (as it must be).

ISTM the big catch for Fortran programmers is when a mutable container
is referenced from multiple places; thus a change via one reference
will confusingly show up via the other one.
As a Fortranner, I agree. Is there an explanation online of why Python
treats lists the way it does? I did not see this question in the Python
FAQs at http://www.python.org/doc/faq/ .

Here is a short Python code and a Fortran 95 equivalent.

a = [1]
c = a[:]
b = a
b[0] = 10
print a,b,c

output: [10] [10] [1]

program xalias
implicit none
integer, target :: a(1)
integer :: c(1)
integer, pointer :: b(:)
a = [1]
c = a
b = a
b(1) = 10
print*,a,b,c
end program xalias

output: 10 10 1

It is possible to get similar behavior when assigning an array (list)
in Fortran as in Python, but one must explicitly use a pointer and "=>"
instead of "=". This works well IMO, causing fewer surprises, and I
have never heard Fortranners complain about it.

Another way of writing the Fortran code so that "a" and "b" occupy the
same memory is to use EQUIVALENCE.

program xequivalence
implicit none
integer :: a(1),b(1)
integer :: c(1)
equivalence (a,b)
a = [1]
c = a
b = a
b(1) = 10
print*,a,b,c
end program xequivalence

output: 10 10 1

EQUIVALENCE is considered a "harmful" feature of early FORTRAN
http://www.ibiblio.org/pub/languages/fortran/ch1-5.html .

Dec 11 '06 #15
In <11************ *********@79g20 00cws.googlegro ups.com>, Beliavsky wrote:
>ISTM the big catch for Fortran programmers is when a mutable container
is referenced from multiple places; thus a change via one reference
will confusingly show up via the other one.

As a Fortranner, I agree. Is there an explanation online of why Python
treats lists the way it does? I did not see this question in the Python
FAQs at http://www.python.org/doc/faq/ .
This question sounds as if lists are treated somehow special. They are
treated like any other object in Python. Any assignment binds an object
to a name or puts a reference into a "container" object.

Ciao,
Marc 'BlackJack' Rintsch
Dec 11 '06 #16

greg wrote:
You need to stop using the term "copy by reference",
because it's meaningless. Just remember that assignment
I agree that "copy by reference" is a bad choice of words. I meant pass
by reference and assign by reference. But the effect is to make a
virtual copy, so although the phrase is perhaps misleading, it is not
"meaningles s."

<cut>
Again, this is something you'll find easier when
you've had more experience with Python. Generally,
you only need to copy something when you want an
independent object that you can manipulate without
affecting anything else, although that probably
doesn't sound very helpful.
Yes, I realize that, but deciding *when* you need a copy is the hard
part.

Dec 11 '06 #17

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
2865
by: Johann Blake | last post by:
I can hardly believe I'm the first one to report this, but having gone through the newsgroup, it appears that way. I would like to open a solution in the VS.NET IDE that consists of multiple DLLs and a test application (an .EXE). The .EXE is my startup application. All the DLLs are shared components. This means that they contain a key and are stored in the GAC. When I go to run the test application in the IDE, it will not execute...
11
1312
by: Alexei | last post by:
Hello, The following doesn't compile due to absence of the copy constructor in class FileStream: FileInfo ^ fi = ...; FileStream fs = fi->OpenRead(); The compiler is Beta 2. Is this supported? Planned to be supported?
0
1912
by: RadekP | last post by:
Hi Gurus .. I would really appreciate some insights for the problem that bugs me for quite some time. I keep my custom controls in their own shared (private/public key signed) assembly. I need to have it shared. Usually I work with a Web application project that has a reference to my shared, custom controls assembly. Web project is a "Startup Project". I need to be able to set a breakpoint in my Custom Controls (shared) project. VS.NET...
3
1723
by: Brian Bischof | last post by:
I'm having troubles getting the debugging process to work consistenly for external classes. I got it to work once and then I turned it off. But now I can't get re-enabled. Here is what I'm doing. If someone could tell me what I'm missing that would be great. 1. Create an external class and call it Test.dll. 2. Create a test Asp.net app called App.sln. 3. For App.sln I set a reference to Test.dll. 4. Compile App.sln and run it. The web...
6
2209
by: Brian Bischof | last post by:
I'm having troubles getting the debugging process to work consistenly for external classes. I got it to work once and then I turned it off. But now I can't get re-enabled. Here is what I'm doing. If someone could tell me what I'm missing that would be great. 1. Create an external class and call it Test.dll. 2. Create a test Asp.net app called App.sln. 3. For App.sln I set a reference to Test.dll. 4. Compile App.sln and run it. The web...
0
1817
by: Ken Allen | last post by:
The MSDN documentation on remote debugging is a bit sparse, to say the least, and there is almost no information available on the 'best' way to configure this. I should note that my development system is Windows XP Pro and the test machine is Windows 2003, both running the latest upgrades; Visual Studio 2005 is installed on the development system and the .Net 2.0 framework is installed on the test system. The two systems do not have the...
1
1456
by: Ethan Strauss | last post by:
Hi, I have a C#.net Web application which calls a web service (http://eutils.ncbi.nlm.nih.gov/entrez/eutils/soap/eutils.wsdl). It has run just fine for months. Recently I started getting "System.Net.WebException: The underlying connection was closed" errors. I started the application up in debugging mode and it ran fine. After fiddling with things for awhile I now find that it works while I am running in debug mode, but does not when I am...
9
2907
by: puzzlecracker | last post by:
From my understanding, if you declare any sort of constructors, (excluding copy ctor), the default will not be included by default. Is this correct? class Foo{ public: Foo(int); // no Foo() is included, i believe. };
6
28879
kenobewan
by: kenobewan | last post by:
Congratulations! You are one of the few who realise that over 80% of errors are simple and easy to fix. It is important to realise this as it can save a lot of time. Time that could be wasted making unnecessary changes, that in turn can cause further problems. Programming is a humbling experience. An experience that causes one to reflect on human error. One major cause of these errors is syntax, syntax, syntax. We tend not to notice when we...
0
9839
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9682
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10871
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10564
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
7806
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5668
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4474
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4039
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3123
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.