Weakrefs to classes that derive from str

[Ron Garret]

Why doesn't this work?

from weakref import ref
class C(str): pass .... ref(C()) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: cannot create weak reference to 'C' object
Note that this does work:
class D(int): pass .... ref(D()) <weakref at 0x53e10; dead>

Likewise for floats, lists, etc. Everything but strs.

rg

[Steven Bethard]

Ron Garret wrote:

Why doesn't this work?

from weakref import ref
class C(str): pass ...ref(C())

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: cannot create weak reference to 'C' object

Note that you don't need the class redirection:

py> ref('')
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
TypeError: cannot create weak reference to 'str' object

But I don't know why strings aren't valid arguments to ref...

STeVe

[Ron Garret]

In article <bv************ ********@comcas t.com>,
Steven Bethard <st************ @gmail.com> wrote:

Ron Garret wrote:

Why doesn't this work?

>from weakref import ref
>class C(str): pass

...

>ref(C())

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: cannot create weak reference to 'C' object

Note that you don't need the class redirection:

py> ref('')
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
TypeError: cannot create weak reference to 'str' object

But I don't know why strings aren't valid arguments to ref...

None of the native types (int, float, list, tuple, etc.) can have weak
references, but wrapping them in a class is supposed to get around that.
And it does -- for all classes except str.

rg

[Steven Bethard]

Ron Garret wrote:

Note that you don't need the class redirection:

py> ref('')
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
TypeError: cannot create weak reference to 'str' object

But I don't know why strings aren't valid arguments to ref...

None of the native types (int, float, list, tuple, etc.) can have weak
references, but wrapping them in a class is supposed to get around that.
And it does -- for all classes except str.

Interesting. Is the wrapping thing documented somewhere? I didn't see
it in the documentation for weakref.ref (though I have been known to be
blind occasionally) ;)

STeVe

[Peter Hansen]

Steven Bethard wrote:

Ron Garret wrote:

None of the native types (int, float, list, tuple, etc.) can have weak
references, but wrapping them in a class is supposed to get around
that. And it does -- for all classes except str.

Interesting. Is the wrapping thing documented somewhere? I didn't see
it in the documentation for weakref.ref (though I have been known to be
blind occasionally) ;)

I believe it's here: http://docs.python.org/lib/module-weakref.html
if you search for the string "Not all" and read the next two
paragraphs.

On the other hand, it says (there) only that "several builtin
types such as list and dict ... can add support through
subclassing", and does not say anything about int, str, etc...

-Peter

[Steven Bethard]

Peter Hansen wrote:

I believe it's here: http://docs.python.org/lib/module-weakref.html
if you search for the string "Not all" and read the next two
paragraphs.

On the other hand, it says (there) only that "several builtin
types such as list and dict ... can add support through
subclassing", and does not say anything about int, str, etc...

Ahh, thanks for the help. I guess there are at least two solutions to
the OP's problem then:

(1) Document that str and subclasses of str can't be weakreffed (easy)
(2) Change str so that it (or at least its subclasses) can be weakreffed
(hard)

Probably either way a feature request should be filed.

STeVe

[Ron Garret]

In article <tu************ ********@powerg ate.ca>,
Peter Hansen <pe***@engcorp. com> wrote:

Steven Bethard wrote:

Ron Garret wrote:

None of the native types (int, float, list, tuple, etc.) can have weak
references, but wrapping them in a class is supposed to get around
that. And it does -- for all classes except str.

Interesting. Is the wrapping thing documented somewhere? I didn't see
it in the documentation for weakref.ref (though I have been known to be
blind occasionally) ;)

I believe it's here: http://docs.python.org/lib/module-weakref.html
if you search for the string "Not all" and read the next two
paragraphs.

On the other hand, it says (there) only that "several builtin
types such as list and dict ... can add support through
subclassing", and does not say anything about int, str, etc...

Empirically:

from weakref import ref
def foo(c): .... class C(c): pass
.... ref(C())
.... foo(int)
foo(float)
foo(dict)
foo(list)
foo(str) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 3, in foo
TypeError: cannot create weak reference to 'C' object foo(tuple) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 3, in foo
TypeError: cannot create weak reference to 'C' object foo(long) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 3, in foo
TypeError: cannot create weak reference to 'C' object

Ah, it appears that non-immediate immutable types don't support
weakrefs. Hm...

rg

[Peter Hansen]

Ron Garret wrote:

foo(int)
foo(float )
foo(dict)
foo(list)
foo(str) TypeError: cannot create weak reference to 'C' object
foo(tuple ) TypeError: cannot create weak reference to 'C' object
foo(long)

TypeError: cannot create weak reference to 'C' object

Ah, it appears that non-immediate immutable types don't support
weakrefs. Hm...

I see the same results you do, and yet I don't understand
the comment. Can you please explain what "immediate"
means in this context?

-Peter

[Raymond Hettinger]

[Ron Garret]

Why doesn't this work?

from weakref import ref
class C(str): pass ... ref(C())

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: cannot create weak reference to 'C' object

. . . Everything but strs.

Also subclasses of tuple are not weak referencable.

The issue is in the design of the C structure as a variable-sized immutable
object. Both strings and tuples allocate as a single unit of memory that holds
both the header information and the content information (the characters in a
string or the array of object pointers for a tuple). Since the size varies from
one string or tuple to the next, there is no straight-forward way for a subclass
to add an additional header field pointing to a list of weak references.

For lists and dicts, this is not a problem because the object is allocated in
two sections, a fixed size header component and a pointer to another area of
memory to hold the contents of the collection. This makes it possible for a
subclass to graft-on a weak reference pointer at a known, fixed offset from the
beginning of the header.

There are two ways to fix this. One is to add a weak reference pointer to every
string object -- that way you wouldn't even have to subclass it. Another way is
to break the object into two pieces as is done for mutable containers like dicts
and lists.

Both approaches consume time and space. In general, that is not a big deal, but
fast, memory efficient strings and tuples are at the center of all things
Python. The need to weak reference this objects is somewhat rare in comparison
to the frequency of their other uses. It did not make sense to always pay a
time/space penalty just to create the possibility of weak referencing.

While the design decision is unlikely to change, the docs could certainly be
improved. A doc patch would be welcome.

FWIW, the work-arounds are to weak-reference instances of UserString or to
create a custom class with a has-a relationship instead of an is-a relationship.
Raymond Hettinger

[Ron Garret]

In article <-4************** ******@powergat e.ca>,
Peter Hansen <pe***@engcorp. com> wrote:

Ron Garret wrote:

>foo(int)
>foo(float )
>foo(dict)
>foo(list)
>foo(str)

TypeError: cannot create weak reference to 'C' object

>foo(tuple )

TypeError: cannot create weak reference to 'C' object

>foo(long)

TypeError: cannot create weak reference to 'C' object

Ah, it appears that non-immediate immutable types don't support
weakrefs. Hm...

I see the same results you do, and yet I don't understand
the comment. Can you please explain what "immediate"
means in this context?

"Immediate" means a data type that will fit entirely in a machine
register, and therefore doesn't need to actually be stored in memory.
Ints and floats are the only two immediate types in Python. They are
immutable, but you can still create weakrefs to classes that derive from
them.

rg