473,769 Members | 7,388 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

What is the status of the __subclasses__ method?

The question I have is: how safe / future proof / portable is the
use of the __subclasses__ method that exists for new-style classes?

Background: I thought I had found an easy-to-understand
application for metaclasses: making classes instantly aware
of their subclasses. The context was that I needed a class
hierarchy, and I wanted to be able to instantiate subclasses
by calling the root class. Which exact subclass would be
instantiated would depend on the arguments with which the root
class would be called. In effect using the root class as a factory.

I did not want to build the logic of argument-to-subclass
mapping in the root class. Instead I wanted the root class
to pass the argument (recursively) to its subclasses, and if
a subclass would accept the argument then an object instantiation
of that subclass would be returned. To do this, each class
in the hierarchy obviously needs to be aware of its subclasses.

I also did not want to revisit the base classes each time
I would create a new subclass -- I wanted the mere act
of subclassing a base class to be enough for the base class
to be aware of the subclass. So, being the kind of lazy
person who prefers to spend several hours on an interesting
problem to avoid 5 minutes of routine work, I wrote a metaclass
that would do this for me. Quite straightforward really.
Only after playing around with this for a while, and inspecting
the results, I happened to notice that new-style classes in
Python have a __subclasses__ method. This method does what
the name implies: Klass.__subclas ses__() returns a list of
the direct subclasses of Klass.

I don't really mind the lost time for the creation of the
metaclass -- that was fun, and I learned some new things
along the way. But what bothers me is that there's absolutely
nothing about this method in the Python documentation.
The excellent "Python in a Nutshell" book also does not
mention it. It's almost as if this method is not meant
to be used by mortals.

When I searched python.org for "__subclasses__ " I found
some more information. The __subclasses__ method appears
to exist to allow modifications of the superclass to be
percolated down to its children, mainly for speed reasons,
if I understand Tim Peter's explanation correctly
(http://mail.python.org/pipermail/pyt...t/176360.html).
I also found several warnings that the __subclasses__ method
only returns classes that have been accessed previously (which
would defeat my purpose) See e.g.
http://mail.python.org/pipermail/pyt...ne/018299.html.
However, when I experimented with the __subclasses__ method
for my application, I always found that the subclass was known
as soon as it was defined.

So again, my question. Is it safe to use the __subclasses__
method for the purpose I described above? Or would it be safer
to revert to my home-grown metaclass solution?

Ruud de Jong

Jul 18 '05 #1
4 7098
Ruud de Jong <ru**********@c onsunet.nl> writes:
The question I have is: how safe / future proof / portable is the
use of the __subclasses__ method that exists for new-style classes?
Hmm. I think it's unlikely to go away.
When I searched python.org for "__subclasses__ " I found
some more information. The __subclasses__ method appears
to exist to allow modifications of the superclass to be
percolated down to its children, mainly for speed reasons,
if I understand Tim Peter's explanation correctly
(http://mail.python.org/pipermail/pyt...t/176360.html).
Yep.
I also found several warnings that the __subclasses__ method only
returns classes that have been accessed previously (which would
defeat my purpose)
Hum, you're being overly paranoid here. Python defined types are
PyType_Ready-ed as soon as they are created.
So again, my question. Is it safe to use the __subclasses__
method for the purpose I described above?
Well, I can't see into the future, but I'd feel secure using it.

I'm not sure what you describe is a very tasteful use of it though...
Or would it be safer to revert to my home-grown metaclass solution?


You might want to keep it around, I guess...

Cheers,
mwh

--
I would hereby duly point you at the website for the current pedal
powered submarine world underwater speed record, except I've lost
the URL. -- Callas, cam.misc
Jul 18 '05 #2
Michael Hudson schreef:
Ruud de Jong <ru**********@c onsunet.nl> writes:

The question I have is: how safe / future proof / portable is the
use of the __subclasses__ method that exists for new-style classes?

Hmm. I think it's unlikely to go away.

When I searched python.org for "__subclasses__ " I found
some more information. The __subclasses__ method appears
to exist to allow modifications of the superclass to be
percolated down to its children, mainly for speed reasons,
if I understand Tim Peter's explanation correctly
(http://mail.python.org/pipermail/pyt...t/176360.html).

Yep.


What I wanted to avoid is that I would rely on something that is
not part of the Python *language*, but rather of a Python
*implementation *. But then, the distinction between these two
is not always clear -- if a construct is present in every
implementation, what is the difference with it being a part
of the language?

Well, raising the question is answering it -- as long as a
construct is not officially part of the language, that construct
can in principle change or disappear if the language maintainers
find that necessary.

I also found several warnings that the __subclasses__ method only
returns classes that have been accessed previously (which would
defeat my purpose)

Hum, you're being overly paranoid here. Python defined types are
PyType_Ready-ed as soon as they are created.


Well, I guess that such cautious behavior has become sort of
second nature for me -- a consequence of having worked some
twenty years on a huge software system (think multi-million lines
of C code), with thousands of colleagues adding, deleting and
modifying code simultaneously. In spite of strict processes and
procedures, too often somebody would make themselves dependent on
some construct in another part of the system that was never
meant for public usage. And when that construct was modified
to add e.g. new functionality or for performance improvement,
all hell would break loose -- because it would break the
functionality that "illegally" used the construct (and that
had been working fine for the customers)

With this conditioning I tend to be *very* careful about
using undocumented features, like __subclasses__.

Anyway, if I understand you correctly, as long as the subclasses
are implemented in Python, they will be known to their parent
immediately. And from your other remarks I get the
impression that the __subclasses__ method is a rather permanent
feature in every Python implementation.
So again, my question. Is it safe to use the __subclasses__
method for the purpose I described above?

Well, I can't see into the future, but I'd feel secure using it.

I'm not sure what you describe is a very tasteful use of it though...

Well, not being a native english speaker, I don't quite know how to
interpret this latter remark. Please elaborate. Are you objecting
to the use of the __subclasses__ method? Or to the general mechanism
I described? Or maybe I really am paranoid, and I am reading things
that aren't there :-)

Here is the essence of the use of __subclasses__ in the application
I referred to. Note that is a variant of the Chain-of-Responsibility
pattern, applied to the Factory Method. I know that I abused the
NotImplementedE rror -- in the actual application it is a custom-
defined exception. But for illustration purposes I wanted to avoid
such non-essential details.
class Event(object): def __new__(cls, line):
for subclass in cls.__subclasse s__():
try:
return subclass.__new_ _(subclass, line)
except NotImplementedE rror:
continue
raise NotImplementedE rror

class MsgEvent(Event) : pass
class IncomingMsgEven t(MsgEvent): def __new__(cls, line):
if line.startswith ('RCV:'):
return object.__new__( cls)
raise NotImplementedE rror

class OutgoingMsgEven t(MsgEvent): def __new__(cls, line):
if line.startswith ('SND:'):
return object.__new__( cls)
raise NotImplementedE rror

x = Event('RCV: blah blah')
x <__main__.Incom ingMsgEvent object at 0x00AD2D30>


The thing to note is that I can add subclasses at will, and
never have to revisit the Event root class. This means better
maintainability and extensibility.
Or would it be safer to revert to my home-grown metaclass solution?

You might want to keep it around, I guess...

I will. I joins my collection of APL one-liners and my FORTRAN V
preprocessor :-)

Regards,

Ruud de Jong

Jul 18 '05 #3
Ruud de Jong <ru**********@c onsunet.nl> writes:
Michael Hudson schreef:
Ruud de Jong <ru**********@c onsunet.nl> writes:
The question I have is: how safe / future proof / portable is the
use of the __subclasses__ method that exists for new-style classes? Hmm. I think it's unlikely to go away.
When I searched python.org for "__subclasses__ " I found
some more information. The __subclasses__ method appears
to exist to allow modifications of the superclass to be
percolated down to its children, mainly for speed reasons,
if I understand Tim Peter's explanation correctly
(http://mail.python.org/pipermail/pyt...t/176360.html).

Yep.


What I wanted to avoid is that I would rely on something that is
not part of the Python *language*, but rather of a Python
*implementation *. But then, the distinction between these two
is not always clear


Indeed.
-- if a construct is present in every implementation, what is the
difference with it being a part of the language?

Well, raising the question is answering it -- as long as a
construct is not officially part of the language, that construct
can in principle change or disappear if the language maintainers
find that necessary.
I'm not sure this attitude really applies to Python. There's no real
definition of what "officially part of the language" means.

We don't take things away for the fun of it. __subclasses__( ) would
only disappear if some major internal restructuring happened and it
becamse massively inconvenient to keep it. But this applies (probably
most of the time with less force) to just about anything else!
I also found several warnings that the __subclasses__ method only
returns classes that have been accessed previously (which would
defeat my purpose)

Hum, you're being overly paranoid here. Python defined types are
PyType_Ready-ed as soon as they are created.


Well, I guess that such cautious behavior has become sort of
second nature for me -- a consequence of having worked some
twenty years on a huge software system (think multi-million lines
of C code), with thousands of colleagues adding, deleting and
modifying code simultaneously. In spite of strict processes and
procedures, too often somebody would make themselves dependent on
some construct in another part of the system that was never
meant for public usage.


It's exposed to Python. It it was really meant to be internal, that
wouldn't have happened.
Anyway, if I understand you correctly, as long as the subclasses
are implemented in Python, they will be known to their parent
immediately.
Yes.
And from your other remarks I get the impression that the
__subclasses__ method is a rather permanent feature in every Python
implementation.
Well, it's only present in one implementation at the moment, for what
that's worth...
So again, my question. Is it safe to use the __subclasses__
method for the purpose I described above?

Well, I can't see into the future, but I'd feel secure using it.
I'm not sure what you describe is a very tasteful use of it
though...

Well, not being a native english speaker, I don't quite know how to
interpret this latter remark. Please elaborate. Are you objecting
to the use of the __subclasses__ method?


No.
Or to the general mechanism I described?
I didn't really spend long trying to understand what you said you were
trying to do, but it sounded a little strange. That's all.
Here is the essence of the use of __subclasses__ in the application
I referred to. Note that is a variant of the Chain-of-Responsibility
pattern, applied to the Factory Method. I know that I abused the
NotImplementedE rror -- in the actual application it is a custom-
defined exception. But for illustration purposes I wanted to avoid
such non-essential details.
>>> class Event(object): def __new__(cls, line):
for subclass in cls.__subclasse s__():
try:
return subclass.__new_ _(subclass, line)
except NotImplementedE rror:
continue
raise NotImplementedE rror

>>> class MsgEvent(Event) : pass
>>> class IncomingMsgEven t(MsgEvent): def __new__(cls, line):
if line.startswith ('RCV:'):
return object.__new__( cls)
raise NotImplementedE rror

>>> class OutgoingMsgEven t(MsgEvent): def __new__(cls, line):
if line.startswith ('SND:'):
return object.__new__( cls)
raise NotImplementedE rror

>>> x = Event('RCV: blah blah')
>>> x <__main__.Incom ingMsgEvent object at 0x00AD2D30> >>>

That's cute.
The thing to note is that I can add subclasses at will, and never
have to revisit the Event root class. This means better
maintainability and extensibility.


Yes. I've actually used __subclasses__ for a somewhat similar
purpose, having a Resource class whose subclasses know how to find all
instances of each resource type and so being able to find all
resources by looking through subclasses.

I will note that there's some thing *slightly* odd going on here,
which perhaps is highlighted by considering what happens if you
subclass OutgoingMsgEven t. In the code you posted, this subclass
wouldn't be picked up. You could traverse the subclass graph easily
enough, but it seems to me that "is able to instantiate events" is
more like a "class-instance" relationship (flat) than a
"class-subclass" relationship (possibly nested) and so it might be
more appropriate to make all your Event classes instances of a custom
metaclass that keeps track of its instances and then create events by
calling a method on the metaclass.

But, whatever.
Or would it be safer to revert to my home-grown metaclass solution?

You might want to keep it around, I guess...

I will. I joins my collection of APL one-liners and my FORTRAN V
preprocessor :-)


:-)

Cheers,
mwh

--
Linux: Horse. Like a wild horse, fun to ride. Also prone to
throwing you and stamping you into the ground because it doesn't
like your socks. -- Jim's pedigree of operating systems, asr
Jul 18 '05 #4

"Michael Hudson" <mw*@python.net > wrote in message
news:m3******** ****@pc150.math s.bris.ac.uk...
I'm not sure this attitude really applies to Python. There's no real
definition of what "officially part of the language" means.


I think 'in the Python Reference Manual' qualifies pretty well as defining
the language. The Python Library Reference fleshes out the current PSF
CPython distribution. Any discrepancies between these and the (C)Python
implementation are considered bugs to be fixed. Guido has (as I remember)
occasionally omitted (and rejected patches about) items that he wants to
remain unofficial implementation details subject to possible change.

Someone using an undocumented implementation details like __subclasses__( )
might especially want to test code against beta releases as they come out
to either protest a change or start adjusting. But the OP can relex
somewhat since code-breaking changes are not made intentionally without
some clear benefit otherwise.

Terry J. Reedy


Jul 18 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
1453
by: Andrew | last post by:
Hi, friends, We need to programm a .dll (C#) which will collect all selected files from various locations, such as a DB, a file system, a VSS, etc. So, it may take long time. I created a method GetAllFiles() to iterate/fetch each file, and it works fine. However, since this method is called from a Windows app (VC#), and users want to see real time status for each file. What is the best approach to implement this requirement? Create...
6
6080
by: John Salerno | last post by:
I understand how they work (basically), but I think maybe the examples I'm reading are too elementary to really show their value. Here's one from Programming C#: #region Using directives using System; using System.Collections.Generic; using System.Text;
0
3062
by: Ken Yee | last post by:
Anyone know how to do this in C#? It's pretty trivial in VB, but is being a PITA in C#. I can call the Start/Stop methods w/o any problems, but I can't figure out how to read the current status in C#? Code looks like this: DirectoryEntry obDirEntry = new DirectoryEntry ("IIS://localhost/W3svc/1"); obDirEntry.Invoke("Stop", new object {}); // Stop is not a synchronous call, so you have to poll status
4
1389
by: John Salerno | last post by:
Here's my method, and I get an error on the array initialization line. I've gotte this same error before, and it was fixed by adding the first line of the method, so I'm not sure what to do now. private static Status ConvertToFlags(int allPanels) { Status panel = new Status; for (int i = 0; i < 4; i++)
18
4147
by: mistral | last post by:
Is there some other (more advanced) effects for status bar, other than standard Scroller Bar, TypeWriter Scroller, Flashing Bar, Decrypter, Ticker, World Clock?
3
16761
by: bharathreddy | last post by:
This article will explain you how to find the status of the fax operation. (Using FAXCOM.dll). Author: Bharath Reddy VasiReddy Reference to the FAXCOM.DLL Reference to import FAXCOM import FAXCOMLib Tracking The Status of Faxes:
1
3121
by: Roland | last post by:
Hello, I am writing modal dialog box to display progress of downloading file. I am starting download of a file in constructor of dialog using some asynchronous method call which returns me an asynchronous object. I could use this object to get status. I will be using this status to update status message on dialog. Which method should I override to update the status message ? Can I follow the approach just like
9
4122
by: tshad | last post by:
I have a Windows App that is doing some work and then writing a "Now Processing..." line to the status line of the window as well as the Textbox on the form. But the problem is that the work is in another class from the main class. So it couldn't access the Status Line or textbox. So what we did was set them up as properties: string IStatusDisplay.Status
9
1967
by: tvnaidu | last post by:
This is just small plain HTML program wtote manually (since I am new to web programming, may be some tools I can use I think), this program loaded onto microcontroller with small foot print web server, when program prints, half-way it prints, then it struck, any issue with this?. when user clicks "submit" again it loaded same page. but it sends some sytring to server (plug1=ON&......), that string will be processed inside server. any open...
0
9423
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10049
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9997
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9865
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7413
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6675
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5310
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5448
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
3565
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.