473,396 Members | 1,933 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

Extending the dict class

Hello Pythoneers:

I need to pass a list of named arguments to a function in a given
order,
and make sure these named arguments are retrieved using keys() in the
same order they were given. Example:

keyargs={}
keyargs['one']=1
keyargs['two']=2
keyargs['three']=3

myfunc(**keyargs)
-myfunc would retrieve key arguments with keys() in the same order
as they were set, i.e. keyargs.keys() == ['one', 'two', 'three']

To achieve that, I subclassed dict and added the required lines
in __init__(), __setitem__() and keys(). I then assigned dict to
my new class but only get the desired behaviour for dictionaries
created like:
d=dict()
but not for dictionaries created like:
d={}
or
myfunc(**keyargs)

Is it possible to force dictionary creation in these case to use
my own dict class instead of the default one?

I guess we can formulate this as a more generic question: if I
want to modify the behaviour of the dictionary class, is there
any way to do it interpreter-wide so that special dict constructors
like those mentioned above use the modified version?

Thanks for helping

Aug 29 '06 #1
16 4055
chosechu wrote:
Is it possible to force dictionary creation in these case to use
my own dict class instead of the default one?
No
I guess we can formulate this as a more generic question: if I
want to modify the behaviour of the dictionary class, is there
any way to do it interpreter-wide so that special dict constructors
like those mentioned above use the modified version?
Not without modifying and recompiling the interpreter.
Aug 29 '06 #2

Duncan Booth wrote:
Is it possible to force dictionary creation in these case to use
my own dict class instead of the default one?

No
Ouch. I was expecting something like that, thanks for confirming it.

If I may: this seems inconsistent to me. I have created an augmented
version of the class with no possibility to extend 2 of the 3 legal
constructors: {} and func(**d). While I agree changing base class
behaviour is a bad idea, since we have the possibility to re-assign
dict to another class we should be able to do it consistently, i.e.
re-assing the two special constructors too.

Just my 2c on the topic... I have found workarounds anyway.

Aug 29 '06 #3
chosechu wrote:
Hello Pythoneers:

I need to pass a list of named arguments to a function in a given
order,
and make sure these named arguments are retrieved using keys() in the
same order they were given. Example:

keyargs={}
keyargs['one']=1
keyargs['two']=2
keyargs['three']=3

myfunc(**keyargs)
-myfunc would retrieve key arguments with keys() in the same order
as they were set, i.e. keyargs.keys() == ['one', 'two', 'three']
I'm not sure to understand why you want to do so - perhaps you could
tell more about your real use case ?

Anyway, and since it's not directly possible, a possible workaround
could be to pass a sequence of (key, value) tuples instead (possibly as
*args). This of course requires that you can modify the implementation
of myfunc().
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"
Aug 29 '06 #4
I'm not sure to understand why you want to do so - perhaps you could
tell more about your real use case ?
Without diving into too many details: I am calling SOAPpy to build
SOAP requests. When calling a proxy, named arguments are used
to build up the corresponding XML like:

proxy.call(alpha=1, beta=2, gamma=3)
would end up like:
<alpha>1</alpha<beta>2</beta<gamma>3</gamma>

Unfortunately, since the arguments as retrieved with **k
their order is lost in the call. The corresponding XML file
re-arranges the parameter order randomly and the final
XML request is unordered, which is not supported by a
number of braindead SOAP servers (including one produced
by a well-known software company I will not name here).

SOAPpy cannot know in advance the argument names since
they are server-dependent and SOAPpy is generic, so retrieving
named arguments with **k seems like the sensible thing to do.
Unfortunately this gets converted to a dict so looses all ordering
when being received. Extending the base dict class to support ordering
was one possible idea.
Anyway, and since it's not directly possible, a possible workaround
could be to pass a sequence of (key, value) tuples instead (possibly as
*args). This of course requires that you can modify the implementation
of myfunc().
Yes, if I could simply modify myfunc() I would have workarounds.
This would mean me modifying SOAPpy and specializing it for
my needs.

There are other ways to implement SOAP clients anyway so
no real need. Just wanted to try out some metaclass stuff.

Aug 29 '06 #5
chosechu wrote:
SOAPpy cannot know in advance the argument names since
they are server-dependent and SOAPpy is generic, so retrieving
named arguments with **k seems like the sensible thing to do.
Unfortunately this gets converted to a dict so looses all ordering
when being received. Extending the base dict class to support ordering
was one possible idea.
If the order of the argument names matters, then it seems to me that should
be handled by the SOAP library, not pushed onto the end user whoch should
just be calling the function with the named arguments in the most
convenient order.

Shouldn't SOAPpy be able to get this information out of the WSDL?

Aug 29 '06 #6
"chosechu" wrote:
Yes, if I could simply modify myfunc() I would have workarounds.
This would mean me modifying SOAPpy and specializing it for
my needs.
maybe you could fake it:

class fakedict(dict):
def __init__(self, *data):
self.data = list(data)
for k, v in data:
self[k] = v
def items(self):
return self.data

d = fakedict(("a", 1), ("b", 2), ("c", 3))

print d ={'a': 1, 'c': 3, 'b': 2}
print d.items() =[('a', 1), ('b', 2), ('c', 3)]
print isinstance(d, dict) =True

(the exact set of methods you need to override depends on how SOAPpy
fetches the members).

</F>

Aug 29 '06 #7

Duncan Booth wrote:
If the order of the argument names matters, then it seems to me that should
be handled by the SOAP library, not pushed onto the end user whoch should
just be calling the function with the named arguments in the most
convenient order.

Shouldn't SOAPpy be able to get this information out of the WSDL?
Yes, SOAPpy could extract this from the WSDL specs.
SOAPpy could also find another way to pass call parameters, but I kinda
like
the named parameters (seems really Python-like). Microsoft could also
build SOAP services that parse XML without making ordering mandatory
where
nobody said it was.

.... but we are living in a different dimension, one where I can extend
the Python dict class but not touch 2 of its 3 constructors (of course
the most
useful ones).

If you really want to know, I ended up at the socket level pushing
templatized
strings. Wish me luck with future server changes.

Aug 29 '06 #8
(the exact set of methods you need to override depends on how SOAPpy
fetches the members).
[fakedict class]
This I did. And checking out the source it seems SOAPpy retrieves named
parameters through keys(), which is the method I tried to overload.
Trouble is:
something happens to my fakedict object when getting through this:
d = fakedict(...)
SOAPmethod(**d)
I have the impression **d acts like a dict constructor, taking over my
fakedict class. I know I could find the answer by scrutinizing the
source,
call me lazy.

Aug 29 '06 #9
chosechu wrote:
>
Duncan Booth wrote:
>If the order of the argument names matters, then it seems to me that
should be handled by the SOAP library, not pushed onto the end user
whoch should just be calling the function with the named arguments in
the most convenient order.

Shouldn't SOAPpy be able to get this information out of the WSDL?

Yes, SOAPpy could extract this from the WSDL specs.
SOAPpy could also find another way to pass call parameters, but I
kinda like
the named parameters (seems really Python-like). Microsoft could also
build SOAP services that parse XML without making ordering mandatory
where
nobody said it was.
Indeed, the spec says that parameterOrder is a hint and may be safely
ignored.
>
... but we are living in a different dimension, one where I can extend
the Python dict class but not touch 2 of its 3 constructors (of course
the most
useful ones).
No, you weren't able to extend the builtin dict class nor touch any its
constructor.

All you did was to create a subclass with its own constructor and hide the
name for the builtin dictionary type. The original type was still unchanged
as you can see since anything which constructed a dictionary without using
the name you had overwritten still got the original type.

If you had looked at type(dict()) and type({}) after your subclassing, you
would see that they are different types.
Aug 29 '06 #10

Duncan Booth wrote:
No, you weren't able to extend the builtin dict class nor touch any its
constructor.
Yes, sorry. Forgot the negation.
All you did was to create a subclass with its own constructor and hide the
name for the builtin dictionary type. The original type was still unchanged
as you can see since anything which constructed a dictionary without using
the name you had overwritten still got the original type.

If you had looked at type(dict()) and type({}) after your subclassing, you
would see that they are different types.
.... which prompted my question.

And prompts yet another one: seems like it
is not possible with Python to modify behaviour
for base classes without recompiling the
interpreter. Forgive me for asking what must
surely have been asked already, but are there
plans to implement something like that,
<teasing>like Ruby</teasing>?

I would not feel too safe navigating in a source
where base object behaviour might have been
re-defined, but it sure is a powerful way of
adding behaviour to third-party code which you
may not have possibility to modify.

Aug 29 '06 #11

Fredrik Lundh wrote:
"chosechu" wrote:
Yes, if I could simply modify myfunc() I would have workarounds.
This would mean me modifying SOAPpy and specializing it for
my needs.

maybe you could fake it:
No you cannot fake it, because the ** form of passing arguments
constructs a new dictionary (and that dictionary is the standard
built-in type of dictionary, not your own fake-dictionary).

Just did some tests, redefining the dict-type:
>>native_dict = dict
type({}) == native_dict
True
>>class my_dict(native_dict):
.... pass
....
>>dict = my_dict
type({}) == native_dict
True
>>type(dict()) == native_dict
False
>>def foo(**kwargs):
.... print type(kwargs) == native_dict
....
>>foo(x=1)
True
>>foo(**dict())
True
>>>
(NB: I also tried changing __builtins__.dict, no effect)

As the OP said, when passing keyword-arguments, a new instance of a
built-in dict is always generated and not the dict type bound to the
dict-constructor.

I do actually think that it would be useful in many circumstances to be
able to replace the default-constructor of a particular so that custom
objects are created for that type.

OTOH, there are probably many more cases where doing so would be a very
bad idea, not a good idea, and we would begin seeing an overwhelming
number of cases of mis-use of such feature.

Cheers,

--Tim

Aug 29 '06 #12

chosechu wrote:
Duncan Booth wrote:
If the order of the argument names matters, then it seems to me that should
be handled by the SOAP library, not pushed onto the end user whoch should
just be calling the function with the named arguments in the most
convenient order.

Shouldn't SOAPpy be able to get this information out of the WSDL?

Yes, SOAPpy could extract this from the WSDL specs.
SOAPpy could also find another way to pass call parameters, but I kinda
like
the named parameters (seems really Python-like). Microsoft could also
build SOAP services that parse XML without making ordering mandatory
where
nobody said it was.
Are you sure that the order is not defined in the WSDL still? I don't
know WSDL, but in XML Schema for instance, order is defined when using
'xsd:sequence' yet many parsers don't care for receiving elements in a
different order from what's defined in the XSD which they're supposed
to validate against (working perhaps with the axiom 'be liberal in what
yo except, but strict in what you send'). Many people don't actually
realize that 'xsd:sequence' dictates the order of elements and use it
just for 'any order is good enough'.

Might be a similar case with the WSDL: Order is defined, but due to
mis-interpretation of the spec it's not realized / enforced by all /
some of the components.
Anyways, modifiying SOAPpy might not be a bad idea: submit your changes
to the project, and you can write on your CV that you have contributed
to open-source projects! ;-)

I do hope for you that you will find something better to do than
pushing template-instances down a socket-hole.

Cheers,

--Tim

Aug 29 '06 #13
Tim N. van der Leeuw wrote:
>maybe you could fake it:
No you cannot fake it, because the ** form of passing arguments
constructs a new dictionary (and that dictionary is the standard
built-in type of dictionary, not your own fake-dictionary).
yeah, I thought he was passing a dictionary to a SOAPpy method. passing
custom types through the argument handling layer doesn't work.

</F>

Aug 29 '06 #14

Tim N. van der Leeuw wrote:
Anyways, modifiying SOAPpy might not be a bad idea: submit your changes
to the project, and you can write on your CV that you have contributed
to open-source projects! ;-)
Been there, done that. Don't worry, my contributions
to open-source projects is largely positive (though under
different aliases).

About SOAPpy: the author mentions something about
ordering (or not) parameters in the XML request. Quoting:

"There is an incompatibility with the way that Python and
SOAP handle method arguments: SOAP requires that
all arguments have names and that they are presented
in the same order as the method signature. [...]"

See SOAPpy/docs/MethodParameterNaming.txt for more
details.
I do hope for you that you will find something better to do than
pushing template-instances down a socket-hole.
It is not the first time I end up implementing some kind of
support for ASCII-based protocols with templatized strings
You would be surprised how easy it is to maintain when
the emitter is required to be as stable as possible. Just
have to be aware that you have a potential maintenance
nightmare somewhere out there.

Aug 29 '06 #15
chosechu wrote:
Duncan Booth wrote:
>No, you weren't able to extend the builtin dict class nor touch any its
constructor.

Yes, sorry. Forgot the negation.
>All you did was to create a subclass with its own constructor and hide the
name for the builtin dictionary type. The original type was still unchanged
as you can see since anything which constructed a dictionary without using
the name you had overwritten still got the original type.

If you had looked at type(dict()) and type({}) after your subclassing, you
would see that they are different types.

... which prompted my question.

And prompts yet another one: seems like it
is not possible with Python to modify behaviour
for base classes without recompiling the
interpreter. Forgive me for asking what must
surely have been asked already, but are there
plans to implement something like that,
<teasing>like Ruby</teasing>?

I would not feel too safe navigating in a source
where base object behaviour might have been
re-defined, but it sure is a powerful way of
adding behaviour to third-party code which you
may not have possibility to modify.
It's usually possible to modify third-parts classes behaviour on the fly
(googling for 'monkey-patching' should get you started). But true, this
doesn't work with builtins.

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"
Aug 29 '06 #16

Bruno Desthuilliers wrote:
It's usually possible to modify third-parts classes behaviour on the fly
(googling for 'monkey-patching' should get you started). But true, this
doesn't work with builtins.
I guess this stems from the fact that Python dictionaries
are C creatures and they are themselves used for internal
housekeeping, so no high-level modifications can be allowed
to disturb language internals. This all makes sense. It is sad
not to have the possibility to extend built-ins but on the other
hand it guarantees a constant (predictable) behaviour, which
I agree is something we should protect.

Thanks to all contributors for your nearly instantaneous
reaction times and help on the topic.

Aug 29 '06 #17

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Thomas Guettler | last post by:
Hi! If I pickle a class which inherits from dict, I don't get the same data again, if I unpickle it. def test_pickle(): home="..." server=WorkflowServer(home) server.save()
2
by: GrelEns | last post by:
hello, i would like if this behaviour can be obtained from python : trap an attributeError from inside a subclassing dict class... (here is a silly examples to explain my question) class...
1
by: Alexander Kervero | last post by:
Hi ,today i was reading diveinto python book,in chapter 5 it has a very generic module to get file information,html,mp3s ,etc. The code of the example is here :...
3
by: Bengt Richter | last post by:
Has anyone found a way besides not deriving from dict? Shouldn't there be a way? TIA (need this for what I hope is an improvement on the Larosa/Foord OrderedDict ;-) I guess I can just document...
15
by: Cruella DeVille | last post by:
I'm trying to implement a bookmark-url program, which accepts user input and puts the strings in a dictionary. Somehow I'm not able to iterate myDictionary of type Dict{} When I write print...
5
by: vbgunz | last post by:
Hello everyone. I own two books. Learning Python and Python in a nutshell. When cross referencing the two books to try and clarify the ideas behind extending methods and delegates, this is where...
15
by: George Sakkis | last post by:
Although I consider dict(**kwds) as one of the few unfortunate design choices in python since it prevents the future addition of useful keyword arguments (e.g a default value or an orderby...
8
by: Almad | last post by:
Hello, I discovered this behaviour in dictionary which I find confusing. In SneakyLang, I've tried to extend dictionary so it visits another class after something is added: class...
12
by: jeremito | last post by:
Please excuse me if this is obvious to others, but I can't figure it out. I am subclassing dict, but want to prevent direct changing of some key/value pairs. For this I thought I should override...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.