473,881 Members | 1,567 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Programmaticall y replacing an API for another module

I've got a bunch of code that runs under a bunch of unit tests. It'd
be really handy if when testing I could supply replacement
functionality to verify that the right things get called without those
things actually getting called, since frequently those calls take a
long time to execute and do things that, well, I don't really want to
do.

E.g. imagine you've got a routine which takes a filespec, generates a
list of files, and copies those files from one location to another. (I
realize that this example could be multiple functions that should be
tested independently, but please bear with me.) You don't actually
want to do the file copy, but you want to make sure that the file copy
API is appropriately called.

Using win32com, I might then use win32file.CopyF ile, but from testing
code I'd like to replace that with a different call, a test function
that I supply. Ideally I could replace the whole module, after
importing the module under test, I could do something like 'win32com =
__dict__', and assuming I had a CopyFile function defined in the test
module, it'd hook right in. However, I'm suspicious that this
wouldn't actually override the win32com seen by the other module that
will actually do the execution (and I don't yet understand Python's
execution model sufficiently).

At present, ModuleTest.py does 'from Module import *' and then makes
calls on Module without the "Module." decoration. Since typically a
file named SomeClass.py will only define the SomeClass class, this is
more of a convenience than anything. If this is a barrier to
replacing APIs, though, well, I can change it.

Any insight is helpful, thanks.
-tom!
Nov 20 '06 #1
10 1709
Tom i think i know what you are talking about. If i do then i have recently
had to deal with something similar.

One approach could be to create a module that acts as an interface between
your code and the api whos functions you wish to use.

The interface would consist of wrapper functions for all of the functions
you wish to use. This interface will only include the wrappers for the api
functions you will actually use.

Now you have created the interface you will need to import the module whose
functions you want to wrap up in the interface.

At this point you can either import a stub or the real thing. The code
snippet would look like this

try:
import stub_api as api
except:
import real_api as api

def get():
return api.get()

............... .......etc
So when you are testing you make sure that the stub_api module is available
to the interface, the try will succeed and you will call stub functions. If
you do not make the stub_api available then the try will fail, the exception
will be handled by importing the real_api. In either case functions will be
callable by using the api. as you have effectively renamed what is being
imported to api.

The advantages of this are that you only make visible to your code the
fuctions of the api that you are going to use. And you are isolating changes
to code to the interface only.

Ok hope this helps.

Steve

"Tom Plunket" <to***@fancy.or gwrote in message
news:ag******** *************** *********@4ax.c om...
I've got a bunch of code that runs under a bunch of unit tests. It'd
be really handy if when testing I could supply replacement
functionality to verify that the right things get called without those
things actually getting called, since frequently those calls take a
long time to execute and do things that, well, I don't really want to
do.

E.g. imagine you've got a routine which takes a filespec, generates a
list of files, and copies those files from one location to another. (I
realize that this example could be multiple functions that should be
tested independently, but please bear with me.) You don't actually
want to do the file copy, but you want to make sure that the file copy
API is appropriately called.

Using win32com, I might then use win32file.CopyF ile, but from testing
code I'd like to replace that with a different call, a test function
that I supply. Ideally I could replace the whole module, after
importing the module under test, I could do something like 'win32com =
__dict__', and assuming I had a CopyFile function defined in the test
module, it'd hook right in. However, I'm suspicious that this
wouldn't actually override the win32com seen by the other module that
will actually do the execution (and I don't yet understand Python's
execution model sufficiently).

At present, ModuleTest.py does 'from Module import *' and then makes
calls on Module without the "Module." decoration. Since typically a
file named SomeClass.py will only define the SomeClass class, this is
more of a convenience than anything. If this is a barrier to
replacing APIs, though, well, I can change it.

Any insight is helpful, thanks.
-tom!

Nov 20 '06 #2
Tom Plunket wrote:
I've got a bunch of code that runs under a bunch of unit tests. It'd
be really handy if when testing I could supply replacement
functionality to verify that the right things get called without those
things actually getting called, since frequently those calls take a
long time to execute and do things that, well, I don't really want to
do.

E.g. imagine you've got a routine which takes a filespec, generates a
list of files, and copies those files from one location to another. (I
realize that this example could be multiple functions that should be
tested independently, but please bear with me.) You don't actually
want to do the file copy, but you want to make sure that the file copy
API is appropriately called.

Using win32com, I might then use win32file.CopyF ile, but from testing
code I'd like to replace that with a different call, a test function
that I supply. Ideally I could replace the whole module, after
importing the module under test, I could do something like 'win32com =
__dict__', and assuming I had a CopyFile function defined in the test
module, it'd hook right in. However, I'm suspicious that this
wouldn't actually override the win32com seen by the other module that
will actually do the execution (and I don't yet understand Python's
execution model sufficiently).

At present, ModuleTest.py does 'from Module import *' and then makes
calls on Module without the "Module." decoration. Since typically a
file named SomeClass.py will only define the SomeClass class, this is
more of a convenience than anything. If this is a barrier to
replacing APIs, though, well, I can change it.

Any insight is helpful, thanks.
-tom!
Whatever points to something in Python is what is called.

example

def abc(a, b):
z=a+b
return z

def efg(a, b):
z=a*b
return z
f=abc
print f(1,2)
f=efg
print f(1,2)

As long as you implement the full API, you can replace functions
with others quite easily. Note: this also works with class methods.

Observation: IMHO it is going to be really hard to keep your functional
shims up to date as you change your REAL functions/methods. You also
are testing different code than you are executing which doesn't make
the unit tests nearly as valuable.

-Larry
Nov 20 '06 #3
Larry Bates wrote:
Whatever points to something in Python is what is called.
Right.
f=abc
print f(1,2)
f=efg
print f(1,2)
Sure. I realized after posting that what I really wanted to ask was
how to replace an imported module. Steve's earlier response is akin
to what I'm looking for, although with far too much work. ;)
As long as you implement the full API, you can replace functions
with others quite easily. Note: this also works with class methods.
Right, I can replace a function at a time:

import os

def my_join(*args):
return '/'.join(*args)

os.path.join = my_join

....but I'm wondering if there's a straight-forward way to inject the
local list of functions into another module.

What I ran into after posting is that I don't know how to get the
__dict__ for the module's scope; is it available? I.e. I can do
'print os.path.__dict_ _', but 'print __dict__' at module scope results
in a NameError since __dict__ is undefined at that scope. However, if
I could do 'os.path = local_module' somehow, I'd be getting basically
what I want. Maybe I could just have a my_os_path.py, and in my test
code import it as os.path, and then the other code would see that
os.path is already loaded and not bother doing so again? Haven't
tried it, am interested in the answer, but I do have enough to work
with for now... Replacing a module in this way also ensures that I'm
only calling the parts of the API that I intend to call, which is a
handy side-effect.
Observation: IMHO it is going to be really hard to keep your functional
shims up to date as you change your REAL functions/methods. You also
are testing different code than you are executing which doesn't make
the unit tests nearly as valuable.
Depends on what all I'm mocking. If I'm mocking an expensive service
across the internet that is time consuming to connect to, or even
mocking a well-defined system API call that is similarly time
consuming to call, then I can stub something in that doesn't do
anything but validate the arguments. Additionally, I can "document"
the caveats and possibly-errant behavior with my mock, and
subsequently get automatic testing that I'm dealing with API bugs
properly.

These aren't functional shims, which would imply that they sit in
between two other bits of interesting code under my control. They're
functional leaves, which are the end point of the operational chain.
E.g. I queue up all of the commands that I'm sending to the database
and verify that they're what I intended to send, and I accept that the
database will interpret those commands as it should.

Thanks for the feedback, regardless,
-tom!
Nov 21 '06 #4
At Monday 20/11/2006 21:52, Tom Plunket wrote:
>What I ran into after posting is that I don't know how to get the
__dict__ for the module's scope; is it available? I.e. I can do
'print os.path.__dict_ _', but 'print __dict__' at module scope results
in a NameError since __dict__ is undefined at that scope. However, if
I could do 'os.path = local_module' somehow, I'd be getting basically
what I want. Maybe I could just have a my_os_path.py, and in my test
code import it as os.path, and then the other code would see that
os.path is already loaded and not bother doing so again? Haven't
tried it, am interested in the answer, but I do have enough to work
with for now... Replacing a module in this way also ensures that I'm
only calling the parts of the API that I intend to call, which is a
handy side-effect.
You can do this:
import my_os_path
os.path = my_os_path

That shows also that you can use the module as its own namespace. No
need to use __dict__ explicitely. dir(module) gives you the names
defined in the module. From inside the module self, use globals().
--
Gabriel Genellina
Softlab SRL

_______________ _______________ _______________ _____
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
Nov 21 '06 #5
Tom Plunket wrote:
I've got a bunch of code that runs under a bunch of unit tests. It'd
be really handy if when testing I could supply replacement
functionality to verify that the right things get called without those
things actually getting called, since frequently those calls take a
long time to execute and do things that, well, I don't really want to
do.
Congratulation, you just rediscovered the MockObject.

(snip)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom. gro'.split('@')])"
Nov 21 '06 #6
Depends on what all I'm mocking. If I'm mocking an expensive service
across the internet that is time consuming to connect to, or even
mocking a well-defined system API call that is similarly time
consuming to call, then I can stub something in that doesn't do
anything but validate the arguments. Additionally, I can "document"
the caveats and possibly-errant behavior with my mock, and
subsequently get automatic testing that I'm dealing with API bugs
properly.
When I find myself doing what you describe, I often introduce
a "testing" mode switch into the equation. It is normally a
keyword argument to functions that I then interrogate in the
function. If I'm running in testing mode, I execute the less
"expensive" testing code. If I'm not in test mode, I don't.
There may be better ways, but this has worked well for me.

-Larry
Nov 21 '06 #7
Bruno Desthuilliers wrote:
I've got a bunch of code that runs under a bunch of unit tests. It'd
be really handy if when testing I could supply replacement
functionality to verify that the right things get called without those
things actually getting called, since frequently those calls take a
long time to execute and do things that, well, I don't really want to
do.

Congratulation, you just rediscovered the MockObject.
How do I rediscover something that I'm explicitly implementing? I had
set out specifically to mock an API, there was no rediscovery going
on.

That does allow me to discover via Google the Python Mock Module,
though, which may well do what I want:

from TestedModule import *

my_win32file = Mock({
'CopyFile': 'CopyFile called'
}

win32file = my_win32file

....perhaps.

I'm just a bit loathe to download and install more stuff when
something simpler appears to be near-at-hand. ...especially
considering the page describing this module doesn't offer any download
links! http://python-mock.sourceforge.net/
-tom!
Nov 22 '06 #8
On Tue, Nov 21, 2006 at 05:00:32PM -0800, Tom Plunket wrote:
Bruno Desthuilliers wrote:
I've got a bunch of code that runs under a bunch of unit tests. It'd
be really handy if when testing I could supply replacement
functionality to verify that the right things get called without those
things actually getting called, since frequently those calls take a
long time to execute and do things that, well, I don't really want to
do.
Congratulation, you just rediscovered the MockObject.

How do I rediscover something that I'm explicitly implementing? I had
set out specifically to mock an API, there was no rediscovery going
on.

That does allow me to discover via Google the Python Mock Module,
though, which may well do what I want:

from TestedModule import *

my_win32file = Mock({
'CopyFile': 'CopyFile called'
}

win32file = my_win32file

...perhaps.

I'm just a bit loathe to download and install more stuff when
something simpler appears to be near-at-hand. ...especially
considering the page describing this module doesn't offer any download
links! http://python-mock.sourceforge.net/
How about mini-mock:
http://blog.ianbicking.org/minimock.html
>

-tom!
--
http://mail.python.org/mailman/listinfo/python-list
Nov 22 '06 #9
I'm just a bit loathe to download and install more stuff when
something simpler appears to be near-at-hand. ...especially
considering the page describing this module doesn't offer any download
links! http://python-mock.sourceforge.net/
How about mini-mock:
http://blog.ianbicking.org/minimock.html
Oh yeah, and the download link for python-mock is:
http://sourceforge.net/project/showf...roup_id=138804
Nov 22 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
15295
by: yaipa | last post by:
What would be the common sense way of finding a binary pattern in a ..bin file, say some 200 bytes, and replacing it with an updated pattern of the same length at the same offset? Also, the pattern can occur on any byte boundary in the file, so chunking through the code at 16 bytes a frame maybe a problem. The file itself isn't so large, maybe 32 kbytes is all and the need for speed is not so great, but the need for accuracy in the...
2
5730
by: Prakash Wadhwani | last post by:
I have an Access Database called "TEMP" On Opening the "TEMP" Database, I need to use VBA (programmatically): a) to connect to another database called "PMF" b) the PMF database requires a password on opening called "allow" c) to link 5 Tables in the PMF Database ... Table1,2,3,4,5 How do I accomplish the above programmatically using VBA ? Thx & Best Rgds,
4
3997
by: u7djo | last post by:
Hi, I'm currently building an application in Access and as part of this need to import forms and modules from another database. Some of the imports will be revisions of existing forms/modules so I need to check and see they already exist before importing them so I can remove the old one. I've figured out how to remove the classes and modules before importing the revised version but I'm having problems with the forms. I was hoping to use...
5
6834
by: Carlo Marchesoni | last post by:
From an aspx page (A.aspx) I open another one (B.aspx - for table lookup). When the user selects an entry in B.aspx I would like to force a button's event in A.aspx to be fired. I guess the only way is using javascript - does anybody have a sample for this ? Thanks
3
9182
by: Codex Twin | last post by:
Hello apologies if this is the wrong newsgroup to be sending this to. Basically, I have an ASP.NET application that I am trying to force to use a proxy server settings. This can be done by accessing machine.config, but I do not have access to the client machine.config machine. Therefore every request made from this application must have these proxy server settings. But I am having problems writing the code for this.
1
1627
by: WT | last post by:
Hello, I was using hastable to store collection of objects of the same class MyClass. I start replacing this hastable by a Dictionary<string,MyClass>....but I was storing this hastable in cache an accessing it from another component in an Provider. This component was developped in a separate assembly and loaded dynamically. There was no need for it to know Myclass, it was simply retrieving a hastable and using the ToString() method on...
1
2455
by: jcz_ch | last post by:
I have to programmatically configure IIS from a windows form application. I tried the code I found here: ttp://weblogs.asp.net/jezell/archive/2003/09/17/27869.aspx and it works fine. The thing is that I have to use another logging module instead of the default one... I spent hours reading the metabase documentation but everything I've tried diten't work... Does anyone has an idea hoe to do that? P.S.
2
2391
by: James Fifth | last post by:
Hello and God Bless, I am stumped trying to get a simple xml database replacing certain data with other data programmatically. This is what my xml looks like. **************************************** <Root_Element> <Topic index="1"> <Sub_Topic> <Notes></Notes>
3
2019
by: aine_canby | last post by:
Hi, I'm working with a number of scripts which were written years ago for my company for Python 2.2, and I'd like to update for Python 2.5. I have written a script to add # -*- coding: cp1252 -*- to the beginning of all my scripts, and that has fixed the encoding issues. Another issue was the use of - from _xmlplus.dom import minidom
0
9930
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9776
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10720
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10403
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9555
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5781
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4597
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4196
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3225
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.