473,722 Members | 2,295 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Rookie Speaks

I'm a python rookie, anyone have and suggestions to streamline this
function? Thanks in advance.....
def getdata(myurl):
sock = urllib.urlopen( myurl)
xmlSrc = sock.read()
sock.close()

xmldoc = minidom.parseSt ring(xmlSrc)

def getattrs(weathe rAttribute):
a = xmldoc.getEleme ntsByTagName(we atherAttribute)
return a[0].firstChild.dat a

currname = getattrs("name" )
currtemp = getattrs("fahre nheit")
currwind = getattrs("wind" )
currdew = getattrs("dewpo int")
currbarom = getattrs("relat ive_humidity")
currhumid = getattrs("barom etric_pressure" )
currcondi = getattrs("condi tions")

print "%13s\t%s\t%s\t %s\t%s\t%s\t%s" % (currname, currtemp,
currwind, currbarom, currdew, currhumid, currcondi)

Jul 18 '05 #1
10 1766
|Thus Spake William S. Perrin On the now historical date of Wed, 07 Jan
2004 17:37:57 -0600|
I'm a python rookie, anyone have and suggestions to streamline this
function? Thanks in advance.....


Please define "streamline " in this context.

Do you mean:
faster
smaller
easier to read
etc.

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #2
Sorry, I guess is it efficient? That is if I called it 1000 times.....

Samuel Walters wrote:
|Thus Spake William S. Perrin On the now historical date of Wed, 07 Jan
2004 17:37:57 -0600|

I'm a python rookie, anyone have and suggestions to streamline this
function? Thanks in advance.....

Please define "streamline " in this context.

Do you mean:
faster
smaller
easier to read
etc.

Sam Walters.


Jul 18 '05 #3
|Thus Spake William S. Perrin On the now historical date of Wed, 07 Jan
2004 17:45:38 -0600|
Sorry, I guess is it efficient? That is if I called it 1000 times.....


Ponders... Even "efficient" has a loose meaning here. Since you describe
running it a thousand times, I'll assume you mean speed of execution.

There's a saying "Premature optimization is the root of all evil." Which
is another way of saying "Try it, and if it's too slow, figure out what
the hold up is. If it's not too slow, don't mess with it." So, try
running it in the context you need it in. Nothing about your code screams
"bad implementation. " In fact, it's quite clearly written. Still, you
won't know if it's too slow until you try it.

There's a way to get a definitive answer on how how fast it's running
through the profile module in python. Do some research on that module. If
you're confused about it, come back and ask more questions then.

Take into consideration that it may not be your code that's slow, but
rather the way you're getting your information. This is called being "I/O
bound." The holdup might not be the program, but instead the disk or the
network. After all, you can't process information until you have the
information. The profile module will help you to see if this is the
problem.

One of the slicker solutions to slow code is the psyco module. It can
give an amazing speed boost to many processing intensive functions, but it
can sometimes even slow down your problem.

If you'd like to see an example of both the psyco and profile modules in
action, let me know and I'll give you some more understandable code that I
once wrote to see what types of things psyco is good at optimizing.

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #4
sdd
William S. Perrin wrote:
I'm a python rookie, anyone have and suggestions to streamline this
function? Thanks in advance.....

... currname = getattrs("name" )
currtemp = getattrs("fahre nheit")
currwind = getattrs("wind" )
currdew = getattrs("dewpo int")
currbarom = getattrs("relat ive_humidity")
currhumid = getattrs("barom etric_pressure" )
currcondi = getattrs("condi tions")

print "%13s\t%s\t%s\t %s\t%s\t%s\t%s" % (currname, currtemp, currwind,
currbarom, currdew, currhumid, currcondi)


How about:
name, temp, wind, dew, barom, humid, condi = map(getattrs,
"name fahrenheit wind dewpoint relative_humidi ty "
" barometric_pres sure conditions".spl it())

print "%13s\t%s\t%s\t %s\t%s\t%s\t%s" % (name, temp, wind,
barom, dew, humid, condi)
-Scott David Daniels
Sc***********@A cm.Org
Jul 18 '05 #5
Anytime you find yourself repeating the same pattern of
code (i.e. the getattrs bit), there's usually a more elegant
way of doing it.

def getdata(myurl):
sock = urllib.urlopen( myurl)
xmlSrc = sock.read()
sock.close()

xmldoc = minidom.parseSt ring(xmlSrc)

def getattrs(weathe rAttribute):
a = xmldoc.getEleme ntsByTagName(we atherAttribute)
return a[0].firstChild.dat a

attributes = ['name', 'fahrenheit', 'wind',
'dewpoint', 'relative_humid ity',
'barometric_pre ssure', 'conditions']

current = {}

for a in attributes:
current[a] = getattrs(a)

format_str = "%13s"+"\t%s"*( len(attributes)-1)
print format_str % tuple([current[a] for a in attributes])
OR, if all you want is to print your numbers, skip the dictionary-

attributes = ['name', 'fahrenheit', 'wind',
'dewpoint', 'relative_humid ity',
'barometric_pre ssure', 'conditions']

format_str = "%13s"+"\t%s"*( len(attributes)-1)
print format_str % tuple([getattrs(a) for a in attributes])
Jul 18 '05 #6
William S. Perrin wrote:

I thinke your function has a sane design :-) XML is slow by design, but in
your case it doesn't really matter, because is probably I/O-bound, as
already pointed out by Samuel Walters.

Below is a slightly different approach, that uses a class:

class Weather(object) :
def __init__(self, url=None, xml=None):
""" Will accept either a URL or a xml string,
preferrably as a keyword argument """
if url:
if xml:
# not sure what would be the right exception here
# (ValueError?), so keep it generic for now
raise Exception("Must provide either url or xml, not both")
sock = urllib.urlopen( url)
try:
xml = sock.read()
finally:
sock.close()
elif xml is None:
raise Exception("Must provide either url or xml")
self._dom = minidom.parseSt ring(xml)

def getAttrFromDom( self, weatherAttribut e):
a = self._dom.getEl ementsByTagName (weatherAttribu te)
return a[0].firstChild.dat a

def asRow(self):
# this will defeat lazy attribute lookup
return "%13s\t%s\t%s\t %s\t%s\t%s\t%s" % (self.name,
self.fahrenheit , self.wind, self.barometric _pressure,
self.dewpoint, self.relative_h umidity, self.conditions )
return

def __getattr__(sel f, name):
try:
value = self.getAttrFro mDom(name)
except IndexError:
raise AttributeError(
"'%.50s' object has no attribute '%.400s'" %
(self.__class__ , name))
# now set the attribute so it need not be looked up
# in the dom next time
setattr(self, name, value)
return value

This has a slight advantage if you are interested only in a subset of the
attributes, say the temperature:

for url in listOfUrls:
print Weather(url).fa hrenheit

Here getAttrFromDom( ) - the equivalent of your getattrs() - is only called
once per URL. The possibility to print a tab-delimited row is still there,

print Weather(url).as Row()

but will of course defeat this optimization scheme.

Peter
Jul 18 '05 #7
Samuel Walters <sw************ *@yahoo.com> writes:
If you'd like to see an example of both the psyco and profile modules in
action, let me know and I'll give you some more understandable code that I
once wrote to see what types of things psyco is good at optimizing.


I think this is generally interesting, and would be curious to see it,
if you'd care to share.
Jul 18 '05 #8
|Thus Spake Jacek Generowicz On the now historical date of Thu, 08 Jan
2004 11:43:01 +0100|
Samuel Walters <sw************ *@yahoo.com> writes:
If you'd like to see an example of both the psyco and profile modules
in action, let me know and I'll give you some more understandable code
that I once wrote to see what types of things psyco is good at
optimizing.


I think this is generally interesting, and would be curious to see it,
if you'd care to share.


Sure thing. The functions at the top are naive prime enumeration
algorithms. I chose them because they're each of a general "looping"
nature and I understand the complexity and methods of each of them. Some
use lists (and hence linearly indexed) methods and some use dictionary(
and hence are has bound). One of them, sieve_list is commented out because
it has such dog performance that I decided I wasn't interested in
how well it optimized.

These tests are by no means complete, nor is this probably a good example
of profiling or the manner in which psyco is useful. It's just from an
area where I understood the algorithmic bottlenecks to begin with.

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

from math import sqrt
def primes_list(Lim its = 1,KnownPrimes = [ 2 ]):
RetList = KnownPrimes
for y in xrange(2,Limits + 1):
w = y
p, r = 0,0
for x in RetList:
if x*x > w:
RetList.append( w)
break
p,r = divmod(y,x)
if r == 0:
w = p
return RetList

def primes_dict(Lim its = 1,KnownPrimes = [ 2 ]):
RetList = KnownPrimes
RetDict = {}
for x in KnownPrimes:
RetDict[x] = 1
w = x + x
n = 2
while w <= Limits + 1:
RetDict[w] = n
w += x
n += 1
p, r = 0,0
for y in xrange(2, Limits + 1):
for x, z in RetDict.iterite ms():
if x*x > y:
RetDict[y] = 1
break
p,r = divmod(y,x)
if r == 0:
RetDict[y] = p
break
return RetList

def sieve_list(Limi ts = 1, KnownPrimes = [ 2 ]):
RetList = KnownPrimes
CompList = [ ]
for y in xrange(2, Limits + 1):
if y not in CompList:
w = y
n = 1
while w <= Limits:
CompList.append (w)
w += y
n += 1
return RetList

def sieve_list_2(Li mits = 1, KnownPrimes = [ 2 ]):
SieveList = [ 1 ]*(Limits )
RetList = [ ]
for y in xrange(2, Limits + 1):
if SieveList[y-2] == 1:
RetList.append( y)
w = y + y
n = 2
while w <= Limits + 1:
SieveList[w - 2] = n
w += y
n += 1
return RetList

def sieve_dict(Limi ts = 1, KnownPrimes = [ 2 ]):
SieveDict = { }
RetList = KnownPrimes
for x in KnownPrimes:
SieveDict[x] = 1
w = x + x
n = 2
while w <= Limits + 1:
SieveDict[w] = n
n += 1
w += x

for y in xrange(2, Limits + 1):
if not SieveDict.has_k ey(y):
RetList.append( y)
w = y
n = 1
while w <= Limits + 1:
SieveDict[w] = n
w += y
n += 1
return RetList

if __name__ == "__main__":
import sys
import profile
import pstats

import psyco

#this function wraps up all the calls that we wish to benchmark.
def multipass(numbe r, args):
for x in xrange(1, number + 1):
primes_list(arg s, [ 2 ])
print ".",
sys.stdout.flus h()
primes_dict(arg s, [ 2 ])
print ".",
sys.stdout.flus h()
#Do not uncomment this line unless you have a *very* long time to wait.
#sieve_list(arg s)
sieve_dict(args , [ 2 ])
print ".",
sys.stdout.flus h()
sieve_list_2(ar gs, [ 2 ])
print "\r \r%i/%i"%(x, number),
sys.stdout.flus h()
print "\n"

#number of times through the test
passes = 5
#find all primes up to maximum
maximum = 1000000

#create a profiling instance
#adjust the argument based on your system.
pr = profile.Profile ( bias = 7.5e-06)

#run the tests
pr.run("multipa ss(%i, %i)"%(passes,ma ximum))
#save them to a file.
pr.dump_stats(" primesprof")

#remove the profiling instance so that we can get a clean comparison.
del pr

#create a profiling instance
#adjust the argument based on your system.
pr = profile.Profile ( bias = 7.5e-06)

#"recompile" each of the functions under consideration.
psyco.bind(prim es_list)
psyco.bind(prim es_dict)
psyco.bind(siev e_list)
psyco.bind(siev e_list_2)
psyco.bind(siev e_dict)

#run the tests
pr.run("multipa ss(%i, %i)"%(passes,ma ximum))
#save them to a file
pr.dump_stats(" psycoprimesprof ")

#clean up our mess
del pr

#load and display each of the run-statistics.
pstats.Stats('p rimesprof').str ip_dirs().sort_ stats('cum').pr int_stats()
pstats.Stats('p sycoprimesprof' ).strip_dirs(). sort_stats('cum ').print_stats( )

Jul 18 '05 #9
On Fri, 2004-01-09 at 05:25, Samuel Walters wrote:
|Thus Spake Jacek Generowicz On the now historical date of Thu, 08 Jan
2004 11:43:01 +0100|
Samuel Walters <sw************ *@yahoo.com> writes:
If you'd like to see an example of both the psyco and profile modules
in action, let me know and I'll give you some more understandable code
that I once wrote to see what types of things psyco is good at
optimizing.


I think this is generally interesting, and would be curious to see it,
if you'd care to share.


Sure thing. The functions at the top are naive prime enumeration
algorithms. I chose them because they're each of a general "looping"
nature and I understand the complexity and methods of each of them. Some
use lists (and hence linearly indexed) methods and some use dictionary(
and hence are has bound). One of them, sieve_list is commented out because
it has such dog performance that I decided I wasn't interested in
how well it optimized.


Out of curiosity I ran your code, and obtained these results:

Fri Jan 9 08:30:25 2004 primesprof

23 function calls in 2122.530 CPU seconds

....

Fri Jan 9 08:43:24 2004 psycoprimesprof

23 function calls in -3537.828 CPU seconds

Does that mean that Armin Rigo has slipped some form of Einsteinian,
relativistic compiler into Psyco? I am reminded of the well-known
limerick:

There once was a lady called Bright,
Who could travel faster than light.
She went out one day,
In a relative way,
And came back the previous night.

--

Tim C

PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere
or at http://members.optushome.com.au/tchur/pubkey.asc
Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQA//dVyeJFGqer5k9AR AsuKAKDOA3t41Zq Qy9QNIp9pZ2uuDE 8yQACgo0wM
1w6Kzm37Xp/c3k5SaNk9iv4=
=XnLz
-----END PGP SIGNATURE-----

Jul 18 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

114
7717
by: muldoon | last post by:
Americans consider having a "British accent" a sign of sophistication and high intelligence. Many companies hire salespersons from Britain to represent their products,etc. Question: When the British hear an "American accent," does it sound unsophisticated and dumb? Be blunt. We Americans need to know. Should we try to change the way we speak? Are there certain words that sound particularly goofy? Please help us with your advice on this...
11
2279
by: Don Bruder | last post by:
Got a stumper here. I imagine that for someone experienced in C++, this is too pathetic for words. For a rookie, using this project as a sort of "midterm exam" in his self-taught "how to program in C++" course, it's seeming to be an insurmountable problem. Can anyone assist? I'm at wit's end here. Everything *LOOKS* (to this rookie, anyway) correct, but the compiler is barfing on it. Here's the class declaration for Bar class Bar {
6
5794
by: bigjmt | last post by:
Sorry to bother you guys with what I though would be an easy task. I have a table in my database were I would like one of the rows to increment a number for each row. I want the first row to start at 1000 and keep on incrementing by 1 'till the end of the rows (about 2.7 million rows). I though this would be a piece of cake, but I just can't seem to find anything like it on the internet...weird. Anyways, I'm just a rookie in sql, any help...
8
4669
by: Tom | last post by:
Please help. I need a quick little scrpit to place on a web page that will count how many days have passed since January 1, 1970. I have ZERO experience writing ANY scripts. Anyone have any suggestions? As for what is displayed on the screen, I just want it to say: Today is: 12578 (or however many days have passed since that date.)
3
2522
by: Dimitris \(GIS\) | last post by:
Hi I am a JavaScript rookie. I have an applet and want to change it's parameters with multiple links in my page. How can I do it? this is my code: <applet code="GIS.class"> <param id="moneyID" name="money" value="100"> <param id="vicimsID" name="vicims" value="2"> <param id="countryID" name="country" value="UK">
2
1462
by: Daveg | last post by:
Hello, Rookie here. I am new at c# and put together a script which is working fine grabbing some elements from an XML file that I created. The problem is that I would like to put at the top of the xml file some comments which document time, date, etc. If I open the file with the comments in a web browser it displays fine, but if I use the GetElementsByTagName ("Some Element"); I get an error as if the method does not ignore the...
0
1202
by: EZboy | last post by:
Rookie question ..... can anybody please send me a sample code that illustrates : how to display an mpg video clip on a new window that is being opened from within an application ? Many Thanks, EZboy
3
1258
by: Dst | last post by:
Hi i'm trying to make a very simple web site using visual studio 2005. I'm completely noob at this so i need some pointers to get me started. As i understand frames should not be used in asp.net ? But what are the alternatives ? My main page is very simple, it is created using a table and looks like this --------------------------------------------------
21
4260
by: AsheeG87 | last post by:
Hey Everyone~ I'm still a C++ Rookie so please bear with me on this. I'm doing a temperature conversion program with prototype functions. Basicly, I was wondering if some of you would take a look at my code and critique it for me. I'm mostly concerned with how prototype functions work and if I designed them correctly in my code. Your participation would be greatly appreciated! My Code:
0
8863
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9384
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9238
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9088
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8052
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
4502
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4762
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3207
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2602
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.