473,560 Members | 2,970 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

bettering my python

7 New Member
Hi guys

Firstly I would like to thank all those people who answer these questions on Bytes. Really you guys are fantastic.

Now I have seen guys post code on this site and some gurus come along and reduce it to a few lines of readable, frankly sexy code. I would love to see how optimised you guys can get this, just so I can learn from you.

General overview:

I create a folder with different files extracted from an incoming mail readying to be processed by different open office programs. Ie .xls for calc .doc for writer.
Later in the process I will make use of unoconv to handle this but:
I have created a whitelist of files I accept, though should there be an attachment with a file extension not supported the script then copies the downloaded email to an ignore folder for manual processing. But I will still process the remaining attachments, that is why i dont throw and exception.

I would love to optimise _locate_not_sup ported and _file_extension _check and get it to lovely readable pythonic

Expand|Select|Wrap|Line Numbers
  2.     def _file_extension_check(self):
  3.         '''check for ext validity, if none still return true just'''
  4.         self.logger.info("checking file extension validity")
  5.         whitelist = "*.pdf","*.rtf","*.doc","*.xls","*.tif","*.tiff"
  6.         root = root=self.currentEmailAttachmentsPath
  7.         pdfgen  = self._locate(whitelist[0], root)
  8.         rtfgen  = self._locate(whitelist[1], root)
  9.         docgen  = self._locate(whitelist[2], root)
  10.         xlsgen  = self._locate(whitelist[3], root)
  11.         tifgen  = self._locate(whitelist[4], root)
  12.         tiffgen = self._locate(whitelist[5], root)
  13.         self._locate_not_supported(whitelist, root)
  14.         return 1
  16.     def _locate_not_supported(self, patterns, root=None):
  17.         for path, dirs, files in os.walk(os.path.abspath(root)):
  18.             for file in files:
  19.                 name, ext = os.path.splitext(file)
  20.                 if ext != patterns[0][1:] and \
  21.                     ext != patterns[1][1:] and \
  22.                     ext != patterns[2][1:] and \
  23.                     ext != patterns[3][1:] and \
  24.                     ext != patterns[4][1:] and \
  25.                     ext != patterns[5][1:]:
  26.                     self._email_to_ignore_folder()
  27.                 else:
  28.                     return
  30.     def _locate(self,pattern, root=None):
  31.         for path, dirs, files in os.walk(os.path.abspath(root)):
  32.             for filename in fnmatch.filter(files, pattern):
  33.                 yield os.path.join(path, filename)
  35.     def _email_to_ignore_folder(self):
  36.         self.logger.info("email sent to ignore folder")
  37.         print self._emailpath + self._ignorefolder
  38.         shutil.move(self.currentEmailPath, self._emailpath + self._ignorefolder)
Thank you so much!
Oct 13 '08 #1
5 1540
2,851 Recognized Expert Moderator Specialist
You should call os.walk() once instead of multiple times. This example does just that. Maybe you could do something similar in your code.
Expand|Select|Wrap|Line Numbers
  1. import os
  3. def dir_list(dir_name, whitelist):
  4.     outputList = []
  5.     for root, dirs, files in os.walk(dir_name):
  6.         for f in files:
  7.             if os.path.splitext(f)[1] in whitelist:
  8.                 outputList.append(os.path.join(root, f))
  9.             else:
  10.                 # send to ignore folder??
  11.     return outputList
  13. dir_name = 'your_directory'
  14. print '\n'.join(dir_list(dir_name, ['.txt', '.tif']))
Oct 13 '08 #2
7 New Member
fantastic, thank you so much that is exactly what I was looking for!

I never realised that:
Expand|Select|Wrap|Line Numbers
  1. if os.path.splitext(f)[1] in whitelist:
was possible, but now it makes complete sense!

Kind regards
Oct 14 '08 #3
7 New Member
How about this?

Expand|Select|Wrap|Line Numbers
  1.     def _email_to_(self, where):
  2.         self.logger.info("sent email to %s folder" % where)
  3.         if self.emailDestination == None:
  4.             self.emailDestination = where
  6.     def _pidgeon_hole_email(self):
  7.         if self.emailDestination == "completed":
  8.             shutil.move(self.currentEmailPath, self._emailpath + self._completedfolder)
  9.         elif self.emailDestination == "ignore":
  10.             shutil.move(self.currentEmailPath, self._emailpath + self._ignorefolder)
  11.         elif self.emailDestination == "error":
  12.             shutil.move(self.currentEmailPath, self._emailpath + self._errorfolder)
  14.         self.emailDestination = None
What cool way is there to pythonise the above?

this is the main method making use of these classes methods above
Expand|Select|Wrap|Line Numbers
  1.     def process_inbox(self):
  2.         self._getconfig()    
  3.         self._setup_logger()
  4.         self.logger.info("starting to digest emails")
  5.         self._contains_emails()
  6.         for self.currentEmailPath in self.emailsPathsInInbox:
  7.             if self._extract_attachments():
  8.                 self._file_extension_check()
  9.                 if self._convert():
  10.                     if self._update_database():
  11.                         self._email_to_("completed")
  12.                     else:
  13.                         self._email_to_("error")
  14.                 else:
  15.                     self._email_to_("error")
  16.             self._pidgeon_hole_email()
Thanks so much
Oct 14 '08 #4
2,851 Recognized Expert Moderator Specialist
Not that there is anything "uncool" about your code, you could do this:
Expand|Select|Wrap|Line Numbers
  1. def _pidgeon_hole_email(self):
  2.     # possible values 'completed', 'ignore', 'error'
  3.     folder = getattr(self, "_%sfolder" % self.emailDestination, None)
  4.     if folder:
  5.         shutil.move(self.currentEmailPath, self._emailpath + folder)
  6.     self.emailDestination = None
Oct 19 '08 #5
7 New Member
oh i got you, actually your way is better!

Thanks for that!
Oct 20 '08 #6

Sign in to post your reply or Sign up for a free account.

Similar topics

by: Emile van Sebille | last post by:
QOTW: "If we get 2.3.3c1 out in early December, we could release 2.3.3 final before the end of the year, and start 2004 with a 100% bug-free codebase <wink>." -- Tim Peters "cjOr proWe vbCould vbSettle prpFor noEnglish prpIn adjHungarian noNotation :-)" -- noPeter Jack Jensen reveals that python on mac continues as an osx option only...
by: Emile van Sebille | last post by:
QOTW: "Have you ever used the copy module? I am *not* a beginner, and have used it *once* (and I can't remember what for, either)." -- Michael Hudson "It will likely take a little practice before this stuff rattles off your fingertips easily." -- Paul Rubin BJ MacNevin wants to run python from a CD....
by: Emile van Sebille | last post by:
QOTW (in the OS agnostic category): "There is a (very popular) Python package out there which exposes the win32 api. I'm not sure what it's called. (win32api? pythonwin? win32all?)" -- Francis Avila QOTW (in the popular vote category): "So far, python has been the easiest language to learn I've ever come across. I tried learning perl,...
by: Emile van Sebille | last post by:
QOTW (advanced interfaces track): "I'm firmly in favour of any language that can DWIMNWIS." -- Tim Delaney QOTW (MS roadkill track): "Underestimate MS at your own risk. It is one thing to not like MS, and possibly a reasonable position to take. However, underestimating them is for fools." -- Mark Hammond, on Microsoft's commitment to...
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.