473,385 Members | 1,693 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Multithreaded python script calls the COMMAND LINE

I have python script does ftp download in a multi threaded way. Each
thread downloads a file, close the file, calls the comman line to
convert the .doc to pdf. Command line should go ahead and convert the
file. My question is, when each thread calls the command line, does one
command line process all the request, or each thread creates a one
command line process for themselves and executes the command? For some
reason I am getting "File '1' is alread exists, Do you want to
overwrite [y/n]?" I have to manually enter 'y' or 'n' for the python
script to complete.

Dec 7 '06 #1
3 1660
On 7 dic, 17:36, "johnny" <rampet...@gmail.comwrote:
I have python script does ftp download in a multi threaded way. Each
thread downloads a file, close the file, calls the comman line to
convert the .doc to pdf. Command line should go ahead and convert the
file. My question is, when each thread calls the command line, does one
command line process all the request, or each thread creates a one
command line process for themselves and executes the command? For some
reason I am getting "File '1' is alread exists, Do you want to
overwrite [y/n]?" I have to manually enter 'y' or 'n' for the python
script to complete.
That depends on how you invoke it: os.system creates a new shell which
in turn creates a new process; the spawn* functions do that directly.
Anyway, if you have many conversion processes running, you should pass
them unique file names to avoid conflicts.
Where does the '1' name come from? If it's you, don't use a fixed name
- the tempfile module may be useful.

Dec 7 '06 #2

gagsl...@yahoo.com.ar wrote:
>
That depends on how you invoke it: os.system creates a new shell which
in turn creates a new process; the spawn* functions do that directly.
I am using os.system. Here is my code

import ftplib, posixpath, threading
from TaskQueue import TaskQueue

def worker(tq):
while True:
host, e = tq.get()

c = ftplib.FTP(host)
c.connect()
try:
c.login()
p = posixpath.basename(e)
ps_dir = r'H:/ftp_download/'
filename = download_dir+p
fp = open(filename, 'wb')
try: c.retrbinary('RETR %s' % e, fp.write)
finally: fp.close()
finally: c.close()
if (p.lower().endswith('.ps') ):
partFileName = p.split('.', 1)
movedFile = download_dir + p
#movedFile = p
finalFile = ps_dir + partFileName[0]+'.pdf'
encode_cmd = r'ps2pdf '+ movedFile + ' '+ finalFile
os.system(encode_cmd)

tq.task_done()

if __name__ == '__main__':
main()

def main():
q = TaskQueue()
#host = 'ftp.microsoft.com'
host = 'mysite.com'
c = ftplib.FTP(host)
c.connect()
try:
#c.login()
c.login()

#folder = '/deskapps/kids/'
folder = ''
for n in c.nlst(folder):
if n.lower().endswith('.ps'):
q.put((host, n))
finally: c.close()

numworkers = 4
for i in range(numworkers):
t = threading.Thread(target=worker, args=(q,))
t.setDaemon(True)
t.start()

q.join()
print 'Done.'
Anyway, if you have many conversion processes running, you should pass
them unique file names to avoid conflicts.
Where does the '1' name come from? If it's you, don't use a fixed name
- the tempfile module may be useful.
I am giving unique name to the converted file with .pdf. I think
something up with the thread. I am new to python, I am not sure what's
wrong.

Dec 7 '06 #3
At Thursday 7/12/2006 19:13, johnny wrote:
Anyway, if you have many conversion processes running, you should pass
them unique file names to avoid conflicts.
Where does the '1' name come from? If it's you, don't use a fixed name
- the tempfile module may be useful.

I am giving unique name to the converted file with .pdf. I think
something up with the thread. I am new to python, I am not sure what's
wrong.
Unless the ps2pdf program uses explicitely '1' as a filename, I don't
see where it may come from. Your python code doesn't generate it -
unless you have a 1.ps file.

[Quoting from a previous message]
>For some
reason I am getting "File '1' is alread exists, Do you want to
overwrite [y/n]?" I have to manually enter 'y' or 'n' for the python
script to complete.
Is that '1' just an example, or you always get that?
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ˇgratis!
ˇAbrí tu cuenta ya! - http://correo.yahoo.com.ar
Dec 8 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

52
by: Olivier Scalbert | last post by:
Hello , What is the python way of doing this : perl -pi -e 's/string1/string2/' file ? Thanks Olivier
1
by: Elbert Lev | last post by:
I started with Python two weeks ago and already saved some time and efforts while writing 2 programs: 1. database extraction and backup tool, which runs once a month and creates a snapshot of...
2
by: Michael Zhang | last post by:
My project uses Python-2.3.4 + Tkinter + PIL-1.1.4 to retrieve images from server and display those images. I created a thread (also a separate toplevel window) for displaying images and another...
1
by: Baggs | last post by:
Guys/Gals I've messed up my Python 2.4 install. I'm really new to Python (and Linux for that matter) and really wanted to try Python out. I have mandrake 10.1. The first thing I did was...
10
by: A.M | last post by:
Hi, I am having difficulty with shell scripting in Python. I use the following command to run a DOS command and put the return value in a Python variable:
4
by: pmcgover | last post by:
I enjoyed Paul Barry's September article in Linux Journal entitled, "Web Reporting with MySQL, CSS and Perl". It provides a simple, elegant way to use HTML to display database content without any...
3
by: Tristan | last post by:
Hello community: I post this because I could not find satisfactory answers in the posts generated by this nice group. I work on winXP. I have many little python applications in different folders,...
3
by: Jorgen Bodde | last post by:
Hi All, I am trying to make a debian package. I am following the tutorial by Horst Jens (http://showmedo.com/videos/video?name=linuxJensMakingDeb&fromSeriesID=37) and it is very informative....
9
by: Catherine Moroney | last post by:
I have one script (Match1) that calls a Fortran executable as a sub-process, and I want to write another script (Match4) that spawns off several instances of Match1 in parallel and then waits...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.