I'm running a python script using apache and modpython on a Linux environment. It integrates in a web tool which allow file processing. The part of my script that includes file processing can have very long execution time. In the original version of my code, the script waits for the files to be processed, and at the end it returns some html with a link to download the resulting file.
<html> <body> <form enctype="multipart/form-data" action="./my_script.py/main" method="post"> <div> <input type="file" name="file"> </div> <div> <input type="submit" value="Upload"> </div> </body> </html>
def main(file): process(file) return "<p> Download your file <a href='./%s'></a>", % file def process(file) #some file treatment here, and a resulting file is stored in current directory
I want to write a feature which would allow the user to receive the resulting file by email. In that case once he has uploaded his file, I would like to redirect him to a page, he could keep on using the web tool, while his file is being processed server sided and thus the user is not being Unix forks. I have made several tests with these 3 options but I always get blocked by the running script. For what I have understood, multiprocessing is best suited for my case, so I have tried this:
def main(file, receiver_mail_address): p = Process(target=process_and_email, args=(file, receiver_mail_address) p.start() return "<p> The resulting files will be emailed to you at %s.</p>" % receiver_mail_address def process_and_email(file, receiver_mail_address): #some file processing here, and emailing. these functions work perfectly as expected.
In this situation I have skipped the
p.join() step which is said in the python docs to
"Block the calling thread until the process whose join() method is called terminates or until the optional timeout occurs."
But in my case, it is still blocked. It means I have to wait for my process
p to being over before reaching the
return statement. How could I do that ?
I have tried to change to the
subprocess module. So I have put the
process_and_email function into a new file called process_and_email.py and I modified the main script :
def main(file, receiver_mail_address): directory = os.path.firname(__file__) path = os.path.join(directory, 'process_and_email.py') subprocess.Popen(['python2.7', path, file, receiver_mail_address], shell=True) return "<p> The resulting files will be emailed to you at %s.</p>" % receiver_mail_address
I still have the same problem : I cannot reached the
return statement before the process_and_email.py file has been totally executed.