2 years ago
#73497
Si si
Good practice to manage multiple processes with python
I have multiple processes I need to run.
Currently I just run them all over the windows process manager:
I would like to create one python script to manage them all with "stop"/"start" and read each output.
Any recommended open-source process manager that can do that ?
I have tried to do it by myself but getting many issues like sudden death of process, process stay alive after script shutdown and many more...
MY process manager:
def work(infomation):
command = ['python', filename]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
shell=False, encoding='utf-8', cwd=full_path)
while True:
try:
realtime_output = process.stdout.readline()
file_obj.refresh_from_db(fields=['enabled'])
if not file_obj.enabled:
print(f'{file_obj.phone_id} killing ....')
process.kill() # TODO: event.set() add process.wait() (until terminated)\
process.wait()
# the poll will stop the loop
if realtime_output or process.poll() is not None:
# process been terminated
if realtime_output == '' and process.poll() is not None:
break
# process having output
if realtime_output and realtime_output != '\n':
# print / handle output
except:
print('traceback WHILE LOOP' + traceback.format_exc())
# wait for termination of process on error
if process:
process.wait()
now = datetime.now().strftime('%m_%d_%Y_%H-%M-%S')
# dump error information to file
with open(f"{DUMPS_DIR}{now}.log", "w+") as logf:
logf.write("SCK_DEBUG_ERROR:\n time:{}\n\n {} \n\n".format(now, str(traceback.format_exc())))
break
```
I triger "work" as a thread.
python
multiprocessing
0 Answers
Your Answer