2 years ago

#73497

test-img

Si si

Good practice to manage multiple processes with python

I have multiple processes I need to run.

Currently I just run them all over the windows process manager:

enter image description here

I would like to create one python script to manage them all with "stop"/"start" and read each output.

Any recommended open-source process manager that can do that ?

I have tried to do it by myself but getting many issues like sudden death of process, process stay alive after script shutdown and many more...

MY process manager:

def work(infomation):

    command = ['python', filename]
    process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
        shell=False, encoding='utf-8', cwd=full_path)

    while True:
        try:
            realtime_output = process.stdout.readline()

            file_obj.refresh_from_db(fields=['enabled'])
            if not file_obj.enabled:
                print(f'{file_obj.phone_id} killing ....')
                process.kill()  # TODO: event.set() add process.wait() (until terminated)\
                process.wait()
                # the poll will stop the loop

            if realtime_output or process.poll() is not None:

                # process been terminated
                if realtime_output == '' and process.poll() is not None:
                    break

                # process having output
                if realtime_output and realtime_output != '\n':
                    # print / handle output

        except:
            print('traceback WHILE LOOP' + traceback.format_exc())

            # wait for termination of process on error
            if process:
                process.wait()

            now = datetime.now().strftime('%m_%d_%Y_%H-%M-%S')
            # dump error information to file
            with open(f"{DUMPS_DIR}{now}.log", "w+") as logf:
                logf.write("SCK_DEBUG_ERROR:\n time:{}\n\n {} \n\n".format(now, str(traceback.format_exc())))

            break


```

I triger "work" as a thread.

python

multiprocessing

0 Answers

Your Answer

Accepted video resources