How to avoid all possible errors by 'try... except' function

Question

I am running a python code to do continuous web scraping (on linux mint with Python 2.7). Due to some reasons, the code breaks down from time to time. What I have done so far is to manually re-run the code whenever an error occurred.

I want to write another python code to replace me to do this 'check status, if break, then re-run' job.

I am not sure where to start. Could anyone give me a hint?

Answer 1

You want something like this:

from my_script import main

restart = True
while restart:
    try:
        main()
        # This line will allow the script to end if main returns. Leave it out
        # if you want main to get restart even when it returns with no errors. 
        restart = False
    except Exception as e:
        print("An error in main: ")
        print(e.message)
        print("Restarting main...")

This requires that your script, my_script.py in this example, be set up something like this:

def foo():
    raise ValueError("An error in foo")

def main():
    print("The staring point for my script")
    foo()

if __name__ == "__main__":
    main()