I want to schedule cron job for my file webScraping.py. When I execute manually everything work perfectly.
#!/bin/bash
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
datetime="$(gdate +'%Y-%m-%d %H:%M:%S.%3N')"
LOG_FILE=webScraping.log
LOG_FORMAT="${datetime} | INFO | $(BASENAME "$0"):${FUNCNAME[0]}:${LINENO} -"
exec >> $LOG_FILE 2>&1
echo "${LOG_FORMAT} Running the bash script"
echo "${LOG_FORMAT} The script bash you are running has :"
echo "${LOG_FORMAT} basename: [$(BASENAME "$0")]"
echo "${LOG_FORMAT} pwd : ['$SCRIPT_DIR']"
export TZ="Europe/Paris"
(crontab -l ; echo "30 23 * * * $SCRIPT_DIR/webScraping.py") | crontab -
python3 $SCRIPT_DIR/webScraping.py >> $PWD/$LOG_FILE 2>&1
echo "${LOG_FORMAT} The script bash was successfully executed"
When I run crontab -l, it display it :
30 23 * * * /Users/kevin/PycharmProjects/Projects2024/nov23_continu_mlops_meteo/src/data/webScraping.py
I don't know where is the issue
This is my LOG_FILE=webScraping.log, it only show when the script is manually executed but not at 23:30 when the Cron job should run
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - Running the bash script
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - The script bash you are running has :
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - basename: [runWebScraping.sh]
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - pwd : ['/Users/kevin/PycharmProjects/Projects2024/nov23_continu_mlops_meteo/src/data']
2024-03-18 23:26:12.814 | INFO | __main__:<module>:128 - Lancement du script
2024-03-18 23:26:12.903 | INFO | __main__:UpdateData:102 - Data is already up to date
2024-03-18 23:26:12.904 | INFO | __main__:<module>:139 - Fin du script
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - The script bash was successfully executed