Bash script and Crontab is not working as expected

42 Views Asked by At

I want to schedule cron job for my file webScraping.py. When I execute manually everything work perfectly.

#!/bin/bash

SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
datetime="$(gdate +'%Y-%m-%d %H:%M:%S.%3N')"

LOG_FILE=webScraping.log
LOG_FORMAT="${datetime} | INFO | $(BASENAME "$0"):${FUNCNAME[0]}:${LINENO} -"

exec >> $LOG_FILE 2>&1

echo "${LOG_FORMAT} Running the bash script"

echo "${LOG_FORMAT} The script bash you are running has :"
echo "${LOG_FORMAT} basename: [$(BASENAME "$0")]"
echo "${LOG_FORMAT} pwd : ['$SCRIPT_DIR']"

export TZ="Europe/Paris"

(crontab -l ; echo "30 23 * * * $SCRIPT_DIR/webScraping.py") | crontab -

python3 $SCRIPT_DIR/webScraping.py >> $PWD/$LOG_FILE 2>&1

echo "${LOG_FORMAT} The script bash was successfully executed"

When I run crontab -l, it display it :

30 23 * * * /Users/kevin/PycharmProjects/Projects2024/nov23_continu_mlops_meteo/src/data/webScraping.py

I don't know where is the issue

This is my LOG_FILE=webScraping.log, it only show when the script is manually executed but not at 23:30 when the Cron job should run

2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - Running the bash script
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - The script bash you are running has :
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - basename: [runWebScraping.sh]
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - pwd : ['/Users/kevin/PycharmProjects/Projects2024/nov23_continu_mlops_meteo/src/data']
2024-03-18 23:26:12.814 | INFO     | __main__:<module>:128 - Lancement du script
2024-03-18 23:26:12.903 | INFO     | __main__:UpdateData:102 - Data is already up to date
2024-03-18 23:26:12.904 | INFO     | __main__:<module>:139 - Fin du script
2024-03-18 23:26:07.489 | INFO | runWebScraping.sh::7 - The script bash was successfully executed
0

There are 0 best solutions below