Cron: Difference between revisions

From Cheaha
Jump to navigation Jump to search
(remove crontab file)
(give first example description)
 
(11 intermediate revisions by the same user not shown)
Line 68: Line 68:
<pre>
<pre>
$ crontab -i -r
$ crontab -i -r
</pre>
To remove specified user cron entries
<pre>
$ crontab -r -u USERNAME
</pre>
</pre>


Line 90: Line 95:
</pre>
</pre>


== Examples ==
== Rudimentary Examples ==


=== Bash Script (cronjob.sh) To Build Environment (via Singularity) & Run Python Script ===
=== Setup ===
 
First off, make sure you have the $USER environment variable set in order to run these scripts. You can check by entering the following command in Cheaha's shell:


<pre>
<pre>
#!/bin/bash
$ echo $USER
#
</pre>
#SBATCH --job-name=dtn-cronjob
 
#
The output of this command should yield your blazerID.  
# Number of tasks needed for this job. Generally, used with MPI jobs
#SBATCH --ntasks=1
#SBATCH --partition=short
#
# Time format = HH:MM:SS, DD-HH:MM:SS
#SBATCH --time=3:30:00
#
# Number of CPUs allocated to each task.
#SBATCH --cpus-per-task=1
#
# Mimimum memory required per allocated  CPU  in  MegaBytes.  
#SBATCH --mem-per-cpu=1000
#
# Send mail to the email address when the job fails
#SBATCH --mail-type=FAIL
#SBATCH --mail-user=YOUR_EMAIL_ADDRESS


#Set your environment here
In the case that the output was blank, you'll need to set your blazerid to the $USER environment variable. You can do so by entering the following command in Cheaha's shell:
export PATH="`cat path`:$PATH"
module load Singularity
# execute python code
CLIENT=$1
TOKEN=$2
singularity exec -B . dtn_image_latest.sif python generate_data.py $CLIENT /datasets/ $PWD/../TEST_TRANSFERS/ -l 01 04 06 12 -t $TOKEN -r /perftest/uab_rc/ -wc
[ ! -d "./data" ] && mkdir ./data
mv ./*.csv ./data/


<pre>
$ export $USER="<your_blazerid>"
</pre>
</pre>


=== Bash Script to Run cronjob.sh Every 4 Hours Every Day ===
Be sure to replace <your_blazerid> with your actual blazerid. Here's an short example where alice01 is the blazerid:


<pre>
<pre>
$ export $USER="alice01"
</pre>
More on the syntax of sbatch scripts can be found [https://github.com/wwarriner/slurm_cheatsheets/blob/master/sbatch_cheat_sheet.pdf here].
In case you're new to slurm, feel free to checkout their documentation: [https://slurm.schedmd.com/documentation.html]
=== Hello World Using Cron ===
In this example, the bash script is used to tell Slurm how many resources to allocate for the given job and to output redirect I/O "hello" to a text file. The crontab file submits the bash script (crontab.sh) to Slurm as an sbatch job every minute.
==== Bash Script to Submit Sbatch job to Slurm ====


<pre>
#!/bin/bash
#!/bin/bash


Line 150: Line 148:
#SBATCH --mail-user=$USER@uab.edu
#SBATCH --mail-user=$USER@uab.edu


module load Singularity
 
singularity pull docker://mmoo97/dtn_image:latest
# redirects the output to a .txt file
# Build crontab
echo "Hello World" > hello.txt
sbatch_dir=$(which sbatch)
 
echo "#!/usr/bin/bash
</pre>
 
To expand more on I/O redirection, navigate to the following link: https://homepages.uc.edu/~thomam/Intro_Unix_Text/IO_Redir_Pipes.html
 
==== Crontab File ====
 
<pre>
#!/usr/bin/env


MAILTO=$USER@uab.edu
MAILTO=$USER@uab.edu


# runs cronjob.sh every 4 hours every day
# submit script to queuing system
00 */4 * * * cd $USER_DATA/DTN_tests && $sbatch_dir $USER_DATA/DTN_tests/cronjob.sh $CLIENT $TOKEN" > dtn-crontab
# runs cronjob.sh every minute of every day
echo $PATH > path
* * * * * /cm/shared/apps/slurm/18.08.9/bin/sbatch /data/user/$USER/Hello_Cron/cronjob.sh
 
</pre>
</pre>


</pre>
The first line in the cron configuration file (crontab file) is the shebang. When this file is executed, if the file content beings with #!, the kernel executes the file specified on the #! line and passes the original file as an argument. Essentially, the shebang tells the terminal which program to use to run your scripts if your script is an executable file.  
The first line in the cron configuration file (crontab file) is the shebang. When this file is executed, if the file content beings with #!, the kernel executes the file specified on the #! line and passes the original file as an argument. Essentially, the shebang tells the terminal which program to use to run your scripts if your script is an executable file.  
<pre>
<pre>
#!/usr/bin/env
#!/usr/bin/env
</pre>
</pre>
== Advanced Examples ==
Navigate to this [https://gitlab.rc.uab.edu/mmoo97/DTN_tests repo] for a nice advanced example on how to use cron on Cheaha.
For more info on this application: [https://www.globusworld.org/tour/data-mobility-exhibition]

Latest revision as of 15:54, 18 March 2021

Cron

What Is Cron?

Cron is a unix time based job scheduler tool used to schedule command execution at a specified time interval. These jobs are referred to as cronjobs, and are a great way of automating tasks or scripts so that they can be executed at a specific time.

Overview

A crontab file houses instructions to the cron daemon of the basic form: "run this command at this time on this date". Each user can have their own crontab file.

Crontab Syntax

crontab [ -u user ] file
crontab [ -u user ] { -l | -r | -e }
crontab [ -u user ] [ -i ] { -e | -l | -r }
crontab [ -u user ] [ -l | -r | -e ] [-i] [-s]

Cron Expression Table

Cron actions are driven by a crontab file, also known as a "cron table". The user's crontab file is a configuration file that specifies shell commands to run at a given schedule.

Every crontab file consists of two parts: a schedule and a command. Here is a look into the scheduling syntax:

# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │                                   7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * * <command to execute>

A crontab file has five fields; each field is represented by an asterisk to determine the data and time of a certain task.

Crontab Characters

Asterisk (*) - defines all the scheduling parameters.

Comma (,) - maintains two or more execution times of a single command.

Hyphen (-) - determines the range of time when setting several execution times of a single command.

Slash (/) - creates predetermined intervals of time in a specific range.

Last (L) - determines the last day of the week given in. a month. Example: 4L means the last Thursday.

Weekday (W) - determines the day of the week, followed by a number ranging from 1 to 5. Example: 1#2 means the second Monday

Question mark (?) - to leave blank

Crontab Options

To install, update, or edit a job in crontab, use the -e option.

$ crontab -e

To list crontab entires, use the -l option

$ crontab -l

To remove ALL jobs from crontab, use the -r option

$ crontab -r

To remove confirm removing a job from crontab, use the -i option

$ crontab -i -r

To remove specified user cron entries

$ crontab -r -u USERNAME

To add SELINUX security to a crontab file,

$ crontab -s

To edit another user's crontab file, use the user -u option and specify the username

$ crontab -u username -e

To list other user crontab entries

$ crontab -u username -l

NOTE: By default, cron uses vim to edit the crontab file. If you're not familiar with Vi/Vim, use the following command in Cheaha to go through a brief tutorial:

$ vimtutor

Rudimentary Examples

Setup

First off, make sure you have the $USER environment variable set in order to run these scripts. You can check by entering the following command in Cheaha's shell:

$ echo $USER

The output of this command should yield your blazerID.

In the case that the output was blank, you'll need to set your blazerid to the $USER environment variable. You can do so by entering the following command in Cheaha's shell:

$ export $USER="<your_blazerid>"

Be sure to replace <your_blazerid> with your actual blazerid. Here's an short example where alice01 is the blazerid:

$ export $USER="alice01"


More on the syntax of sbatch scripts can be found here.

In case you're new to slurm, feel free to checkout their documentation: [1]

Hello World Using Cron

In this example, the bash script is used to tell Slurm how many resources to allocate for the given job and to output redirect I/O "hello" to a text file. The crontab file submits the bash script (crontab.sh) to Slurm as an sbatch job every minute.

Bash Script to Submit Sbatch job to Slurm

#!/bin/bash

# resources 
#SBATCH cpus-per-task=1
#SBATCH mem-per-cpu=4G
#SBATCH array=-1%1
#SBATCH partition=express

# job name, error and output files
#SBATCH --job-name=cron-job
#SBATCH error=err_%j_4a.log
#SBATCH output=out_%j_%4a.log
#SBATCH ntasks=1

# email address to request notifications when the job is complete or if it fails
#SBATCH --mail-type=FAIL
#SBATCH --mail-user=$USER@uab.edu


# redirects the output to a .txt file
echo "Hello World" > hello.txt

To expand more on I/O redirection, navigate to the following link: https://homepages.uc.edu/~thomam/Intro_Unix_Text/IO_Redir_Pipes.html

Crontab File

#!/usr/bin/env

MAILTO=$USER@uab.edu

# submit script to queuing system 
# runs cronjob.sh every minute of every day
* * * * * /cm/shared/apps/slurm/18.08.9/bin/sbatch /data/user/$USER/Hello_Cron/cronjob.sh

The first line in the cron configuration file (crontab file) is the shebang. When this file is executed, if the file content beings with #!, the kernel executes the file specified on the #! line and passes the original file as an argument. Essentially, the shebang tells the terminal which program to use to run your scripts if your script is an executable file.

#!/usr/bin/env

Advanced Examples

Navigate to this repo for a nice advanced example on how to use cron on Cheaha.

For more info on this application: [2]