Cron: Difference between revisions

From Cheaha
Jump to navigation Jump to search
(add crontab syntax synopsis)
(give first example description)
 
(23 intermediate revisions by the same user not shown)
Line 2: Line 2:


=== What Is Cron? ===
=== What Is Cron? ===
'''Cron''' is a unix time based job scheduler tool.  
'''Cron''' is a unix time based job scheduler tool used to schedule command execution at a specified time interval. These jobs are referred to as cronjobs, and are a great way of automating tasks or scripts so that they can be executed at a specific time.  


== Overview ==
== Overview ==
A crontab file houses instructions to the cron daemon of the basic form: "run this command at this time on this date". Each user can have their own crontab file.
=== Crontab Syntax ===
=== Crontab Syntax ===


Line 17: Line 19:
Cron actions are driven by a crontab file, also known as a "cron table". The user's crontab file is a configuration file that specifies shell commands to run at a given schedule.  
Cron actions are driven by a crontab file, also known as a "cron table". The user's crontab file is a configuration file that specifies shell commands to run at a given schedule.  


Each line of a crontab file represents a job, and looks like this:
Every crontab file consists of two parts: a schedule and a command. Here is a look into the scheduling syntax:
<pre>
<pre>
# ┌───────────── minute (0 - 59)
# ┌───────────── minute (0 - 59)
Line 29: Line 31:
# * * * * * <command to execute>
# * * * * * <command to execute>
</pre>
</pre>
A crontab file has five fields; each field is represented by an asterisk to determine the data and time of a certain task.
=== Crontab Characters ===
'''Asterisk''' (*) - defines all the scheduling parameters.
'''Comma''' (,) - maintains two or more execution times of a single command.
'''Hyphen''' (-) - determines the range of time when setting several execution times of a single command.
'''Slash''' (/) - creates predetermined intervals of time in a specific range.
'''Last''' (L) - determines the last day of the week given in. a month. Example: 4L means the last Thursday.
'''Weekday''' (W) - determines the day of the week, followed by a number ranging from 1 to 5. Example: 1#2 means the second Monday
''' Question mark''' (?) - to leave blank
=== Crontab Options ===
=== Crontab Options ===
To install, update, or edit a job in crontab, use the -e option.
<pre>
$ crontab -e
</pre>
To list crontab entires, use the -l option
<pre>
$ crontab -l
</pre>
To remove ALL jobs from crontab, use the -r option
<pre>
$ crontab -r
</pre>


== Examples ==
To remove confirm removing a job from crontab, use the -i option
=== Using Cron to Initiate An Sbatch To Run A Bash Script ===
<pre>
$ crontab -i -r
</pre>


==== Bash Script ====
To remove specified user cron entries
<pre>
$ crontab -r -u USERNAME
</pre>
 
To add SELINUX security to a crontab file,
<pre>
$ crontab -s
</pre>
 
To edit another user's crontab file, use the user -u option and specify the username
<pre>
$ crontab -u username -e
</pre>
 
To list other user crontab entries
<pre>
$ crontab -u username -l
</pre>
 
NOTE: By default, cron uses vim to edit the crontab file. If you're not familiar with Vi/Vim, use the following command in Cheaha to go through a brief tutorial:
<pre>
$ vimtutor
</pre>
 
== Rudimentary Examples ==
 
=== Setup ===
 
First off, make sure you have the $USER environment variable set in order to run these scripts. You can check by entering the following command in Cheaha's shell:
 
<pre>
$ echo $USER
</pre>
 
The output of this command should yield your blazerID.
 
In the case that the output was blank, you'll need to set your blazerid to the $USER environment variable. You can do so by entering the following command in Cheaha's shell:
 
<pre>
$ export $USER="<your_blazerid>"
</pre>
 
Be sure to replace <your_blazerid> with your actual blazerid. Here's an short example where alice01 is the blazerid:
 
<pre>
$ export $USER="alice01"
</pre>
 
 
More on the syntax of sbatch scripts can be found [https://github.com/wwarriner/slurm_cheatsheets/blob/master/sbatch_cheat_sheet.pdf here].
 
In case you're new to slurm, feel free to checkout their documentation: [https://slurm.schedmd.com/documentation.html]
 
=== Hello World Using Cron ===
 
In this example, the bash script is used to tell Slurm how many resources to allocate for the given job and to output redirect I/O "hello" to a text file. The crontab file submits the bash script (crontab.sh) to Slurm as an sbatch job every minute.
==== Bash Script to Submit Sbatch job to Slurm ====


<pre>
<pre>
Line 53: Line 146:
# email address to request notifications when the job is complete or if it fails
# email address to request notifications when the job is complete or if it fails
#SBATCH --mail-type=FAIL
#SBATCH --mail-type=FAIL
#SBATCH --mail-user=$blazerid@uab.edu
#SBATCH --mail-user=$USER@uab.edu
 


# redirects the output to a .txt file
echo "Hello World" > hello.txt


# store text file in the home directory
echo "hi" > $HOME/test.txt
</pre>
</pre>
To expand more on I/O redirection, navigate to the following link: https://homepages.uc.edu/~thomam/Intro_Unix_Text/IO_Redir_Pipes.html


==== Crontab File ====
==== Crontab File ====
Line 64: Line 160:
<pre>
<pre>
#!/usr/bin/env
#!/usr/bin/env
# run generate_data.py 
# git add, commit, and push


MAILTO=blazerid@uab.edu
MAILTO=$USER@uab.edu
 
#git add data.csv
#git commit -m "update data"
#git push


# submit script to queuing system  
# submit script to queuing system  
# runs cronjob.sh every minute of every day
# runs cronjob.sh every minute of every day
* * * * * /cm/shared/apps/slurm/18.08.9/bin/sbatch /data/user/user_id/project_dir/cronjob.sh
* * * * * /cm/shared/apps/slurm/18.08.9/bin/sbatch /data/user/$USER/Hello_Cron/cronjob.sh
</pre>
</pre>


Line 82: Line 172:
#!/usr/bin/env
#!/usr/bin/env
</pre>
</pre>
== Advanced Examples ==
Navigate to this [https://gitlab.rc.uab.edu/mmoo97/DTN_tests repo] for a nice advanced example on how to use cron on Cheaha.
For more info on this application: [https://www.globusworld.org/tour/data-mobility-exhibition]

Latest revision as of 15:54, 18 March 2021

Cron

What Is Cron?

Cron is a unix time based job scheduler tool used to schedule command execution at a specified time interval. These jobs are referred to as cronjobs, and are a great way of automating tasks or scripts so that they can be executed at a specific time.

Overview

A crontab file houses instructions to the cron daemon of the basic form: "run this command at this time on this date". Each user can have their own crontab file.

Crontab Syntax

crontab [ -u user ] file
crontab [ -u user ] { -l | -r | -e }
crontab [ -u user ] [ -i ] { -e | -l | -r }
crontab [ -u user ] [ -l | -r | -e ] [-i] [-s]

Cron Expression Table

Cron actions are driven by a crontab file, also known as a "cron table". The user's crontab file is a configuration file that specifies shell commands to run at a given schedule.

Every crontab file consists of two parts: a schedule and a command. Here is a look into the scheduling syntax:

# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │                                   7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * * <command to execute>

A crontab file has five fields; each field is represented by an asterisk to determine the data and time of a certain task.

Crontab Characters

Asterisk (*) - defines all the scheduling parameters.

Comma (,) - maintains two or more execution times of a single command.

Hyphen (-) - determines the range of time when setting several execution times of a single command.

Slash (/) - creates predetermined intervals of time in a specific range.

Last (L) - determines the last day of the week given in. a month. Example: 4L means the last Thursday.

Weekday (W) - determines the day of the week, followed by a number ranging from 1 to 5. Example: 1#2 means the second Monday

Question mark (?) - to leave blank

Crontab Options

To install, update, or edit a job in crontab, use the -e option.

$ crontab -e

To list crontab entires, use the -l option

$ crontab -l

To remove ALL jobs from crontab, use the -r option

$ crontab -r

To remove confirm removing a job from crontab, use the -i option

$ crontab -i -r

To remove specified user cron entries

$ crontab -r -u USERNAME

To add SELINUX security to a crontab file,

$ crontab -s

To edit another user's crontab file, use the user -u option and specify the username

$ crontab -u username -e

To list other user crontab entries

$ crontab -u username -l

NOTE: By default, cron uses vim to edit the crontab file. If you're not familiar with Vi/Vim, use the following command in Cheaha to go through a brief tutorial:

$ vimtutor

Rudimentary Examples

Setup

First off, make sure you have the $USER environment variable set in order to run these scripts. You can check by entering the following command in Cheaha's shell:

$ echo $USER

The output of this command should yield your blazerID.

In the case that the output was blank, you'll need to set your blazerid to the $USER environment variable. You can do so by entering the following command in Cheaha's shell:

$ export $USER="<your_blazerid>"

Be sure to replace <your_blazerid> with your actual blazerid. Here's an short example where alice01 is the blazerid:

$ export $USER="alice01"


More on the syntax of sbatch scripts can be found here.

In case you're new to slurm, feel free to checkout their documentation: [1]

Hello World Using Cron

In this example, the bash script is used to tell Slurm how many resources to allocate for the given job and to output redirect I/O "hello" to a text file. The crontab file submits the bash script (crontab.sh) to Slurm as an sbatch job every minute.

Bash Script to Submit Sbatch job to Slurm

#!/bin/bash

# resources 
#SBATCH cpus-per-task=1
#SBATCH mem-per-cpu=4G
#SBATCH array=-1%1
#SBATCH partition=express

# job name, error and output files
#SBATCH --job-name=cron-job
#SBATCH error=err_%j_4a.log
#SBATCH output=out_%j_%4a.log
#SBATCH ntasks=1

# email address to request notifications when the job is complete or if it fails
#SBATCH --mail-type=FAIL
#SBATCH --mail-user=$USER@uab.edu


# redirects the output to a .txt file
echo "Hello World" > hello.txt

To expand more on I/O redirection, navigate to the following link: https://homepages.uc.edu/~thomam/Intro_Unix_Text/IO_Redir_Pipes.html

Crontab File

#!/usr/bin/env

MAILTO=$USER@uab.edu

# submit script to queuing system 
# runs cronjob.sh every minute of every day
* * * * * /cm/shared/apps/slurm/18.08.9/bin/sbatch /data/user/$USER/Hello_Cron/cronjob.sh

The first line in the cron configuration file (crontab file) is the shebang. When this file is executed, if the file content beings with #!, the kernel executes the file specified on the #! line and passes the original file as an argument. Essentially, the shebang tells the terminal which program to use to run your scripts if your script is an executable file.

#!/usr/bin/env

Advanced Examples

Navigate to this repo for a nice advanced example on how to use cron on Cheaha.

For more info on this application: [2]