Deep dive into Shell Scripting by Pratul Patel

LINUX

I'm excited to continue my DevOps learning journey alongside my new role as a Level 2 Server Engineer. This position will not only allow me to gain hands-on industry experience but also help me deepen my understanding of technologies directly related to my job.

By integrating the knowledge, I gain from both DevOps practices and server management, I can better align my skills with the evolving demands of modern infrastructure, ensuring continuous growth and expertise in both areas. I'll also maintain consistency by sharing my insights through blogs, bridging the gap between my learning and real-world application.

This week I propose to learn Shell Scripting in LINUX, CICD using Jenkins.

LINUX - Shell Scripting

I started my preparation by learning the concept of shell & Kernel, and how the user applications & hardware are related to it.

Kernel is an interface between software (user applications) & hardware.
Shell is like a container. It is the interface between users & Kernel/OS.
Eg: Windows GUI, LINUX KDE GUI like ‘sh‘, ‘bash‘ etc are shells.

To Find out the shell of our OS:

→ echo $0

→ Available Shells “cat /etc/shells”

→ Your Shell? /etc/passwd

Below is the complete hardware & software interconnection to get idea on Shell & Kernel.

What is a Shell Script?

A. Shell script is an executable file containing multiple shell commands that are executed sequentially. The file can contain:

Shell (#!/bin/bash) //shebang (more details below…)
Comments (# comments)
Commands (echo, cp, grep etc.)
Statements (if, while, for etc.)

NOTE: A shell must have below criterias:

Must have executable permissions (e.g. -rwx(u) r-x(g) r-x(o))
Shell script has to be called from absolute path. (e.g /home/userdir/script.bash).
If called from current location, then ./script.bash.
Output to screen using echo.
Filter/Text processors through scripts (cut, awk, grep, sort, uniq, wc).

My Lab

I started my Shell Scripting my setting up a VM in VMware Workstation Player having OS as CentOS with 7.0 version.

Started my lab by taking root user for which I configured the LINUX (CentOS) VM basic hardware configuration like setting up network configurations with ports & protocols.

I created a primary user named Pratul using useradd command & secondary user named Atul for my lab. Both of these users were given administrator privileges by adding them to the wheel group as mentioned below which defines they are part of sudoers.

I also created an additional user for my lab named Prachi. But it’s not given admin privilege acting as a normal user.

My First Flight: A Basic Shell Script

Created a demo script file named myshell-demo within /home/atul as user Atul. Written below script which needed output as

The Current User
Hostname
List of files in directory in reverse order with respective permissions.
A comment

I have given my script inputs using vim command where I enabled these features - Indentation :set ai & Numbering :set nu.

#!/bin/bash
#Define small tasks
whoami 
echo 
hostname 
echo 
ls -ltr 
echo "I am cat"

NOTE: echo command is used in-between lines for blank spaces.

I successfully executed the script using “. /myshell-demo” command for which I got the below output.

[atul@localhost ~]$ ./myshell-demo
atul

localhost.localdomain

total 52
-rw-r--r--. 1 atul atul    41 Dec 15 16:15 zad
-rw-r--r--. 1 root root 42980 Dec 15 17:10 mesg-new
-rwxrwxr-x. 1 atul atul    71 Dec 15 23:11 myshell-demo
I am cat

I also used parameters & variables concept in Shell Scripting where I defined my name in variables & called it using parameters values within comments.

#!/bin/bash
#This script contains defining variables

a="Pratul"
b="Kumar"
c="Patel"

echo "My First name is $a"
echo "My Second name is $b"
echo "My Third name is $c"
echo "My Full name is $a $b $c"

NOTE: #!/bin/bash at the top of the script is called a shebang. It specifies the interpreter that should be used to execute the script. It tells the operating system to use the Bash shell to interpret the script. Without this, the OS might not know how to run the script or may use a default shell that might not support Bash-specific features.

Without #!/bin/bash the script looks in this way:

I give the execute permission using chmod command at Others scope level & executed the script using “./<filename>”.

CHALLENGES #1

On executing my script using “. /myshell-demo” command I was getting Permission Denied.

I checked the file permission where I got to know that the script required execute permission at everyone/others scope.

Using chmod command,

chmod a+x <filename>

chmod a+x myshell-demo

The script was able to be executed successfully using “. /myshell-demo” with desired output as described above.

CHALLENGES #2

I was not getting the desired output:

On checking I found that there were spaces in defining variables as below:

#ERROR

#!/bin/bash
#This script contains defining variables

a = "Pratul"    ##Error: Spaces inbetween
b = "Kumar"     ##Error: Spaces inbetween
c = "Patel"    ##Error: Spaces inbetween

echo "My First name is $a"
echo "My Second name is $b"
echo "My Third name is $c"
echo "My Full name is $a $b $c"

#TROUBLESHOOTED

#!/bin/bash
#This script contains defining variables

a="Pratul"    ##Removed Spaces
b="Kumar"    ##Removed Spaces
c="Patel"    ##Removed Spaces

echo "My First name is $a"
echo "My Second name is $b"
echo "My Third name is $c"
echo "My Full name is $a $b $c"

I then executed the script using “./<filename>” to get output of the script.

Using the above logic, I created a script which is a baby step to create an automation!

In order to achieve this, I used read command to take user inputs when asked to type it.

#!/bin/bash
#Author
#Date

echo "Hello! My name is Pratul Patel"
echo
echo "What is your Name?"
read myName                #used 'myName' to take inputs as variables
echo
echo "Hello $myName"
echo

OUTPUT-

Refining the above with more detailed script for which I added the LINUX command as well & used it as calling parameters. Linux commands like hostname, pwd, date&time etc.

OUTPUT-

CHALLENGES #3

While running the above script I was not getting the desired output, for hostname it was not giving me the output of hostname as given below.

On debugging the script, there need to be Backticks (`) if any Linux command is called to be run during execution of script.

#ERROR

#!/bin/bash
#conversation
#automate

h=hostname    #missing (`) on both ends
d=`pwd`
echo
echo "Welcome to this $h"
echo
echo "what is your name?"
read myName
echo
echo "Hello $myName, Thanks for joining this host - $h"
echo
echo "You are currently in $d directory!"
echo
echo

#TROUBLESHOOTED

#!/bin/bash
#conversation
#automate

h=`hostname`    #missing (`) on both ends
d=`pwd`
echo
echo "Welcome to this $h"
echo
echo "what is your name?"
read myName
echo
echo "Hello $myName, Thanks for joining this host - $h"
echo
echo "You are currently in $d directory!"
echo
echo

NOTE: Always use Backticks (`) at start & end when using LINUX commands in Shell Script to call it as parameters within script.

e.g., `hostname`, `pwd`

if-else

Next I had dived into the implementation of if-else in Shell scripting.

I created a file names ifelse-script1 & ran the script in the same way as before, within the script I developed a demo code using if-else logic.

#!/bin/bash

count=500
if [ $count -eq 50 ]
then
        echo Count is 50
else
        echo Count is not 50
fi

OUTPUT-

Changed the scripts input multiple times to check whether the given script works or not.

For reference, within the script I changed $count value.

CHALLENGES #4

During the execution of the script, I was facing the below error:

On researching I found the ‘[ ]‘ need spaces as shown below,

#ERROR

count=500
if [$count -eq 50]    ##there's no spaces inbetween '[]' before & after
then
        echo Count is 50
else
        echo Count is not 50
fi

#TROUBLESHOOTED

count=500
if [ $count -eq 50 ]       ##spaces provided
then
        echo Count is 50
else
        echo Count is not 50
fi

NOTE: Always put spaces before & after ‘[ ]‘ to make the interpreter recognize the command and execute the script with desired output.

I also tweaked the if-else with more set of codes & executed accordingly with desired output as below:

To check if a file is present or not in any directory:

 #!/bin/bash

 clear
 echo
 echo Please type the Filepath
 read filep    ##User Input

 ##OR##
 # echo '/<filepath>'
 # echo '/etc/var/log' 

 echo
 if [ -e $filep ]
         then
         echo "File exist"
         else
         echo "File doesn't exist"
 fi

OUTPUT-

To check variable value is met (Eg, to know the present day)

#!/bin/bash

##You can use 'date' command to view the output and configure the script accordingly.
#OUTPUT-> 'Wed Dec 18 23:29:40 IST 2024'


a=`date | awk '{print $1}'`  ##date -> prints as 'Wed Dec 18 23:29:40 IST 2024' 
                             ##"awk '{print $1}'" -> prints as 'Wed' representing as first coloumn 
if [ "$a" == Wed ]
        then
        echo Today is $a
        else
        echo Today is not Mon
fi

OUTPUT-

I referred to other tokens or expressions that is used in if-else which will be helpful in achieving tasks using scripting,

Comparisons:

-eq → equal to for numbers

\== → equal to for letters

-ne → not equal to

! == → not equal to for letters

-lt → less than

-le → less than or equal to

-gt → greater than

-ge → greater than or equal to
File Operations:

-s → file exists and is not empty

-f → file exists and is not a directory

-d → directory exists

-x → file is executable

-w → file is writable

-r → file is readable

for

I then expanded my learning by getting into for loop command.

Starting with a very basic script & then using it I configured my script in such a way I get my desired output on one run.

To perform my lab practice I created a new directory named ‘For’ within Atul directory

A Simple For loop file

 #!/bin/bash

 for i in 1 2 3 4 5
 do
 echo "Welcome $i times"    ##prints echo '5' times on execution of the script.
 done

OUTPUT-

CREATE & DELETE multiple files using for

 #!/bin/bash

 for i in {1..5}
 do
  touch myFile$i    ##where 'i' denotes 1 to 5
 done

OUTPUT-

Similarly, for DELETE replace rm command in place of touch.

 #!/bin/bash

 for i in {1..5}
 do
  rm myFile$i    ##DELETES the existing files present in this file format.
 done

Specify days in for loop

 #!/bin/bash
 i=1    ##initialization for loop to incrementing; i=1, i=2, i=3 etc..  

 for day in Mon Tue Wed Thu Fri
 do
 echo "Weekday $((i++)) : $day"        ##prints days from Mon till Fri
 done

OUTPUT-

List all users one by one from /etc/passwd file using For loop

 #!/bin/bash

 i=1
 for username in `awk -F: '{print $1}' /etc/passwd`
 do
 echo "Username $((i++)) : $username"    ##prints all users present in the 'passwd' file
 done

OUTPUT-

NOTE: -F: is an option passed to the awk command. Here's what it does and why it's important:

The -F flag in awk specifies the field delimiter or separator for parsing each line of input.
: is used as the field delimiter in this case.

NEED: By specifying -F: you instruct awk to treat : as the separator. This allows it to correctly extract the username (the first field) from each line. Without -F:, awk defaults to treating whitespace (spaces and tabs) as the delimiter. In this case, it wouldn't correctly extract the username field from the /etc/passwd file.
This is how it looks without -F: after execution of script,

Prints entire properties of usernames present in passwd.

Case

In this command, I got to know the feature of taking multiple inputs and based on our input we get our desired output which more like MCQ!

I started with lab with same practice by creating a directory named ‘Case’ directory within ‘Atul’ folder.

#!/bin/bash

echo
echo Please chose one of the options below
echo
echo "1 = Display Date and Time"        ##List of questions thrown after execution of script
echo '2 = List file and directories'
echo '3 = List users logged in'
echo '4 = Check System uptime'
echo

 read choices        ##user input taken during after execution of script
 case $choices in
1) date;;            ##LINUX commands -> date, ls, who & uptime are called based on its number inputs before ')'
2) ls;;
c) who;;
d) uptime;;
*) echo Invalid choice - Bye.    ##For invalid input throw this output
 esac

OUTPUT-

Other than “options → 1, 2, c & d”, for any other inputs it will give output as ‘Invalid choice - Bye’.

Remote Servers Connectivity

Until now, I have practiced the shell scripts using various commands & logics used. Culmination of all these I wanted to use the above concepts in system administration.

Therefore, I started a new directory named ‘check-ip’ within ‘atul‘ directory for my test labs.

REQUIREMENTS: I took 4 IPs, out of which 2 IPs are assigned (primary & secondary IPs of the VM NIC cards - “virbr0” & “ens33“) & 2 IPs are unassigned (random IPs).

Pingable IPs:
192.168.122.1    //Primary IP (pingable)
192.168.60.133   //Secondary IP (pingable)

Non pingable/Random IPs:
192.168.122.2
192.168.165.60

Using same method to execute the shell script using vim/vi, chmod & ./<filename> in sequential way as used for other scripts before.

A basic ping script

 #!/bin/bash
 #

 ping -c1 192.168.122.1

     if [ $? -eq 0 ]
     then
     echo OK
     else
     echo NOT OK
     fi

OUTPUT-

For any random IP:

#!/bin/bash 
# Author: Pratul Patel 
# Date: 22/12/2024 
# Description: This script will ping a remote host & notify! 
# Modified  22/12/2024

ping -c1 192.168.122.2        ## After Changing to random IP 
        if [ $? -eq 0 ] 
        then 
        echo OK 
        else 
        echo NOT OK 
        fi

Output-

I got non-pingable message!

Print only Comments for ping

 #!/bin/bash 
 # Author: Pratul Patel 
 # Date: 22/12/2024 
 # Description: This script will ping a remote host & notify! 
 # Modified  22/12/2024 

 #Don’t show the Output 
 ping -c1 192.168.122.1 &> /dev/null     ##Will only prints below comment without displaying ping status. 
         if [ $? -eq 0 ] 
         then 
         echo OK 
         else 
         echo NOT OK 
         fi

OUTPUT-

For any random IP:

 #!/bin/bash 
 # Author: Pratul Patel 
 # Date: 22/12/2024 
 # Description: This script will ping a remote host & notify! 
 # Modified 22/12/2024 

 #Don’t show the Output 
 ping -c1 192.168.122.2 &> /dev/null     ##Will only prints below comment without displaying ping status.
                                         ##Random IP 
         if [ $? -eq 0 ] 
         then 
         echo OK 
         else 
         echo NOT OK 
         fi

Output-

NOTE:

 ping -c1 192.168.122.2 &> /dev/null

ping -c1 192.168.122.2 sends a single ICMP echo request (-c1) to the current IP address ($ip).
&> /dev/null redirects both standard output (stdout) and standard error (stderr) to /dev/null, effectively discarding the output.

    if [ $? -eq 0 ]

$? holds the exit status of the last command executed (in this case, ping).
- 0 indicates success (the host responded to the ping).
- Any other value indicates failure.
[ $? -eq 0 ] checks if the exit status equals 0 (ping was successful).

Print Comments with its IP for ping to be more specific

 #!/bin/bash 
 # Author: Pratul Patel 
 # Date: 22/12/2024 
 # Description: This script will ping a remote host & notify! 
 # Modified 22/12/2024 

 hosts="192.168.122.1"           ##storing IP in a variable 
 ping -c1 $hosts &> /dev/null 
         if [ $? -eq 0 ] 
         then 
         echo $hosts OK          ##calling variable & giving very specific output 
         else 
         echo $hosts NOT OK 
         fi

OUTPUT-

For any random IP:

 #!/bin/bash 
 # Author: Pratul Patel 
 # Date: 22/12/2024 
 # Description: This script will ping a remote host & notify! 
 # Modified 22/12/2024 

 hosts="192.168.165.60"           ##storing IP in a variable 
 ping -c1 $hosts &> /dev/null 
         if [ $? -eq 0 ] 
         then 
         echo $hosts OK          ##calling variable & giving very specific output 
         else 
         echo $hosts NOT OK 
         fi

Output-

Using File as a reference consisting of IPs for ping

 #!/bin/bash 
 # Author: Pratul Patel 
 # Date: 22/12/2024 
 # Description: This script will ping multiple remote hosts & notify! 
 # Modified  22/12/2024 

 hosts='/home/atul/check-ip/ip-devices'     ##File Reference
 for ip in $(cat $hosts) 
 do 
         ping -c1 $ip &> /dev/null 
         if [ $? -eq 0 ] 
         then 
         echo $ip ping passed 
         else
         echo $ip ping failed 
         fi
 done

Here,

 for ip in $(cat $hosts)

cat $hosts reads the content of the file specified in the hosts variable.
$(cat $hosts) executes the cat command and provides its output as a list of IP addresses.
for ip in $(...) loops through each IP address in this list, assigning one IP at a time to the variable ip.

OUTPUT-

CHALLENGES #5

I accidentally used $hosts within For loop instead of $ip for which I got error as below:

    #!/bin/bash 
    # Author: Pratul Patel 
    # Date: 22/12/2024 
    # Description: This script will ping multiple remote hosts & notify! 
    # Modified  22/12/2024 

    hosts='/home/atul/check-ip/ip-devices' 
    for ip in $(cat $hosts) 
    do
            ping -c1 $hosts &> /dev/null 
            if [ $? -eq 0 ] 
            then 
            echo $hosts ping passed 
            else 
            echo $hosts ping failed 
            fi 
    done

#ERROR

Issue:

    ping -c1 $hosts &> /dev/null

     echo $hosts ping passed 
     else 
     echo $hosts ping failed

This causes the script to incorrectly try to ping the file path instead of each IP address, which will fail unless the file path happens to resolve as a valid hostname or IP.

#TROUBLESHOOTED

→ Reverted back to original one.

Getting Started with Shell Scripting

LINUX

LINUX - Shell Scripting

My Lab

if-else

for

Case

Remote Servers Connectivity

REFERENCES