Linux Command Line and Shell Scripting Bible

The GNU long parameters 91 Real-time process monitoring 93 Stopping processes 95 The kill command 95 The pkill command 96 Monitoring Disk Space 96 Mounting media 97 The mount command 97 The umount command 99 Using the df command 100 Using the du command 101 Working with Data Files 102 Sorting data 102 Searching for data 106 Compressing data 108 Archiving data 109 Summary 111 Chapter 5: Understanding the Shell 113 Investigating Shell Types 113 Exploring Parent and Child Shell Relationships 117 Looking at process lists 121 Creatively using subshells 123 Investigating background mode 123 Putting process lists into the background 125 Looking at co-processing 126 Understanding External and Built-In Commands 127 Looking at external commands 128 Looking at built-in commands 129 Using the history command 130 Using command aliases 134 Summary 135 Chapter 6: Using Linux Environment Variables 137 Exploring Environment Variables 137 Looking at global environment variables 138 Looking at local environment variables 140 Setting User-Defined Variables 141 Setting local user-defined variables 141 Setting global environment variables 142 Removing Environment Variables 144 Uncovering Default Shell Environment Variables 145 Setting the PATH Environment Variable 150 Locating System Environment Variables 152 Understanding the login shell process 152 Viewing the /etc/profi le file 152 Viewing the $HOME startup files 156 Understanding the interactive shell process 157 Understanding the noninteractive shell process 158 Making environment variables persistent 159 Learning about Variable Arrays 159 Summary 161 Chapter 7: Understanding Linux File Permissions 163 Exploring Linux Security 163 The /etc/passwd file 164 The /etc/shadow file 165 Adding a new user 166 Removing a user 169 Modifying a user 169 usermod 170 passwd and chpasswd 170 chsh, chfn, and chage 171 Using Linux Groups 173 The /etc/group file 173 Creating new groups 174 Modifying groups 175 Decoding File Permissions 176 Using file permission symbols 176 Default file permissions 178 Changing Security Settings 179 Changing permissions 180 Changing ownership 181 Sharing Files 182 Access Control Lists 184 Summary 186 Chapter 8: Managing Filesystems 189 Exploring Linux Filesystems 189 Exploring the Linux filesystem evolution 190 Looking at the ext filesystem 190 Looking at the ext2 filesystem 190 Digging into journaling filesystems 191 Looking at the ext3 filesystem 191 Looking at the ext4 filesystem 191 Looking at the JFS filesystem 192 Looking at ReiserFS 192 Looking at XFS 192 Understanding the volume-managing filesystems 192 Looking at the ZFS filesystem 193 Looking at the Btrfs filesystem 193 Looking at the Stratis filesystem 193 Working with Filesystems 194 Creating partitions 194 Looking at the fdisk utility 194 Working with gdisk 197 The GNU parted command 198 Creating a filesystem 199 Checking and repairing a filesystem 201 Managing Logical Volumes 202 Exploring LVM layout 203 Physical volume 203 Volume group 203 Logical volume 203 Understanding the LVM in Linux 203 Create the PVs 204 Create a VG 204 Create an LV 205 Using the Linux LVM 206 Format and mount an LV 206 Growing or shrinking your VGs and LVs 206 Summary 207 Chapter 9: Installing Software 209 Exploring Package Management 209 Inspecting the Debian-Based Systems 210 Managing packages with apt 211 Installing software packages with apt 213 Upgrading software with apt 215 Uninstalling software with apt 216 The apt repositories 218 The Red Hat-Based Systems 220 Listing installed packages 220 Installing software with dnf 221 Upgrading software with dnf 223 Uninstalling software with dnf 223 Dealing with broken dependencies 223 RPM repositories 224 Managing Software Using Containers 225 Using snap containers 225 Using flatpak containers 227 Installing from Source Code 229 Summary 232 Chapter 10: Working with Editors 235 Visiting the vim Editor 235 Checking your vim package 236 Exploring vim basics 237 Editing data 240 Copying and pasting 240 Searching and substituting 241 Navigating the nano Editor 242 Exploring the Emacs Editor 244 Checking your Emacs package 245 Using Emacs on the console 246 Exploring the basics of Emacs 247 Editing data 248 Copying and pasting 249 Searching and replacing 249 Using buffers in Emacs 250 Using windows in console mode Emacs 251 Using Emacs in a GUI 252 Exploring the KDE Family of Editors 253 Looking at the KWrite editor 253 Looking at the Kate editor 259 Exploring the GNOME Editor 263 Starting gedit 264 Understanding basic gedit features 265 Managing plugins 268 Summary 271 Part II: Shell Scripting Basics 273 Chapter 11: Basic Script Building 275 Using Multiple Commands 275 Creating a Script File 276 Displaying Messages 278 Using Variables 279 Environment variables 280 User variables 281 Command substitution 283 Redirecting Input and Output 284 Output redirection 285 Input redirection 285 Employing Pipes 287 Performing Math 289 The expr command 290 Using brackets 292 A floating-point solution 293 The basics of bc 293 Using bc in scripts 295 Exiting the Script 297 Checking the exit status 297 The exit command 298 Working through a Practical Example 300 Summary 301 Chapter 12: Using Structured Commands 303 Working with the if-then Statement 303 Exploring the if-then-else Statement 306 Nesting ifs 307 Trying the test Command 311 Using numeric comparisons 313 Using string comparisons 314 Looking at string equality 315 Looking at string order 316 Looking at string size 318 Using file comparisons 320 Checking directories 320 Checking whether an object exists 321 Checking for a file 322 Checking for read access 324 Checking for empty files 325 Checking whether you can write to a file 326 Checking whether you can run a file 327 Checking ownership 328 Checking default group membership 329 Checking file date 330 Considering Compound Testing 331 Working with Advanced if-then Features 332 Using single parentheses 332 Using double parentheses 333 Using double brackets 335 Considering the case Command 335 Working through a Practical Example 337 Summary 343 Chapter 13: More Structured Commands 345 Looking at the for Command 345 Reading values in a list 346 Reading complex values in a list 347 Reading a list from a variable 349 Reading values from a command 350 Changing the field separator 351 Reading a directory using wildcards 352 Trying the C-Style for Command 354 The C language for command 355 Using multiple variables 356 Exploring the while Command 357 Basic while format 357 Using multiple test commands 358 Using the until Command 359 Nesting Loops 361 Looping on File Data 364 Controlling the Loop 365 The break command 365 Breaking out of a single loop 365 Breaking out of an inner loop 366 Breaking out of an outer loop 367 The continue command 368 Processing the Output of a Loop 371 Working through a Few Practical Examples 372 Finding executable files 373 Creating multiple user accounts 374 Summary 375 Chapter 14: Handling User Input 377 Passing Parameters 377 Reading parameters 377 Reading the script name 380 Testing parameters 381 Using Special Parameter Variables 382 Counting parameters 382 Grabbing all the data 384 Being Shifty 386 Working with Options 388 Finding your options 388 Processing simple options 388 Separating options from parameters 389 Processing options with values 391 Using the getopt command 392 Looking at the command format 392 Using getopt in your scripts 393 Advancing to getopts 395 Standardizing Options 398 Getting User Input 399 Reading basics 399 Timing Out 401 Reading with no display 402 Reading from a file 403 Working through a Practical Example 404 Summary 408 Chapter 15: Presenting Data 411 Understanding Input and Output 411 Standard file descriptors 412 STDIN 412 STDOUT 413 STDERR 414 Redirecting errors 414 Redirecting errors only 414 Redirecting errors and data 415 Redirecting Output in Scripts 416 Temporary redirections 416 Permanent redirections 417 Redirecting Input in Scripts 418 Creating Your Own Redirection 419 Creating output file descriptors 419 Redirecting file descriptors 420 Creating input file descriptors 421 Creating a read/write file descriptor 422 Closing file descriptors 423 Listing Open File Descriptors 424 Suppressing Command Output 426 Using Temporary Files 427 Creating a local temporary file 427 Creating a temporary file in /tmp 428 Creating a temporary directory 429 Logging Messages 430 Working through a Practical Example 431 Summary 433 Chapter 16: Script Control 435 Handling Signals 435 Signaling the Bash shell 435 Generating signals 436 Interrupting a process 436 Pausing a process 437 Trapping signals 438 Trapping a script exit 440 Modifying or removing a trap 441 Running Scripts in Background Mode 443 Running in the background 443 Running multiple background jobs 445 Running Scripts without a Hang-up 447 Controlling the Job 448 Viewing jobs 448 Restarting stopped jobs 450 Being Nice 451 Using the nice command 452 Using the renice command 453 Running like Clockwork 454 Scheduling a job using the at command 454 Understanding the at command format 454 Retrieving job output 455 Listing pending jobs 457 Removing jobs 457 Scheduling regular scripts 458 Looking at the cron table 458 Building the cron table 459 Viewing cron directories 460 Looking at the anacron program 460 Starting scripts with a new shell 462 Working through a Practical Example 463 Summary 469 Part III: Advanced Shell Scripting 471 Chapter 17: Creating Functions 473 Exploring Basic Script Functions 473 Creating a function 474 Using functions 474 Returning a Value from a Function 477 The default exit status 477 Using the return command 478 Using function output 479 Using Variables in Functions 480 Passing parameters to a function 480 Handling variables in a function 482 Global variables 483 Local variables 484 Investigating Array Variables and Functions 485 Passing arrays to functions 485 Returning arrays from functions 487 Considering Function Recursion 488 Creating a Library 489 Using Functions on the Command Line 491 Creating functions on the command line 491 Defining functions in the bashrc file 492 Directly defining functions 492 Sourcing function files 493 Working Through a Practical Example 494 Downloading and installing 494 Building the library 495 The shtool library functions 496 Using the library 497 Summary 497 Chapter 18: Writing Scripts for Graphical Desktops 499 Creating Text Menus 499 Create the menu layout 499 Create the menu functions 501 Add the menu logic 502 Putting it all together 502 Using the select command 504 Doing Windows 505 The dialog package 506 The msgbox widget 507 The yesno widget 508 The inputbox widget 508 The textbox widget 510 The menu widget 511 The fselect widget 512 The dialog options 513 Using the dialog command in a script 515 Getting Graphic 516 The KDE environment 517 kdialog widgets 517 Using kdialog 519 The GNOME environment 521 zenity Widgets 521 Using zenity in scripts 523 Working Through a Practical Example 525 Summary 529 Chapter 19: Introducing sed and gawk 531 Manipulating Text 531 Getting to know the sed editor 532 Defining an editor command in the command line 533 Using multiple editor commands in the command line 534 Reading editor commands from a file 534 Getting to know the gawk program 535 Visiting the gawk command format 536 Reading the program script from the command line 536 Using data field variables 537 Using multiple commands in the program script 538 Reading the program from a file 539 Running scripts before processing data 539 Running scripts after processing data 540 Looking at the sed Editor Basic Commands 542 Introducing more substitution options 542 Substituting flags 542 Replacing characters 543 Using addresses 544 Addressing the numeric line 544 Using text pattern filters 545 Grouping commands 546 Deleting lines 547 Inserting and appending text 549 Changing lines 551 Transforming characters 552 Printing revisited 553 Printing lines 554 Printing line numbers 555 Listing lines 555 Using files with sed 556 Writing to a file 556 Reading data from a file 557 Working Through a Practical Example 559 Summary 564 Chapter 20: Regular Expressions 565 Exploring Regular Expressions 565 A definition 565 Types of regular expressions 567 Defining BRE Patterns 567 Plain text 567 Special characters 569 Anchor characters 570 Starting at the beginning 570 Looking for the ending 571 Combining anchors 572 The dot character 572 Character classes 573 Negating character classes 575 Using ranges 576 Special character classes 577 The asterisk 578 Trying Out Extended Regular Expressions 579 The question mark 579 The plus sign 580 Using braces 581 The pipe symbol 582 Grouping expressions 583 Working Through Some Practical Examples 584 Counting directory files 584 Validating a phone number 585 Parsing an email address 587 Summary 589 Chapter 21: Advanced sed 591 Looking at Multiline Commands 591 Navigating the next command 592 Using the single-line next command 592 Combining lines of text 593 Navigating the multiline delete command 595 Navigating the multiline print command 596 Holding Space 598 Negating a Command 599 Changing the Flow 602 Branching 603 Testing 605 Replacing via a Pattern 606 Using the ampersand 606 Replacing individual words 607 Placing sed Commands in Scripts 608 Using wrappers 608 Redirecting sed output 609 Creating sed Utilities 610 Spacing with double lines 610 Spacing files that may have blanks 611 Numbering lines in a file 612 Printing last lines 613 Deleting lines 614 Deleting consecutive blank lines 615 Deleting leading blank lines 616 Deleting trailing blank lines 616 Removing HTML tags 617 Working Through a Practical Example 619 Summary 624 Chapter 22: Advanced gawk k 627 Using Variables 627 Built-in variables 628 The field and record separator variables 628 Data variables 631 User-defined variables 634 Assigning variables in scripts 634 Assigning variables on the command line 635 Working with Arrays 636 Defining array variables 636 Iterating through array variables 637 Deleting array variables 638 Considering Patterns 638 Regular expressions 639 The matching operator 639 Mathematical expressions 640 Structured Commands 641 The if statement 641 The while statement 643 The do-while statement 644 The for statement 645 Printing with Formats 645 Using Built-in Functions 648 Mathematical functions 649 String functions 650 Time functions 652 Trying Out User-Defined Functions 653 Defining a function 653 Using your functions 654 Creating a function library 654 Working Through a Practical Example 655 Summary 657 Chapter 23: Working with Alternative Shells 659 Considering the Dash Shell 659 Looking at the Dash Shell Features 661 The Dash command-line parameters 661 The Dash environment variables 662 Default environment variables 662 Positional parameters 663 User-defined environment variables 663 The Dash built-in commands 664 Scripting in Dash 665 Creating Dash scripts 665 Things that won't work 665 Using arithmetic 666 The test command 666 The function command 667 Exploring the zsh Shell 668 Viewing Parts of the zsh shell 668 Shell options 669 Built-in commands 670 Core built-in commands 670 Add-in modules 673 Viewing, adding, and removing modules 673 Scripting with zsh 674 Mathematical operations 674 Performing calculations 674 Mathematical functions 675 Structured commands 676 Functions 677 Working Through a Practical Example 677 Summary 678 Part IV: Creating and Managing Practical Scripts 679 Chapter 24: Writing Simple Script Utilities 681 Performing Backups 681 Backing up files daily 682 Obtaining the required functions 682 Creating a daily archive location 685 Creating a daily backup script 686 Running the daily archive script 688 Creating an hourly archive script 689 Running the hourly archive script 692 Managing Account Deletion 693 Obtaining the required functions 693 Getting the correct account name 693 Creating a function to get the correct account name 695 Verifying the entered account name 696 Determining whether the account exists 698 Removing any account processes 699 Finding account files 702 Removing the account 702 Creating the script 703 Running the script 708 Monitoring Your System 710 Obtaining the default shell audit functions 710 Obtaining the permission audit functions 714 Creating the script 716 Running the script 719 Summary 721 Chapter 25: Getting Organized 723 Understanding Version Control 723 Working directory 724 Staging area 725 Looking at the local repository 725 Exploring a remote repository 726 Branching 726 Cloning 726 Using Git for VCS 727 Setting Up Your Git Environment 727 Committing with Git 732 Summary 739 Appendix A: Quick Guide to Bash Commands 741 Appendix B: Quick Guide to sed and gawk 755 Index 767