Chapter 6 ⏱️ 45 min read 📚 Intermediate

Tune System Performance

Master system performance monitoring and tuning. Learn to use performance analysis tools, manage process priorities, and configure tuned profiles for optimal system performance.

🎯 Introduction

System performance tuning involves monitoring resource usage, adjusting process priorities, and applying performance profiles to optimize system behavior for specific workloads. RHEL provides powerful tools to monitor CPU, memory, disk, and network usage, plus the tuned daemon for automated performance tuning.

In this chapter, you'll learn to:

  • Monitor system performance with top, ps, uptime, and other tools
  • Adjust process priorities using nice and renice
  • Manage tuned profiles for different workload types
  • Identify performance bottlenecks
  • Optimize system settings for specific use cases (desktop, server, virtual machine, etc.)
📘 Note

Performance tuning is about finding the right balance for your specific workload. There's no one-size-fits-all solution.

📊 Performance Monitoring Tools

top - Real-Time Process Viewer

The top command provides a dynamic real-time view of running processes:

# Launch top
top

# Sample output:
top - 14:23:15 up  3:42,  2 users,  load average: 0.52, 0.58, 0.59
Tasks: 234 total,   1 running, 233 sleeping,   0 stopped,   0 zombie
%Cpu(s):  5.3 us,  2.1 sy,  0.0 ni, 92.2 id,  0.3 wa,  0.0 hi,  0.1 si,  0.0 st
MiB Mem :   3947.8 total,    234.5 free,   2456.7 used,   1256.6 buff/cache
MiB Swap:   2048.0 total,   2048.0 free,      0.0 used.   1234.3 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 1234 student   20   0 1234567  89012   3456 S   2.3   2.2   0:45.67 firefox
 5678 root      20   0  123456  12345   6789 S   1.0   0.3   1:23.45 httpd

Understanding top Output

Field Description
load average 1, 5, and 15-minute load averages (processes waiting for CPU)
us (user) Time spent running user processes
sy (system) Time spent in kernel space
id (idle) CPU idle time
wa (I/O wait) Time waiting for I/O operations (disk)
PR Priority (lower is higher priority)
NI Nice value (-20 to 19)
VIRT Virtual memory used (total)
RES Resident memory (physical RAM)

Interactive Commands in top

# While top is running:
k       # Kill a process (enter PID)
r       # Renice a process (change priority)
M       # Sort by memory usage
P       # Sort by CPU usage (default)
1       # Toggle individual CPU cores display
h       # Help screen
q       # Quit

# Start top with specific options
top -u student      # Show only student's processes
top -d 5            # Update every 5 seconds (default is 3)
top -n 3            # Run 3 iterations then exit
top -b > top.log    # Batch mode for logging

uptime - System Load

# Check how long system has been running and load average
uptime
14:23:15 up 3:42, 2 users, load average: 0.52, 0.58, 0.59

# Interpreting load average:
# - Numbers represent 1, 5, and 15-minute averages
# - On 4-core system: load of 4.0 = 100% utilization
# - Load > number of cores = system overloaded

free - Memory Usage

# Display memory usage
free

# Human-readable format
free -h
              total        used        free      shared  buff/cache   available
Mem:          3.9Gi       2.4Gi       234Mi       156Mi       1.2Gi       1.2Gi
Swap:         2.0Gi          0B       2.0Gi

# Key fields:
# available: estimate of memory available for starting new applications
# buff/cache: memory used for caching (can be reclaimed if needed)

ps - Process Status

# View all processes with full details
ps aux

# View processes in tree format
ps auxf

# View processes for specific user
ps -u student

# Custom output format
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -10

# Find specific process
ps aux | grep httpd

vmstat - Virtual Memory Statistics

# Display system statistics
vmstat

# Update every 2 seconds, 5 times
vmstat 2 5

# Sample output:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 234567  12345 678901    0    0   123   456  789 1234  5  2 92  1  0

⚙️ Process Management & Priority

Linux uses a priority system to determine which processes get CPU time. Understanding and adjusting process priorities is crucial for performance tuning.

Priority and Nice Values

Each process has two related values:

  • Priority (PR): Actual priority used by kernel (calculated value)
  • Nice (NI): User-controllable value that influences priority
Nice Value Priority Who Can Set
-20 (highest priority) Most CPU time Root only
0 (default) Normal priority Everyone
19 (lowest priority) Least CPU time Everyone
📘 Understanding "Nice"

The term "nice" reflects the concept of being "nice" to other users. A higher nice value means the process is being nicer by using less CPU priority.

🎚️ Nice and Renice Commands

nice - Start Process with Different Priority

Use nice to start a new process with a specific nice value:

# Start process with default nice (0)
./my_program

# Start process with nice value of 10 (lower priority)
nice -n 10 ./my_program

# Start with nice value of -10 (higher priority, requires root)
sudo nice -n -10 ./important_program

# Start background process with low priority
nice -n 15 ./backup_script.sh &

# Check the nice value
ps -o pid,ni,cmd -p $(pgrep my_program)

renice - Change Priority of Running Process

Use renice to change the priority of an already running process:

# Renice by PID
renice -n 5 -p 1234

# Renice all processes for a user (requires root)
sudo renice -n 10 -u student

# Renice all processes in a group
sudo renice -n 5 -g developers

# Lower priority of CPU-intensive process (nice to others)
renice -n 19 -p $(pgrep heavy_computation)

# Increase priority (requires root)
sudo renice -n -5 -p 1234

# Verify the change
ps -o pid,ni,cmd -p 1234

Practical Examples

# Run backup with low priority so it doesn't slow down system
nice -n 19 tar -czf /backups/home.tar.gz /home &

# Run database maintenance with slightly elevated priority
sudo nice -n -5 /usr/bin/mysql_optimize

# Lower priority of someone's runaway process
sudo renice -n 15 -p $(pgrep -u problematic_user)

# Start compilation with low priority
nice -n 10 make -j4
💡 Best Practices
  • Use nice for batch jobs and backups (nice 10-19)
  • Never use nice -20 unless absolutely necessary
  • Regular users can only increase nice values (lower priority)
  • Only root can decrease nice values (higher priority)

🔧 Tuned Profiles

tuned is a daemon that monitors system usage and adjusts settings to optimize performance for different workload types. It provides pre-configured profiles for common scenarios.

Installing and Starting Tuned

# Install tuned (usually pre-installed)
sudo dnf install tuned

# Start and enable tuned service
sudo systemctl start tuned
sudo systemctl enable tuned

# Check tuned status
sudo systemctl status tuned

Working with Tuned Profiles

# List all available profiles
tuned-adm list

# Sample output:
Available profiles:
- balanced
- desktop
- latency-performance
- network-latency
- network-throughput
- powersave
- throughput-performance
- virtual-guest
- virtual-host
Current active profile: balanced

# Check current active profile
tuned-adm active

# Get recommended profile for your system
tuned-adm recommend

# Switch to a different profile
sudo tuned-adm profile throughput-performance

# Switch to profile optimized for virtual guest
sudo tuned-adm profile virtual-guest

# Turn off tuned (not recommended)
sudo tuned-adm off

Common Tuned Profiles

Profile Use Case Optimizations
balanced Default, good compromise Balance between performance and power saving
throughput-performance Servers, maximum throughput Maximum performance, no power saving
latency-performance Low-latency requirements Minimize latency at cost of power
virtual-guest Virtual machines (guest) Optimized for VMs
virtual-host Hypervisor hosts Optimized for running VMs
desktop Desktop/laptop systems Responsive, some power saving
powersave Battery-powered devices Maximum power savings

Verifying Tuned Profile

# Check which profile is active
tuned-adm active
Current active profile: throughput-performance

# Verify profile was applied correctly
tuned-adm verify
Verification succeeded, current system settings match the preset profile.

# If verification fails, it means manual changes conflict with profile
# Reapply the profile:
sudo tuned-adm profile throughput-performance

Procedure: Optimizing a Database Server

  1. Check current profile:
    tuned-adm active
  2. Get system recommendation:
    tuned-adm recommend
  3. Apply throughput-performance profile:
    sudo tuned-adm profile throughput-performance
  4. Verify the change:
    tuned-adm active
    tuned-adm verify
  5. Monitor database performance improvements

📝 Practice Questions

Question 1: What does a load average of 4.0 mean on a 4-core system?

  • A) System is idle
  • B) System is at 100% CPU utilization
  • C) System is overloaded
  • D) System has 4 processes running
Answer: B) System is at 100% CPU utilization
Load average represents processes waiting for CPU. On a 4-core system, a load of 4.0 means all cores are fully utilized. Load > number of cores indicates overloading.

Question 2: Which nice value gives a process the HIGHEST priority?

  • A) 19
  • B) 0
  • C) -20
  • D) 10
Answer: C) -20
Nice values range from -20 (highest priority) to 19 (lowest priority). Only root can set negative nice values. Lower nice values = higher priority.

Question 3: How do you permanently change a running process's priority?

  • A) nice -n 10 -p 1234
  • B) renice -n 10 -p 1234
  • C) top then press 'r'
  • D) Both B and C
Answer: D) Both B and C
renice changes the priority of a running process. You can also use top's interactive 'r' command. nice is only for starting new processes, not changing existing ones.

Question 4: Which tuned profile is best for a virtual machine guest?

  • A) balanced
  • B) throughput-performance
  • C) virtual-guest
  • D) virtual-host
Answer: C) virtual-guest
The virtual-guest profile is specifically optimized for virtual machine guests. virtual-host is for hypervisor hosts. Apply with: sudo tuned-adm profile virtual-guest

Question 5: What command shows the active tuned profile?

  • A) tuned-adm status
  • B) tuned-adm active
  • C) systemctl status tuned
  • D) tuned-adm list
Answer: B) tuned-adm active
tuned-adm active shows the currently active profile. tuned-adm list shows all available profiles. tuned-adm recommend suggests the best profile for your system.