Debug Processes with strace and perf on VPS

June 2026 · 8 min read

When a process on your VPS hangs, slows down unexpectedly, or consumes more resources than it should, tools like strace and perf help you see what the process is actually doing at the system call level — without stopping your service or modifying code. Both are standard tools available on every Linux distribution and work perfectly on real production servers.

Install strace and perf

sudo apt update
sudo apt install strace linux-tools-common linux-tools-$(uname -r) -y

# Check versions
strace --version
perf --version

Using strace to View System Calls

# Attach to a running process by PID
sudo strace -p 1234

# View only file I/O
sudo strace -p 1234 -e trace=read,write,open,close

# View network calls
sudo strace -p 1234 -e trace=network

# Log output to file + show timestamps
sudo strace -p 1234 -T -tt -o /tmp/strace.log

# Run a new command with strace
strace -T ls /var/log

strace Summary of Most-Used System Calls

# -c = count mode shows summary
sudo strace -c -p 1234
# Press Ctrl+C to stop and see summary

# Example output:
# % time     seconds  usecs/call     calls    syscall
# 45.23       0.054        54      1000    epoll_wait
# 23.11       0.027        27      1000    read
# 18.45       0.022        22       500    write

View Files Opened by a Process with lsof

# View all files the process has open
sudo lsof -p 1234

# View only network connections
sudo lsof -p 1234 -i

# View files any process has open in a directory
sudo lsof +D /var/log/nginx/

Using perf to Find Performance Bottlenecks

# View top functions consuming the most CPU
sudo perf top -p 1234

# Record 10 seconds then show report
sudo perf record -g -p 1234 sleep 10
sudo perf report

# View CPU events across the system for 30 seconds
sudo perf stat -a sleep 30

Debug Hanging Processes

# See what system call the process is waiting on
sudo cat /proc/1234/wchan
# "futex" = waiting on mutex/semaphore
# "poll_schedule_timeout" = waiting on network/file
# "disk_sleep" = waiting on I/O (dangerous)

# View stack trace in real-time
sudo cat /proc/1234/stack

# View open file descriptors
ls -la /proc/1234/fd/

Verify Tools Are Working Correctly

# Test strace with a simple command
strace -c ls /etc 2>&1 | head -20

# Test perf on the system (requires root)
sudo perf stat ls /etc

# Check if strace can attach to processes
sudo strace -p 1 -e trace=none 2>&1
# No error = ptrace permission is granted

# View system load with easier-to-use tools
sudo apt install htop iotop -y
sudo iotop -oPa  # View I/O per process

Common Issues & Troubleshooting

strace error: "attach: ptrace(PTRACE_SEIZE): Operation not permitted"

The kernel has disabled ptrace for other processes. You must run as root or temporarily disable the restriction:

sudo strace -p 1234
# If still failing:
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
# Restore the setting when done:
echo 1 | sudo tee /proc/sys/kernel/yama/ptrace_scope

strace slows down the process significantly

strace has high overhead when tracing every syscall. Use filters to reduce the impact:

# View only syscalls taking longer than 1ms
sudo strace -T -p 1234 2>&1 | awk '/<[0-9]/ && $NF > 0.001'

# Or use perf instead, which has much lower overhead
sudo perf top -p 1234

perf error: "No permission to collect stats"

You must run perf as root or adjust the perf_event_paranoid setting:

sudo perf stat ls
# Or reduce restriction:
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
# Restore when done:

Process using 100% CPU but can't find the bottleneck

Use flamegraph to visualize the call stack:

sudo perf record -F 99 -p 1234 -g sleep 30
sudo perf script > /tmp/out.perf
# Get FlameGraph tools
git clone https://github.com/brendangregg/FlameGraph
perl FlameGraph/stackcollapse-perf.pl /tmp/out.perf | perl FlameGraph/flamegraph.pl > /tmp/flame.svg

Frequently Asked Questions

What's the difference between strace and ltrace?

strace shows system calls (kernel interface) while ltrace shows library calls (like libc functions). If a process is slow due to algorithm logic in user space, use perf or ltrace. If it's slow due to I/O or network, use strace.

Can I detect memory leaks with strace?

Partially — you can watch for continuous increases in mmap/brk syscalls. However, better tools are Valgrind (valgrind --leak-check=full) or AddressSanitizer for C/C++ programs.

Can I use strace with Docker containers?

Yes, if the container runs with --cap-add SYS_PTRACE or --security-opt seccomp=unconfined. For already-running containers:

docker exec -it container_name bash
# Inside container:
apt install strace -y
strace -p $(pgrep myapp)