Rr

rr is the Record and Replay Framework, developped by Mozilla, and is a replacement for and powerful enhancement of gdb.

References

• rr homepage.
• Using rr in an IDE
• Building and Installing and rr Usage, from Mozilla wiki.

• Pernosco A nice front-end for rr (free for users on Github, see HN thread).

Install and configuration

On Debian, rr is available as a package:

sudo apt install rr

rr also requires the linux performance analysis package for the current kernel.

# Install linux perf
sudo apt install linux-perf-4.19

# Enable perf for user land (temporary)
sudo sysctl kernel.perf_event_paranoid=1

# Enable perf permanently
echo 'kernel.perf_event_paranoid=1' | sudo tee '/etc/sysctl.d/51-rr-enable-perf-events.conf'

# To squeeze max perf on laptop (even plugged on AC!), disable CPU scaling:
sudo apt-get install cpufrequtils
sudo cpufreq-set -g performance          # Must be redone at each reboot !!!
# CPU scaling can be disabled permanently in BIOS

See rr usage page for more performance tuning and setting perf flag permanently.

See also troubleshooting below (for VM...).

Integration with cgdb

rr works nicely with cgdb:

rr replay -d cgdb

CGDB output will be messed up if target produces any output [1]. Here a few options to circumvent that:

• Mute any output with rr replay -q -d cgdb
• Start rr in one window, and attach cgdb to it.

rr replay -s 1234               # In one terminal
cgdb -ex "target remote :1234"  # In second terminal

Example of use

Start recording the application:

rr record my_app

Then replay it with:

rr replay

In case the application fires several processes, we can tell rr to fast-forward until child is forked. First we use rr ps to get the list of recorded processes, then we start rr with -f:

rr ps
rr replay -f 12345

During replay, rr behaves likes gdb, with the usual reverse debugging command (rc, rn, rs...). See referenced links for more information.

NOTE: rr seems to ignore breakpoints if we do run instead of cont...

Troubleshoot

Potential issues in VM

• See rr page [2].
• See this bug in VMWare [3]. Fix is to add monitor_control.disable_hvsim_clusters = true.
• See this bug in Xen [4].
• Discuss on rr in Hacker News.