Gdb: Difference between revisions

Revision as of 18:20, 29 November 2016

References

7.2 Example Debugging Session: Segmentation Fault Example (unkownroad.com)
Debugging with GDB (delorie.com)
GDB commands (tutorialspoint.com)
An Interactive Guide to Faster, Less Frustrating Debugging
Norm Matloff's Debugging Tutorial (also guide to fast editing and gui debuggers DDD, GVD)
8 gdb tricks you should know (oracle.com)

GDB front-ends

Reference list:

Ideally front-ends must use the GDB/MI2 interface. There is also the built-in Text User Interface to GDB (C-x C-a: http://davis.lbl.gov/Manuals/GDB/gdb_21.html

Good candidates

Mainly from SO post above:

Vim:

https://github.com/vim-scripts/Conque-GDB

Use old shell implementation, not the new neovim terminal.

http://pyclewn.sourceforge.net/

Read some comment it was not convenient, and would require extra plugin like https://github.com/notEvil/vim-debug.

Neovim:

https://github.com/critiqjo/lldb.nvim

Only for LLDB.

GDBInit hacks

Using colout one may color almost any gdb output.

Old front-ends

These are using the old "annotation" mechanisms:

DDD
KDBG
cgdb (text mode with vi keymap)

GDB configuration

GDB reads file ~/.gdbinit at start.

Some references:

http://stackoverflow.com/questions/209534/how-to-highlight-and-color-gdb-output-during-interactive-debugging

Bare minimum configuration

From StackOverflow [1]:

set history save on
set print pretty
set output-radix 16
set height 0

GDB dashboard

GDB dashboard is a modular visual interface for GDB in Python.

To install simply copy .gdbinit as ~/.gdbinit

cp gdb-dashboard/.gdbinit ~/.gdbinit

Alternatively, source it from ~/.gdbinit:

source ~/.gdbinit-dashboard

Install pygments

Install pygments to get source highlighting

sudo pip install Pygments        # Globally
pip install Pygments             # Locally

If GDB uses python3 (ldd $(which gdb))), you'll need to install with pip3:

sudo pip3 install Pygments       # Globally
pip3 install Pygments            # Locally

To get the list of available styles:

python from pygments.styles import get_all_styles as styles
python for s in styles(): print(s)

Alternative styles:

gruvbox

Install Voltron manually

This is the procedure to follow to install Voltron manually, for instance on a custom instance of gdb or on an offline pc. This procedure follows the file install.sh from voltron repository.

Install apt dependencies

# From voltron/install.sh
sudo apt-get -y install libreadline6-dev python-dev python-setuptools python-yaml python-pip

# To fix error "missing library libncursesw.so.5"
sudo apt-get -y install libncursesw5       # On 32-bit python, use libncursesw5:i386

Install python dependencies

Here we install in user local site (--user).

# Extra dependency for requests_unixsocket
pip install --user pbr

# To fix error "ImportError: No module named pkg_resources" when importing scruffy
pip install --user setuptools

# From voltron/install.sh
pip install --user 'scruffington>=0.3.6' flask flask_restful blessed pygments requests requests_unixsocket six pysigset pygments

Check dependencies

We can check some dependencies before installing voltron. Start your gdb and run the following commands. These should run without errors. If not, check the troubleshooting section.

>>> pi import scruffy
>>> pi import flask

Install voltron module

Now, we are ready to install voltron module. We use --no-deps since we installed the dependencies ourselves. Use developer mode if you plan to modify voltron.

cd path/to/voltron
pip install --user --no-deps .
pip install --user --no-deps -e .     # To install in "developer mode"

Source voltron from ~/.gdbinit. For the path, either point to the site-package, or to the voltron repository if installed in developer mode.

source /path/to/your/voltron/entry.py

Finally, add the following to ~/.bashrc:

export PATH=$PATH:~/.local/bin
# Or alternatively:
# alias voltron=~/.local/bin/voltron

Installing on offline PC

Use apt-offline to transfer apt packages.
Use pip download to download pip packages (see Python).

Troubleshooting

Error ImportError: cannot import name md5.

This occurs on import flask. Looking at the `_hashlib.so` dependencies, we have:

locate _hashlib.so
ldd /path/to/your/_hashlib.so
# linux-gate.so.1 =>  (0xf77c3000)
# libssl.so.6 => not found
# libcrypto.so.6 => not found
# libpython2.7.so.1.0 => not found
# libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf776a000)
# libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf75b3000)
# /lib/ld-linux.so.2 (0x5659b000)

Looking on the internet, we find reference of libssl.so.6 and {{file|libcrypto}.so.6} on RedHat 6, related to OpenSSL 0.9.8e. On Ubuntu, we find similar libraries in package libsslfrom Ubuntu Precise LTS 12.04 32-bit.

wget http://security.ubuntu.com/ubuntu/pool/universe/o/openssl098/libssl0.9.8_0.9.8o-7ubuntu3.2_i386.deb
dpkg -x libssl0.9.8_0.9.8o-7ubuntu3.2_i386.deb tmp
sudo cp ./tmp/lib/i386-linux-gnu/libssl.so.0.9.8 /lib/i386-linux-gnu/libssl.so.6
sudo cp ./tmp/lib/i386-linux-gnu/libcrypto.so.0.9.8 /lib/i386-linux-gnu/libcrypto.so.6

Error ImportError: No module named pkg_resources.

Install python package setuptools (see Python).

Install Python modules for GDB

From pwndbg [2]:

# Find the Python version used by GDB.
PYVER=$(gdb -batch -q --nx -ex 'pi import platform; print(".".join(platform.python_version_tuple()[:2]))')
PYTHON=$(gdb -batch -q --nx -ex 'pi import sys; print(sys.executable)')
PYTHON="${PYTHON}${PYVER}"

# Find the Python site-packages that we need to use so that
# GDB can find the files once we've installed them.
SITE_PACKAGES=$(gdb -batch -q --nx -ex 'pi import site; print(site.getsitepackages()[0])')
# or to install in user mode:
SITE_PACKAGES=$(gdb -batch -q --nx -ex 'pi import site; print(site.getusersitepackages())')

# Install Python dependencies
sudo ${PYTHON} -m pip install --target ${SITE_PACKAGES} -Ur requirements.txt

Example of file requirements.txt

pip
pycparser
psutil>=3.1.0

Prepare debug session

Compile with debug symbols, use option -g:

gcc -g program.c               # -g : debug symbols
gcc -g -O0 program.c           #  ... -O0: disable optimization

Force core dumps (see bash help ulimit):

ulimit -c unlimited
./a.out
# Segmentation fault (core dumped)

GDB invocation

gdb a.out
gdb a.out core.1234           # If coredump available

GDB commands

Reference:

GDB manual
https://beej.us/guide/bggdb/

Break points and watch points
`b [+-][NUMBER]` `break [+-][NUMBER]`	Set a breakpoint at current line, at given line NUMBER or NUMBER lines after/before current line.
`b LOCATION` `break LOCATION`	Set breakpoint at LOCATION. `b main` sets a breakpoint at beginning of function `main()`. `b foo.c:42` sets a breakpoint at file foo.c, line 42.
`watch EXPR`	Stop execution when EXPR changes
`awatch EXPR`	Stop execution when EXPR is accessed
`i b` `info b` `info break`	list breakpoints
`cl LOCATION` `clear LOCATION`	Clear breakpoint by LOCATION
`d` `delete`	Delete all breakpoints
`d NUMBER` `delete NUMBER`	Clear breakpoint by NUMBER (as listed by `i b`)
`dis NUMBER` `disable NUMBER`	Disable breakpoint by NUMBER (as listed by `i b`)
`save b FILE` `save breakpoints FILE`	Save current breakpoints as script FILE. Use `source` to reload.

Execute program
`r [ARGS]` `run [ARGS]`	Start (or restart) program. Arguments may include wildcards (*) and redirections (<, <<...)
`kill`	Kill current program.
`c` `cont`	Continue and interrupted program.
`s [NUMBER]` `step [NUMBER]`	Step (into) current line, or NUMBER lines.
`n` `next`	Run to next line (over current line)
`fin` `finish`	Execute till returning from current selected frame.
`advance LOCATION`	Run until temporary breakpoint set at LOCATION.
`u [NUMBER]` `until [NUMBER]`	Execute until the program reaches a source line greater than current (very handy).

View stack
`bt` `bt [COUNT]` `backtrace [COUNT]` `where [COUNT]`	Print backtrace of all stack frames, or innermost (outermost) COUNT frames if COUNT>0 (COUNT<0)
`f [FRAME]` `frame [FRAME]`	Select frame FRAME and print stack frame
`up`	Go up a level in the stack (frame calling current frame).
`do` `down`	Go down a level in the stack (frame called by current frame).

View memory
`disp EXPR` `display EXPR`	Display EXPR at each prompt (if within scope).
`x/FMT ADDRESS`	Examine memory at ADDRESS as FMT(`x`)
`i locals` `i args` `info locals` `info args`	Print information on local variables / function arguments in the current frame
`print EXPR`	print EXPR.
`undisplay NUMBER`	Undisplay expression by NUMBER.

View code
`l` `list`	List (10 by default) lines of current frame
`disassemble /m` `disassemble /s` `disassemble 'foo.c'::bar`	Disassemble a specified section of memory

Miscellaneous
`q` `quit`	Quit gdb.
`help COMMAND` `apropos WORD`	Get help on COMMAND, or search commands related to WORD.
`source FILE`	Source script FILE.
`RETURN`	Repeat last command.

Tips

Define a custom label for breakpoint in C/C++

Say we want to set a breakpoint at a specified location in source file, but this position may move over time. The easiest is to use an asm statement to define the label [3]:

#include <stdio.h>

int main () {
  void *ret_p = &&ret;
  printf("ret: %p\n", ret_p);
  goto *ret_p;

  return 1;

ret:
  asm("RET:")

    return 0;
}

This will add a symbol table entry as follows.

gcc  -Wl,--export-dynamic t.c  -ldl
readelf -s a.out | grep RET
# 41: 0804858a     0 NOTYPE  LOCAL  DEFAULT   13 RET

Use `help` to test a command abbreviation

Don't know if f stands for finish or frame? Just use help f:

help f
# Select and print a stack frame.
# ...

8 gdb tricks you should know

From https://blogs.oracle.com/ksplice/entry/8_gdb_tricks_you_should:

Use break WHERE if COND

For instance break context_switch if next == init_task.

Use command

This sets commands to be executed when a breakpoint is hit. For instance

b do_mmap_pgoff 
# Breakpoint 1 at 0xffffffff8111a441: file mm/mmap.c, line 940.
command 1
# Type commands for when breakpoint 1 is hit, one per line.
# End with a line saying just "end".
>print addr
>print len
>print prot
>end

Use gdb --args to specify runtime arguments

gdb --args pizzamaker --deep-dish --toppings=pepperoni
# ...
show args
# Argument list to give program being debugged when it is started is
#  " --deep-dish --toppings=pepperoni".
b main
# ...

Finding source files

Use directory to add directory to search for source files.

list main
# 1192    ls.c: No such file or directory.
#     in ls.c
directory ~/src/coreutils-7.4/src/
# Source directories searched: /home/nelhage/src/coreutils-7.4:$cdir:$cwd
list main

Use set substitute-path to fix absolute paths

list schedule
# 5519    /build/buildd/linux-2.6.32/kernel/sched.c: No such file or directory.
#     in /build/buildd/linux-2.6.32/kernel/sched.c
set substitute-path /build/buildd/linux-2.6.32 /home/nelhage/src/linux-2.6.32/
list schedule

Use gcc -ggdb3 to add MACRO's symbol as gdb macro

Use gdb's variable like $1 but also define your own

set $foo = 4
p $foo
# $3 = 4

Use register variables

All CPU registers are available as variables like $R0, $R1, but gdb also defines cross-architectures ones like $sp and $pc. For instance, if $rsi register is used to pass the 1st parameter,

break write if $rsi == 2

The x command

x/s 0xffffffff81946000
# ffffffff81946000 <>:     "root=/dev/sda1 quiet"

# Use x/i as a quick way to disassemble memory
x/5i schedule
#   0xffffffff8154804a <schedule>:   push   %rbp
#   0xffffffff8154804b <schedule+1>: mov    $0x11ac0,%rdx
#   0xffffffff81548052 <schedule+8>: mov    %gs:0xb588,%rax
#   0xffffffff8154805b <schedule+17>:    mov    %rsp,%rbp
#   0xffffffff8154805e <schedule+20>:    push   %r15

# Print code surround current pc
x/20i $ip-40

Use the @ symbol

The following works if source code is something like int a[ 10 ] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };:

p a
# $1 = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

But if the array is much bigger, or array is a C++ vector, we can use @:

p *&a[0]@10
# $1 = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
p *&a[550]@4
# $1 = {551, 552, 553, 554}

Use value history variables `$`

See summary list below. For more details, check GDB manual on Value History.

$1          # History value 1
$2          # History value 2
$           # MOST RECENT value in the history
$$          # ... and value before that
$$2         # ... and value before that
$$0         # Same as $
$$1         # Same as $$

For instance:

p *$        # Dereference last value
p *$.next   # ... follow the chain
<RETURN>    # ... again
<RETURN>    # ... again

show values    # Print the last 10 values.

Examples

Simple Segmentation Fault Example

(From [4])

Example program segfault.c:

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
  char *buf;

  buf = malloc(1<<31);

  fgets(buf, 1024, stdin);
  printf("%s\n", buf);

  return 1;
}

Compile and launch gdb:

gcc -g segfault.c
gdb a.out

The debug session

run
backtrace
frame 3
print buf
kill
break segfault.c:8
run
print buf
next
print buf

Fix the bug, then start again, watching now buf:

watch buf
# Start again, answer 'y' when asked to start from beginning
run
# Break at watch point, let's _c_ontinue
c

More GDB stuff

display i            # Display i at each stop
display/4i $pc       # Display the 4 next instruction at each stop
u                    # Continue until we reach an higher pc address

Using gdb-dashboard

da                   # Print dashboard again
da reg               # Remove register
da reg               # Add it back
da m w 0x1000 0x10   # Watch memory zone at 0x1000 for 16 bytes
da m w 0x2000 0x10   # Add a second zone at 0x2000

@@ Line 121: / Line 121: @@
 pip install --user --no-deps -e .     # To install in "developer mode"
 </source>
+Source voltron from {{file|~/.gdbinit}}. For the path, either point to the site-package, or to the voltron repository if installed in ''developer mode''.
+<source lang=python>
+source /path/to/your/voltron/entry.py
+</source>
 Finally, add the following to {{file|~/.bashrc}}:
 <source lang=bash>

Gdb: Difference between revisions

Revision as of 18:20, 29 November 2016

Contents

References

GDB front-ends

Good candidates

GDBInit hacks

Old front-ends

GDB configuration

Bare minimum configuration

GDB dashboard

Install Voltron manually

Install Python modules for GDB

Prepare debug session

GDB invocation

GDB commands

Tips

Define a custom label for breakpoint in C/C++

Use `help` to test a command abbreviation

8 gdb tricks you should know

Use value history variables `$`

Examples

Simple Segmentation Fault Example

More GDB stuff

Using gdb-dashboard

Navigation menu

Gdb: Difference between revisions

Revision as of 18:20, 29 November 2016

References

GDB front-ends

Good candidates

GDBInit hacks

Old front-ends

GDB configuration

Bare minimum configuration

GDB dashboard

Install Voltron manually

Install Python modules for GDB

Prepare debug session

GDB invocation

GDB commands

Tips

Define a custom label for breakpoint in C/C++

Use help to test a command abbreviation

8 gdb tricks you should know

Use value history variables $

Examples

Simple Segmentation Fault Example

More GDB stuff

Using gdb-dashboard

Navigation menu

Search

Use `help` to test a command abbreviation

Use value history variables `$`