Gdb: Difference between revisions

From miki
Jump to navigation Jump to search
Line 687: Line 687:


If the remote executable uses dynamic libraries, these must be sent to the client.
If the remote executable uses dynamic libraries, these must be sent to the client.
To accelerate the loading, copy first the libraries locally on the client, then tell GDB where to find them:
To accelerate the loading, copy first the libraries locally on the client, then tell GDB where to find them [https://www.fayewilliams.com/2013/01/31/gdb-unable-to-find-dynamic-linker-breakpoint-function/]:
<source lang="text">
<source lang="text">
set sysroot /d/st/courses/training_obfu/formation_reverse/dynamic/tp/qmessage/pi_root
set sysroot /d/st/courses/training_obfu/formation_reverse/dynamic/tp/qmessage/pi_root

Revision as of 17:32, 5 October 2019

References

GDB front-ends

Reference list:

Ideally front-ends must use the GDB/MI2 interface. There is also the built-in Text User Interface to GDB (C-x C-a: http://davis.lbl.gov/Manuals/GDB/gdb_21.html

Good candidates

Mainly from SO post above:

Vim:

Use old shell implementation, not the new neovim terminal.
Read some comment it was not convenient, and would require extra plugin like https://github.com/notEvil/vim-debug.

Neovim:

Only for LLDB.
GDBInit hacks
  • Using colout one may color almost any gdb output.
Old front-ends

These are using the old "annotation" mechanisms:

GDB configuration

GDB reads file ~/.gdbinit at start.

Some references:

Bare minimum configuration

From StackOverflow [1]: 

set history save on
set print pretty
set output-radix 16
set height 0

Install Python modules for GDB

From pwndbg [2]: 

# Find the Python version used by GDB.
PYVER=$(gdb -batch -q --nx -ex 'pi import platform; print(".".join(platform.python_version_tuple()[:2]))')
PYTHON=$(gdb -batch -q --nx -ex 'pi import sys; print(sys.executable)')
PYTHON="${PYTHON}${PYVER}"

# Find the Python site-packages that we need to use so that
# GDB can find the files once we've installed them.
SITE_PACKAGES=$(gdb -batch -q --nx -ex 'pi import site; print(site.getsitepackages()[0])')
# or to install in user mode:
SITE_PACKAGES=$(gdb -batch -q --nx -ex 'pi import site; print(site.getusersitepackages())')

# Install Python dependencies
sudo ${PYTHON} -m pip install --target ${SITE_PACKAGES} -Ur requirements.txt

Example of file requirements.txt 

pip
pycparser
psutil>=3.1.0

Front-ends

GDB dashboard

GDB dashboard is a modular visual interface for GDB in Python.

To install simply copy .gdbinit as ~/.gdbinit 

cp gdb-dashboard/.gdbinit ~/.gdbinit


Alternatively, source it from ~/.gdbinit: 

source ~/.gdbinit-dashboard
Install pygments

Install pygments to get source highlighting 

sudo pip install Pygments        # Globally
pip install Pygments             # Locally

If GDB uses python3 (ldd $(which gdb))), you'll need to install with pip3: 

sudo pip3 install Pygments       # Globally
pip3 install Pygments            # Locally

To get the list of available styles: 

python from pygments.styles import get_all_styles as styles
python for s in styles(): print(s)

Alternative styles:

Issues
  • Cannot write the dashboard: [Errno 10] No child process issue #56
This is a bug in the gdb vendor. Fix file platform.py:
@@ -1009,7 +1009,10 @@
     except (AttributeError,os.error):
         return default
     output = string.strip(f.read())
-    rc = f.close()
+    try:
+        rc = f.close()
+    except:
+        rc = 0
     if not output or rc:
         return default
     else:
  • Cannot write the dashboard: Invalid character '?' in expression.
Likely due to incorrect character in info reg. The register is evaluated via parse_and_eval

Voltron

See Voltron.

Prepare debug session

  • Compile with debug symbols, use option -g:
gcc -g program.c               # -g : debug symbols
gcc -g -O0 program.c           #  ... -O0: disable optimization
  • Force core dumps (see bash help ulimit):
ulimit -c unlimited
./a.out
# Segmentation fault (core dumped)

GDB invocation

gdb a.out
gdb a.out core.1234           # If coredump available

GDB commands

Reference

Break points and watch points

b [+-][NUMBER]
break [+-][NUMBER]

Set a breakpoint at current line, at given line NUMBER or NUMBER lines after/before current line.

b LOCATION
break LOCATION

 Set breakpoint at LOCATION.
b main sets a breakpoint at beginning of function main().
b foo.c:42 sets a breakpoint at file foo.c, line 42.
b *0x400120 sets a breakpoint at address 0x400120.

watch EXPR

Stop execution when EXPR changes

awatch EXPR

Stop execution when EXPR is accessed

i b
info b
info break

 list breakpoints

cl LOCATION
clear LOCATION

Clear breakpoint by LOCATION

d
delete

Delete all breakpoints

d NUMBER
delete NUMBER

Clear breakpoint by NUMBER (as listed by i b)

dis NUMBER
disable NUMBER

Disable breakpoint by NUMBER (as listed by i b)

save b FILE
save breakpoints FILE

Save current breakpoints as script FILE. Use source to reload.
Registers

info reg
i r

Display registers

Execute program

r [ARGS]
run [ARGS]

Start (or restart) program. Arguments may include wildcards (*) and redirections (<, <<...)

start [ARGS]

 Start and break on main

kill

Kill current program.

c
cont

Continue and interrupted program.

s [NUMBER]
step [NUMBER]

Step (into) current line, or NUMBER lines.

si [NUMBER]
stepi [NUMBER]

Step one (assembly) instruction exactly (N times)

n
next

Run to next line (over current line)

ni [NUMBER]
nexti [NUMBER]

Step one (assembly) instruction exactly (N times), but over subroutine calls.

fin
finish

Execute till returning from current selected frame.

adv LOCATION advance LOCATION

Run until temporary breakpoint set at LOCATION.

u [NUMBER] until [NUMBER]

Execute until the program reaches a source line greater than current (very handy).
View stack

bt
bt [COUNT]
backtrace [COUNT]
where [COUNT]

Print backtrace of all stack frames, or innermost (outermost) COUNT frames if COUNT>0 (COUNT<0)

f [FRAME]
frame [FRAME]

Select frame FRAME and print stack frame

up

Go up a level in the stack (frame calling current frame).

do
down

Go down a level in the stack (frame called by current frame).
View memory

disp EXPR
display EXPR

Display EXPR at each prompt (if within scope).

x[/nfu] ADDR x/16i ADDR x/16zw ADDR

Examine n memory locations at ADDR, format as n (xduotacfszr) and unit size u (bhwg). If omitted, reuse last format FMT.

  • /i for a single assembly instruction. /16i for 16 instructions
  • /16zw for 16 32-bit words with leading zeroes.
  • x/16i $PC use current PC addr

i locals
i args
info locals
info args

Print information on local variables / function arguments in the current frame

print EXPR

print EXPR.

undisplay NUMBER

Undisplay expression by NUMBER.
View code

l
list

List (10 by default) lines of current frame

disas
disassemble /m
disassemble /s
disassemble 'foo.c'::bar disassemble main,+40 disassemble $pc,+40

Disassemble a specified section of memory
Miscellaneous

q
quit

Quit gdb.

help COMMAND
apropos WORD

Get help on COMMAND, or search commands related to WORD.

source FILE

Source script FILE.
RETURN Repeat last command.

add-symbol-file FILE ADDR

Load symbols from FILE at given ADDR.

Change memory

set VARIABLE = VALUE
For instance, given a program 

int main(void){
    char[] person = "Bob";
    char[] p2 = "Alice";

    printf("Hello %s\n");
}

set main::person = { 'S', 'a', 'm', 0x00 }
set main::person = "Sam"
# To change memory directly:
set {char [4]}0x43f800 = {0x01, 0x02, 0x03, 0x04}

Alternatively we can use strcpy [3]: 

(gdb) p malloc(20)
$3 = (void *) 0x6ce81808
(gdb) p strcpy($3, "my string")
$4 = 1827149832
(gdb) x/s $3
0x6ce81808: "my string"

Dump / restore

For instance, to load an Intel hex file: 

restore firmware.hex

To generate the same file: 

dump ihex memory firmware.hex 0x00400000 0x0040FFFF

Note that dump can also save the result of an expression: dump [format] memory filename start_addr end_addr dump [format] value filename expr

Python extension

See Python in GDB manual.

Troubleshooting

Tips

Use RETURN to repeat last command

Pressing RETURN repeats all last command:

  • Commands like s (step) or n (next). Very handy to step in the code.
  • Command like x/16w 0x10100000 or x/16i 0x10100000, each time printing 16 new words.

Define a custom label for breakpoint in C/C++

Say we want to set a breakpoint at a specified location in source file, but this position may move over time. The easiest is to use an asm statement to define the label [4]: 

#include <stdio.h>

int main () {
  void *ret_p = &&ret;
  printf("ret: %p\n", ret_p);
  goto *ret_p;

  return 1;

ret:
  asm("RET:")

    return 0;
}

This will add a symbol table entry as follows. 

gcc  -Wl,--export-dynamic t.c  -ldl
readelf -s a.out | grep RET
# 41: 0804858a     0 NOTYPE  LOCAL  DEFAULT   13 RET

Use help to test a command abbreviation

Don't know if f stands for finish or frame? Just use help f: 

help f
# Select and print a stack frame.
# ...

8 gdb tricks you should know

From https://blogs.oracle.com/ksplice/entry/8_gdb_tricks_you_should:

  • Use break WHERE if COND
For instance break context_switch if next == init_task.
  • Use command
This sets commands to be executed when a breakpoint is hit. For instance
b do_mmap_pgoff 
# Breakpoint 1 at 0xffffffff8111a441: file mm/mmap.c, line 940.
command 1
# Type commands for when breakpoint 1 is hit, one per line.
# End with a line saying just "end".
>print addr
>print len
>print prot
>end
  • Use gdb --args to specify runtime arguments
gdb --args pizzamaker --deep-dish --toppings=pepperoni
# ...
show args
# Argument list to give program being debugged when it is started is
#  " --deep-dish --toppings=pepperoni".
b main
# ...
  • Finding source files
Use directory to add directory to search for source files.
list main
# 1192    ls.c: No such file or directory.
#     in ls.c
directory ~/src/coreutils-7.4/src/
# Source directories searched: /home/nelhage/src/coreutils-7.4:$cdir:$cwd
list main
Use set substitute-path to fix absolute paths
list schedule
# 5519    /build/buildd/linux-2.6.32/kernel/sched.c: No such file or directory.
#     in /build/buildd/linux-2.6.32/kernel/sched.c
set substitute-path /build/buildd/linux-2.6.32 /home/nelhage/src/linux-2.6.32/
list schedule
  • Use gcc -ggdb3 to add MACRO's symbol as gdb macro
  • Use gdb's variable like $1 but also define your own
set $foo = 4
p $foo
# $3 = 4
  • Use register variables
All CPU registers are available as variables like $R0, $R1, but gdb also defines cross-architectures ones like $sp and $pc. For instance, if $rsi register is used to pass the 1st parameter,
break write if $rsi == 2
  • The x command
x/s 0xffffffff81946000
# ffffffff81946000 <>:     "root=/dev/sda1 quiet"

# Use x/i as a quick way to disassemble memory
x/5i schedule
#   0xffffffff8154804a <schedule>:   push   %rbp
#   0xffffffff8154804b <schedule+1>: mov    $0x11ac0,%rdx
#   0xffffffff81548052 <schedule+8>: mov    %gs:0xb588,%rax
#   0xffffffff8154805b <schedule+17>:    mov    %rsp,%rbp
#   0xffffffff8154805e <schedule+20>:    push   %r15

# Print code surround current pc
x/20i $ip-40
  • Use the @ symbol
The following works if source code is something like int a[ 10 ] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };:
p a
# $1 = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
But if the array is much bigger, or array is a C++ vector, we can use @:
p *&a[0]@10
# $1 = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
p *&a[550]@4
# $1 = {551, 552, 553, 554}

Use value history variables $

See summary list below. For more details, check GDB manual on Value History. 

$1          # History value 1
$2          # History value 2
$           # MOST RECENT value in the history
$$          # ... and value before that
$$2         # ... and value before that
$$0         # Same as $
$$1         # Same as $$

For instance: 

p *$        # Dereference last value
p *$.next   # ... follow the chain
<RETURN>    # ... again
<RETURN>    # ... again

show values    # Print the last 10 values.

Disassemble at an arbitrary address

disas FUNCTION command only works for defined FUNCTION. To disassemble at an arbitrary address:

  • x/i 0x10100000 one instruction at address 0x10100000,
  • x/16i 0x10100000 (16 instruction) at address Ox10100000,
  • x/16i *0x10100000 (16 instruction) at address contained at Ox10100000,
  • x/16i $PC disassemble 16 instructions at PC.

Redirect STDIN

Redirect STDIN with the run command: 

gdb ./myprog
run <input.txt            # Run with stdin redirected

One can also pass parameters with the run command (or use gdb --args command-line parameter).

Exit gdb if run fails

Use python script 

def tryRun():
    try:
        gdb.execute("run")
    except:
        print("Exception occured during gdb command: run")
        gdb.execute("quit 1")

Define custom gdb command

class MyCommand( gdb.Command ):
    """MyCommand"""

    def __init__(self):
        super(MyCommand, self).__init__( "mycommand", gdb.COMMAND_USER )

    def invoke(self, arg, from_tty):
        print("My great command is alive!")
        print "R0=0x%08X" %( gdb.parse_and_eval("$R0") )

MyCommand()

Load symbols from another ELF file

This is useful in embedded systems where some part of the code are already loaded or stored in a ROM. When the debugger stops in a function for which it doesn't have the symbols, it cannot decode the stack and provide a correct backtrace. The fix is to load the symbols for that function using the command add-symbol-file. 

add-symbol-file a.out 0               # Add symbol from file a.out, at offset 0

Typically the offset to give is the lowest address in the ELF file.

Modify memory content

Use set [5]. Simplest is to modify a variable: 

set variable i = 10

But GDB can write in any memory location: 

set {int}0x83040 = 4
set *((int *)0x83040) = 4

Remote debug with gdbserver

Start the gdb server on the remote machine: 

gdbserver --multi 192.168.20.1:6666     # CLIENT_ADDR:REMOTE_PORT

This will listen on port 6666, waiting for connection from client 192.168.20.1

On the client, start the GDB client: 

gdb-multiarch ./my_exec                 # multiarch necessary if client/server are different arch

Then, in GDB: 

target extended-remote {REMOTE_ADDR}:{REMOTE_PORT}
set remote exec-file /path/on/remote/to/my_exec

If the remote executable uses dynamic libraries, these must be sent to the client. To accelerate the loading, copy first the libraries locally on the client, then tell GDB where to find them [6]: 

set sysroot /d/st/courses/training_obfu/formation_reverse/dynamic/tp/qmessage/pi_root
set solib-absolute-prefix /d/st/courses/training_obfu/formation_reverse/dynamic/tp/qmessage/pi_root
set solib-search-path /d/st/courses/training_obfu/formation_reverse/dynamic/tp/qmessage/pi_root

Examples

Simple Segmentation Fault Example

(From [7])

Example program segfault.c: 
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
  char *buf;

  buf = malloc(1<<31);

  fgets(buf, 1024, stdin);
  printf("%s\n", buf);

  return 1;
}
Compile and launch gdb: 
gcc -g segfault.c
gdb a.out

The debug session 

run
backtrace
frame 3
print buf
kill
break segfault.c:8
run
print buf
next
print buf

Fix the bug, then start again, watching now buf: 

watch buf
# Start again, answer 'y' when asked to start from beginning
run
# Break at watch point, let's _c_ontinue
c

More GDB stuff

display i            # Display i at each stop
display/4i $pc       # Display the 4 next instruction at each stop
u                    # Continue until we reach an higher pc address

Using gdb-dashboard

da                   # Print dashboard again
da reg               # Remove register
da reg               # Add it back
da m w 0x1000 0x10   # Watch memory zone at 0x1000 for 16 bytes
da m w 0x2000 0x10   # Add a second zone at 0x2000

Frequently used commands

info reg
x/i $pc

GDB script

Breakpoint command lists

See help commands (or here)

Example script: 

break foo if x>0
commands
silent
printf "x is %d\n",x
cont
end
  • Use silent to make the breakpoint silent.
  • Use condition (here if x>0) on breakpoint to break conditionally.

Python script

Tbc

Retrieved from "https://miki.immie.org/mediawiki/index.php?title=Gdb&oldid=11209"