Archive for May, 2014

How to: kgdb + QEMU + FreeBSD 10 kernel debugging

First boot up the Freebsd QEMU guest image as per normal:

qemu-system-x86_64 -net user -net nic -m 2048 -hda freebsd10.img -boot c -cdrom FreeBSD-10.0-RELEASE-amd64-dvd1.iso

And copy out the kernel and kernel.symbols file from /boot/kernel directory to the host machine.

Then shut the guest image down, and reboot with QEMU serial debugging enabled:

qemu-system-x86_64 -s -S -net user -net nic -m 2048 -hda freebsd10.img -boot c -cdrom FreeBSD-10.0-RELEASE-amd64-dvd1.iso

When QEMU is "stopped" as displayed, open a separate terminal and use "gdb" in the host machine to connect to the guest:

gdb kernel

where the "kernel" above is the Freebsd kernel file copy out as mentioned earlier. Now followed by two statement below:

target remote localhost:1234

cont

After the guest image has booted up, go back to the gdb terminal and enter "ctrl-C", and there you can set breakpoints.

For example, enter:

rbreak vm_pageout*

And using "bt" for displaying the backtrace after the breakpoint is reached, you can learn a lot of the APIs implementations internals:

For example:

Breakpoint 43, vm_page_requeue_locked (m=0x7e1ed138)
at /usr/src/sys/vm/vm_page.c:2124
2124 /usr/src/sys/vm/vm_page.c: No such file or directory.
(gdb) bt
#0 vm_page_requeue_locked (m=0x7e1ed138) at /usr/src/sys/vm/vm_page.c:2124
#1 0xffffffff80b2187b in vm_pageout_scan (vmd=<optimized out>,
pass=<optimized out>) at /usr/src/sys/vm/vm_pageout.c:1384
#2 vm_pageout_worker (arg=<optimized out>)
at /usr/src/sys/vm/vm_pageout.c:1625
#3 vm_pageout () at /usr/src/sys/vm/vm_pageout.c:1696
#4 0xffffffff8088198a in fork_exit (callout=0x6178490, arg=0x0, frame=0x4)
at /usr/src/sys/kern/kern_fork.c:995
#5 0xffffffff80c758ce in fork_trampoline ()
at /usr/src/sys/amd64/amd64/exception.S:606
#6 0x0000000000000000 in ?? ()
(gdb)

Breakpoint 67, vm_page_insert_radixdone (m=0x7d373f98, object=0x6563c00,
mpred=0x0) at /usr/src/sys/vm/vm_page.c:1015
1015 in /usr/src/sys/vm/vm_page.c
(gdb) bt
#0 vm_page_insert_radixdone (m=0x7d373f98, object=0x6563c00, mpred=0x0)
at /usr/src/sys/vm/vm_page.c:1015
#1 0xffffffff80b1d251 in vm_page_insert_after (object=<optimized out>,
mpred=<optimized out>, m=<optimized out>, object=<optimized out>,
pindex=<optimized out>, mpred=<optimized out>)
at /usr/src/sys/vm/vm_page.c:998
#2 vm_page_alloc (object=0x6563c00, pindex=0, req=<optimized out>)
at /usr/src/sys/vm/vm_page.c:1611
#3 0xffffffff80b0ac13 in vm_fault_hold (map=0x2000000, vaddr=0,
fault_type=2 '02', fault_flags=0, m_hold=0x0)
at /usr/src/sys/vm/vm_fault.c:432
#4 0xffffffff80b0a917 in vm_fault (map=0x0, vaddr=<optimized out>,
fault_type=0 '00', fault_flags=0) at /usr/src/sys/vm/vm_fault.c:224
#5 0xffffffff80c8e83b in trap_pfault (frame=0x937aa970, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:775
#6 0xffffffff80c8e0f6 in trap (frame=0x0)
at /usr/src/sys/amd64/amd64/trap.c:463
#7 0xffffffff80c75392 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:232
#8 0xfffffe0093800000 in ?? ()
#9 0x0000000801808021 in ?? ()
#10 0x0000000000000400 in ?? ()
#11 0xfffffe00937aacc0 in ?? ()
---Type <return> to continue, or q <return> to quit---
#12 0x0000000000000400 in ?? ()
#13 0xfffffe00937aaa50 in ?? ()
#14 0x00007ff7fe7f7f2f in ?? ()
#15 0x0000000000000000 in ?? ()

#0 vm_page_alloc (object=0x6563c00, pindex=0, req=64)
at /usr/src/sys/vm/vm_page.c:1451
#1 0xffffffff80b0ac13 in vm_fault_hold (map=0x2000000,
vaddr=18446741877160935424, fault_type=2 '02', fault_flags=0, m_hold=0x0)
at /usr/src/sys/vm/vm_fault.c:432
#2 0xffffffff80b0a917 in vm_fault (map=0x93800000, vaddr=<optimized out>,
fault_type=32 ' ', fault_flags=0) at /usr/src/sys/vm/vm_fault.c:224
#3 0xffffffff80c8e83b in trap_pfault (frame=0x937aa970, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:775
#4 0xffffffff80c8e0f6 in trap (frame=0x2)
at /usr/src/sys/amd64/amd64/trap.c:463
#5 0xffffffff80c75392 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:232
#6 0xfffffe0093800000 in ?? ()
#7 0x0000000801808021 in ?? ()
#8 0x0000000000000400 in ?? ()
#9 0xfffffe00937aacc0 in ?? ()
#10 0x0000000000000400 in ?? ()
#11 0xfffffe00937aaa50 in ?? ()
#12 0x00007ff7fe7f7f2f in ?? ()
#13 0x0000000000000000 in ?? ()

#0 vm_page_activate (m=<optimized out>, m=<optimized out>)
at /usr/src/sys/vm/vm_page.c:2146
#1 0xffffffff80b0c232 in vm_fault_hold (map=0x2000000, vaddr=<optimized out>,
fault_type=2 '02', fault_flags=<optimized out>, m_hold=0x0)
at /usr/src/sys/vm/vm_fault.c:922
#2 0xffffffff80b0a917 in vm_fault (map=0x93800000, vaddr=<optimized out>,
fault_type=32 ' ', fault_flags=0) at /usr/src/sys/vm/vm_fault.c:224
#3 0xffffffff80c8e83b in trap_pfault (frame=0x937aa970, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:775
#4 0xffffffff80c8e0f6 in trap (frame=0x2)
at /usr/src/sys/amd64/amd64/trap.c:463
#5 0xffffffff80c75392 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:232
#6 0xfffffe0093800000 in ?? ()
#7 0x0000000801808021 in ?? ()
#8 0x0000000000000400 in ?? ()

%d bloggers like this: