Debugging by using QEMU trace-events

Hello folks! In this post, I am going to talk a bit about QEMU trace-events. I found this mechanism during the development of a virtio driver. Roughly speaking, trace-events are generated by different components of QEMU like the virtio queues, the ioapic, lapics, etc. To enable it, you have to compile QEMU with the following option:

--enable-trace-backends=simple

Then, you have to add the following line to command-line:

-trace events=./events

The file named events contains a list of events that we are interesting to observe. For example, this is mine:

apic_*

ioapic_*

virtio_*

In my case, I am interested in checking if irqs are correctly acknowledged. To see this, I get all the events related with apic, ioapic and virtio. To output the logs in a file, I have to get QEMU monitor and first do 'trace-file on' and second 'trace-file flush'. I am not sure why this is not automatically done. You end up getting a file named 'trace-PID' in which PID is the corresponding PID of the QEMU process. To read this file, you have just to run the following python script:

python3 ~/qemulast/scripts/simpletrace.py ~/qemulast/build/trace-events-all trace-30572   

You will get something like:

virtio_mmio_read 131.447 pid=2451 offset=0x60

virtio_mmio_write_offset 141.046 pid=2451 offset=0x64 value=0x1

virtio_mmio_setting_irq 8.345 pid=2451 level=0x0

ioapic_set_irq 4.359 pid=2451 vector=0xc level=0x0

ioapic_eoi_broadcast 29.005 pid=2451 vector=0x2c

ioapic_clear_remote_irr 1.683 pid=2451 n=0xc vector=0x2c

In this example, we can see that when an IRQ is captured, the handler reads the status register and writes it to ack the irq. Then, the virtio sets the irq level at 0x0. The handler ends up by sending the EOI to the LAPIC. You can find more information about trace-events at: 

https://git.qemu.org/?p=qemu.git;a=blob_plain;f=docs/devel/tracing.txt;hb=HEAD.

http://blog.vmsplice.net/2011/02/observability-using-qemu-tracing.html

http://blog.vmsplice.net/2011/03/how-to-write-trace-analysis-scripts-for.html