Debugging TORO with BOCHS and GDB

In the following lines I'll tray to explain how debug Toro using bochs+gdb. I often used qemu instance of bochs but the latest days I figured out some rare behaviors so I replaced qemu for bochs with good results. 

Before to start we have to compile bochs for such purpose, after download the source run the next command in order to compile it:

> ./configure --enable-cpu-level=6 --enable-all-optimizations --enable-x86-64 --enable-pci --enable-vmx --enable-disasm --enable-debugger-gui --enable-logging --enable-fpu --enable-3dnow --enable-sb16=dummy --enable-cdrom --enable-x86-debugger --enable-iodebug --disable-plugins --disable-docbook --enable-gdb-stub     

> make install

It is not possible to enable the smp support and gdb stub at the same time. If the compilation was right we'll be able to run bochs with gdb support.

Now It is needed to compile the toro application within the debug symbols, if there're any old .o or .ppu file they must be deleted because they don't have symbols information. We should execute in the toro source directory the next commands:

> fpc ToroHello.pas -g -oToroHello -Fu../rtl/ -Fu../rtl/drivers

> ./build 4 ToroHello boot.o ToroHello.img

This procedure is for ToroHello.pas but it's the same for other toro's app.

So far, we've the bochs and the Toro's image, now we have to build the .bochsrc file and launch bochs.  The following lines may be useful:

megs: 32

ata0-master: type=disk, path="ToroHello.img"

boot: c

log: bochsout.txt

gdbstub: enabled=1

Check that we've indeed to enable the gdbstub in the bochs' source file.

The next step is to run bochs and then GDB:

> bochs

> gdb ToroHello

If we run gdb in the toro/test directory .gdbinit will be used, otherwise we have to connect to bochs manually as follow:

> (gdb) target remote localhost:1234 

If everything goes well we are able to set breakpoints and uses all the tools of gdb. For instance we could do:

> (gdb) b KERNELSTART

> (gdb) c

In the first line a breakpoint is set at KERNELSTART and then the virtual machine continues until comes back when the breakpoint is reached.

Many commands could be usefull in this point like n, for running line by line, step for stepping into, info registers and soon on.

There're a lot of information that we can get at this point but that's for another tutorial ;)

Matias E. Vara

www.torokernel.org

Git workflow for Toro

"Git workflow" is very known subject, It is possible find thousands documents talking about and with enough information that it makes unnecessary write other tutorial. Anyway, when I have to do any change in Toro's source, I do not follow any patter and I just make the changes and commit them. 

In order to make cleaner this procedure I decide to write a kind of  "git protocol" to be follow every time that I make a change on Toro.

I figured out how important is to be clear when we change the source, moreover if the project is huge and every little modification can become it unstable. It helps for understand the design and also for finding faster a possible bug. 

For implementing a new feature or fix a bug, we should start creating a new branch. It will keep our job independent of the master and we can mistake without care.

> git checkout -b new_branch

This will create a local branch and it will switch to it.

Makes the changes and then commits when you made enough changes, try to avoid very short commit that they aren't adding important information. 

> git commit -a 

Note about commit message:

"- Write in present

 - First line of the description is a short summary of the change:

      - If I’m fixing a bug, a small example of the bug I fixed or issue I was changing.

      - If I’m changing a feature, or adding a new one, why I think it the right choice and why I think it’s  

 acceptable or make that sort of change in the given part of the product cycle we are in."

For sharing your job before finish it, push the branch to the remote repo.

> git push -u origin new_branch

Then, in order to show all the branches either local or remote, first execute:
> git branch -a


And after that, to perform a local checkout run:
> git checkout -b new_branch origin/new_branch

It could be useful write a test in toro/tests/ in order to check the new feature and the stability of the kernel.

When you are sure about the changes, merge to master:

> git checkout master

> git merge --no-ff new_branch

A comment about --no-ff flags extracted from here:

"The --no-ff flag causes the merge to always create a new commit object, even if the merge could be performed with a fast-forward. This avoids losing information about the historical existence of a feature branch and groups together all commits that together added the feature"

Finally, we should remove the remote and local branch:

> git push origin :new_branch

> git branch -d new_branch


Matias E. Vara
www.torokernel.org

x86 rings protection on Toro

In x86 arch there are 4 ring levels. Ring 0 is the most privileged level and ring 3 is the least. In a OS the kernel runs in ring 0 and the user application runs in ring 3.

Ring 0 descriptors are used by the kernel and ring 3 descriptors are used by the user. The GDT supports up to 8192 descriptors but the OS just uses 4, two for kernel´s text and data, and two for user´s text and data. With these descriptors the kernel can access to all memory, for example 4GB in 32 bits.

When the OS uses privileged levels the processor has to check if every operation is valid, these mechanisms adds latencies. In a multitasking OS protection is essential and it protects the kernel code and data.

But what happens with dedicated multithread application? It runs alone in the system and it was written carefully, we need protection in this case? guessing that, we can reduce a lot the OS.
For example, if we want to implement syscalls, we don´t need traps. If the kernel and user application are in same ring, we just may use “call” instruction to kernel´s function. Actually, OS are using interruptions for support syscalls but they are too expensive. Don´t forget that we are jumping from ring 0 to ring 3.

In other way, when we are running user application and a interruption happens, the processor has to jump from ring 3 to ring 0, that is too expensive. In general cases a kernel procedure handles the interruption. If the user application runs in the same level that the kernel, we don´t spend time in latencies.

On TORO, the kernel and user application run in ring 0. And cause the kernel and app are compiling together, the syscalls are implemented easy. It just uses “call” instruction.

Matias E. Vara

Threads migration without Lock in Toro

In a Multicore environment, the programmer needs to create local and remote threads. In TORO create a remote threads is easy, you just have to use BeginThread() with the appropriate CPU identification. On that basis, there are two important procedures in TORO:

- Thread Emigrating: is when the threads are created in a remote processor.
- Thread Inmigrating: is when the guest processor enque in its scheduler the threads that they are comming from others processors.

This is the unique kernel point which needs syncronization between the cores. The mechanism is called "Exchange Slot" and it works without any atomic operation. In this case it used for send and receiv threads but it works with any kind of data.

For every processor in TORO there is an structure called TSchedulerExchangeSlot:

TSchedulerExchangeSlot = record
DispatcherArray: array[0..MAX_CPU-1] of PThread;
EmigrateArray: array[0..MAX_CPU-1] of PThread;
end;

Where MAX_CPU is the number of processors and PThread is a pointer to TThread structure. From the structure declaration we can see that every processor has two arrays

(DispatcherArray y EmigrateArray), and every entry in the array is a pointer to a thread´s queu.

The procedure to send threads to remote processor has three stages:
1-The user calls to BeginThread()for create a new one, if the parameter CPUID is different to local CPU then the kernel enque it to DispatcherArray[CPUID].
2-During Scheduling (cause SysThreadSwitch syscall). The procedure Emigrating()moves all threads from DispatcherArray[] to EmigrateArray[] (only if EmigrateArray[] is nil)
3-During Scheduling of the Remote CPU, the procedure Inmigrating() look for a not nil entry in EmigrateArray[LocalCPUID] in every TSchedulerExchangeSlot processor structure. If it is not nil Then import all the threads to local scheduler and become EmigrateArray[LocalCpuid] to nil.
Local processor just writes and read to DispatcherArray[]. While the local and remote processor write and read to EmigrateArray[], but the access is synchronized using nil pointer.
The “Exchange Slot” doesn´t need "LOCK" instruction.


The Inmigrating and Emigrating procedures are called from the Scheduler. The scheduler makes a few system task, for example in the picture, we can see the scheduler´s flow diagram. There, first it calls Inmigrating(), after that it calls Emigrating() and At the end a new thread is scheduling.


Matias E. Vara
www.torokernel.org