Recently I came back to look into coreboot. Mainly because low level is fun and skills related to firmware (even coreboot) starting get attention on freelance portals (first odesk job, second odesk job). I was surprised that under the wings of Google coreboot team start to support ARM (BTW ARM programming is IMHO next great skill to learn). So I cloned latest, code compiled QEMU armv7 mainboard model and tried to kick it in latest qemu-system-arm. Unfortunately it didn’t boot. Below you can find my TL;DR debugging story.

coreboot qemu-armv7 mainboard compilation – very quick steps

Set: Mainboard -> Mainboard model -> QEMU armv7 (vexpress-a9)

NOTE: To prevent annoying warning about XML when running gdb from coreboot crossgcc utilities:

libexpat1-dev should be installed.

buildgcc will provide armv7 toolchain with debugger (-G) and compilation will use 8 parallel jobs.

qemu-system-arm compilation – very quick steps

Debugging hint

Use good gdbinit, so with every instruction executed gdb will automatically provide most useful informations. IMHO good choice is fG! gdbinit shared on github. It contain support for ARM and x86. To switch to ARM mode inside gdb simple use arm command. Output looks pretty awesome:

gdbinit

Noob dead end

Command for running qemu that I found in early qemu-armv7 commit log:

It ends with qemu error:

At the beginning I thought that it is a mistake so I tried:

What ends with:

Obviously qemu complains on value in R15 (PC – Program Counter), which is the address of current instruction (like EIP in x86).

Stepping through assembler instructions using cross-compiled debugger (util/crossgcc/xgcc/bin/armv7-a-eabi-gdb) points to:

ldmia will load from stack values of all given registers. This cause that PC goes to 0x0 and then run instruction from zeroed memory, which in ARM instructions means:

It happens till PC reach 0x4000000 which is out of ‘RAM or ROM’ for qemu. Unfortunately there is no sign about ldmia instruction with above range of registers in coreboot and qemu code.

Bisection

I knew that at some point qemu worked with coreboot. I tried few versions and it leads me to some commit between v2.1.0-rc1 and v2.1.0-rc0. For -kernel switch I was able to narrow down problem to one commit that change VE_NORFLASHALIAS option for vexpress-a9 to 0 (6ec1588). It looks like for vexpress-a9 qemu place kernel at 0x60000000 (vexpress.highmem), which is aliased to range 0x0-0x3ffffff. VE_NORFLASHALIAS=0 cause mapping of vexpress.flash0 to the same region as kernel and because flash (-bios) was not added we have empty space (all zeros) what gives andeq r0, r0, r0.

Right now I have working version of coreboot but only with -kernel and VE_NORFLASHALIAS=-1 set in hw/arm/vexpress.c. The main questions are:

  • what is the correct memory map for qemu-armv7 and how coreboot should be mapped ?
  • what’s going on with coreboot or qemu that I can’t go through bootblock ?

Debugging

I tried to debug coreboot executed from flash:

Coreboot as UEFI has few phases. For UEFI we distinguish SEC, PEI, DXE and BDS (there are also TSL, RT and AL, but not important for this considerations). On coreboot side we have bootblock, romstage, ramstage and payload.

qemu-armv7 bootblock failure

qemu-armv7 booting procedure start from _rom section which contain hardcoded jump to reset procedure. After that go through few methods like on below flow:

At the end of dcache_foreach we experience failure because ldmia instruction tries to restore registers from stack, which should be stored at the beginning of dcache_foreach, by:

Unfortunately for some reason stack doesn’t contain any reasonable values (all 0xffffffff) after stmdb. Why is that ?

Obvious things are not so obvious

As I point above everything seems to be related with memory map for vexpress-a9. I wrote question to qemu developers mailing list describing all the problems. You can read it here. So the answer is that ARM Versatile Express boards in general have two different memory maps. First is legacy with RAM in low memory and second is modern with flash in low memory instead of RAM. Since qemu v2.1.0 modern memory map was used. That’s why I saw change in behavior. Obviously flash in qemu is read only, so no matter what pushing on stack didn’t work.

coreboot stack location fix

I though that fix would be easy. One thing that I have to do is change stack address. The question is where to place the stack ? So I took a look at qemu memory map:

SRAM is temporary storage where I decide to put stack. The change in coreboot looks like below:

`c src/mainboard/emulation/qemu-armv7/Kconfig config STACK_TOP hex default 0x4803ff00

config STACK_BOTTOM hex default 0x48000000

config STACK_SIZE hex default 0x0003ff00

It was not the end because another error popped up:

memcpy during CBFS decompression

Problem was with storing registers stmia during memcpy. Backtrace:

For some reason R0 (to which we store), contain strange address 0x10000. No value was stored in this memory range, because again it was read only flash. Address is passed from upper layers – cbfs_get_file_content. During debugging I realize that this address means ROMSTAGE_BASE. So I changed ROMSTAGE_BASE to somewhere in SRAM.

c src/mainboard/emulation/qemu-armv7/Kconfig config ROMSTAGE_BASE hex default 0x48040000

What I saw when trying to boot coreboot with this fix was wonderful log proved that coreboot boots without problems.

Conclusion

Above debugging session was all about memory map. It was really fun to experience all those issues because I had to understand lot of ARM assembly instructions, track memory, read the spec, read coreboot and qemu code. It gave me a lot of good experience. If you have any questions or comments please let me know. And finally what is most important it was next thing done on my list.

I think next challenge could be experiment with Linux kernel booting. Coreboot can boot kernel directly or through payload with bootloader.

Thanks for reading.