KVM on ARM Cortex A15 (OMAP5432 UEVM)

Hi! In this post I'll summarize the steps I needed to do in order to get KVM working on the OMAP5 ARM board using the virtualization extensions.

ARM A15 HYP mode.

In Cortex-A15, ARM have introduced a new operating mode called HYP (hypervisor). It has lower permissions than TruztZone. In fact, HYP splits the "insecure" world into two parts, one for hypervisor and the other one for the guests. By default on most boards the system boots into the insecure non-HYP mode. To enter the HYP mode, one needs to use platform-specific ways. For OMAP5 this involves making a call to the TrustZone which will restart the insecure mode cores.

A good overview of how virtualization support for ARM was added to Linux is available at LWN.
http://lwn.net/Articles/557132/

Ingo Molnar HYP patch

There was a patch for u-boot to enable entering HYP mode on OMAP5 by Ingo Molnar. Too bad, it was either written for an early revision of omap5 or poorly tested. It did not work for my board, so I had to learn about OMAP5 TrustZone SCM commands from various sources and put up a patch (which is integrated to my u-boot branch).
If you're interested, you can take a look at the corresponding mailing list entry.

http://u-boot.10912.n7.nabble.com/RFD-OMAP5-Working-HYP-mode-td163302.html

Preparing u-boot SD card

Get the android images from TI or build them yourself. You can use the usbboot tool to boot images from the PC. Or even better, you can build u-boot (this is the preferred way) and then you won't need android images. But you may need the TI GLSDK for the x-loader (MLO). Creating an SD card with u-boot is the same as for omap3 and omap4, so I'll leave this out. There is some magic with creating a proper partition table, so I advise that you get some prebuilt image (like ubuntu for pandaboard) and then replace the files in the FAT partition.

http://software-dl.ti.com/omap/omap5/omap5_public_sw/OMAP5432-EVM/5AJ_1_5_Release/index_FDS.html
http://www.omappedia.com/wiki/6AJ.1.1_Release_Notes

Please consult the OMAP5432 manual on how to set up the DIP switches to boot from SD card.

Source code

For u-boot:
https://github.com/astarasikov/uboot-tegra/tree/omap5_hyp_test

For linux kernel:
https://github.com/astarasikov/linux/tree/omap5_kvm_hacks

Linux kernel is based on the TI omapzoom 3.8-y branch. I fixed a null pointer in the DWC USB3 driver and some issues with the 64-bit DMA bitmasks (I hacked the drivers to work with ARM LPAE, but this probably broke them for anything else. The upstream has not yet decided on how this should be handled).

Compiling stuff

First, let's build the u-boot
#!/bin/bash
export PATH=/home/alexander/handhelds/armv6/codesourcery/bin:$PATH
export ARCH=arm
export CROSS_COMPILE=arm-none-eabi-
U_BOARD=omap5_uevm
make clean
make distclean
make ${U_BOARD}_config
make -j8

you'll get the u-boot.bin and the u-boot.img (which can be put to the SD card). Besides, that will build the mkimage tool that we'll need later.

Now, we need to create the boot script for u-boot that will load the kernel and the device tree file to RAM.

i2c mw 0x48 0xd9 0x15
i2c mw 0x48 0xd4 0x05
setenv fdt_high 0xffffffff
fdt addr 0x80F80000
mmc rescan
mmc part
fatload mmc 0:1 0x80300000 uImage
fatload mmc 0:1 ${fdtaddr} omap5-uevm.dtb
setenv mmcargs setenv bootargs console=ttyO2,115200n8 root=/dev/sda1 rw rootdelay=5 earlyprintk nosmp
printenv
run mmcargs
bootm 0x80300000 - ${fdtaddr} 

Now, compile it to the u-boot binary format:
./tools/mkimage -A arm -T script -C none -n "omap5 boot.scr" -d boot.txt boot.scr

Building linux:

export PATH=/home/alexander/handhelds/armv6/linaro-2012q2/bin:$PATH
export ARCH=arm
export CROSS_COMPILE=/home/alexander/handhelds/armv6/linaro-2012q2/bin/arm-none-eabi-
export OMAP_ROOT=/home/alexander/handhelds/omap
export MAKE_OPTS="-j4 ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE"

pushd .
cd ${OMAP_ROOT}/kernel_omapzoom
make $MAKE_OPTS omap5uevm_defconfig
make $MAKE_OPTS zImage
popd

Now, we need to compile the DTS (device tree source code) using the dtc tool. If you choose to use the usbboot instead of u-boot, you can enable the config option in kernel and simply append the DTB blob to the end of zImage.
(Boot Options -> Use appended device tree blob to zImage)

./scripts/dtc/dtc arch/arm/boot/dts/omap5-uevm.dts -o omap5-uevm.dtb -O dtb
cat kernel_omapzoom/arch/arm/boot/zImage omap5-uevm.dtb > kernel
./usbboot -f; fastboot -c "console=ttyO2 console=tty0 rootwait root=/dev/sda1" -b 0x83000000 boot kernel

VirtualOpen

For userspace part, I've followed the manual from VirtualOpenSystems for versatile express. The only tricky part was building qemu for the ArchLinux ARM host, and the guest binaries are available for download.
http://www.virtualopensystems.com/media/kvm-resources/kvm-arm-guide.pdf


P.S. please share your stories on how you're using or plan to use virtualization on ARM

Read More

An update on OSX Touchscreen driver

After playing with the HID interface in OS X, I have found out there exists an API for simulating input events from user space, so I've implemented the touchscreen driver using it.

One unexpected caveat was that you need to set up an increasing event number to be able to click the menu bar. Even worse, when launched from XCode, the app would hang if you clicked the XCode menu bar. If you click any other app or launch it outside XCode, everything's fine. Caused some pain while debugging.

Now I still want to implement the driver as a kernel module. On the other hand, the userspace version is also fine. I want to add the support for multitouch gestures, though I'm not sure it is possible to provide multiple pointers to applications.

<iframe width="560" height="315" src="//www.youtube.com/embed/ccMOPNel-2k" frameborder="0" allowfullscreen></iframe>

The code is at github https://github.com/astarasikov/osxhidtouch

You can get the binary OSX touchscreen driver for the Dell S2340T screen from the link below. Basically, you can replace the HidTouch binary with your own, and if you put it to "HidTouch.app/Contents/MacOS/HidTouch", you will get an app bundle that can be installed to Applications and added to the auto-start.
https://drive.google.com/file/d/0B7wcN-tOkdeRRmdYQmhJSWsta1U/edit?usp=sharing

Read More

Multitouch touchscreen support in OS X

Hi there!

I happen to have a Dell S2340T multitouch monitor (quite an expensive toy btw) which has a touch controller from 3M. It works fine in Windows (which I don't have any plans to use), sort of works in linux (which is my primary work environment) and does not work at all in OS X (not a big deal but it would be cool to have it).

So I set out on the search for a driver and have figured out the following:

• There is some old driver from TouchBase that sucks. I mean, it's using some stinky installer that pollutes the OS X root file system, it has the UI from 90s. More than that, it's not signed and does not support my touchscreen. Even adding the USB Vendor ID to its Info.plist did not fix a thing
• There is some new trial driver from TouchBase that should work for most devices but is limited to 100 touches (a demo version). Well, I didn't want to grovel before them and sign up at their stupid site. And still, even if I patched the driver to remove the trial limitations, I would have to sign it with my certificate and there would be no legal way for me to distribute it on the internetz
• There is a full version of the TouchBase driver that costs $100. Are they fucking crazy? On the other hand, I would do the same. See, home users don't care, multitouch displays are very rare, and people are used to pirating sofrware. But one could raise quite some money selling the driver to the workshops that build customized Apple computers (like ModBook) or car PCs.
• There are some drivers for other touchscreens, but they're for the old single-touch devices

Doesn't look promising. Now, one may wonder "WTF ain't it working out of the box? It's a HID device, should work everywhere". Well, there are two problems:

• Multitouch devices have the new HID event type (called HID usages) for the Digitizer class which the OS X does not support in IOKit.  Actually, since it uses a new HID class (different to single-touch monitors), it is not even classified as a touchscreen by the OS (rather as a multitouch trackpad)
• The touchscreen gets recognized by the generic HID driver as a mouse. And here comes the problem - when a finger is released, the device reports the pointer to be at the origin (point <0, 0>) which causes the cursor to be stuck at the top left corner

I decided to learn about the HID system in OS X and write a driver. I started with the sample code from the HID documentation and added some event debugging. I found out that the device reports the events both as a digitizer and as a pointing device. So for now I prepared a demo app in Qt5 that can recognize multiple touches and draw points in different colors when the screen is touched. Have a look at it in action:

The source code is at https://github.com/astarasikov/osx-multitouch-demo .

Well, looks like I should figure out how to turn this into a device driver. I will probably need to figure out how to enumerate HID devices based on their class and make a kext file (Kernel EXTension) for XNU (the kernel used in OS X) and then.. sell my driver for some $20, right?

Read More

Using a hybrid graphics laptop on linux

Introduction.

I have a laptop with so-called hybrid graphics. That is, it has two GPUs - one of them is part of the SoC (Intel HD3000 GPU), the other one is the "discrete" PCIe Radeon HD 6630M from AMD. Typicaly, older models of dual-GPU laptops (and new Apple Macbook Pro machines) have a multiplexer that switches the output from the GPU to display. As the majority of modern laptops, my features a "muxless" combination of the GPUs - that is, the more powerful GPU does not have any physical connection to the display panel. Instead, it can write to the framebuffer memory of the less powerful GPU (or it can perform only the computational part of OpenGL/DirectX and let the primary GPU handle 2D drawing and blitting).

Linux support for the hybrid graphics if far from perfect. To be honest, it just sucks even now that such kind of laptop have been dominating the market for some 4-5 years already. With the recent advent of the "DRI-PRIME" interface the linux kernel and userspace now supports using the secondary GPU for offloading intensive graphic workloads. Right now to utilize the separate GPU, an application has to be launched with a special environment variable (namely, "DRI_PRIME=1") and the system is supposed to dynamically power-up the GPU when it is needed and turn it off to reduce power consumption and heating at other times. Unfortunately, the support for power management is still deeply broken. In this post I summarize my findings and scripts which allow to turn off the external GPU permanently to save power and increase the battery life of the laptop. I am not using the secondary GPU because OpenGL and X11 drivers for the Intel GPU are more stable, and the open-source Radeon driver/mesa does not yet support OpenCL (the computational API for GPUs).

vga_switcheroo

The kernel interface for controlling the power for the external GPU is called "vga_switcheroo" and it allows to power-down the GPU. Unfortunately, I have found out that my laptop (and most others) enable the GPU after a suspend-resume cycle. I think this behaviour is intentional, because ACPI calls should be used to control the power of the GPU. However, it confuses the vga_switcheroo which thinks the card is still powered down. Meanwhile, the card drains some 10-30 watts of power, effectively reducing the laptop battery life from 7 hours to 2 hours or so.

My first stab at this problem was a patch that forced the vga_switcheroo to disable all the cards that were not held by the userspace at resume. That did solve the problem for a while, but was hackish and never made it into the mainline kernel. However, it is still useful for linux kernels up to 3.9.
http://lkml.indiana.edu/hypermail/linux/kernel/1204.3/02530.html

kernel 3.10+

As of linux version 3.10, several changes were made in regards to the hybrid graphics power management. First is the introduction of the dynamic power management (in the form of clock control code and parsing the predefined power states provided by the OEM BIOS) for the Radeon chips. Second one is the change to the vga_switcheroo which allowed it to power down the card when unused without the user interaction. It does work to some extent, but the problem with the card being powered on after a sleep-resume cycle remains.

The problem is that now, when I manually disable the card via the vga_switcheroo, the PCI device is gone - it is removed and never appears again. The same behaviour could be exhibited on pre-3.10 kernels if one did issue the "remove" command to the DRM node (/sys/class/drm/card1/). Besides, my hack to the vga_switcheroo stopped working since these upgrades. Now, this did not make me happy and I set out to figure out the solution.

acpi_call

Linux powers off the radeon GPU using the PCIe bridge and GPU PCIe registers. A "portable" and "official" way to control the power of a GPU is using an ACPI call. ACPI is an interface for the Intel-X86 based computers (although now also implemented for the ARM CPUs in an attempt to provide the support for UEFI, Windows RT and an unified linux kernel binary capable of running on any ARM SoC) intended to provide abstraction of hardware enumeration and power management. It contains tables with lists of PCI and other PNP (plug-and-play) peripherals which can be used by the OS kernel instead of the unsafe "probing" mechanism. Moreover, it contains a specification for the interpreted bytecode language. Some methods are implemented by the OEM inside the BIOS ACPI tables to perform certain functions - query battery status, power up- and down the devices etc.

Folks have since long figured out to use ACPI calls in linux to control the power of the discrete GPU. Although it could interfere with the vga_switcheroo interface, in case we either completely disable the external GPU or power it off via vga_switcheroo first, we're safe to use it. Moreover, I have found out that I can use an ACPI call to power on the device and make it visible to the system after it is removed as described in the previous paragraph!

There exist a module for the linux kernel that allows to perform arbitrary ACPI calls from the userspace. Turns out that a direct ACPI call can even work around the new vga_switcheroo. There's a good guide on installing the acpi_call module into the DKMS subsystem so that it is automatically packaged and built any time you upgrade the linux kernel on your machine.

http://garyservin.wordpress.com/2012/01/06/disabling-discrete-gpu-in-debian-gnulinux-wheezy/

The module contains the turn_off_gpu.sh script in the examples folder which can be used to power down the GPU. I ran it and took a notice of the method used for my laptop - it was the "\_SB.PCI0.PEG0.PEGP._OFF" (which means South Bridge -> PCI controller 0 -> Pci Express Graphics -> Pci Express Graphics Port).

Now, I did the following magical trick:
echo "\_SB.PCI0.PEG0.PEGP._ON" > /proc/acpi/call
And BANG! after being gone when turned off via the vga_switcheroo, the GPU was identified by linux and enabled again. Neato.

Putting it all together.
Now the real showstopper is that the card still drains power after resume. I figured I had to write a script which would power down the card. It turned out that sometimes if the laptop was woken up and then it immediately went to sleep due to some race condition, systemd did not execute the resume hook.

Instead, I used the udev. Udev is a daemon that listens for the events that the kernel sends when hardware state changes. Each time the laptop wakes up, the power supply and battery drivers are reinitialized. Besides, the primary GPU also sends wakeup events. Another two observations are that I'm not using the discrete GPU and it is safe to call the ACPI "OFF" method multiple times. So, it is not a problem if we call the script too many times. I decided to just call it on any event from the two aforementioned subsystems. Here is how it can be done (note that you need to have root permissions to edit the files in /etc. If you are a newcomer to linux, you can become root by issuing a "sudo su" command):

Create the file "/etc/udev/rules.d/10-battery-hook.rules" with the following contents:
ACTION=="change", SUBSYSTEM=="power_supply", RUN+="/etc/gpu_poweroff.sh"
ACTION=="change", SUBSYSTEM=="drm", RUN+="/etc/gpu_poweroff.sh"

Now, create the /etc/gpu_poweroff.sh script with the contents mentioned below. You can uncomment the "echo" calls to debug the script and verify it is getting called. By the way, the power_profile part is not necessary but is an example of how to put a radeon GPU into a low-power state without disabling it.

gpu_poweroff.sh

#!/bin/bash

#this is a script for lowering the power consumption
#of a radeon GPUs in laptops. comment out whatever portion
#of it you don't need

#echo "gpu script @`date`" >> /tmp/foo

if [ -e /sys/class/drm/card0 ] ;then
for i in /sys/class/drm/card*; do
if [ -e $i/device/power_method ]; then
echo profile > $i/device/power_method
fi
if [ -e $i/device/power_profile ]; then
echo low > $i/device/power_profile
fi
done
fi

if [ -d /sys/kernel/debug/vgaswitcheroo ]; then
echo OFF > /sys/kernel/debug/vgaswitcheroo/switch 
fi

acpi_methods="
\_SB.PCI0.P0P1.VGA._OFF
\_SB.PCI0.P0P2.VGA._OFF
\_SB_.PCI0.OVGA.ATPX
\_SB_.PCI0.OVGA.XTPX
\_SB.PCI0.P0P3.PEGP._OFF
\_SB.PCI0.P0P2.PEGP._OFF
\_SB.PCI0.P0P1.PEGP._OFF
\_SB.PCI0.MXR0.MXM0._OFF
\_SB.PCI0.PEG1.GFX0._OFF
\_SB.PCI0.PEG0.GFX0.DOFF
\_SB.PCI0.PEG1.GFX0.DOFF
\_SB.PCI0.PEG0.PEGP._OFF
\_SB.PCI0.XVR0.Z01I.DGOF
\_SB.PCI0.PEGR.GFX0._OFF
\_SB.PCI0.PEG.VID._OFF
\_SB.PCI0.PEG0.VID._OFF
\_SB.PCI0.P0P2.DGPU._OFF
\_SB.PCI0.P0P4.DGPU.DOFF
\_SB.PCI0.IXVE.IGPU.DGOF
\_SB.PCI0.RP00.VGA._PS3
\_SB.PCI0.RP00.VGA.P3MO
\_SB.PCI0.GFX0.DSM._T_0
\_SB.PCI0.LPC.EC.PUBS._OFF
\_SB.PCI0.P0P2.NVID._OFF
\_SB.PCI0.P0P2.VGA.PX02
\_SB_.PCI0.PEGP.DGFX._OFF
\_SB_.PCI0.VGA.PX02
\_SB.PCI0.PEG0.PEGP.SGOF
\_SB.PCI0.AGP.VGA.PX02
"

# turn off the dGPU via an ACPI call
if [ -e /proc/acpi/call ]; then
for i in $acpi_methods; do
echo $i > /proc/acpi/call
done
#echo "turned gpu off @`date`" >> /tmp/foo
fi

exit 0

Make it executable by issuing a "chmod +x /etc/gpu_poweroff.sh" command.
Also, take a look at the "/etc/rc.local" script. If it does not exist, simply create it with the following contents and make it executable:

#!/bin/sh -e

bash /etc/gpu_poweroff.sh

exit 0

If it does exist, insert the call to the "/etc/gpu_poweroff.sh" before the "exit" line.

Bonus

To silence the fans on the Sony Vaio laptop, you can add the following to your rc.local script (I advise against putting it to the gpu_poweroff script because this command has a noticeable delay):

if [ -e /sys/devices/platform/sony-laptop/thermal_control ]; then
echo silent > /sys/devices/platform/sony-laptop/thermal_control
fi

Read More

Sleeping considered harmful

Intrabduction

Synchronization is a very controversial topic in computer science and software engineering. We're caught up in a vicious circle where everyone is told syncronization is difficult and therefore tries to stay away from the topic, and when one has to deal with threading, they prefer to stay away from reading about synchronization because reading is even more difficult.

One thing I have noticed while developing linux software for various devices is that drivers written by hardware vendors inevitably suck. Today I will tell you about one thing that freaks me out (sure there's a whole universe of things that freak me out but that's beyond the scope of this post).

Now, sometimes, when dealing with hardware, one has to wait (sleep) some time after writing to registers to let the device change its state. One prominent example are the drivers for the LED controllers. Yeah, your fancy new Android paperweight probably has a color LED to notify when your friends sign on to FaceBoob. LEDs can have various triggers, for example, one can set a LED to blink when memory is accessed, just like on old desktop boxes with HDDs. Somehow every time you try to set a trigger to a LED in Android kernel (as opposed to using the Android's proprietary ioctls), the device just freezes dead. What actually happens?

If we take a look at the LED class definition in the linux kernel, we'll see that the functions controlling the LED state (which are to be implemented by the hardware vendor) must not use sleeping primitives like msleep or usleep. Why is such a requirement posed? There are two reasons. One is that LEDs can be used in a context that does not allow sleeping - for example, inside the scheduler code to indicate the CPU load. Sleeping while scheduling could be handled by kernel developers, of course, but such an operation will cause time drifting and increase the complexity of the scheduler. Another reason is that LED control code may be called thousands of times per second, and any delay will cause the huge CPU load.

Anyway, even the performance critical code needs to be correct, therefore the need for the synchronization. The lightweight synchronization primitive used throughout the whole kernel is the spinlock. As you may guess from its name, it does not block the calling thread (like a semaphore or a mutex would do), but busy-waits till it can get into critical section. Polling (or busy-waiting) has always been discouraged because it consumes CPU time. However, it allows the algorithm to progress instead of blocking (therefore, most lock-free algorithms use memory barriers and busy waiting which is equivalent to using a spin lock in kernel). Either way, what you should need about spinlocks is that in most cases they are used to guard writing to an IO register which is a very fast operation, and while a spinlock is held, interrupts are disabled, therefore if you called sleep(), your thread would have no way to resume.

And yeah, most newbie driver developers don't know that some external busses (like I2C) require sleeping due to the nature of hardware. So, when one issues an I2C transfer inside the spinlock, they just want to see the kernel freeze.

Now that we've figured out what causes freezing, let's think of what we can do about it. On the one hand, if we have enough money, we can just assume customers will accept any crap we sell since they have no alternative. And that does work for Samsung, Qualcomm, Sony and many other vendors. On the other hand, we may want to fix it. We know we can't sleep in a lock, but we need a lock to keep data consistent. Linux has several solutions to the rescue and I'll give an overview of the two of them that are easy and suitable for a typical embedded/desktop system
(i.e., an SMP processor with a small number of cores).

Workqueues

First solution is using a workqueue. A workqueue is an abstraction (on top of kernel threads) which allows to schedule jobs (work) to be executed at some moment later. You can think of a workqueue as of a "future" or "async" concept used in userland.

I will use the example of a LED driver I have written for one smartphone (HTC Kovsky also known as Sony Ericsson Xperia X1) to illustrate the workqueue.

First, we need to include the relevant header. Don't trust me here, stuff changes all the time in linux, so you'd rather use grep to find the actual header.

1. #include <linux/workqueue.h>

We need to declare a "work" that we want to schedule. A work is essentially a routine which would be called, that accepts the instance of a "struct work_struct" class. We can use the "DECLARE_WORK" macro to define a work.

1. static DECLARE_WORK(colorled_wq, htckovsky_update_color_leds);

Now, let's look at how to implement the work function.

1. static void htckovsky_update_button_light(struct work_struct* work) {
2. char buffer[3] = {MICROP_KEYPAD_BRIGHTNESS_KOVS, 0, 0};
3. char brightness = kovsky_leds[BUTTONS].brightness;
4. if (brightness) {
5. buffer[1] = 0x94;
6. buffer[2] = brightness >> 2;
7. }
8. microp_ng_write(client, buffer, 3);
9. }

As one may notice, we're using the microp_ng_write function which is actually initiating an I2C transfer. We could also add an msleep() call or even add a mutex and that would still work.

Now, how do we call the code? Simple, in your non-sleeping routine (which may be an interrupt handler or an implementation of a function from the LED class), call the "schedule_work" routine.

1. static void htckovsky_set_brightness_color(struct led_classdev *led_cdev,
2. enum led_brightness brightness)
3. {
4. schedule_work(&colorled_wq);
5. }

Now, what if you wanted to pass some data to the work routine without using the global variables? The macro "container_of" comes to rescue! Container_of is essentially an implementation of the OOP-like classes in C based on routines and composition instead of inheritance. It works like the following: assume you have a routine which only accepts the "struct work_struct" pointer, and you want to pass arbitrary data. Modifying linux code and creating a copy of the workqueue implementation each time would be silly and boring (although that's what windows developers would do anyway). Instead, we create a structure to hold our data, and add the pointer to the workqueue as a member to it. The container_of macro calculates the offset of a workqueue in our structure, and substracts it from the workqueue pointer. Woops, we have a pointer to our structure and can unmarshal our data from it. This works like the following:

struct work_struct *our_work = kmalloc(sizeof(struct work_struct), GFP_KERNEL);
/* Init the work structure */
INIT_WORK(our_work, handler_routine);
struct my_work_data {
struct work_struct *cool_work;
} work = {
cool_work = our_work;
};

The full source code of the driver is available at the corresponding linux tree at gitorious. It does not synchronize the LED state, though. One should be careful when writing code without synchronization: even though for a LED which updates thousands of times per second, displaying a mixed state is not an issue (a human eye will not notice glitches), if you go this way, only keep your "virtual", driver state global and concurrently accessible, but make sure that the data written to the hardware registers can not be modified concurrently!
https://gitorious.org/linux-on-qualcomm-s-msm/alex-linux-xperia/source/844e6f8bed194d6947d966233f411e8997484091:drivers/leds/leds-microp-htckovsky.c

Kernel Threads
Another alternative is using the kernel threads. In fact, kernel threads are almost like a userspace process running in kernel mode. It can receive signals and do many other things, but programming kernel threads is more difficult and almost all the time you can get away with using workqueues. Here's some sample code showing how to spawn a thread and do your dirty business inside:

static int my_thread(void *v) {
 struct completion exit;
 siginfo_t info;

 daemonize("mydriver_baddaemond");
 allow_signal(SIGKILL);
 init_completion(&exit);

 while (1) {
  if (signal_pending(current)) {
   if (dequeue_signal_lock(current, &current->blocked, &info) == SIGKILL) {
    goto cleanup;
   }
  }
  else {
   //Do your very bad stuff here
  }
 }

cleanup:
 complete_and_exit(&exit, 0);
 return 0;
}

//should be somewhere in the driver data
struct task_struct *my_task = NULL;

static void start_thread(void) {
 void *arg = NULL; //Can be used to pass data to the handler
 my_task = kthread_create(my_thread, arg, "mydriver_baddaemond");
}

static void stop_thread(void) {
 send_sig(SIGKILL, my_task, 0);
 my_thread_task = NULL;
}

Conclusions
Now you know how to make your drivers more stable and less power-hungry. Almost every time you are thinking of an ad-hoc implementation of a common concept, be sure linux has a clean and working solution already.

Read More

[russian] Уязвимость загрузки APK в Android

Это - перевод статьи-анализа уязвимости. Оригинал - http://blog.sina.com.cn/s/blog_be6dacae0101bksm.html

Не успели интернет-массы отбурлить после недавно обнаруженной группой Bluebox Security уязвимости в ОС Android, позволяющей заменить код установленного приложения без нарушения цифровой подписи, как в системе загрузки Apk была обнаружена еще одна дырка.

Немного теории.

Многие думают, что сам факт написания программы в управляемой среде типа Java автоматически делает их код неуязвимым и безопасным. Но забывают о том, что у JVM есть довольно строгая спецификация на низкоуровневые взаимодействия с памятью и арифметику. Давайте рассмотрим простой пример. Скорее всего, внимательные программисты сразу поймут детали уязвимости, а для тех, кто не поймет - будет пояснение.

public class JavaTest {
    public static void main (String [] args) {
        short a = (short) 0xFFFF;
        int b;
        b = a;
        System.out.println (b);
        b = a & 0xFFFF;
        System.out.println (b);
    }
}

Просто, да? Если нет, то давайте посмотрим, во что это реально исполняется. Я тут быстренько написал в REPL Clojure - он на JVM, да и результаты будут одинаковы в большинстве не-JVM языков.

Clojure 1.1.0
user=> (def a (short 0xffff))
#'user/a
user=> (def b (int a))
#'user/b
user=> (def c (int (and a 0xffff)))
#'user/c
user=> a
-1
user=> b
-1
user=> c
65535
user=>

В чем дело? В расширении знаковых типов (sign extension). Казалось бы, основа основ, которую вбивают со школьной скамьи, ан нет - эта бяка ответственна за половину известных уязвимостей (за остальную половину - неинициализированная память). 0xffff, когда мы кладем его в 16-битный short, превращается в -1 (дополнительный код). Согласно правилам расширения знака, если мы присваиваем более широкому типу (в данном случае - 32-битному int) отрицательное значение, оно преобразуется так, чтобы представление в новом формате хранило то же значение, что исходная переменная. То есть, мы получаем int b = 0xFFFFFFFF. Старший (знаковый) бит стоит => лополнительный код => -1. Когда мы отсекаем старшие биты (b & 0xffff), старший бит обнуляется, и мы получаем 0xffff. Но у нас же int, 32 бита - это всего лишь 65535 в прямом коде.

Формат ZIP

Давайте рассмотрим формат архивов ZIP, которым на самом деле являются Jar и Apk архивы (примечание переводчика - на платформе Android Apk обычно часть данных типа картинок не сжимается, и выравнивается на 4 байта утилитой ZipAlign, чтобы можно было отображать ресурсы в память через системный вызов mmap, но это не влияет на совместимость с другими реализациями ZIP). Каждая запись в Zip архиве (абстракцией для которой является Java класс ZipEntry) имеет заголовок следующего вида:

        local file header signature     4 bytes  (0x04034b50)
        version needed to extract       2 bytes
        general purpose bit flag        2 bytes
        compression method              2 bytes
        last mod file time              2 bytes
        last mod file date              2 bytes
        crc-32                          4 bytes
        compressed size                 4 bytes
        uncompressed size               4 bytes
        filename length                 2 bytes
        extra field length              2 bytes

        filename (variable size)
        extra field (variable size)

Обратите внимание на то, что два последних поля (имя файла и дополнительное поле, которое хранится после данных файла), имеют переменную длину.

Проверка Apk в системе Android

При загрузке apk файла происходит проверка контрольной суммы. Функция getInputStream, которая позволяет получить поток для чтения данных из ZipEntry, выглядит примерно следующим образом:

RAFStream rafstrm = new RAFStream(raf, entry.mLocalHeaderRelOffset + 28);
DataInputStream is = new DataInputStream(rafstrm);
int localExtraLenOrWhatever = Short.reverseBytes(is.readShort());
is.close();
//skip the name and this "extra" data or whatever it is: 

rafstrm.skip(entry.nameLength + localExtraLenOrWhatever);
rafstrm.mLength = rafstrm.mOffset + entry.compressedSize;
if (entry.compressionMethod == ZipEntry.DEFLATED) {
    int bufSize = Math.max(1024, (int) Math.min(entry.getSize(), 65535L));
    return new ZipInflaterInputStream(rafstrm, new Inflater(true), bufSize, entry);
}
else {
    return rafstrm;
}

Обратите внимание на выделенные фрагменты кода. Как мы видим, максимальная длина "экстра" данных - 65535 байт (для хранения используется двухбайтовое беззнаковое число). Проблема в том, что в Java нет беззнаковых типов данных, и максимальная длина - 32768 (2 ^ 15). А что, если записать в заголовок Zip большее значение? Правильно, старший бит будет интерпретирован как знаковый, и в соответствии с расширением знаковых типов, переменная localExtraLenOrWhatever получит отрицательное значение.

Таким образом, вектор атаки очевиден - записать отрицательное значение в длину поля "extra", тем самым получив возможность перезаписать имя DEX файла. Небольшое ограничение - для проведения такой атаки необходимо, чтобы размер DEX кода в исходном APK был меньше 64кб.

На сегодняшний день уязвимость исправлена, но ошибки, связанные с переполнением целочисленных типов, по-прежнему являются причиной многих уязвимостей. С учётом того, что больше и больше программистов не знают не только о различиях big/little endian, а даже о знаковых типах данных - ситуация будет только ухудшаться :)

Read More

Mobile phones suck!

Here we go again, another rant about mobile operating systems.

I'm fed up with Android. But there's no alternative for me right now. So I have to moan. To make this moaning more constuctive, let's go over what I hate about Android, what I like about other OSs, and what I want to see.

Android sucks

• Multimedia [audio] is not realtime. Seriously, this is stupid. A dual-core 1.5GHz CPU, and I still hear clicks and pops every now and then when I launch the browser
• Multimedia scanner notifications are broken. So, I download a piece of music, and what happens next? Right, the media scanner kicks in and kills the music player. Why? Oh, because the cool kids are going mad about OOP  and design patterns nowadays, but implementing a proper live configuration updating is beyond the scope of the abilities of those Senior Developers. Sure, burning everything to the ground and creating the new instance of the class is the only way to go   
• No, I ain't going to replace my phone with an mp3 player because I'm lazy. I don't want to bother copying music over to it, I just want to download it over Wi-Fi from warez sites (d'oh)
• UI lags. Even on a quad-core CPU with 2Gb ram
• Apps crash every now and then. Oh well. I don't even know why. In java you have structured exception handling and stuff like that. Even on Windows Mobile where everything was written in plain C with manual memory management, there were not that many crashes. Does your destkop crash every 30 seconds? No? Ok, I'll blame Java. Android's Dalvik JVM, to be more precise. The application life cycle is implemented in Java, the launcher is implemented in Java, but for some multimedia apps the performance of Java is not enough, and IntBuffer's create a considerable overhead. Besides, parsing C structures without JNA is just painful. So, developers have to integrate their C and C++ code into the Java environment. Every Java-to-C call and C-to-Java involves lots of manual memory manipulations, and I suppose most developers don't do that correctly. Still,  it puzzles me why pure Java apps crash so often
• No proper fsck and dead file removal. Seriously, if your data file system is corrupted, and the settings database for an app becomes unwritable, guess what happens? Right, the app launches, crashes with the IOException, and the cycle repeats indefinitely
• Android's using a crippled custom libc and is binary incompatible with GNU/Linux. Okay, libhybris allows us to steal Android's OpenGL drivers, I'm not moaning anymore

Android unsucks

There are positive points, of course

• The system is open-source, it can be easily adapted for any custom piece of hardware [I'm more interested in the HW engineer's point of view rather than the end user's one]
• It is based on linux kernel, allowing to reuse many software via chroot or doable porting [like Qt4]
• That's about all
• Oh, no. There are a lot of commercial devices available, the user has a lot of choices

What next

I've used many mobile platforms and here's what I've liked and disliked about them

Windows Mobile

• Damn slow UI. No compositing or 3D support whatsoever
• Low memory footprint
• Very stable
• Lots of apps
• Native C SDK
• Ability to load kernel-mode drivers
• Good driver model, with stable ABI and configuration via registry 

Windows Phone

• Locked down tightly. No fun for hacking
• Too expensive to build a custom device
• But on the positive side, capability-based resource management and app isolation via containers (security!)

iOS

• It just works
• But no source code
• And SDKs are tied to OSX

Maemo/Meego

• Based on Qt and X11 (Wayland soon), lots of reusable code
• Using traditional GNU/Linux stack and buildsystem
• Open-source
• Native code (except for some parts of UI written in QML).
• Lower resource requirements than Android

I'm looking forward to Jolla's Sailfish OS and trying to port the Nemo Mobile (the unbranded open-source stack behind Sailfish) to my omap4 devices.

Some ideas

Here's what I think a new kind of mobile OS must be

• Realtime scheduling [at least, proper priority handling for multimedia and UI, but policy is a complex question]
• Using capability security model
• Ability to proxify services and override methods
• Built with power optimization in mind
• Microkernel architecture with fixed IPC buffer size [to prevent buffer overflow exploits, to ease debugging]
• Support para- and hw- virtualization
• Dynamic resource paths. I want to query irqs like XPath. For example, |irq = Irq_session "/soc/gic/12"| or "/bus/i2c/omap4-1/0x30/irq_expander/10". Maybe a solution is some kind of a platform driver that will just map strings to irq at runtime.
• Reuse existing linux API and code

So far some of these features are implemented by Fiasco.OC and Genode, but there are some areas in which I want a completely different approach

• Replace XML and LUA configs with... an XML with schema or, better, a DSL with static checking
• Avoid runtime memory allocation. I want that as much data structures as possible (at least in drivers) are allocated at compile-time. This will reduce memory trashing, bootup time and simplify the out-of-memory handling

Use a decent language. I mean, something like Scala or Haskell.

• First one is no uninitialized data. During my C coding experience, 80% of "WTF" problems are uninitialized variables.
• Next, immutability. Functions without side effects can be verified. Of course, verifying 100% code is very expensive (and well, absolute verification is impossible because we cannot verify the universe), but at least splitting the code to the "dangerous" and "debuggable" is a good start".
• Strict type checking and lack of implicit type casts. Will save a lot of problems with the C "everything is an int" paradigm
• Option types. Like haskell's Maybe/Just and OCaml's Some. They eliminate the "NULL" pointers, the source of all crashes and undefined behaviour in C++ and Java, by forcing the programmer to pattern-match the value

As for reusing linux code, I think a starting point could be microkernelizing linux. We could make every module a separate process and replace all global non-static variables and exported symbols with IPC calls for a start. 

Open questions

I am still unsure on some points, and would be happy to see comments on them

• Whether implementing heterogenous CPU support is a feasible task
• Can we make a better bytecode than JVM? why does JVM suck? why is it slow? or is it not?
• What systems of such kind exists already? Phantom OS? Inferno OS?

Read More

Porting Genode to commercial hardware. Episode1: B&N Nook HD+

Hi there!

In this post I will briefly describe my endeavours in the process of making the Genode OS Framework running on the B&N Nook HD+ tablet.

General thoughts on microkernels

While during my work at Ksys Labs LLC I have to work on developing Genode, this was my free-time project. I got fascinated by the microkernel conception and the Genode framework, and I want to have a fully paravirtualized linux on omap4 with the PowerVR GPU working. It would be nice to see how kernelizing the system core will improve stability and debuggability. Though, currently linux on ARM is very optimized in terms of performance and power consumption (due to the immense efforts of 10+ years of development and ingenious algorithms for scheduling, memory management and power state machine), and only a few closed-source solutions offer comparable or better service (namely, QNX, VxWorks and Windows Compact Embedded). Most microkernels and runtimes (L4Re, Genode) lack any kind of power management, moreover scheduling and memory allocation algorithms are primitive, with implementation simplicity and reliability valued over efficiency, therefore they are better off used as hypervisors instead of general-purpose OSs. Besides, driver for linux are written by hardware manufacturers, while they are not particularly interested in open-source microkernel development, therefore driver support for new technologies will certainly lag behind conventional OSs. Nevertheless, I still think it is interesting to develop a FOSS software stack for embedded devices.

Motivation

So, why the Nook HD+? While I have alrealy ported Genode to the Samsung Galaxy Nexus phone for the FOSDEM 2013 demo session, I decided to give another device a try for the following reasons:

• As a tablet, it has much less hardware (at least, no phone) to support in order to be a daily driver
• Some peripherals and hardware setup varies between it and the Nexus, so it may help exposing new bugs or hardcodery
• It has the microSD slot connected to OMAP4 mmc0 with standard voltage setup (the VMODE bit). This is extremely useful because it allows to directly use the Genode MMC driver and it is easy to setup a MBR/VFAT filesystem on the microSD. Galaxy Nexus, on the contrary, has no external memory card slot, and to access the internal memory, it is necessary to implement a small change in the MMC driver and implement the EFI GPT partition parser
• It has a Full HD display, and I have passion for hi-res screens. Besides, hi-res is a way to stress-test memory subsystem and framebuffer performance
• Because I can

U-boot

Booting custom software on a commercial device is usually connected with some obstacles, like breaking the chain of trust. A typical approach (which I have utilized for many devices, including Acer Iconia A500, Samsung Galaxy S2 and Samsung Galaxy Nexus) is porting the u-boot bootloader to be an intermediate chainloader, flashed instead of the linux kernel.

The B&N Nook HD+ does feature the signed kernel-based chain of trust for the internal EMMC memory, but it allows booting unsigned MLO, xloader and u-boot from the external microSD card. That is to say, there already exists the u-boot port, but to make it boot Genode, numerous changes were needed.

First, I've obtained the android cwm recovery image for the sd card from the B&N Nook HD+ forum on xda-developers.com [credits go to the forum members verygreen and fat-tire]. After writing the image to the SD card and fixing partition layout in fdisk, I ended up with a VFAT partition containing the MLO, u-boot.bin and uImage. The MLO is the header file for the omap4 CPU which combines the memory initialization table with the x-loader bootloader. The u-boot.bin is the almost vanilla B&N u-boot which initializes the display and boots the uImage [which in the case of sd booting is also u-boot]. We'll be replacing the uImage with the customized u-boot.

You can obtain the source code for my u-boot from https://github.com/astarasikov/uboot-bn-nook-hd-fastboot and below is the list of problems I've solved

• Removing the [unneeded for us] emmc and sd boot options.
• Enabling fastboot. The bootloader listens on usb for the fastboot connection. "fastboot continue" allows to boot the "kernel" and "ramdisk" files from the sd card
• Fixed display initialization. Turns out, the code in the uImage was broken, and did not reinit the display properly. The reasons were that it lacked one power GPIO (which would cause it to never come up after reset on some hardware revisions, my tablet being one of the unlucky ones), and the typo in one of the frame sync borders (which caused all font symbols to be one pixel tall). The display initialization code was scattered around 4 files, contained hardcoded definitions for another board. The framebuffer initialization was done in the MMC driver (sic!). I think I did write a rant about it about a year ago, but nothing ever changes. Most commercial embedded software is crap. Well, most software is crap either way, but the level of crappiness in open-source software is reduced when someone finds themselves in a need to add the support for a new configuration, and the community does not welcome hardcoding and breaking old code.
• Fixed booting "ANDROID!" boot images over fastboot with the "booti" command. The code contained many incorrect assumptions about the image layout.
• Moved the u-boot base in RAM and increased the fastboot buffer to allow downloading huge images (up to 496M). This allows me to boot Genode images with the built-in ramdisk with the root file system while I've not completed the GPT support
• Enabled the framebuffer console for debugging

Genode

I had to do some changes to make Genode run. I'll briefly discuss some notable items.

Fiasco.OC cross-compiling

Recently, Genode crew have updated to the latest revision of the Fiasco.OC microkernel and it seems to contain some hardcoded cross-compiler name for ARM. I was reluctant to either fix it or download the 'proper' compiler (especially knowing that the one from the Genode toolchain does work for omap4).

So, here is what I've done:

• Made a new directory and added it to the PATH variable (append "export PATH=/path/to/that/very/dir/:$PATH" to your .bashrc if you have no slightest idea what I'm talking about)
• Made symbolic links for the fake "arm-linux" toolchain pointing to genode with "for i in `ls /usr/local/genode-gcc/bin/genode-arm*`;do  ln -s $i ${i//*genode-arm/arm-linux} ;done"

Increasing nitpicker memory quota.

Currently nitpicker [window manager providing virtual framebuffers] is hardcoded for some 1024x768 screens (I believe because no one, even at genode, seriously considers using Genode as a daily driver today), so we need to fix the memory limit constant in the following way:

--- a/os/include/nitpicker_session/connection.h

+++ b/os/include/nitpicker_session/connection.h

@@ -43,8 +43,8 @@ namespace Nitpicker {

                                char argbuf[ARGBUF_SIZE];

                                argbuf[0] = 0;

 

-                               /* by default, donate as much as needed for a 1024x768 RGB565 screen */

-                               Genode::size_t ram_quota = 1600*1024;

+                               /* by default, donate as much as needed for a Full HD RGB565 screen */

+                               Genode::size_t ram_quota = 1920*1280*2;

Adding LCD support to omap4 framebuffer

Currently, omap4 framebuffer only supports the TV interface for HDMI. To make it reset and configure the LCD interface (which is used in smartphones and tablets), we need to add some register definitions [and fix the incorrect definition of the base address register while we're at it] to the code according to the omap4 DSS subsystem manual.

diff --git a/os/src/drivers/framebuffer/omap4/dispc.h b/os/src/drivers/framebuffer/omap4/dispc.h

index 23f80df..ea9f602 100644

--- a/os/src/drivers/framebuffer/omap4/dispc.h

+++ b/os/src/drivers/framebuffer/omap4/dispc.h

@@ -18,8 +18,14 @@ struct Dispc : Genode::Mmio

         */

        struct Control1 : Register<0x40, 32>

        {

+               struct Lcd_enable : Bitfield<0, 1> { };

                struct Tv_enable : Bitfield<1, 1> { };

 

+               struct Go_lcd : Bitfield<5, 1>

+               {

+                       enum { HW_UPDATE_DONE    = 0x0,   /* set by HW after updating */

+                              REQUEST_HW_UPDATE = 0x1 }; /* must be set by user */

+               };

                struct Go_tv : Bitfield<6, 1>

                {

                        enum { HW_UPDATE_DONE    = 0x0,   /* set by HW after updating */

@@ -46,11 +52,17 @@ struct Dispc : Genode::Mmio

                struct Width  : Bitfield<0, 11>  { };

                struct Height : Bitfield<16, 11> { };

        };

+       struct Size_lcd : Register<0x7c, 32>

+       {

+               struct Width  : Bitfield<0, 11>  { };

+               struct Height : Bitfield<16, 11> { };

+       };

 

        /**

         * Configures base address of the graphics buffer

         */

-       struct Gfx_ba1 : Register<0x80, 32> { };

+       struct Gfx_ba0 : Register<0x80, 32> { };

+       struct Gfx_ba1 : Register<0x84, 32> { };

 

        /**

         * Configures the size of the graphics window

diff --git a/os/src/drivers/framebuffer/omap4/driver.h b/os/src/drivers/framebuffer/omap4/driver.h

index d754a97..53517a3 100644

--- a/os/src/drivers/framebuffer/omap4/driver.h

+++ b/os/src/drivers/framebuffer/omap4/driver.h

@@ -203,6 +203,7 @@ bool Framebuffer::Driver::init(Framebuffer::Driver::Mode   mode,

        }

        _dispc.write<Dispc::Gfx_attributes::Format>(pixel_format);

 

+       _dispc.write<Dispc::Gfx_ba0>(phys_base);

        _dispc.write<Dispc::Gfx_ba1>(phys_base);

 

        _dispc.write<Dispc::Gfx_size::Sizex>(width(mode) - 1);

Hacking in Nook HD+ display support.

I wanted to make the display work in a quick and dirty way, so I've commented out the HDMI init code and replaced with a simple code that reconfigured the framebuffer address for the LCD (we piggyback on the u-boot to initialize the screen). Remember, kids, never ever think of doing this in production. I am deeply ashamed of havind done that. Either way, I'll show you the code, and the framebuffer driver badly wants some changes:

• Adding proper LCD initailization
• Configurable resolution via the config
• Support for DSI/DPI interface initialization and custom panel drivers
• Rotation and HW blitting

Ok, enough talk, here's the patch

diff --git a/os/src/drivers/framebuffer/omap4/driver.h b/os/src/drivers/framebuffer/omap4/driver.h

index 53517a3..9287cdc 100644

--- a/os/src/drivers/framebuffer/omap4/driver.h

+++ b/os/src/drivers/framebuffer/omap4/driver.h

@@ -76,6 +76,7 @@ class Framebuffer::Driver

 

                static size_t width(Mode mode)

                {

+                       return 1920; //XXX: fix config parsing

                        switch (mode) {

                        case MODE_1024_768: return 1024;

                        }

@@ -84,6 +85,7 @@ class Framebuffer::Driver

 

                static size_t height(Mode mode)

                {

+                       return 1280;

                        switch (mode) {

                        case MODE_1024_768: return 768;

                        }

@@ -117,12 +119,17 @@ bool Framebuffer::Driver::init(Framebuffer::Driver::Mode   mode,

                                Framebuffer::addr_t         phys_base)

 {

        /* enable display core clock and set divider to 1 */

+       #if 0

        _dispc.write<Dispc::Divisor::Lcd>(1);

        _dispc.write<Dispc::Divisor::Enable>(1);

+       #endif

+

+       _dispc.write<Dispc::Control1::Lcd_enable>(0);

 

        /* set load mode */

        _dispc.write<Dispc::Config1::Load_mode>(Dispc::Config1::Load_mode::DATA_EVERY_FRAME);

 

+       #if 0

        _hdmi.write<Hdmi::Video_cfg::Start>(0);

 

        if (!_hdmi.issue_pwr_pll_command(Hdmi::Pwr_ctrl::ALL_OFF, _delayer)) {

@@ -196,6 +203,10 @@ bool Framebuffer::Driver::init(Framebuffer::Driver::Mode   mode,

        _dispc.write<Dispc::Size_tv::Height>(height(mode) - 1);

 

        _hdmi.write<Hdmi::Video_cfg::Start>(1);

+       #endif

+

+       _dispc.write<Dispc::Size_lcd::Width>(width(mode) - 1);

+       _dispc.write<Dispc::Size_lcd::Height>(height(mode) - 1);

 

        Dispc::Gfx_attributes::access_t pixel_format = 0;

        switch (format) {

@@ -212,6 +223,7 @@ bool Framebuffer::Driver::init(Framebuffer::Driver::Mode   mode,

        _dispc.write<Dispc::Global_buffer>(0x6d2240);

        _dispc.write<Dispc::Gfx_attributes::Enable>(1);

 

+       #if 0

        _dispc.write<Dispc::Gfx_attributes::Channelout>(Dispc::Gfx_attributes::Channelout::TV);

        _dispc.write<Dispc::Gfx_attributes::Channelout2>(Dispc::Gfx_attributes::Channelout2::PRIMARY_LCD);

 

@@ -223,6 +235,9 @@ bool Framebuffer::Driver::init(Framebuffer::Driver::Mode   mode,

                PERR("Go_tv timed out");

                return false;

        }

+       #endif

+       _dispc.write<Dispc::Control1::Lcd_enable>(1);

+       _dispc.write<Dispc::Control1::Go_lcd>(1);

 

        return true;

 }

Configuration file

Genode is configured via the XML configuration file. Here are some notes

• We're using the dde_kit usb_drv driver to provide stubs for networking and input drivers
• The nic_bridge proxifies the networking for two l4linux instances
• nitpicker and nit_fb are used to split the display into virtual framebuffers
• both nic_bridge and nit_fb are using the Genode concepts of the service interfaces and service routing. We're configuring the services in such a way that they're using the specific service if needed, and rely on the parent to provide the default service if we dont' care. For example, take a look at how nic_bridge is configured. The usb_drv features the "provides" section that declares which interfaces the service is allowed to provide. These may be used in the "route" section of the client services. By default, Genode features a deny-all policy, so if you don't configure something, you have no access to it.
• usb_drv has some memory leak (or had back in winter and I was lazy to look into it) so I've increased the RAM quota and hoped it would survive. It did.

<start name="usb_drv">

<resource name="RAM" quantum="40M"/>

<provides>

<service name="Input"/>

<service name="Nic"/>

</provides>

<config>

<hid/>

<nic mac="2e:60:90:0c:4e:01" />

</config>

</start>

<start name="nic_bridge">

<resource name="RAM" quantum="2M"/>

<provides><service name="Nic"/></provides>

<route>

<service name="Nic"> <child name="usb_drv"/> </service>

<any-service> <parent/> <any-child/> </any-service>

</route>

</start>

Compiling and running

So, how about trying it out yourself?

Install the Genode toolchain (consult the Genode website and sourceforge project) and create the symlinks as explained above.

Get the u-boot source code

git clone git://github.com/astarasikov/uboot-bn-nook-hd-fastboot.git

Compile U-boot. I recommend using the codesourcery 2010q1-188 toolchain (because not all toolchains produce working code)

export PATH=/path/to/codesourcery/toolchain/bin:$PATH

export ARCH=arm

export CROSS_COMPILE=arm-none-eabi-

U_BOARD=bn_ovation

make clean

make distclean

make ${U_BOARD}_config

make -j8 

./tools/mkimage -A arm -O linux -T kernel -C none -a 0x88000000 -e 0x88000000 -n foo -d u-boot.bin uImage

Get the genode framework source code

git clone git://github.com/astarasikov/genode.git

git checkout nook_staging

Now, for each directory in the genode tree (base-foc, and non-base directories), go to them and execute "make prepare" to download the required libraries. Well, libports is heavily broken and many packages (openssl, zlib, any more?) fail to download. You can skip libports and qt4 for now.

Prepare the build directory

./tool/create_builddir foc_panda BUILD_DIR=/my/build/dir

Edit the /my/build/dir/etc/build.conf

Uncomment the "REPOSITORIES += " entries to allow building l4linux and nitpicker

Add the "MAKE += -j8" to the file to build in parallel utilizing all CPU cores.

Add "SPECS += uboot" to the /my/build/dir/etc/specs.conf to force creating the raw image.bin.gz binary.

Compile the Genode framework

cd /my/build/dir

make run/nookhdp_l4linux

Actually running the image

Now, you may wonder how to run the image. The tablet must be in the fastboot mode. Genode expects itself to be loaded at 0x81000000, and the u-boot does make a stupid assumption that linux kernel must be shifted 0x8000 bytes (i.e., 2 pages) from the base address. It should be fixed eventually, but for now, we're manually substracting the offset from the boot address

gunzip var/run/nookhdp_l4linux/image.bin.gz

fastboot -b 0x80ff8000 boot var/run/nookhdp_l4linux/image.bin

Results

Here is a picture of the Genode running. You can see the screen split into four parts with the help of the Nitpicker window manager. Each screen chunk is a virtual framebuffer provided by the Nit_fb service. Genode is running the Launchpad server (top right corner), the LOG written to the framebuffer (bottom left) and two instances of L4Linux.

So, now it does not do much useful work, but remember it was an overnight proof of concept hacked together with the sole purpose of demonstrating the capabilities of Genode Framework, and this tutorial is basically a collection of anti-patterns.

Future plans

Since I want to have a fully-functional port on both the Nook HD+ and Samsung Galaxy Nexus, here are some areas of interest for me and anyone who would like to contribute

• Upstream OMAP4 and I.MX53 I2C drivers (we at Ksys Labs have written them almost a year ago and they're working fine, but had no time to suggest them to Genode Labs)
• Upstream GPIOMUX interface for OMAP4
• Rework and upstream voltage regulator and TWL6030 code
• Improve OMAP4 Framebuffer Driver
• EFI GPT Partition table parsing
• EXT2 file system driver
• File System to Block interface adapter for Genode (for doing loop mounts)
• TWL6040 Sound Codec support
• Virtual SDHCI driver for L4Linux (for prototyping wifi)
• Ressurect and fix the MUSB OTG driver for DDE_LINUX
• Nook HD+ Touchscreen support
• Refactor and upstream Google Nexus touchscreen
• Multitouch support in Genode and L4Android
• GPIO Buttons driver
• Battery drivers
• Charging driver
• HSI Serial for OMAP4 (for Nexus modem)
• PWM and backlight support for OMAP4
• Sensors (Accelerometer, Gyroscope, Light)
• UI for observing hardware state and controlling drivers
• Basic power management
• Native Genode linux port (without l4linux and Fiasco.OC). Besides dropping the huge messy pile of L4 support code from linux, this will allow to break the dependency on Fiasco.OC and switch to the native port (base-hw)

Read More

UEFI and ARM

Introduction

UEFI [Universal Extensible Firmware Interface] is a standard for implementing the bootloaders and the interface between the bootloader and the OS.

UEFI defines a standard for executable images (second-stage loaders which load the OS - like rEFIt for OS X, bootx64.efi for Windows, grub-efi and elilo for linux). Which is actually a PE/COFF (windows "MZ" exe files).

Advantages

• Fixed API and ABI
• Extended type annotations [like, IN/OUT/INOUT for function arguments]. This can theoretically help spot some coding mistakes at compilation time.
• Providing IO range and IRQ descriptions (like ACPI) for ARM systems

Disadvantages

• It was designed by Microsoft and Intel (therefore, unnecessary code bloat)
• All the code runs in the same address space, MMIO access is not protected by capabilities or any other security mechanism. While the situation is the same with other bootloaders and using the one-to-one memory mapping shared between all components [processes or libraries] allows to use the bootloader in the systems without MMU, UEFI was devised to provide a secure chain  of trust for the bootup process and 

UEFI Services

Since UEFI services are PE/COFF binaries, they export the symbols via the import/export tables. This allows to lookup the needed functions in the binary modules. UEFI defines a number of services. For example, the initial bootloader can rely on the UEFI bootloader for reading data from the disk. The part which confuses me is that most of these services are destroyed with the ExitBootServices() call, and the only usable service available at runtime is the RTC/Timer service. Since the OS cannot piggy-back on the bootloader for all its driver routines, why introduce a complex bootloader at all? It does deliver potential vulnerabilities but does not have advantages over BIOS in terms of hardware initialization.

TianoCore EDK2 and UEFI on ARM

TianoCore EDK2 is a reference UEFI implementation from Intel. It comes with various interesting packages and can be used to build both UEFI bootloaders and standalone applications for systems already running UEFI (like, most consumer-grade motherboards and laptops available on the market)

• ArmPkg - contains the Linux Loader and the drivers for CPUS (Cortex A8, A9, A15) - cache, interrupts (GIC), Timer
• ArmPlatformPkg - contains the Uart, GPIO and Nor drivers for the ARM reference platforms, TrustZone setup routines, Exception handling and stack switching code
• CryptoPkg - contains the wrappers for OpenSSL to allow using cryptography (for example, for UEFI Secure Boot)
• DuetPkg - the package to test UEFI on an X86 computer
• EdkShellPkg - the UEFI shell which allows browsing mass storage driver, booting custom images and interacting with drivers via the configurable variables
• MdePkg - contains the runtime services and HAL (Hardware Abstraction Layer) for PCI and other busses.
• NetworkPkg - contains the support for IPv6, DHCP, TFTP and SCSI (obviously, for network booting)
• Nt32Pkg - Bootloader services for Windows 7 Embedded
• OvmfPkg - ACPI emulation and Virtio
• PcAtChipsetPkg - obviously the x86 support - Timer, PIC, HPET and PCI bridge drivers.
• StdLib - EFI library and sockets

I think this particular implementation sucks. I could bear with the unreadable code, the fact that you have to put all definitions in like three files (platform, board files and .ini files). But. The reference implementation only works for ARM BeagleBoard. For PandaBoard, the source code clearly states that "the display driver is broken and uncommenting it leads to mysterious freeze". I've spent some time hacking on it but still didn't manage to get the display working. Neither the MMC. On PandaBoard, it just failed to work. On Galaxy Nexus, it started working after I switched it from the block mode to the "unsupported" streaming mode in the generic mmc host code. Weird.

Windows RT Bootup Process

Windows RT is a port of Windows 8 to the ARM architecture. Here, the UEFI is used to emulate an X86 setup on ARM: the ACPI tables and power states. Besides, UEFI is used to query the IO ranges for peripheral controllers (it is known that most ARM SoCs use memory-mapped IO for accessing peripherals and the only things an OS has to care about are the IO range and the IRQ pin. This is what UEFI provides). So, the advantage of the UEFI is that it allows to compile the OS kernel and drivers once and use that on various boards.

For Windows RT and Windows Phone 8, the boot process starts with the UEFI bootloader loading the bootarm.efi file. It then tries to read the BCD (Boot Configuration Data) table and mount the root partition. It does rely on UEFI for reading the drive at this stage. Next, control is transferred to the Windows NT kernel which calls the ExitBootServices() routine and loads the native block driver. So far, I've managed to partition the usb thumb drive properly and boot the bootarm.efi and make it recognize the partition on qemu emulating the BeagleBoard, but that's about all.

Alternatives

Linux kernel has recently also gained the support for building mutiple SoCs in one kernel. You can have a single kernel that boots on OMAP, Tegra and what not. The advantage is that you can have a kernel-side board driver that will initialize the drivers with needed data eliminating the need for ACPI emulation and complex binary table parsers.

Suppose you're an evil hardware manufacturer wanting to hide the details about your board and not contribute code to linux. What do you do then? Right, use the DTS or Device Tree. Originated on the PowerPC MACs, the flattened device tree was subsequently ported to ARM and is now supported by the u-boot bootloader and both Linux and FreeBSD kernels. It allows to describe all peripherals and driver parameters in a hierarchial text format which will then either be passed to kernel as is or compiled to a human-unreadable binary format (which is security by obscurity of course but that's what proprietary developers think is cool).

Let's take a look at the FTD file for a tegra board taken from u-boot. I find it neat that you can specify everything - IO ranges, gpio and irq pins. If the BSP code is written properly, you can get away with writing no board code, only a declarative IO map description.

/dts-v1/;

#include "tegra114.dtsi"

/ {
 model = "NVIDIA Dalmore";
 compatible = "nvidia,dalmore", "nvidia,tegra114";

 aliases {
  i2c0 = "/i2c@7000d000";
  i2c1 = "/i2c@7000c000";
  i2c2 = "/i2c@7000c400";
  i2c3 = "/i2c@7000c500";
  i2c4 = "/i2c@7000c700";
  sdhci0 = "/sdhci@78000600";
  sdhci1 = "/sdhci@78000400";
 };

 memory {
  device_type = "memory";
  reg = <0x80000000 0x80000000>;
 };

 i2c@7000c000 {
  status = "okay";
  clock-frequency = <100000>;
 };

 i2c@7000c400 {
  status = "okay";
  clock-frequency = <100000>;
 };

 i2c@7000c500 {
  status = "okay";
  clock-frequency = <100000>;
 };

 i2c@7000c700 {
  status = "okay";
  clock-frequency = <100000>;
 };

 i2c@7000d000 {
  status = "okay";
  clock-frequency = <400000>;
 };

 spi@7000da00 {
  status = "okay";
  spi-max-frequency = <25000000>;
 };

 sdhci@78000400 {
  cd-gpios = <&gpio 170 1>; /* gpio PV2 */
  bus-width = <4>;
  status = "okay";
 };

 sdhci@78000600 {
  bus-width = <8>;
  status = "okay";
 };
};

Conclusion

UEFI is evil. It does not solve any particular problems on X86 and brings bug-ridden ACPI to ARM. Besides, on ARM Microsoft devices, Secure Boot cannot be turned off to boot custom images. This effectively locks the user out of their device. In other words, it is a typical DRM (digital rights management) system with a chain of trust (or, rather, a chain of distrust) which is aimed at preventing the "bad" user from running custom software. This allows the vendor to remove all the freeware content from the application market, and users will be forced to buy it because they have no alternative.

On X86, most vendors have not implemented the support for choosing the UEFI image to load. While this does not prevent you from renaming your binary to windozish bootx64.efi, it does show the attitude and the direction we're heading into.

Let us developers just ignore this crap!

And btw http://arstechnica.com/information-technology/2013/02/linus-torvalds-i-will-not-change-linux-to-deep-throat-microsoft/

Read More