LLVM is a great tool to port applications to embedded architectures like ARM, the Advanced Risc Machine. I have spent days and weeks to setup an environment for efficiently building the so called “toy” language from the LLVM website. ARM can’t work with the standard JIT, so take the MCJIT example from the LLVM blog. It does work well, but is slightly slower than the old JIT, but please take a look on the blog entry for more details.
One might think that cross compiling LLVM from a x86 machine to ARM should no big deal these days, as there is potential directly support by the LLVM team through cmake. Unfortunately, running cmake and make afterwards for cross compiling on x86 to build LLVM for ARM is with LLVM 3.3 broken. Adding a patch hadn’t a too positive influence on my build process.
Building on ARM itself, for instance I own a raspberry PI, an ARM 1176JZF-S, with 512 MB memory, takes up to 20h and if it comes to the linking the system aborts with a memory failure, even with a big swap partition. But there are other ARM targets, which can’t be used to built on itself at all. So, this would be no solution for all compiling scenarios.
Another approach is to use a “chroot”, mounting your SD card image and running that image with QEMU in an emulation mode. It works, but it is slow, too. But you need to setup a chroot to compile the LLVM libraries to ARM binaries. See for instance this blog entry how to do so.
The best way is of course to get all done on your x86 box by clang and LLVM. In the following I will guide you how to do so. The first step is to compile clang that it also can cross compile to ARM. By default all targets are selected, but if you don’t plan to cross compile to different targets I propose only to call
configure –enable-optimized –enable-targets=x86,ARM
You should have downloaded the LLVM core sources and also the compiler-rt and clang packages and put them into the projects and tools subfolder, respectively. Then you need to call make. Please consider to call configure and make from an additionally sub folder below the llvm source folder, for instance build. As I use also BOOST for some configuring and command line options, plase set REQUIRES_RTTI=1. So, call make REQUIRES_RTTI=1. This enables runtime type information for clang. Otherwise, compiling isn’t possible, at least with my enhanced “toy” language.
This should, depending on the memory and CPU power available, take 1-3h. After making the sources, hopefully without any incidents, you need also to install the includes, binaries and libs. Just call make install.
I’ve spllited the “toy” language a bit, so it is more reusable. There is a lexer, a parser, some utility functions, a (configuration) reader and the main routine. This isn’t important at all to understand how to cross compile, so we suppress here the details. To make compilation easy for me and more reusable, I have written a small Makefile.
I will explain the compiling and linking steps here. Anything else is pretty simple and straight forward and I assume, you, the reader, will take easily the next steps once understood how to step the difficult cross compilation and linking steps. So let’s have a look on the respective Makefile lines:
# The lexer
lexer.o: $(SRC)/lexer.cpp init
@echo ‘Compiling lexer.cpp’; cd $(BIN); $(CC) -c $(CFLAGS) ../$(SRC)/lexer.cpp $(LLVM_CONFIG_COMPILE); cd ..;
There are multiple rules for compiling and one for linking. Above is the example of compiling the lexer. The first line contains the rule name and the dependencies. In this case a dependency to target init and the availability of the source code. The second line switches off the echo and prints out that some stuff is going to be compiled here. Then I jump into the binary folder and call the C compiler (in this case this is clang++). The most interesting part is here behind the variable CFLAGS. This variable is defined at the very top of the Makefile and has for the case of a cross compilation following values:
- -target armv7l-unknown-linux-gnueabihf
- -ccc-gcc-name armv7l-linux-gnueabihf-gcc
The first argument defines the overall architecture, in our case it’s an ARM. The second argument is the Application Binary Interface (ABI). This is for raspberry PI a hard float, this means no software emulated floating point (fp) unit. But there are other ARM targets with a soft fp. The third argument is inherently important, though you will get your code compiled, but your ARM target will give you an “Illegal Instruction”, if not properly specified. So better put in here your correct CPU id. The target is defined by the following entry. The next three include files are very important, otherwise you will get noticed by the compiler that some header files couldn’t be found. This files will be provided by the build process of LLVM. The last entry is needed in some scenarios to force that the linker will take hard float, but in my case it had no effect.
Behind the source file there is also yet the variable LLVM_CONFIG_FILE. This line contains some switches for the C compiler. The value of this variable is: llvm-config –cppflags. This expands to:
This values are pretty standard and not very special to cross compile. So we don’t dive here into more explanations. For compiling these are the basic definitions.
Let’s look at the Makefile code for the linking:
# Linking altogether
all: init tl.o parser.o ast.o helper.o util.o error_util.o config_reader.o lexer.o
@echo ‘Linking altogether’; $(CC) $(CFLAGS) $(BIN)/lexer.o $(BIN)/config_reader.o $(BIN)/error_util.o $(BIN)/util.o $(BIN)/helper.o $(BIN)/ast.o $(BIN)/parser.o $(BIN)/tl.o $(THREAD_LIB_DIR) $(LLVM_CONFIG_LINK) -O2 -o $(BIN)/tl
The interesting points are here the variable THREAD_LIB_DIR and LLVM_CONFIG_LINK. The variable THREAD_LIB_DIR does hold all the needed linker entries:
THREAD_LIB_DIR=-L/usr/lib/arm-linux-gnueabihf -L/usr/arm-linux-gnueabihf/lib -L$(LLVM_LIB_DIR) -L$(LLVM_BOOST_LIB_DIR) -lz -lpthread -lrt -ldl -lm
Beneath the linux base libs also the ARM binary compatible LLVM libs needs to be added. Therefore you should have them compiled for ARM prior. I recommend to use a chroot and then copy or just mount the chroot and then reference to the respective folders. Moreover, as I use BOOST also the respective BOOST libraries needs to be linked. The variable LLVM_CONFIG_LINK does hold the enumeration of all required LLVM libs:
LLVM_CONFIG_LINK=`llvm-config –libs $(LLVM_LIBS)`.
Afterwards you only need to copy the binary file (and maybe some configuration files, if any exist) to your ARM device. It’s statically bound. Congratulations, you can now compile in a very efficient way your ARM code on your development machine.