Difference between revisions of "High level system modelling, hands-on computer session"

From F-Si wiki
Jump to navigation Jump to search
 
(20 intermediate revisions by 3 users not shown)
Line 3: Line 3:
* web: https://www.greensocs.com
* web: https://www.greensocs.com


==Slides==
==Downloads==
[[Media:fsic-greensocs.pdf|fsic-greensocs.pdf]]
* [[:File:fsic-greensocs.pdf|Slides]]
==Abstract==
* [https://peertube.f-si.org/videos/watch/78e5cdfd-9856-41df-a0c4-ffa2a4b52717 Video recording]
 
=Abstract=
GreenSocs® was formed in 2005 to offer a complete range of solutions for the implementation of Model Based Design Engineering for embedded systems.
GreenSocs® was formed in 2005 to offer a complete range of solutions for the implementation of Model Based Design Engineering for embedded systems.
Those solutions range from implementation of the design cycle to the delivery of pre-developed models through support of modeling teams of customers and to complete outsourcing of modeling.
Those solutions range from implementation of the design cycle to the delivery of pre-developed models through support of modeling teams of customers and to complete outsourcing of modeling.
Line 14: Line 16:
During this tutorial, we will show a simple example of QEMU+SystemC integration of a HW/SW system using the current, publicly available, GreenSocs integration solution.
During this tutorial, we will show a simple example of QEMU+SystemC integration of a HW/SW system using the current, publicly available, GreenSocs integration solution.


==Tutorial==
=Tutorial=
Soon here !
Thanks to Clement Deschamps <code><clement.deschamps@greensocs.com></code> for this great tutorial!
== Activity 1 - Build a Simple System ==
 
=== Overview: ===
 
* Create a simple virtual platform
** Implement a simple memory (with DMI support)
** Add a CPU
** Add a UART
* Build and run a simple firmware
* Build more advanced firmware
** Add a libc and run Dhrystone
** Build and run Linux
** Create a minimal filesystem using Builroot
 
=== 1. Presentation ===
 
During this course, we will build the following simple virtual platform.
 
We will then use it to explore some of the features of SystemC and QBox.
 
[[File:platform-1.png]]
 
The virtual platform will contain the following components:
 
* ARM Cortex A57 CPU (provided by QBox)
* PL011 UART (source code provided)
* Simple memory (skeleton provided)
 
=== 2. Create a simple virtual platform ===
 
==== 2.1 Prepare environment ====
 
This tutorial requires either Debian (8 or 9), or Ubuntu (16.04 or 18.04).
 
If you are not using one of these distributions, or if you want a clean environment, you can create a temporary docker container with (first install the package <code>docker.io</code> if docker is not yet present on your system):
 
<source lang="sh">docker run --name gs-tuto -d -i -t --cap-add=SYS_PTRACE ubuntu:18.04
docker attach gs-tuto
<ENTER>
apt update && apt install sudo
cd $HOME</source>
 
Interestingly enough, if you're using docker, the tutorial will be executed while being `root`. This is quite unusual, but funny!
 
Install required Ubuntu packages:
 
<source lang="sh">sudo apt install -y \
    bison flex cmake wget git htop device-tree-compiler socat gcc-aarch64-linux-gnu\
    libboost-dev libboost-filesystem-dev libfdt-dev libglib2.0-dev libgtk-3-dev liblua5.2-dev \
    libncurses5-dev libpixman-1-dev python-dev swig texi2html zlib1g-dev tree vim gdb bc</source>
Build SystemC 2.3.1a and install it in <code>/opt/systemc-2.3.1a</code>
 
<source lang="sh">wget http://www.accellera.org/images/downloads/standards/systemc/systemc-2.3.1a.tar.gz
tar -xf systemc-2.3.1a.tar.gz
sed -i -e 's/auto_ptr/unique_ptr/g' systemc-2.3.1a/src/sysc/packages/boost/get_pointer.hpp
sed -i -e '/using std::gets/d' systemc-2.3.1a/src/systemc.h
mkdir systemc-2.3.1a/objdir
pushd systemc-2.3.1a/objdir
../configure --prefix=/opt/systemc-2.3.1a
make -j$(nproc)
sudo make install
popd</source>
 
==== 2.2 Implement a simple Memory ====
 
The objective is to implement a very simple TLM memory, supporting Loosely Timed (LT) b_transport, and Direct Memory Access (DMI).
 
The memory will be tested with a MemoryTester component that we provide.
 
[[File:Memory-1.png]]
 
Please download the project skeleton from:
 
<pre>wget https://darwin.greensocs.com/s/YF9Bk3yFEdyEqnN/download -O hands-on-vp.tar.gz
tar xvf hands-on-vp.tar.gz</pre>
The project contains the following tree: [[File:memory-tree.png]]
 
* <code>cmake/</code> folder contains a script for finding SystemC
* <code>CMakeLists.txt</code> is the cmake description of the project
* <code>src/main.cc</code> contains the sc_main() that instantiates a Memory and a MemoryTester
* <code>src/memory.h</code> is the skeleton of the memory you need to implement
* <code>src/memory-tester.h</code> is a sc_module for testing the memory
 
We provide a SystemC module named MemoryTester that will access your memory through TLM transactions, and through DMI.
 
===== Build the project =====
 
The build process is based on CMake, and takes place in two stages. First, standard build files are created from configuration files. Then the platform’s native build tools (gcc, binutils, …) are used for the actual building.
 
<pre>cd hands-on-vp
mkdir build
cd build
cmake -DSYSTEMC_PREFIX=/opt/systemc-2.3.1a ..</pre>
And finally, build the rather "thin" virtual platform using <code>make</code> command.
 
'''Please make sure you run <code>cmake</code> into the <code>build</code> directory, otherwise strange things appear later on.''' (As you may have noticed during the 'live' run.)
 
This will generate an executable file named <code>vp</code>.
 
===== Run =====
 
Simply execute the <code>vp</code> binary file.
 
You should see an error as the Memory component is not fully implemented (it’s only a stub):
 
[[File:memory-error-1.png]]
 
===== Implement LT transport =====
 
* '''Links to git repo from where to copy text from:'''
** https://git.greensocs.com/clement/hands-on-help
** https://git.greensocs.com/clement/hands-on-help/blob/master/src/memory.h
 
Implement a <code>b_transport</code> function in <code>memory.h</code>, and register it on the <code>socket</code>.
 
Note: a good memory would support endianness, and take notice of the streaming width. In our case, we will keep things simple and support 1, 2, 4 and 8 byte words only. We’ll be running a Little Endian machine on a Little Endian host, so you can ignore Endianness, and we wont worry about streaming width.
 
Once you’ve finished implementing the b_transport, re-build the platform with <code>make</code> and run it.
 
If the memory test passes, you should see a success, which means your memory is working!
 
Take a look at the memory-tester.h code, to see what it’s testing in your memory. One of the test if a full memory write followed by a full memory read.
 
Now measure the time it takes to run your virtual platform with the following command:
 
<pre>time ./vp</pre>
Note the time somewhere, we will compare with the DMI approach later.
 
===== Implement DMI =====
 
Now implement <code>get_direct_mem_ptr</code> DMI function in <code>memory.h</code>, and register it on the <code>socket</code>.
 
Don’t forget to set the DMI hint in Memory::b_transport, so the initiator knows the memory supports DMI.
 
Build the platform with <code>make</code> and run it.
 
You should see a speed up.
 
We can now construct a simple system using the memory you just implemented!
 
==== 2.3 Add a CPU ====
 
[[File:cpu-1.png]]
 
===== Clone QBox and dependencies =====
 
First we need to clone qbox, tlm2c, greenlib and simple_cpu.
 
Navigate to <code>libs</code> directory and clone the following projects:
 
<pre>cd ~/hands-on-vp/libs
git clone --depth 1 https://git.greensocs.com/tlm/tlm2c.git
git clone --depth 1 https://git.greensocs.com/greenlib/greenlib.git
git clone --depth 1 https://git.greensocs.com/models/simple_cpu.git
git clone --depth 1 https://git.greensocs.com/qemu/qbox.git</pre>
It is possible, and much faster, to only clone the last commit without the full history using option <code>--depth 1</code> right after the <code>clone</code> command.
 
Your <code>libs</code> directory should look like this: [[File:libs.png]]
 
TLM2C is the adapter between Qemu and SystemC, while Simple_cpu wraps the result and provides the actual TLM model. GreenLib contains the critical infrastructure items we’ll be using, and QBox is the actual Qemu itself.
 
Edit the cmake file in <code>libs</code> directory to tell CMake that new projects were added. Add the following lines in the currently empty <code>libs/CMakeLists.txt</code> [[File:CMakeLists.png]]
 
Now you can re-build the platform with the<code>make</code> command executed from <code>build</code> directory. CMake should detect the changes in the configuration and build the new subprojects you added.
 
===== Modify the platform =====
 
Edit the file <code>src/main.cc</code>.
 
Remove the MemoryTester, and add a Cortex-A53 CPU:
 
For instantiating the CPU you need the following include: <code>#include &quot;SimpleCPU/arm/armCortexA53CPU.h&quot;</code>
 
The Cortex-A53 class is called ArmCortexA53CPU. The class is templated with BUSWIDTH. We will use a BUSWIDTH of 32 during this course.
 
Instantiate an ArmCortexA53CPU&lt;32&gt;, named <code>&quot;cpu&quot;</code>, and connect it directly to the memory as pictured above. The cpu socket is named master_socket (Initiator) :
 
<source lang="c">ArmCortexA53CPU<32> *cpu = new ArmCortexA53CPU<32>("cpu");
cpu->master_socket(memory->socket);</source>
It is necessary to add the dependency on this new components in the original top level <code>CMakeLists.txt</code> : [[File:simplecpu.png]]
 
Build the platform to check that there is no syntax error in the code.
 
===== Configure the CPU =====
 
The CPU contains some parameters (gs_params) that you can set either programmatically or by using a Lua description. For example the number of cores is a CPU parameter named <code>ncores</code>.
 
GreenLib provides a Lua parser to simplify the configuration of the parameters.
 
You can find more information about Lua on lua.org.
 
First, we need to add the Lua parser to our code:
 
<pre>#include &quot;greencontrol/config_api_lua_file_parser.h&quot;</pre>
<pre>gs::cnf::LuaFile_Tool luareader(&quot;luareader&quot;);
if (luareader.config(&quot;../platform.lua&quot;) != 0) {
    printf(&quot;lua file not found.\n&quot;);
    exit(EXIT_FAILURE);
}</pre>
This code will load the lua file from <code>../platform.lua</code>. Note the <code>..</code> as we are executing our <code>vp</code> from <code>./build</code> folder, and we want to use the <code>platform.lua</code> in the base directory.
 
Create an empty <code>platform.lua</code> in the base directory, and add the following content:
 
<pre>cpu = {
    library = &quot;libs/qbox/build/aarch64-softmmu/libqbox-cortex-a53.so&quot;,
    kernel = &quot;../fw/hello.elf&quot;,
    ncores = 1,
    cores = {
        { AA64nAA32 = 1, CfgEnd = 0, CfgTe = 0, Rvbaraddr = 0, Vinithi = 0 },
    },
}</pre>
The name ‘cpu’ is the name you gave to the Qbox instance. The library is the qemu library code that will be loaded, and the kernel is the file that qemu will be told to load into memory for you. We’re asking for 1 core, which we configure.
 
Build the platform as before, and this time run it again. The hello.elf firmware will attempt to write to a UART (which for now is not there).
 
To exit the simulation hit ctrl+c
 
Your terminal might be a bit broken because Qemu modifies a bit the configuration. If you experience the issue, use the <code>reset</code> command to set it back to normal.
 
==== 2.4 Add a Router and a UART ====
 
[[File:platform-1.png]]
 
===== Add a router =====
 
GreenLib provides a simple router implementation, we are going to use it in our platform.
 
Include:
 
<pre>#include &quot;greenrouter/simpleRouter.h&quot;</pre>
The router class is SimpleRouter. The router’s sockets are named <code>init_socket</code> (Initiator) and <code>target_socket</code> (Target).
 
Now you can:
 
* Instantiate a router SimpleRouter&lt;32&gt; named router
* Bind the cpu initiator socket to the router target socket
* Bind the memory target socket to the router initiator socket
 
Once the memory is bound, we need to tell the router where the memory is mapped on the Bus, you can do this by calling the following router function '''right after''' the socket binding:
 
<source lang="cpp">Memory *memory = new Memory("memory", 0x40000000);
router->init_socket(memory->socket)
router->assign_address(0, memory->size());</source>
It will map the memory in the range [0, memory-size].
 
===== Add a UART =====
 
First we need to clone the pl011 project.
 
Navigate to <code>libs</code> directory and clone the following project:
 
<pre>https://git.greensocs.com/models/pl011.git</pre>
Your <code>libs</code> directory should look like this: [[File:libs-2.png]]
 
Add the new subproject in the CMakeLists.txt
 
Now you can:
 
* Instantiate a PL011 named pl011
* Bind the pl011 target socket to the router initiator socket
* Bind the pl011 irq socket to the cpu irq socket
 
<pre>#include &quot;PL011/PL011.h&quot;
 
[...]
 
PL011 *pl011 = new PL011(&quot;pl011&quot;);
pl011-&gt;irq_socket(cpu-&gt;irq_socket);
router-&gt;init_socket(pl011-&gt;target_port);</pre>
Once the pl011 is bound, we need to tell the router where it is mapped on the Bus, you can do this by modifying the lua file. NOTE: This is a more elegant way of configuring the address of a component. The gs_params are within the socket (which is a greensocs tlm-2 socket), and will be automatically picked up by the router.
 
<pre>pl011 = {
    irq_number = 1,
    target_port = {
        base_addr = 0xC0000000,
        high_addr = 0xC0001000
    }
}</pre>
It will map the pl011 in the range [0xC0000000, 0xC0001000].
 
It is necessary to add the dependency on this new components in the original top level <code>CMakeLists.txt</code> : [[File:pl011.png]]
 
===== Add a backend to the UART =====
 
The PL011 model needs what we call a <code>backend</code> in order to output the characters.
 
GreenLib provides a <code>ncurses</code> based serial backend, you can create it and connect it to the pl011 using the following code:
 
<pre>#include &quot;greenserialsocket/backends/ncurses_serial_backend.h&quot;
[...]
StdioSerial *backend = new StdioSerial(&quot;serial&quot;);
pl011-&gt;serial_sock(backend-&gt;socket);</pre>
Now re-run the platform, this time the hello.elf firmware should output a message to the UART (which you should see on the terminal).
 
=== 3. Build and run a simple firmware ===
 
In this step we are going to create a simple firmware and run it on our platform.
 
First we need an aarch64 cross compiler in order to compile the firmware.
Hopefully, we installed one at the very beginning of this tutorial.
 
Check that it works with:
<pre>aarch64-linux-gnu-gcc --version</pre>
Now download the firmware skeleton from:
 
<pre>wget https://darwin.greensocs.com/s/kWxJm49PxWm5sYM/download -O skel.tar.gz
</pre>
Explore the different files (Makefile, sources, linker script)
 
You need to:
 
* initialize the stack pointer in boot.s (hope you know a bit of armv8 asm)
* jump to c_entry() from boot.s
* create a kernel.c file containing the c_entry() function
* write a print() function, that print to the UART (this is pure C, just assume that the UART fifo stands at the base address of the UART).
 
Build the firwmare and run it on your platform.
 
=== 4. Build advanced firmware ===
 
The little hello firmware is fine for testing, but now we will move to larger applications.
 
For that we will use a C Standard Library (libc). The C standard library provides macros, type definitions and functions for tasks such as string handling, mathematical computations, input/output processing, memory management, and several other operating system services.
 
As an example we will use a baremetal dhrystone, built using libc. So first we must build libc for the target.
 
==== 4.1 C standard library (libc) ====
 
In this chapter, we are hoing to see how to build <code>Newlib</code>.
 
Newlib is a C library intended for use on embedded systems. It is a conglomeration of several library parts, all under free software licenses that make them easily usable on embedded products.
 
Newlib is only available in source form. It can be compiled for a wide array of processors, and will usually work on any architecture with the addition of a few low-level routines.
 
Download Newlib version 2.5.0.201712 from Newlib website.
 
Extract the archive, and build newlib with:
 
<pre>./configure --target aarch64-elf --prefix=$PWD/prefix
make
make install</pre>
===== Use newlib in your firmware =====
 
We need to modify the Makefile to include libc headers and to link with libc.
 
The includes are in the <newlib-install-prefix>/include.
 
The library to link against is libc.a and is located in <newlib-install-prefix>/lib
 
Add a call to printf in your firmware, and build it.
 
Do you see a linker issue ?
 
You’re missing some function that are required for newlib to work, we’ll explain that in the next section.
 
===== System Calls =====
 
The C subroutine library depends on a handful of subroutine calls for operating system services.
 
We need to implement some of these subroutines.
 
More information here : http://www.sourceware.org/newlib/libc.html#Syscalls
 
Create a file named syscalls.c and add it to the sources files in your Makefile.
 
You need to implement:
 
* _close
* _fstat
* _isatty
* _lseek
* _read
* _write
* _sbrk
 
For <code>_close</code>, <code>_fstat</code>, <code>_isatty</code>, <code>_lseek</code>, <code>_read</code> you can use the stubs provided on Newlib Syscalls page linked above.
 
For <code>_write</code>, you can use the stub provided on Newlib Syscalls page, but you’ll need to declare a subroutine named <code>void outbyte(unsigned char b)</code> that will output the character toward the pl011 uart.
 
For <code>_sbrk</code>, you need to write a small allocator, you could for example allocate a static byte array and use it as a memory pool, so _sbrk would return pointers inside that array.
 
Once you have implemented syscalls.c, you can build and run your program, the printf function provided by newlib should output to the console (where the pl011 writes).
 
==== 4.2 Dhrystone ====
 
Now that we have a libc, porting an existing application is easier.
 
In this section, we will port the original dhrystone application.
 
Dhrystone is a popular benchmark program developed in 1984.
 
You can find more information about dhrystone here : https://en.wikipedia.org/wiki/Dhrystone
 
Download original dhrystone sources from:
 
<pre>https://darwin.greensocs.com/s/FYWqXG5MqkJfMmz</pre>
Extract the archive and add the needed dhrystone files to your Makefile.
 
Dhrystone uses scanf to request from the user the number of iterations to run. The <code>scanf</code> function provided by newlib uses the <code>_read</code> function we stubbed earlier in <code>syscalls.c</code>. It is now time to implement this <code>_read</code> function, by waiting for an available character in the pl011 and returning the read character.
 
Build. You should have a link error.
 
Dhrystone also uses the function <code>gettimeofday</code> to measure the time of execution. The <code>gettimeofday</code> function provided by newlib calls <code>_gettimeofday</code> that we need to implement in <code>syscalls.c</code>. If you have time, implement a SystemC component that returns the SystemC time on register read. Or you can stub this function for now (in this case the time printed at the end will be wrong).
 
Build and run.
 
Thanks to newlib, the porting of dhrystone is pretty easy.
 
==== 4.3 Linux ====
 
Now that we have seen how to build a minimal libc and link a bare metal application to it, we will see how to build more complex applications such as an operating system. The advantage of running an operating system such as linux on a virtual platform is that it allows to test a lot of things (instructions, modes, locks, timers, etc …), plus linux provides a lot of tool for testing a platform. Additionnaly, there are a lot of benchmark application running on top of linux that we can then run easily.
 
In this section, we will build a minimal linux kernel to run on our platform.
 
===== Build the kernel =====
 
Start by downloading the latest stable kernel archive from <code>https://www.kernel.org/</code>.
 
Extract the archive, and prepare the environment:
 
<pre>export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-</pre>
Configure the tiniest possible kernel with:
 
<pre>make tinyconfig</pre>
Now we need to add a few options needed for linux to print messages in our platform.
 
Linux provides a ncurses based interface for modifying the configuration:
 
<pre>make menuconfig</pre>
We need to :
 
* Enable support for printk
* Enable TTY
* Enable ARM AMBA PL011 serial port support
* Enable Support for console on AMBA serial port
 
When you have enabled these 4 options, you can exit and save the config.
 
You can now build the kernel with the <code>make</code> command.
 
After the compilation, you should see your kernel image in <code>arch/arm64/boot/Image</code>.
 
This is the image that we are going to load into QBox.
 
But first we need to write a device tree.
 
===== Device Tree =====
 
The Device Tree is a data structure for describing hardware. Rather than hard coding every detail of a device into an operating system, many aspect of the hardware can be described in a data structure that is passed to the operating system at boot time. For example the memory mapping of a platform is passed to the kernel using a device tree loaded in memory by the bootloader.
 
More information about Device Tree can be found here: https://www.kernel.org/doc/Documentation/devicetree/usage-model.txt
 
In this section we will write a Device Tree Source (dts file), that describes our platform.
 
Start by downloading the skeleton of the device tree:
 
<pre>wget https://darwin.greensocs.com/s/3otDLnbFHLrZfgt/download -O gsboard-skel.dts</pre>
Edit the dts file and set the correct memory mapping for your memory and your uart.
 
Then compile your dts into a dtb (linux kernel understands dtb only).
 
<pre>dtc -I dts -O dtb -o gsboard-skel.dtb gsboard-skel.dts</pre>
Now edit your platform.lua to point <code>cpu.kernel</code> to your built kernel Image, and add new parameter called <code>cpu.dtb</code> with the path to your dtb file.
 
Run the simulation, you should see your kernel start!
 
Your kernel should panic beause it lacks a root filesystem to execute (When linux kernel has finished the initialization, it tries to acess the root filesystem and execute a script called <code>/init</code>).
 
==== 4.4 Buildroot ====
 
We are going to build a minimal filesystem containing busybox, and dhrystone as a userland application.
 
Download buildroot 2018.02.6 archive from buildroot.org
 
Extract the archive.
 
===== Build buildroot =====
 
Buildroot use an interface similar to linux for configuring the options:
 
<pre>make menuconfig</pre>
We need to :
 
* Set the Target Architecture to AArch64 (little endian)
* cpio the root filesystem (for use as an initial RAM filesystem)
** and compress it using gzip
* Enable Dhrystone in Target Packages
 
When you have enabled these options, you can exit and save the config.
 
Now we can build buildroot using <code>make</code> command.
 
Buildroot outputs the rootfs in <code>output/images/rootfs.cpio.gz</code>.
 
===== Piggyback the rootfs on the kernel =====
 
Our platform is very simple and does not have any SD card controller or any SATA controller, which would help us mounting a filesystem.
 
So we are going to use the INITRAMS option of the kernel: our root filesystem will be piggybacked on the kernel Image, and loaded in memory.
 
Enable the INITRAMS option in your kernel, and point to the rootfs.cpio.gz you just built in the previous section.
 
Build, you should see that your kernel Image is a bit bigger than earlier.
 
Run on your platform, login as root, and run dhrystone in userland.
 
Congratulations you have a simple platform running a minimal linux kernel with a minimal root filesystem!

Latest revision as of 17:05, 16 July 2019

  • Speaker(s): Frédéric Pétrot (stepping in in place of Mark Burton, GreenSocs CEO)
  • email: frederic.petrot@univ-grenoble-alpes.fr (mark.burton@greensocs.com)
  • web: https://www.greensocs.com

Downloads

Abstract

GreenSocs® was formed in 2005 to offer a complete range of solutions for the implementation of Model Based Design Engineering for embedded systems. Those solutions range from implementation of the design cycle to the delivery of pre-developed models through support of modeling teams of customers and to complete outsourcing of modeling. We make use of (but not exclusively) open source tools for modeling (GreenSocs Open Source Technology) which multiplies the potential of SystemC. The GreenSocs mission is to enable the ESL community to quickly develop models and tools that can be used together with independence of vendor (whether the vendor is of models or tools). Our scope includes everything from package management for ESL, simple IP blocks, integration's with scripting tools and of course interfaces. GreenSocs aquired Antfield in 2018, brining together the very best in Qemu and SystemC technologies.

During this tutorial, we will show a simple example of QEMU+SystemC integration of a HW/SW system using the current, publicly available, GreenSocs integration solution.

Tutorial

Thanks to Clement Deschamps <clement.deschamps@greensocs.com> for this great tutorial!

Activity 1 - Build a Simple System

Overview:

  • Create a simple virtual platform
    • Implement a simple memory (with DMI support)
    • Add a CPU
    • Add a UART
  • Build and run a simple firmware
  • Build more advanced firmware
    • Add a libc and run Dhrystone
    • Build and run Linux
    • Create a minimal filesystem using Builroot

1. Presentation

During this course, we will build the following simple virtual platform.

We will then use it to explore some of the features of SystemC and QBox.

Platform-1.png

The virtual platform will contain the following components:

  • ARM Cortex A57 CPU (provided by QBox)
  • PL011 UART (source code provided)
  • Simple memory (skeleton provided)

2. Create a simple virtual platform

2.1 Prepare environment

This tutorial requires either Debian (8 or 9), or Ubuntu (16.04 or 18.04).

If you are not using one of these distributions, or if you want a clean environment, you can create a temporary docker container with (first install the package docker.io if docker is not yet present on your system):

docker run --name gs-tuto -d -i -t --cap-add=SYS_PTRACE ubuntu:18.04
docker attach gs-tuto
<ENTER>
apt update && apt install sudo
cd $HOME

Interestingly enough, if you're using docker, the tutorial will be executed while being `root`. This is quite unusual, but funny!

Install required Ubuntu packages:

sudo apt install -y \
     bison flex cmake wget git htop device-tree-compiler socat gcc-aarch64-linux-gnu\
     libboost-dev libboost-filesystem-dev libfdt-dev libglib2.0-dev libgtk-3-dev liblua5.2-dev \
     libncurses5-dev libpixman-1-dev python-dev swig texi2html zlib1g-dev tree vim gdb bc

Build SystemC 2.3.1a and install it in /opt/systemc-2.3.1a

wget http://www.accellera.org/images/downloads/standards/systemc/systemc-2.3.1a.tar.gz
tar -xf systemc-2.3.1a.tar.gz
sed -i -e 's/auto_ptr/unique_ptr/g' systemc-2.3.1a/src/sysc/packages/boost/get_pointer.hpp
sed -i -e '/using std::gets/d' systemc-2.3.1a/src/systemc.h
mkdir systemc-2.3.1a/objdir
pushd systemc-2.3.1a/objdir
../configure --prefix=/opt/systemc-2.3.1a
make -j$(nproc)
sudo make install
popd

2.2 Implement a simple Memory

The objective is to implement a very simple TLM memory, supporting Loosely Timed (LT) b_transport, and Direct Memory Access (DMI).

The memory will be tested with a MemoryTester component that we provide.

Memory-1.png

Please download the project skeleton from:

wget https://darwin.greensocs.com/s/YF9Bk3yFEdyEqnN/download -O hands-on-vp.tar.gz
tar xvf hands-on-vp.tar.gz

The project contains the following tree: Memory-tree.png

  • cmake/ folder contains a script for finding SystemC
  • CMakeLists.txt is the cmake description of the project
  • src/main.cc contains the sc_main() that instantiates a Memory and a MemoryTester
  • src/memory.h is the skeleton of the memory you need to implement
  • src/memory-tester.h is a sc_module for testing the memory

We provide a SystemC module named MemoryTester that will access your memory through TLM transactions, and through DMI.

Build the project

The build process is based on CMake, and takes place in two stages. First, standard build files are created from configuration files. Then the platform’s native build tools (gcc, binutils, …) are used for the actual building.

cd hands-on-vp
mkdir build
cd build
cmake -DSYSTEMC_PREFIX=/opt/systemc-2.3.1a ..

And finally, build the rather "thin" virtual platform using make command.

Please make sure you run cmake into the build directory, otherwise strange things appear later on. (As you may have noticed during the 'live' run.)

This will generate an executable file named vp.

Run

Simply execute the vp binary file.

You should see an error as the Memory component is not fully implemented (it’s only a stub):

Memory-error-1.png

Implement LT transport

Implement a b_transport function in memory.h, and register it on the socket.

Note: a good memory would support endianness, and take notice of the streaming width. In our case, we will keep things simple and support 1, 2, 4 and 8 byte words only. We’ll be running a Little Endian machine on a Little Endian host, so you can ignore Endianness, and we wont worry about streaming width.

Once you’ve finished implementing the b_transport, re-build the platform with make and run it.

If the memory test passes, you should see a success, which means your memory is working!

Take a look at the memory-tester.h code, to see what it’s testing in your memory. One of the test if a full memory write followed by a full memory read.

Now measure the time it takes to run your virtual platform with the following command:

time ./vp

Note the time somewhere, we will compare with the DMI approach later.

Implement DMI

Now implement get_direct_mem_ptr DMI function in memory.h, and register it on the socket.

Don’t forget to set the DMI hint in Memory::b_transport, so the initiator knows the memory supports DMI.

Build the platform with make and run it.

You should see a speed up.

We can now construct a simple system using the memory you just implemented!

2.3 Add a CPU

Cpu-1.png

Clone QBox and dependencies

First we need to clone qbox, tlm2c, greenlib and simple_cpu.

Navigate to libs directory and clone the following projects:

cd ~/hands-on-vp/libs
git clone --depth 1 https://git.greensocs.com/tlm/tlm2c.git
git clone --depth 1 https://git.greensocs.com/greenlib/greenlib.git
git clone --depth 1 https://git.greensocs.com/models/simple_cpu.git
git clone --depth 1 https://git.greensocs.com/qemu/qbox.git

It is possible, and much faster, to only clone the last commit without the full history using option --depth 1 right after the clone command.

Your libs directory should look like this: Libs.png

TLM2C is the adapter between Qemu and SystemC, while Simple_cpu wraps the result and provides the actual TLM model. GreenLib contains the critical infrastructure items we’ll be using, and QBox is the actual Qemu itself.

Edit the cmake file in libs directory to tell CMake that new projects were added. Add the following lines in the currently empty libs/CMakeLists.txt CMakeLists.png

Now you can re-build the platform with themake command executed from build directory. CMake should detect the changes in the configuration and build the new subprojects you added.

Modify the platform

Edit the file src/main.cc.

Remove the MemoryTester, and add a Cortex-A53 CPU:

For instantiating the CPU you need the following include: #include "SimpleCPU/arm/armCortexA53CPU.h"

The Cortex-A53 class is called ArmCortexA53CPU. The class is templated with BUSWIDTH. We will use a BUSWIDTH of 32 during this course.

Instantiate an ArmCortexA53CPU<32>, named "cpu", and connect it directly to the memory as pictured above. The cpu socket is named master_socket (Initiator) :

ArmCortexA53CPU<32> *cpu = new ArmCortexA53CPU<32>("cpu");
cpu->master_socket(memory->socket);

It is necessary to add the dependency on this new components in the original top level CMakeLists.txt : Simplecpu.png

Build the platform to check that there is no syntax error in the code.

Configure the CPU

The CPU contains some parameters (gs_params) that you can set either programmatically or by using a Lua description. For example the number of cores is a CPU parameter named ncores.

GreenLib provides a Lua parser to simplify the configuration of the parameters.

You can find more information about Lua on lua.org.

First, we need to add the Lua parser to our code:

#include "greencontrol/config_api_lua_file_parser.h"
gs::cnf::LuaFile_Tool luareader("luareader");
if (luareader.config("../platform.lua") != 0) {
    printf("lua file not found.\n");
    exit(EXIT_FAILURE);
}

This code will load the lua file from ../platform.lua. Note the .. as we are executing our vp from ./build folder, and we want to use the platform.lua in the base directory.

Create an empty platform.lua in the base directory, and add the following content:

cpu = {
    library = "libs/qbox/build/aarch64-softmmu/libqbox-cortex-a53.so",
    kernel = "../fw/hello.elf",
    ncores = 1,
    cores = {
        { AA64nAA32 = 1, CfgEnd = 0, CfgTe = 0, Rvbaraddr = 0, Vinithi = 0 },
    },
}

The name ‘cpu’ is the name you gave to the Qbox instance. The library is the qemu library code that will be loaded, and the kernel is the file that qemu will be told to load into memory for you. We’re asking for 1 core, which we configure.

Build the platform as before, and this time run it again. The hello.elf firmware will attempt to write to a UART (which for now is not there).

To exit the simulation hit ctrl+c

Your terminal might be a bit broken because Qemu modifies a bit the configuration. If you experience the issue, use the reset command to set it back to normal.

2.4 Add a Router and a UART

Platform-1.png

Add a router

GreenLib provides a simple router implementation, we are going to use it in our platform.

Include:

#include "greenrouter/simpleRouter.h"

The router class is SimpleRouter. The router’s sockets are named init_socket (Initiator) and target_socket (Target).

Now you can:

  • Instantiate a router SimpleRouter<32> named router
  • Bind the cpu initiator socket to the router target socket
  • Bind the memory target socket to the router initiator socket

Once the memory is bound, we need to tell the router where the memory is mapped on the Bus, you can do this by calling the following router function right after the socket binding:

Memory *memory = new Memory("memory", 0x40000000);
router->init_socket(memory->socket)
router->assign_address(0, memory->size());

It will map the memory in the range [0, memory-size].

Add a UART

First we need to clone the pl011 project.

Navigate to libs directory and clone the following project:

https://git.greensocs.com/models/pl011.git

Your libs directory should look like this: Libs-2.png

Add the new subproject in the CMakeLists.txt

Now you can:

  • Instantiate a PL011 named pl011
  • Bind the pl011 target socket to the router initiator socket
  • Bind the pl011 irq socket to the cpu irq socket
#include "PL011/PL011.h"

[...]

PL011 *pl011 = new PL011("pl011");
pl011->irq_socket(cpu->irq_socket);
router->init_socket(pl011->target_port);

Once the pl011 is bound, we need to tell the router where it is mapped on the Bus, you can do this by modifying the lua file. NOTE: This is a more elegant way of configuring the address of a component. The gs_params are within the socket (which is a greensocs tlm-2 socket), and will be automatically picked up by the router.

pl011 = {
    irq_number = 1,
    target_port = {
        base_addr = 0xC0000000,
        high_addr = 0xC0001000
    }
}

It will map the pl011 in the range [0xC0000000, 0xC0001000].

It is necessary to add the dependency on this new components in the original top level CMakeLists.txt : Pl011.png

Add a backend to the UART

The PL011 model needs what we call a backend in order to output the characters.

GreenLib provides a ncurses based serial backend, you can create it and connect it to the pl011 using the following code:

#include "greenserialsocket/backends/ncurses_serial_backend.h"
[...]
StdioSerial *backend = new StdioSerial("serial");
pl011->serial_sock(backend->socket);

Now re-run the platform, this time the hello.elf firmware should output a message to the UART (which you should see on the terminal).

3. Build and run a simple firmware

In this step we are going to create a simple firmware and run it on our platform.

First we need an aarch64 cross compiler in order to compile the firmware. Hopefully, we installed one at the very beginning of this tutorial.

Check that it works with:

aarch64-linux-gnu-gcc --version

Now download the firmware skeleton from:

wget https://darwin.greensocs.com/s/kWxJm49PxWm5sYM/download -O skel.tar.gz

Explore the different files (Makefile, sources, linker script)

You need to:

  • initialize the stack pointer in boot.s (hope you know a bit of armv8 asm)
  • jump to c_entry() from boot.s
  • create a kernel.c file containing the c_entry() function
  • write a print() function, that print to the UART (this is pure C, just assume that the UART fifo stands at the base address of the UART).

Build the firwmare and run it on your platform.

4. Build advanced firmware

The little hello firmware is fine for testing, but now we will move to larger applications.

For that we will use a C Standard Library (libc). The C standard library provides macros, type definitions and functions for tasks such as string handling, mathematical computations, input/output processing, memory management, and several other operating system services.

As an example we will use a baremetal dhrystone, built using libc. So first we must build libc for the target.

4.1 C standard library (libc)

In this chapter, we are hoing to see how to build Newlib.

Newlib is a C library intended for use on embedded systems. It is a conglomeration of several library parts, all under free software licenses that make them easily usable on embedded products.

Newlib is only available in source form. It can be compiled for a wide array of processors, and will usually work on any architecture with the addition of a few low-level routines.

Download Newlib version 2.5.0.201712 from Newlib website.

Extract the archive, and build newlib with:

./configure --target aarch64-elf --prefix=$PWD/prefix
make
make install
Use newlib in your firmware

We need to modify the Makefile to include libc headers and to link with libc.

The includes are in the <newlib-install-prefix>/include.

The library to link against is libc.a and is located in <newlib-install-prefix>/lib

Add a call to printf in your firmware, and build it.

Do you see a linker issue ?

You’re missing some function that are required for newlib to work, we’ll explain that in the next section.

System Calls

The C subroutine library depends on a handful of subroutine calls for operating system services.

We need to implement some of these subroutines.

More information here : http://www.sourceware.org/newlib/libc.html#Syscalls

Create a file named syscalls.c and add it to the sources files in your Makefile.

You need to implement:

  • _close
  • _fstat
  • _isatty
  • _lseek
  • _read
  • _write
  • _sbrk

For _close, _fstat, _isatty, _lseek, _read you can use the stubs provided on Newlib Syscalls page linked above.

For _write, you can use the stub provided on Newlib Syscalls page, but you’ll need to declare a subroutine named void outbyte(unsigned char b) that will output the character toward the pl011 uart.

For _sbrk, you need to write a small allocator, you could for example allocate a static byte array and use it as a memory pool, so _sbrk would return pointers inside that array.

Once you have implemented syscalls.c, you can build and run your program, the printf function provided by newlib should output to the console (where the pl011 writes).

4.2 Dhrystone

Now that we have a libc, porting an existing application is easier.

In this section, we will port the original dhrystone application.

Dhrystone is a popular benchmark program developed in 1984.

You can find more information about dhrystone here : https://en.wikipedia.org/wiki/Dhrystone

Download original dhrystone sources from:

https://darwin.greensocs.com/s/FYWqXG5MqkJfMmz

Extract the archive and add the needed dhrystone files to your Makefile.

Dhrystone uses scanf to request from the user the number of iterations to run. The scanf function provided by newlib uses the _read function we stubbed earlier in syscalls.c. It is now time to implement this _read function, by waiting for an available character in the pl011 and returning the read character.

Build. You should have a link error.

Dhrystone also uses the function gettimeofday to measure the time of execution. The gettimeofday function provided by newlib calls _gettimeofday that we need to implement in syscalls.c. If you have time, implement a SystemC component that returns the SystemC time on register read. Or you can stub this function for now (in this case the time printed at the end will be wrong).

Build and run.

Thanks to newlib, the porting of dhrystone is pretty easy.

4.3 Linux

Now that we have seen how to build a minimal libc and link a bare metal application to it, we will see how to build more complex applications such as an operating system. The advantage of running an operating system such as linux on a virtual platform is that it allows to test a lot of things (instructions, modes, locks, timers, etc …), plus linux provides a lot of tool for testing a platform. Additionnaly, there are a lot of benchmark application running on top of linux that we can then run easily.

In this section, we will build a minimal linux kernel to run on our platform.

Build the kernel

Start by downloading the latest stable kernel archive from https://www.kernel.org/.

Extract the archive, and prepare the environment:

export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-

Configure the tiniest possible kernel with:

make tinyconfig

Now we need to add a few options needed for linux to print messages in our platform.

Linux provides a ncurses based interface for modifying the configuration:

make menuconfig

We need to :

  • Enable support for printk
  • Enable TTY
  • Enable ARM AMBA PL011 serial port support
  • Enable Support for console on AMBA serial port

When you have enabled these 4 options, you can exit and save the config.

You can now build the kernel with the make command.

After the compilation, you should see your kernel image in arch/arm64/boot/Image.

This is the image that we are going to load into QBox.

But first we need to write a device tree.

Device Tree

The Device Tree is a data structure for describing hardware. Rather than hard coding every detail of a device into an operating system, many aspect of the hardware can be described in a data structure that is passed to the operating system at boot time. For example the memory mapping of a platform is passed to the kernel using a device tree loaded in memory by the bootloader.

More information about Device Tree can be found here: https://www.kernel.org/doc/Documentation/devicetree/usage-model.txt

In this section we will write a Device Tree Source (dts file), that describes our platform.

Start by downloading the skeleton of the device tree:

wget https://darwin.greensocs.com/s/3otDLnbFHLrZfgt/download -O gsboard-skel.dts

Edit the dts file and set the correct memory mapping for your memory and your uart.

Then compile your dts into a dtb (linux kernel understands dtb only).

dtc -I dts -O dtb -o gsboard-skel.dtb gsboard-skel.dts

Now edit your platform.lua to point cpu.kernel to your built kernel Image, and add new parameter called cpu.dtb with the path to your dtb file.

Run the simulation, you should see your kernel start!

Your kernel should panic beause it lacks a root filesystem to execute (When linux kernel has finished the initialization, it tries to acess the root filesystem and execute a script called /init).

4.4 Buildroot

We are going to build a minimal filesystem containing busybox, and dhrystone as a userland application.

Download buildroot 2018.02.6 archive from buildroot.org

Extract the archive.

Build buildroot

Buildroot use an interface similar to linux for configuring the options:

make menuconfig

We need to :

  • Set the Target Architecture to AArch64 (little endian)
  • cpio the root filesystem (for use as an initial RAM filesystem)
    • and compress it using gzip
  • Enable Dhrystone in Target Packages

When you have enabled these options, you can exit and save the config.

Now we can build buildroot using make command.

Buildroot outputs the rootfs in output/images/rootfs.cpio.gz.

Piggyback the rootfs on the kernel

Our platform is very simple and does not have any SD card controller or any SATA controller, which would help us mounting a filesystem.

So we are going to use the INITRAMS option of the kernel: our root filesystem will be piggybacked on the kernel Image, and loaded in memory.

Enable the INITRAMS option in your kernel, and point to the rootfs.cpio.gz you just built in the previous section.

Build, you should see that your kernel Image is a bit bigger than earlier.

Run on your platform, login as root, and run dhrystone in userland.

Congratulations you have a simple platform running a minimal linux kernel with a minimal root filesystem!