Refine the docs.
Change-Id: I6808bb7515e39f52053d66af54e5ad2a45b1c159
diff --git a/README.md b/README.md
index a15bee1..5b9ea5f 100644
--- a/README.md
+++ b/README.md
@@ -1,168 +1,15 @@
# Hafnium
+[go/hafnium](https://goto.google.com/hafnium)
+
Hafnium is a hypervisor, initially supporting aarch64 (64-bit ARMv8 CPUs).
Further details are available in the [design doc](https://goto.google.com/hafnium-dd).
-## Getting the source code
+Get in touch and keep up-to-date at
+[g/hafnium-eng](https://groups.google.com/a/google.com/forum/#!forum/hafnium-eng).
-``` shell
-git clone --recurse-submodules sso://hafnium/hafnium && (cd hafnium && f=`git rev-parse --git-dir`/hooks/commit-msg ; curl -Lo $f https://gerrit-review.googlesource.com/tools/hooks/commit-msg ; chmod +x $f)
-```
+## Getting started
-To upload a change for review:
-
-``` shell
-git push origin HEAD:refs/for/master
-```
-
-
-## Building
-
-### Compiling the hypervisor
-
-By default, the hypervisor is built for an aarch64 QEMU target by running:
-
-``` shell
-make
-```
-
-To build for the HiKey board, change the target platform:
-
-``` shell
-PLATFORM=hikey make
-```
-
-To build using gcc instead of clang, the aarch64 variant must be installed:
-
-``` shell
-sudo apt install gcc-aarch64-linux-gnu
-GCC=true make
-```
-
-The compiled binary is stored in `out/{clang,gcc}_aarch64/hafnium.bin`.
-
-## Running on QEMU
-
-You will need at least version 2.9 for QEMU. The following command line can be
-used to run Hafnium on it:
-
-``` shell
-qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -machine virtualization=true -kernel out/aarch64/qemu/hafnium.bin
-```
-
-Though it is admittedly not very useful because it doesn't have any virtual
-machines to run. The next sections describe how to build an image that can be
-run using the following command-line:
-
-``` shell
-qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -machine virtualization=true -kernel out/aarch64/qemu/hafnium.bin -initrd initrd.img -append "rdinit=/sbin/init"
-```
-
-## Hafnium RAM disk
-
-Hafnium expects to find the following files in the root directory of its ramdisk:
-
-* `vmlinuz` -- the kernel of the primary VM.
-* `initrd.img` -- the initial ramdisk of the primary VM.
-* `vms.txt` -- optionally describes the secondary VMs.
-* kernels for the secondary VMs, whose names vary and are described in `vms.txt`.
-
-### Format of `vms.txt` file
-The format is currently one line per secondary VM, with the following format:
-
-``` shell
-<memory-size-in-bytes> <number-of-cpus> <kernel-filename>
-```
-
-For example, the following defines two secondary VMs, the first one with 1MB of
-memory, 2 CPUs and kernel image called `kernel0`, while the second one has 2MB
-of memory, 4 CPUs and a kernel image called `kernel1`.
-
-``` shell
-1048576 2 kernel0
-2097152 4 kernel1
-```
-
-### Creating a hafnium RAM disk
-
-Assuming that a subdirectory called `initrd` contains the files listed in the
-previous section, we can build `initrd.img` with the following command:
-
-```shell
-cd initrd; find . | cpio -o > ../initrd.img; cd -
-```
-
-## Primary VM
-
-### Kernel
-
-The linux kernel for the primary VM can be built using the following
-command-line:
-
-``` shell
-git clone https://github.com/torvalds/linux.git
-cd linux
-ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make defconfig
-ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j24
-```
-
-The compiled image is stored in `arch/arm64/boot/Image`. This should be copied
-to the Hafnium ramdisk's root as `vmlinuz`.
-
-### Kernel Module
-
-From the hafnium root directory, the following commands can be used to compile
-the kernel module, replacing `<kernel-path>` with the path to the kernel built
-in the previous section:
-
-``` shell
-cd hafnium/driver/linux/
-ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- KERNEL_PATH=<kernel-path> make
-```
-
-The compiled module is called `hafnium.ko`, and should be copied into the
-ramdisk described in the next section.
-
-### Busybox RAM disk
-
-An initial ramdisk for the primary VM containing busybox can be built with the
-following commands:
-
-``` shell
-git clone git://busybox.net/busybox.git
-cd busybox
-ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make defconfig
-ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make menuconfig
-```
-
-At this point you should ensure that the option `Settings > Build static binary
-(no shared libs)` is selected. Then you can proceed with the following commands:
-
-``` shell
-ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j24
-ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make install
-cd _install
-mkdir proc
-mkdir sys
-mkdir -p etc/init.d
-cat <<EOF > etc/init.d/rcS
-#!bin/sh
-mount -t proc none /proc
-mount -t sysfs none /sys
-EOF
-chmod u+x etc/init.d/rcS
-grep -v tty ../examples/inittab > ./etc/inittab
-```
-
-At this point you can copy into the current directory additional files you may
-want in the ramdisk, for example, the kernel module built in the previous
-section. Then run the following commands:
-
-``` shell
-find . | cpio -o -H newc | gzip > ../initrd.img
-cd ..
-```
-
-The resulting file is `initrd.img`. It should be copied to the Hafnium ramdisk's
-root.
+To jump in and build Hafnium, follow the [getting
+started](docs/GettingStarted.md) instructions.
diff --git a/docs/GettingStarted.md b/docs/GettingStarted.md
new file mode 100644
index 0000000..4df0d23
--- /dev/null
+++ b/docs/GettingStarted.md
@@ -0,0 +1,75 @@
+# Getting started
+
+## Getting access
+
+Read access to the source is provided to all Googlers (members of
+`nonconf-mirror@prod`, to be precise). Permission to submit CLs requires
+membership of `hafnium-access@prod`. Full administrative rights are only granted
+to members of `hafnium-admin@prod`.
+
+## Getting the source code
+
+``` shell
+git clone --recurse-submodules sso://hafnium/hafnium && (cd hafnium && f=`git rev-parse --git-dir`/hooks/commit-msg ; curl -Lo $f https://gerrit-review.googlesource.com/tools/hooks/commit-msg ; chmod +x $f)
+```
+
+To upload a commit for review:
+
+``` shell
+git push origin HEAD:refs/for/master
+```
+Browse source at [go/hafnium-repo](https://goto.google.com/hafnium-repo).
+Review CLs at [go/hafnium-review](https://goto.google.com/hafnium-review).
+
+## Compiling the hypervisor
+
+By default, the hypervisor is built with clang for an aarch64 QEMU target by
+running:
+
+``` shell
+make
+```
+
+The compiled image can be found at `out/aarch64/qemu/clang_aarch64/hafnium.bin`.
+
+To build for the HiKey board, change the target platform:
+
+``` shell
+PLATFORM=hikey make
+```
+
+To build using gcc instead of clang, the aarch64 variant must be installed:
+
+``` shell
+sudo apt install gcc-aarch64-linux-gnu
+GCC=true make
+```
+## Running on QEMU
+
+You will need at least version 2.9 for QEMU. The following command line can be
+used to run Hafnium on it:
+
+``` shell
+qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -machine virtualization=true -kernel out/aarch64/qemu/clang_aarch64/hafnium.bin
+```
+
+Though it is admittedly not very useful because it doesn't have any virtual
+machines to run. Follow the [Hafnium RAM disk](HafniumRamDisk.md) instructions
+to create an initial RAM disk for Hafnium with Linux as the primary VM.
+
+The following command line will run Hafnium, with the RAM disk just created,
+which will then boot into the primary Linux VM:
+
+``` shell
+qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -machine virtualization=true -kernel out/aarch64/qemu/clang_aarch64/hafnium.bin -initrd initrd.img -append "rdinit=/sbin/init"
+```
+
+## Running tests
+
+After building, presubmit tests can be run with the following command line:
+
+``` shell
+./kokoro/ubuntu/test.sh
+```
+
+Read about [testing](Testing.md) for more details about the tests.
diff --git a/docs/HafniumRamDisk.md b/docs/HafniumRamDisk.md
new file mode 100644
index 0000000..59925ef
--- /dev/null
+++ b/docs/HafniumRamDisk.md
@@ -0,0 +1,37 @@
+# Hafnium RAM disk
+
+Hafnium expects to find the following files in the root directory of its RAM
+disk:
+
+ * `vmlinuz` -- the kernel of the primary VM.
+ * `initrd.img` -- the initial ramdisk of the primary VM.
+ * `vms.txt` -- optionally describes the secondary VMs.
+ * kernels for the secondary VMs, whose names are described in `vms.txt`.
+
+Follow the [preparing Linux](PreparingLinux.md) instructions to produce
+`vmlinuz` and `initrd.img` for a basic Linux primary VM.
+
+## Format of `vms.txt` file
+The format is currently one line per secondary VM, with the following format:
+
+``` shell
+<memory-size-in-bytes> <number-of-cpus> <kernel-filename>
+```
+
+For example, the following defines two secondary VMs, the first one with 1MB of
+memory, 2 CPUs and kernel image called `kernel0`, while the second one has 2MB
+of memory, 4 CPUs and a kernel image called `kernel1`.
+
+``` shell
+1048576 2 kernel0
+2097152 4 kernel1
+```
+
+## Create a RAM disk for Hafnium
+
+Assuming that a subdirectory called `initrd` contains the files listed in the
+previous section, we can build `initrd.img` with the following command:
+
+```shell
+cd initrd; find . | cpio -o > ../initrd.img; cd -
+```
diff --git a/docs/PreparingLinux.md b/docs/PreparingLinux.md
new file mode 100644
index 0000000..881f2f3
--- /dev/null
+++ b/docs/PreparingLinux.md
@@ -0,0 +1,79 @@
+# Preparing Linux
+
+To boot Linux, a kernel image (`vmlinuz`) and a suitable initial RAM disk
+(`initrd.img`) need to be created.
+
+## Build the kernel
+
+The Linux kernel for the primary VM can be built using the following
+command-line:
+
+``` shell
+git clone https://github.com/torvalds/linux.git
+cd linux
+ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make defconfig
+ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j24
+```
+
+The compiled image is stored in `arch/arm64/boot/Image`. This will later be
+copied to the Hafnium RAM disk's root as `vmlinuz`.
+
+## Build the kernel Module
+
+From the Hafnium root directory, the following commands can be used to compile
+the kernel module, replacing `<kernel-path>` with the path to the kernel checked
+out in the previous section:
+
+``` shell
+cd hafnium/driver/linux/
+ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- KERNEL_PATH=<kernel-path> make
+```
+
+The compiled module is called `hafnium.ko`, and will later be copied into the
+RAM disk for Linux.
+
+## Build Busybox
+
+To make Linux useful, it needs a shell. These following instructions will
+construct a file system for the Linux RAM disk with the BusyBox shell as the
+init process.
+
+``` shell
+git clone git://busybox.net/busybox.git
+cd busybox
+ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make defconfig
+ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make menuconfig
+```
+
+At this point you should ensure that the option `Settings > Build static binary
+(no shared libs)` is selected. Then you can proceed with the following commands:
+
+``` shell
+ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j24
+ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make install
+cd _install
+mkdir proc
+mkdir sys
+mkdir -p etc/init.d
+cat <<EOF > etc/init.d/rcS
+#!bin/sh
+mount -t proc none /proc
+mount -t sysfs none /sys
+EOF
+chmod u+x etc/init.d/rcS
+grep -v tty ../examples/inittab > ./etc/inittab
+```
+
+## Create a RAM disk for Linux
+
+At this point you can copy into the current directory additional files you may
+want in the RAM disk, for example, the kernel module built in the previous
+section. Then run the following commands:
+
+``` shell
+find . | cpio -o -H newc | gzip > ../initrd.img
+cd ..
+```
+
+The resulting file is `initrd.img`. It should be copied to the Hafnium RAM
+disk's root.
diff --git a/docs/Testing.md b/docs/Testing.md
new file mode 100644
index 0000000..6521264
--- /dev/null
+++ b/docs/Testing.md
@@ -0,0 +1,48 @@
+# Testing
+
+Testing of Hafnium is currently evolving. There are basic tests running on QEMU
+but we want more and more kinds of tests e.g. unit tests.
+
+## Presubmit
+
+Presubmit builds everything, runs all tests and checks the source for
+formatting and lint errors. This can be run locally with:
+
+``` shell
+./kokoro/ubuntu/build.sh
+```
+
+Or to just run the tests after having built everything manually run:
+
+``` shell
+./kokoro/ubuntu/test.sh
+```
+
+## QEMU tests
+
+These tests boot Hafnium on QEMU and the VMs make calls to Hafnium to test its
+behaviour.
+
+### `hftest`
+
+Having a framework for tests makes them easier to read and write. `hftest` in a
+framework to meet the needs of VM based tests for Hafnium. It consists of:
+
+ * assertions
+ * test declarations
+ * base VM image
+ * driver script
+
+Assertions should be familiar from other testing libraries. They make use of
+C11's `_Generic` expressions for type genericity.
+
+Test declarations name the test and the suite that the test is part of.
+Declarations are converted into descriptors stored in the `.hftest` section of
+the VM image which allows the image to inspect the structure of the tests it
+contains. The linker sorts the descriptors by their symbol name which is how
+descriptors from the same suite are grouped together for easier parsing.
+
+The base VM image offers a command line interface, via the bootargs, to query
+the tests in the image and to run specific tests. The driver script uses this
+interface to execute tests, each with a fresh QEMU boot to give a fresh
+environment.
