Getting and running aarch64-based root filesystem

First is getting the docker image from:

https://hub.docker.com/r/aarch64/ubuntu

And next is to get the QEMU binary emulator:

wget https://github.com/multiarch/qemu-user-static/releases/download/v3.0.0/qemu-aarch64-static.tar.gz

(which is binary rewriting from aarch64 to x86-64) and then:

docker run –rm –privileged multiarch/qemu-user-static:register

docker pull aarch64/ubuntu

docker run -v ~/qemu-aarch64-static:/usr/bin/qemu-aarch64-static -it aarch64/ubuntu

Then do a "apt-get update && apt-get install binutils file " and you can run command to show the following:

Advertisements

How to migrate docker across different machines (or different installation of OS)

As shown below, I have 500GB of docker images, containers and volumes. How to migrate across different installation of the Ubuntu, which is necessitated by the move from BIOS to UEFI mode of runtime execution.

The answer is as follows:

Just install the OS on a new harddisk, and then install docker-engine using the normal method. For me I used the instruction from DigitalOcean:

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04

Now the new docker installation have no images or containers yet, and so the entire subdirectory "/var/lib/docker" can be renamed:

sudo systemctl status docker.service

First stop the docker.service:

sudo systemctl stop docker.service

Then rename the directory:

mv /var/lib/docker /var/lib/docker_old

Then softlink the docker link to point to the old original directory (where the 500G docker directory lies:

cd /var/lib
ln -s /sda4/docker docker

And then start the service again:

sudo systemctl start docker.service

Job done.

How to run ARM binaries in docker

Answering the following question for ARM binaries:

https://tthtlc.wordpress.com/2018/11/27/how-to-run-ppc-binaries-in-docker/

cd $HOME

wget https://github.com/multiarch/qemu-user-static/releases/download/v3.0.0/qemu-arm-static.tar.gz

Unpacked the image in $HOME directory:

tar xvfz qemu-arm-static.tar.gz

docker pull armhf/ubuntu

docker run --rm --privileged multiarch/qemu-user-static:register

docker run -v ~/qemu-arm-static:/usr/bin/qemu-arm-static -it armhf/ubuntu

And testing out (after apt-get update && apt-get install file binutils inside the ARM ubuntu):

Using Haskell to write an OS (and memory safety

Further to this:

https://tthtlc.wordpress.com/2018/10/21/rust-programming/

And all its goodies features:

https://blog.gds-gov.tech/appreciating-rust-memory-safety-438301fee097

https://doc.rust-lang.org/nomicon/meet-safe-and-unsafe.html

https://stackoverflow.com/questions/36136201/how-does-rust-guarantee-memory-safety-and-prevent-segfaults

https://theburningmonk.com/2015/05/rust-memory-safety-without-gc/

http://willcrichton.net/notes/rust-memory-safety/

Now I just discover its ancestor is Haskell:

Image result for rust development history

(from:  https://developer.ibm.com/articles/os-developers-know-rust/)

OS and development-related issues with Haskell:

https://www.reddit.com/r/haskell/comments/69jhsf/haskell_os/

https://www.fpcomplete.com/blog/2017/12/building-haskell-apps-with-docker

https://docs.haskellstack.org/en/v1.1.0/docker_integration/

https://hub.docker.com/_/haskell/

https://robots.thoughtbot.com/easy-haskell-development-and-deployment-with-docker

https://ghc.haskell.org/trac/ghc/wiki/Building/CrossCompiling

http://uhsure.com/halvm3.html

Memory safety (stackoverflow – free, integer-overflow free etc):

https://stackoverflow.com/questions/26780624/how-safe-is-safe-haskell

http://www.haskellforall.com/2015/12/compile-time-memory-safety-using-liquid.html

https://wiki.haskell.org/GHC/Memory_Management

https://tonyarcieri.com/it-s-time-for-a-memory-safety-intervention

https://www.reddit.com/r/haskell/comments/2botc2/the_limitations_of_haskell_for_safety_critical/

Thread-safety (race-condition-free):

http://hackage.haskell.org/package/tslogger

https://wiki.haskell.org/Concurrency

http://www.well-typed.com/blog/53/

How to run 32-bit docker inside 64-bit host, and enabling strace (and other APIs)

As documented here, there is a long list of system call disabled by the seccomp mechanism in Docker:

https://docs.docker.com/engine/security/seccomp/

A few of these system call disabled by default are listed below:

Syscall Description
bpf Deny loading potentially persistent bpf programs into kernel, already gated by CAP_SYS_ADMIN.
clock_adjtime Time/date is not namespaced. Also gated by CAP_SYS_TIME.
clock_settime Time/date is not namespaced. Also gated by CAP_SYS_TIME.
clone Deny cloning new namespaces. Also gated by CAP_SYS_ADMIN for CLONE_* flags, except CLONE_USERNS.
create_module Deny manipulation and functions on kernel modules. Obsolete. Also gated by CAP_SYS_MODULE.
delete_module Deny manipulation and functions on kernel modules. Also gated by CAP_SYS_MODULE.
finit_module Deny manipulation and functions on kernel modules. Also gated by CAP_SYS_MODULE.
get_kernel_syms Deny retrieval of exported kernel and module symbols. Obsolete.
get_mempolicy Syscall that modifies kernel memory and NUMA settings. Already gated by CAP_SYS_NICE.
init_module Deny manipulation and functions on kernel modules. Also gated by CAP_SYS_MODULE.
ioperm Prevent containers from modifying kernel I/O privilege levels. Already gated by CAP_SYS_RAWIO.
iopl Prevent containers from modifying kernel I/O privilege levels. Already gated by CAP_SYS_RAWIO.
kcmp Restrict process inspection capabilities, already blocked by dropping CAP_PTRACE.
kexec_file_load Sister syscall of kexec_load that does the same thing, slightly different arguments. Also gated by CAP_SYS_BOOT.
kexec_load Deny loading a new kernel for later execution. Also gated by CAP_SYS_BOOT.
keyctl Prevent containers from using the kernel keyring, which is not namespaced.
lookup_dcookie Tracing/profiling syscall, which could leak a lot of information on the host. Also gated by CAP_SYS_ADMIN.
mbind Syscall that modifies kernel memory and NUMA settings. Already gated by CAP_SYS_NICE.
mount Deny mounting, already gated by CAP_SYS_ADMIN.
move_pages Syscall that modifies kernel memory and NUMA settings.
name_to_handle_at Sister syscall to open_by_handle_at. Already gated by CAP_SYS_NICE.
nfsservctl Deny interaction with the kernel nfs daemon. Obsolete since Linux 3.1.
open_by_handle_at Cause of an old container breakout. Also gated by CAP_DAC_READ_SEARCH.
perf_event_open Tracing/profiling syscall, which could leak a lot of information on the host.
personality Prevent container from enabling BSD emulation. Not inherently dangerous, but poorly tested, potential for a lot of kernel vulns.
pivot_root Deny pivot_root, should be privileged operation.
process_vm_readv Restrict process inspection capabilities, already blocked by dropping CAP_PTRACE.
process_vm_writev Restrict process inspection capabilities, already blocked by dropping CAP_PTRACE.
ptrace Tracing/profiling syscall, which could leak a lot of information on the host. Already blocked by dropping CAP_PTRACE.

Program like gdb and strace are all using ptrace() to debug another program. Without ptrace() syscall, there is no way to do gdb inside docker container.

To bypass the seccomp restriction:

docker run –rm -it –security-opt seccomp=unconfined 32bit/ubuntu:bionic bash

The above will be running a 32-bit docker container inside the existing 64-bit host.

After installing the appropriate tools, we can do strace:

root@68ba6b846014:~# apt-get update
root@68ba6b846014:~# apt-get install strace ltrace

root@68ba6b846014:~# strace /bin/ls

execve("/bin/ls", ["/bin/ls"], 0xffb2e750 /* 11 vars */) = 0
brk(NULL) = 0x580ed000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7767000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=11433, …}) = 0
mmap2(NULL, 11433, PROT_READ, MAP_PRIVATE, 3, 0) = 0xf7764000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/i386-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0L\0\0004\0\0\0"…, 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=169960, …}) = 0
mmap2(NULL, 179612, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf7738000
mmap2(0xf7761000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x28000) = 0xf7761000
mmap2(0xf7763000, 3484, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xf7763000
close(3) = 0

How to run PPC binaries in docker

Using a QEMU solution:

https://tthtlc.wordpress.com/2018/09/28/how-to-install-ppc-based-ubuntu-inside-the-x86-64-environment/

is one option, but performance is not good. Docker offer possibly 6 times faster performance (ie, 6 docker container = 1 VMware guest) but will require more elaborate setup:

First download the QEMU PPC emulator:

wget https://github.com/multiarch/qemu-user-static/releases/download/v2.7.0/qemu-ppc64le-static.tar.gz

Unpack it into the $HOME directory:

tar tvfz qemu-ppc64le-static.tar.gz

Then pull the PPC64LE image:

docker pull ppc64le/ubuntu:latest

This is a key step – to be repeated each time the host machine is rebooted:

docker run –rm –privileged multiarch/qemu-user-static:register

What it does is to map the types of the binaries to the corresponding QEMU instruction emulator – for PPC the instruction emulator “qemu-ppc64le-static” will be used, and it is available at the $HOME directory as unpacked earlier.

Next start the container:

docker run -v ~/qemu-ppc64le-static:/usr/bin/qemu-ppc64le-static -it ppc64le/ubuntu

After knowing the container ID (via docker ps), we can also start another alternative screen into the same container (assuming 16b74063bd84 is the container ID):

docker exec -it 16b74063bd84 bash

Should the host be rebooted, you just need to repeat the following steps to restart the same container again:

docker run –rm –privileged multiarch/qemu-user-static:register
docker start 16b74063bd84

docker exec -it 16b74063bd84 bash

And verifying it is PPC:

Selection_014

Learning about mbox: virtualization via ptrace()

http://cursuri.cs.pub.ro/pipermail/oss/2014-February/000163.html

https://github.com/tsgates/mbox

https://gts3.org/assets/papers/2013/kim:mbox-slides.pdf

https://news.ycombinator.com/item?id=7214419

https://pdos.csail.mit.edu/archive/mbox/

https://people.csail.mit.edu/nickolai/papers/kim-mbox.pdf

https://www.usenix.org/node/174518

https://www.usenix.org/system/files/conference/atc13/atc13-kim.pdf

%d bloggers like this: