Archive for the ‘kernel_general’ Category

MongoDB / NoSQL Database

Differences:

http://www.oracle.com/technetwork/database/database-technologies/nosqldb/documentation/nosql-vs-mongodb-1961723.pdf

Modelling methods:

Differences between SQL and NoSQL:

Automated conversion of SQL to MongoDB syntax:

MySQL to MongoDB converter:

Update performance:

Advertisements

How kernel modules is loaded and organized in memory?

Just piecing together various resources from internet:

Parallella: booting up parabuntu

https://www.parallella.org/

https://www.parallella.org/wp-content/uploads/2014/11/parallella-board-22-609×400.jpg

https://www.parallella.org/wp-content/uploads/2014/11/P16000-01-A1_1024x10241-609×400.jpg

And manual:

https://www.parallella.org/docs/parallella_manual.pdf

Images ready for flashing to sdcard:

https://github.com/parallella/parabuntu/releases

Installing on the SD card (using “dd” command) and letting it run (output is captured in the UART through usb2serial ftdi cable):

The bootup log is here:

https://gist.github.com/tthtlc/c7e267d7fbf95d3d366146233b28554f

First install “openssh-server” on the device instead:

apt-get update

apt-get install openssh-server

And next is to plugin the network UTP cable into the RJ45 port and start downloading examples.

Downloading examples to try:

(https://www.parallella.org/2016/01/21/creating-an-fpga-accelerator-in-15-minutes/)

The following are needed:

apt-get install iverilog

apt-get install gtkwave

git clone https://github.com/parallella/oh

Setting up env:

source setenv.sh

and then running example “./run.sh accelerator”:

Another example source:

git clone https://github.com/parallella/parallella-examples.git
cat /proc/cpuinfo
 processor : 0
 model name : ARMv7 Processor rev 0 (v7l)
 BogoMIPS : 1332.01
 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpd32
 CPU implementer : 0x41
 CPU architecture: 7
 CPU variant : 0x3
 CPU part : 0xc09
 CPU revision : 0

processor : 1
 model name : ARMv7 Processor rev 0 (v7l)
 BogoMIPS : 1332.01
 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpd32
 CPU implementer : 0x41
 CPU architecture: 7
 CPU variant : 0x3
 CPU part : 0xc09
 CPU revision : 0

Hardware : Xilinx Zynq Platform
 Revision : 0003
 Serial : 0000000000000000

Accessing other CPUs will require some custom programming.

References:

https://www.parallella.org/2015/04/05/parallella-chronicles-part-six-moving-multiple-bytes-simply/

Kernel source code static analysis with clang: lots of false positive

Clicked the above to see the details (below):

Steps in compiling Linux kernel with LLVM scan-build/scan-view:

1. change the "Makefile" – all "gcc" should be replaced by "clang", and similarly for "g++" to "clang++".
2. copy .config from somewhere, (eg, Ubuntu) and "make oldconfig" to autogenerate the .config file.

3. then "make kernel" to make sure compilation completes fine.

4. then "make clean" and "scan-build make kernel" to building using clang.

5. view the output:

scan-view –allow-all-hosts –no-browser –debug /tmp/scan-build-2018-03-21-001413-5298-1

Just focusing on the memory leak:

So LLVM knows alloc_and_copy_xxx() allocate memory resources.

now how is the conclusion of leakage at 27 achieved?

since memory is always freed as the last line indicated, no leakage is possible.

So the reason is because free_ftrace_hash() is not associated with free() of memory as allocated earlier.

Looking at next one:

Similarly, kfree() is not known to do freeing of memory resources.

And the following is clearly wrong too:

(kfree() is deallocator).

References:

https://clang-analyzer.llvm.org/

https://clang-analyzer.llvm.org/scan-build.html

https://clang-analyzer.llvm.org/

http://btorpey.github.io/blog/2015/04/27/static-analysis-with-clang/

powerpc linux kernel board bring-up problem

Reading this forum:

https://forum.kernelnewbies.org/read.php?18,118113,118113#msg-118113

And to requote the problem here:

Hi everyone!

I got lucky and got an embedded linux internshipsmiling smiley

I’m trying to port linux to an existing board that is currently using vxWorks.
The board uses an mpc8572e processor which contains two e500v2 cores. Basically, the entry point in vmlinux
is arch/powerpc/kernel/head_fsl_booke.S

We are using the simpleboot bootwrapper which works just fine (simpleboot passes the device tree address and jump right into the gunziped vmlinux which is located at 0x0 physical).

head_fsl_booke.S include arch/powerpc/kernel/fsl_booke_entry_mapping.S and we just crash at line 223 (rfi instruction)
of fsl_booke_entry_mapping.S

215 lis r7,MSR_KERNEL@h
216 ori r7,r7,MSR_KERNEL@l
217 bl 1f /* Find our address */
218 1: mflr r9
219 rlwimi r6,r9,0,20,31
220 addi r6,r6,(2f – 1b)
221 mtspr SPRN_SRR0,r6
222 mtspr SPRN_SRR1,r7
223 rfi /* start execution out of TLB1[0] entry */

rfi at line 223 causes us to jump to 0xc000_0264 while we want to jump to 0x000_0264.
This is not caused by our mmu, we are basically loading 0xc000_0264 in srr0

It’s related to the following config fields in our defconfig:
CONFIG_PAGE_OFFSET=0xc0000000 <—-
CONFIG_PHYSICAL_START=0x00000000
CONFIG_PHYSICAL_ALIGN=0x04000000
CONFIG_TASK_SIZE=0xc0000000
CONFIG_KERNEL_START=0xc00000000 <—–

I tried to set CONFIG_KERNEL_START and CONFIG_PAGE_OFFSET to 0x0 but this would not compile:
arch/powerpc/include/asm/processor.h:100:2: error: #error User TASK_SIZE overlaps with KERNEL_START address

By the way CONFIG_TASK_SIZE=0xc0000000

Also, my debugger cannot map anything because all the symbols in vmlinux are at 0xcxxx_xxxx but we are at 0x0xxx_xxxx

Pretty much all the ports that uses the simpleboot bootwrapper are gunzipping vmlinux at 0x0 but I can easily change that
and I tried to send vmlinux at 0xc000_0000 and it solved this particular issue but then the mmu mapping just does not work
because simpleboot extract our device tree at about 0x0011_0000 and we need vmlinux and the device tree to be inside a 64Mb page (also tried to hack fsl_booke_entry_mapping.S to create a 4GB tlb entry based at 0x0 and this was causing other issues…)

Got any idea?

The details I cannot help.   But my suggested steps of action are as follows:

First is to get the cross-compiler for e500v2 (ppc):

https://www.nxp.com/docs/en/supporting-information/WBNR_FTF10_NET_F0676.pdf

http://zeromq.org/build:powerpc

https://community.nxp.com/thread/310153

Next is to compile uboot for the board:

https://github.com/theopolis/u-boot-sboot

which is support board itself (mpc8572e).   Next is the emulator, but not sure if it works? (though it indicated support for mpc8572e)

https://github.com/tycho/pearpc

And to understand how uboot can startup the PPC CPU, you will need to know PPC hardware knowledge specific to e500v2 and all the bootup details:
And then only the final steps is the understanding of Linux kernel itself, and its bootup requirements:

 

How to extend the filesystem size in CentOS (default is XFS) running as a VMware guest?

sudo fdisk /dev/sdc –> /dev/sdc1

Then issue "pvcreate /dev/sdc1".

This will add the disk to the physical volume group (pvdisplay), noticed the "VG name" is not assigned yet:

vgextend centos /dev/sdc1 —> after this the following 59.5 G is displayed.

And "sudo pvdisplay" will show that /dev/sdc1 is assigned to the VG group "centos".

But lvdisplay still showing old values:

Finally, "lvextend -l +100%FREE /dev/centos/root" and this will extend the VG extents to the 57G limit.

Finally, boot in rescue mode and do the following (this is for ext4 filesystem):

resize2fs /dev/mapper/centos-root

But if your filesystem is XFS, which is default for CentOS 7.2 1511:

The rootfs is running XFS, and XFS does allow online resizing.

So without any rebooting or rescue mode:

So now the rootfs have been properly resize by the command:

xfs_growfs -d /dev/mapper/centos-root

The option "-d" is for maximum possible size.

So in summary, for LVM only the last part is different between XFS and ext4 filesystem: resize2fs or xfs_growfs command.

How does KCOVtrace worked (in the Linux Kernel)?

We all are familiar with how strace worked: it uses the system call “ptrace” to attach to a process and then setting breakpoints on all the system call.

So how does KCOVTRACE worked?

The tool is available here:

https://github.com/google/syzkaller/blob/master/tools/kcovtrace/kcovtrace.c

It requires the kernel to be compiled with CONFIG_KCOV.

When applied on a binary like “/bin/ls” and let it execute, we get a list of addresses as output:

0xffffffff81a109e4
0xffffffff81a109da
0xffffffff81a109e4
0xffffffff81a109da
0xffffffff81a109e4

To know the address 0xffffffff81a109e4, let use the Linux kernel image file “vmlinux”, do a disassembly via “objdump -d vmlinux”:

Selection_065

As we can see the address is the byte offset immediately following calling the function <__sanitizer_cov_trace_pc>.

This function is inserted using GCC plugin mechanism when the linux kernel is compiled with GCC plugin enabled, and CONFIG_KCOV=y.

Inside kernel/kcov.c, you see that that the address of each basic block that got executed in the kernel is updated into “area[]” array, which get floats to the userspace, which is retrieved via /sys/kernel/debug/kcov interface ioctl() read.

Extracted from above URL and displayed as below:

Since only the process that explicitly open the /sys/kernel/debug/kcov interface will get to receive the output the kcov is therefore specific to the process that open the file descriptor.

(unlike FTRACE, where the interface can be open, and another process generate FTRACE in the kernel, only to be retrieve by final process that read the FTRACE).

%d bloggers like this: