Understanding Android’s atrace/systrace/ftrace

https://github.com/brendangregg/FlameGraph/issues/167

https://android.googlesource.com/platform/prebuilts/simpleperf/+/ndk-r14-beta1/README.md#show-flamegraph

https://android.googlesource.com/platform/prebuilts/simpleperf/+/ndk-r14-beta1/README.md#what-is-simpleperf

https://www.chromium.org/developers/how-tos/trace-event-profiling-tool/recording-tracing-runs

https://subscription.packtpub.com/book/application_development/9781785288951/2/ch02lvl1sec34/running-a-trace

https://elinux.org/images/5/54/Yaghmour-android-platform-debug-dev-clean-131030.pdf

https://medium.com/@bkodirov/android-systrace-2d1b8285cbb3

https://android.googlesource.com/platform/system/extras/+/jb-mr1-dev-plus-aosp/atrace/atrace.c

https://source.android.com/devices/tech/debug/systrace

https://source.android.com/devices/tech/debug/ftrace

Advertisements

eXpress Data Path

The paper:

https://raw.githubusercontent.com/tohojo/xdp-paper/master/xdp-the-express-data-path.pdf

and the video: https://www.youtube.com/watch?v=Y103CWBa1BI

https://blogs.igalia.com/dpino/2019/01/10/the-express-data-path/

https://blog.cloudflare.com/single-rx-queue-kernel-bypass-with-netmap/

https://www.iovisor.org/technology/xdp

https://prototype-kernel.readthedocs.io/en/latest/networking/XDP/

https://developers.redhat.com/blog/2018/12/06/achieving-high-performance-low-latency-networking-with-xdp-part-1/

https://developers.redhat.com/blog/2018/12/03/network-debugging-with-ebpf/

All about Autotools, configure, Makefile.in, Makefile.am, configure.ac

How to use:

https://eklitzke.org/how-to-autotools

https://developer.gnome.org/anjuta-build-tutorial/stable/create-autotools.html.en

http://freesoftwaremagazine.com/articles/brief_introduction_to_gnu_autotools/

https://www.lrde.epita.fr/~adl/autotools.html

https://stackoverflow.com/questions/1970926/whats-the-point-of-aclocal

https://www.gnu.org/software/automake/manual/html_node/amhello_0027s-configure_002eac-Setup-Explained.html

https://github.com/yugui/example/blob/master/configure.ac

https://www.gnu.org/software/autoconf/manual/autoconf-2.61/html_node/Writing-configure_002eac.html

https://www.edwardrosten.com/code/autoconf/

https://developer.gnome.org/anjuta-build-tutorial/stable/create-autotools.html.en

Radare2 Internal Analysis

https://github.com/radare/radare2

To debug a binary "a.out":

r2 -d ./a.out

And subsequently what are all the "p" functions?

And if you thought that all commands starting with "p" are listed above, then you are wrong – since "p?" means one or two characters, so options like "pdj" are not listed.

Now entering "pd?":

More options.

Looking into radare2 source code – libr/core/core.c is one file with all the commands:

And implementation is in cmd_print.c:

and lots of other files for other commands:

More development related blogs:

http://radare.today/posts/background_tasks/

And other references:

https://www.radare.org/get/r2pipe-nn2015.pdf

https://www.megabeets.net/a-journey-into-radare-2-part-1/

https://malwology.com/2018/11/30/intro-to-radare2-for-malware-analysis/

https://radare.gitbooks.io/radare2book/content/

AFL-unicorn: What is it and how to use it?

AFL-unicorn is well explained here:

https://www.youtube.com/watch?v=OheODvF0884

And the source code is here:

https://github.com/jdbirdwell/afl

Installation is well described here:

https://hackernoon.com/afl-unicorn-fuzzing-arbitrary-binary-code-563ca28936bf

Just git clone and make:

and followed by build_unicorn_suport.sh:

Here I will describe how I run AFL unicorn on the following C program:

#include <stdio.h>

int main(){
int i=0, j=0;
for(i=200;i>100;i++) {
j=i+1;
i=j-3;
//printf(“%dn”, i);
}
return i;
}

First compile it and disassemble it using IDA Pro:

Here we can see that at address 0x4004DA we have setup the first breakpoint (in RED).

Next is to setup the debugger:

After there that we click on “Start Process” to enable it to run inside the IDA Pro envionment.

At the breakpoint where it stopped, we capture the process state via “process dumper”: In IDA Pro top menu bar: “File”->”Script File” and select “unicorn_dumper_ida.py” and here is the output:

Notice the directory where the dumper direct the CPU state to. And inside this directory there is a file “_index.json” which record all the CPU/process memory information:

Notice from the above the value of “rip”, which is where we set the breakpoint in IDA Pro to dump out the process information.   Convert this “rip” value to hexadecimal and it will be the same as the “START_ADDRESS” to be used in the Unicorn emulator script.

And so from there just this directory and the input files are all that is needed – the binary and its memory requirements are already captured indie the binary informatin.

From AFL-unicorn_battele copy the file “template_test_harness.py” and modify several things:

a. Since it is x86 64-bit we are going to emulate, therefore we use x86_const.py (which can be seen as part of download from https://github.com/unicorn-engine/unicorn)

b. Generally if is just non-supervisor mode (for ARM) or ring 3 for x86 there is no need to change CPU mode during emulation. But for emulating ring 0 or syscall, it is more complicated and slower, and thus it is preferred that we should emulate by hand-supplying the return results from making these syscalls. This is why when the assembly called functions like malloc(), or free(), or open(), or read() etc – all these has to be emulated, as shown below (notice it is for 32-bit x86 architecture, as compared with the present case which is 64-bit x86) from the original author (but not done here):

Selection_155

https://github.com/unicorn-engine/unicorn/issues/694

https://github.com/unicorn-engine/unicorn/issues/1048

https://www.reddit.com/r/linux/comments/48fted/unicorn_the_ultimate_cpu_emulator/

c. Next is to allocate the stack, heap or any memory as required at the start point of emulation. If it is stack, then the stack pointer (EBP/RBP or ESP/RSP) has to point to the end of the memory block allocated (stack_addr+512).

d. Lastly is the loop that read the next assembly instruction and emulate it:

(Notice the “START_ADDRESS” above?   It is the same value we have highlighted earlier)   And the full python script is here below:

https://gist.github.com/tthtlc/3b9fff8e0e4c016fd5cb09a78dc94d2d

Overall execution in debug mode(-d):

simplec_emulation

For the integration of this python script with AFL-fuzzer – this is not done as the C program has to be modified to receive input through a file mechanism.

References:

More tricks or caveats for Unicorn engine emulation:

https://www.reddit.com/r/linux/comments/48fted/unicorn_the_ultimate_cpu_emulator/

How Unicorn can be used to emulate interrupt processing:

https://github.com/unicorn-engine/unicorn/issues/376

https://github.com/unicorn-engine/unicorn/issues/825

The fuzzing internals are explained here:

https://hackernoon.com/afl-unicorn-part-2-fuzzing-the-unfuzzable-bea8de3540a5

And AFL-unicorn itself is based on the Unicorn and Capstone Engine:

https://www.capstone-engine.org

https://github.com/unicorn-engine/unicorn

https://news.ycombinator.com/item?id=9989609

https://news.ycombinator.com/item?id=11437962

http://eternal.red/2018/unicorn-engine-tutorial/

https://hackmd.io/s/rJTUtGwuW#

Understanding Chrome Browser integration to GLIB

How do we start understanding the implementation internals of Chrome Browser?

By collecting stacktrace.

Upon starting Chrome at the command line, I happened to encounter this stack trace:

#0 0x7f76f9368bc1 base::debug::CollectStackTrace()
#1 0x7f76f91b554d base::debug::StackTrace::StackTrace()
#2 0x7f76f91b5505 base::debug::StackTrace::StackTrace()
#3 0x7f76f91f3d43 logging::LogMessage::~LogMessage()
#4 0x56345616001b SigninViewController::ShowDiceSigninTab()
#5 0x56345615fca6 SigninViewController::ShowSignin()
#6 0x56345603b199 chrome::ShowBrowserSignin()
#7 0x5634544ebdc7 signin_ui_util::internal::EnableSyncFromPromo()
#8 0x5634544ebad0 signin_ui_util::EnableSyncFromPromo()
#9 0x5634560513da ManagePasswordsUIController::EnableSync()
#10 0x5634562c1de9 ManagePasswordsBubbleModel::OnSignInToChromeClicked()
#11 0x5634562c937c PasswordSignInPromoView::Accept()
#12 0x5634562c67a2 PasswordPendingView::Accept()
#13 0x7f76e9ad480e views::DialogClientView::AcceptWindow()
#14 0x7f76e9ad52ba views::DialogClientView::ButtonPressed()
#15 0x7f76e99fd12b views::Button::NotifyClick()
#16 0x7f76e99fc038 views::Button::OnMouseReleased()
#17 0x7f76e9ab7ddc views::View::ProcessMouseReleased()
#18 0x7f76e9ab78ac views::View::OnMouseEvent()
#19 0x7f76f01b2a05 ui::EventHandler::OnEvent()
#20 0x7f76f01c1c1c ui::ScopedTargetHandler::OnEvent()
#21 0x7f76f01afe5a ui::EventDispatcher::DispatchEvent()
#22 0x7f76f01af7b6 ui::EventDispatcher::ProcessEvent()
#23 0x7f76f01af407 ui::EventDispatcherDelegate::DispatchEventToTarget()
#24 0x7f76f01af21b ui::EventDispatcherDelegate::DispatchEvent()
#25 0x7f76e9ac5513 views::internal::RootView::OnMouseReleased()
#26 0x7f76e9acd642 views::Widget::OnMouseEvent()
#27 0x7f76e9afa475 views::NativeWidgetAura::OnMouseEvent()
#28 0x7f76f01b2a05 ui::EventHandler::OnEvent()
#29 0x7f76f01afe5a ui::EventDispatcher::DispatchEvent()
#30 0x7f76f01af7b6 ui::EventDispatcher::ProcessEvent()
#31 0x7f76f01af407 ui::EventDispatcherDelegate::DispatchEventToTarget()
#32 0x7f76f01af21b ui::EventDispatcherDelegate::DispatchEvent()
#33 0x7f76f01b3bdd ui::EventProcessor::OnEventFromSource()
#34 0x7f76f01b3e8c ui::EventProcessor::OnEventFromSource()
#35 0x7f76f01b5583 ui::EventSource::DeliverEventToSink()
#36 0x7f76f01b52e7 ui::EventSource::SendEventToSinkFromRewriter()
#37 0x7f76f01b5021 ui::EventSource::SendEventToSink()
#38 0x7f76e9b0eb41 views::DesktopWindowTreeHostX11::DispatchMouseEvent()
#39 0x7f76e9b0fb58 views::DesktopWindowTreeHostX11::DispatchEvent()
#40 0x7f76f7648997 ui::PlatformEventSource::DispatchEvent()
#41 0x7f76d5327bb1 ui::X11EventSourceGlib::ProcessXEvent()
#42 0x7f76d53199f5 ui::X11EventSource::ExtractCookieDataDispatchEvent()
#43 0x7f76d5319964 ui::X11EventSource::DispatchXEvents()
#44 0x7f76d5327d55 ui::(anonymous namespace)::XSourceDispatch()
#45 0x7f76d6e9204a g_main_context_dispatch
#46 0x7f76d6e923f0 <unknown>
#47 0x7f76d6e9249c g_main_context_iteration
#48 0x7f76f920c49f base::MessagePumpGlib::Run()
#49 0x7f76f92c0e29 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run()
#50 0x7f76f925e466 base::RunLoop::Run()
#51 0x563454035f15 ChromeBrowserMainParts::MainMessageLoopRun()
#52 0x7f76f3fd130b content::BrowserMainLoop::RunMainMessageLoopParts()
#53 0x7f76f3fd76ce content::BrowserMainRunnerImpl::Run()
#54 0x7f76f3fcc82a content::BrowserMain()
#55 0x7f76f539a01b content::RunBrowserProcessMain()
#56 0x7f76f539b3fa content::ContentMainRunnerImpl::RunServiceManager()
#57 0x7f76f539ae99 content::ContentMainRunnerImpl::Run()
#58 0x7f76f5398769 content::ContentServiceManagerMainDelegate::RunEmbedderProcess()
#59 0x7f76d27cf73d service_manager::Main()
#60 0x7f76f5399b13 content::ContentMain()
#61 0x56345263a246 ChromeMain
#62 0x56345263a152 main
#63 0x7f76d536b830 __libc_start_main
#64 0x56345263a02a _start

It captured beautifully the history of the calls made.

First either download the entire source:

git clone https://chromium.googlesource.com/chromium/src.git

or navigate here:

https://chromium.googlesource.com/chromium/src.git/+/refs/heads/master

Starting with ChromeBrowserMainParts::MainMessageLoopRun() from above stacktrace:

https://chromium.googlesource.com/chromium/src.git/+/refs/heads/master/chrome/browser/chrome_browser_main.cc#1830

The function is captured as follows:

First thing we learned is that Android architecture have a different message loop.

Next from the next frame of the stacktrace, we know that the HTML/javascript processing is within the g_run_loop->Run() function, which lead to calling base::RunLoop::Run() (and this is in the base/run_loop.cc file):

After this is the Chrome implementation of the interface to glib’s message pump (unique to Linux and thus not applicable to Windows):

https://github.com/adobe/chromium/blob/master/base/message_pump_glib.cc

“g_source_new” is an API from GLIB’s:

https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html#g-source-new

image.png

image.png

For other information on GLIB:

https://book.huihoo.com/gtk+-gnome-application-development/sec-mainloop.html

https://developer.gnome.org/gtk3/stable/gtk3-General.html

https://developer.gnome.org/glib/stable/glib-Asynchronous-Queues.html

https://developer.gnome.org/glib/stable/glib-Message-Logging.html

New development in Chrome Browswer

There are a lot of new happenings in Chrome Browser development, if you got time to browser slowly through here:

https://cs.chromium.org/chromium/src/third_party/blink/public/platform/modules/?g=0

For example here below choose "Idle":

And you have "Idle Management":

https://github.com/inexorabletash/idle-detection

And if you choose Locks, you will have "Web Locks":

https://wicg.github.io/web-locks/

https://w3c.github.io/payment-request/

And interestingly there is a request for upgrade for insecure resources:

https://cs.chromium.org/chromium/src/third_party/blink/public/platform/modules/payments/web_can_make_payment_event_data.h?g=0

https://w3c.github.io/webappsec-upgrade-insecure-requests/#insecure-requests-policy

Astr0baby's not so random thoughts _____ rand() % 100;

@astr0baby on Twitter for fresh randomness

WildML

Artificial Intelligence, Deep Learning, and NLP

The Data Explorer

playing around with open data to learn some cool stuff about data analysis and the world

Conorsblog

Data | ML | NLP | Python | R

quyv

Just a thought

IFT6266 - H2017 Deep Learning

A Graduate Course Offered at Université de Montréal

Deep Learning IFT6266-H2017 UdeM

Philippe Paradis - My solutions to the image inpainting problem

IFT6266 – H2017 DEEP LEARNING

Pulkit's thoughts on the course project

ML/AI

Machine learning. Artificial Intelligence

the morning paper

an interesting/influential/important paper from the world of CS every weekday morning, as selected by Adrian Colyer

The Spectator

Shakir's Machine Learning Blog

Everything about Data Analytics

big data, data analytics

Microcontrollers Lab

Microcontrollers tutorials and projects

%d bloggers like this: