Archive for September, 2013

How to compile Android NDK samples (in Ubuntu Linux)

Assuming Android NDK is installed in /opt/android-ndk-r8e and Android SDK is in /opt/android-sdk-linux and that “ant” have been installed (via “apt-get install ant” in Ubuntu). Only the following four commands are needed for compiling the NDK samples and generating the APK for installation into Android devices:

/opt/android-ndk-r8e/ndk-build

ant clean

/opt/android-sdk-linux/tools/android update project --path . --target android-17

ant debug

and below are the output:

First is the EGL libraries of the sample:

Inside the directory /opt/android-ndk-r8e/samples/hello-gl2:

/opt/android-ndk-r8e/ndk-build ======> gives:

Compile++ thumb : gl2jni <= gl_code.cpp
StaticLibrary : libstdc++.a
SharedLibrary : libgl2jni.so
Install : libgl2jni.so => libs/armeabi/libgl2jni.so

and then is the compilation of the JAVA into APK:

/opt/android-sdk-linux/tools/android update project --path . --target android-17

Updated and renamed default.properties to project.properties
Updated local.properties
No project name specified, using Activity name ‘GL2JNIActivity’.
If you wish to change it, edit the first line of build.xml.
Added file ./build.xml
Added file ./proguard-project.txt

ant debug

Buildfile: /opt/android-ndk-r8e/samples/hello-gl2/build.xml

XXXXX (lots of output truncated)

-build-setup:
[getbuildtools] Using latest Build Tools: 17.0.0
[echo] Resolving Build Target for GL2JNIActivity…
[gettarget] Project Target: Android 4.2.2
[gettarget] API level: 17
[echo] ———-
[echo] Creating output directories if needed…
[mkdir] Created dir: /opt/android-ndk-r8e/samples/hello-gl2/bin
[mkdir] Created dir: /opt/android-ndk-r8e/samples/hello-gl2/bin/res
[mkdir] Created dir: /opt/android-ndk-r8e/samples/hello-gl2/gen
[mkdir] Created dir: /opt/android-ndk-r8e/samples/hello-gl2/bin/classes

-obfuscate:

-dex:
[dex] input: /opt/android-ndk-r8e/samples/hello-gl2/bin/classes
[dex] Converting compiled files and external libraries into /opt/android-ndk-r8e/samples/hello-gl2/bin/classes.dex…

-crunch:
[crunch] Crunching PNG Files in source dir: /opt/android-ndk-r8e/samples/hello-gl2/res
[crunch] To destination dir: /opt/android-ndk-r8e/samples/h
[crunch] ello-gl2/bin/res
[crunch] Crunched 0 PNG files to update cache

-package-resources:
[aapt] Creating full resource package…

-package:
[apkbuilder] Current build type is different than previous build: forced apkbuilder run.
[apkbuilder] Creating GL2JNIActivity-debug-unaligned.apk and signing it with a debug key…

-post-package:

-do-debug:
[zipalign] Running zip align on final apk…
[echo] Debug Package: /opt/android-ndk-r8e/samples/hello-gl2/bin/GL2JNIActivity-debug.apk
[propertyfile] Creating new property file: /opt/android-ndk-r8e/samples/hello-gl2/bin/build.prop
[propertyfile] Updating property file: /opt/android-ndk-r8e/samples/hello-gl2/bin/build.prop
[propertyfile] Updating property file: /opt/android-ndk-r8e/samples/hello-gl2/bin/build.prop
[propertyfile] Updating property file: /opt/android-ndk-r8e/samples/hello-gl2/bin/build.prop

And after installing and running it:

I/ActivityManager( 275): START u0 {act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10200000 cmp=com.android.gl2jni/.GL2JNIActivity} from pid 392
W/WindowManager( 275): Failure taking screenshot for (246×410) to layer 21005
E/SurfaceFlinger( 36): ro.sf.lcd_density must be defined as a build property
D/dalvikvm( 900): Not late-enabling CheckJNI (already on)
I/ActivityManager( 275): Start proc com.android.gl2jni for activity com.android.gl2jni/.GL2JNIActivity: pid=900 uid=10049 gids={50049, 1028}
I/ActivityManager( 275): START u0 {act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10200000 cmp=com.android.gl2jni/.GL2JNIActivity} from pid 392
I/dalvikvm( 900): Turning on JNI app bug workarounds for target SDK version 5…
E/Trace ( 900): error opening trace file: No such file or directory (2)
V/PhoneStatusBar( 335): setLightsOn(true)
E/SurfaceFlinger( 36): ro.sf.lcd_density must be defined as a build property
I/Choreographer( 275): Skipped 37 frames! The application may be doing too much work on its main thread.
E/SurfaceFlinger( 36): ro.sf.lcd_density must be defined as a build property
D/libEGL ( 900): Emulator without GPU support detected. Fallback to software renderer.
D/libEGL ( 900): loaded /system/lib/egl/libGLES_android.so
I/Choreographer( 900): Skipped 70 frames! The application may be doing too much work on its main thread.
W/dalvikvm( 900): threadid=11: thread exiting with uncaught exception (group=0x40a71930)
E/AndroidRuntime( 900): FATAL EXCEPTION: GLThread 90
E/AndroidRuntime( 900): java.lang.IllegalArgumentException: No configs match configSpec
E/AndroidRuntime( 900): at com.android.gl2jni.GL2JNIView$ConfigChooser.chooseConfig(GL2JNIView.java:168)
E/AndroidRuntime( 900): at android.opengl.GLSurfaceView$EglHelper.start(GLSurfaceView.java:1024)
E/AndroidRuntime( 900): at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1401)
E/AndroidRuntime( 900): at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1240)
D/gralloc_goldfish( 900): Emulator without GPU emulation detected.
W/ActivityManager( 275): Force finishing activity com.android.gl2jni/.GL2JNIActivity
W/WindowManager( 275): Failure taking screenshot for (246×410) to layer 21020
I/ActivityManager( 275): Displayed com.android.gl2jni/.GL2JNIActivity: +1s523ms
I/Choreographer( 392): Skipped 38 frames! The application may be doing too much work on its main thread.
I/Choreographer( 275): Skipped 43 frames! The application may be doing too much work on its main thread.
E/SurfaceFlinger( 36): ro.sf.lcd_density must be defined as a build property
E/SurfaceFlinger( 36): ro.sf.lcd_density must be defined as a build property
I/Process ( 900): Sending signal. PID: 900 SIG: 9
I/ActivityManager( 275): Process com.android.gl2jni (pid 900) has died.
W/InputMethodManagerService( 275): Got RemoteException sending setActive(false) notification to pid 900 uid 10049

moz screenshot

More debugging needed.

Linux Graphics / GPU / X / Wayland Architecture

What is Wayland Architecture?

Homepage:

http://wayland.freedesktop.org/

From wiki:

http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)

To quote:

Wayland is a protocol that specifies the communication between the display server and its clients.[3] It was initially designed by Kristian Høgsberg as a replacement for the X Window System. In stark contrast to the latter, Wayland specifies that the clients are responsible for the drawing of their window borders and their decorations and the clients will render without detour directly into their own buffer located in the graphics memory, through the use of EGL with some additional Wayland-specific extensions to EGL.

And the details are partly shown below:


① The evdev module of the Linux kernel gets an event and sends it to the Wayland compositor.
② The Wayland compositor looks through its scenegraph to determine which window should receive the event. The scenegraph corresponds to what’s on screen and the Wayland compositor understands the transformations that it may have applied to the elements in the scenegraph. Thus, the Wayland compositor can pick the right window and transform the screen coordinates to window local coordinates, by applying the inverse transformations. The types of transformation that can be applied to a window is only restricted to what the compositor can do, as long as it can compute the inverse transformation for the input events.
③ As in the X case, when the client receives the event, it updates the UI in response. But in the Wayland case, the rendering happens by the client via EGL, and the client just sends a request to the compositor to indicate the region that was updated.
④ The Wayland compositor collects damage requests from its clients and then re-composites the screen. The compositor can then directly issue an ioctl to schedule a pageflip with KMS

More resources below:

Wayland Input Method System Proposal

https://wiki.maliit.org/Wayland_Input_Method_System_Proposal

The Wayland Compositor Architecture

https://01.org/linuxgraphics/sites/default/files/documentation/wayland-osts.pdf

What does EGL do in the Wayland stack

http://ppaalanen.blogspot.sg/2012/03/what-does-egl-do-in-wayland-stack.html

Introduction to Wayland

https://www.tizen.org/sites/default/files/lfcs2012_barnes.pdf

How to embrace wayland for Tizen

http://download.tizen.org/misc/media/conference2013/slides/TDC2013-How_to_Embrace_Wayland_for_Tizen.pdf

Why Wayland is better:

http://askubuntu.com/questions/11537/why-is-wayland-better

Compositing after X KWin on the Road to Wayland

https://community.kde.org/images.community/0/01/KWin_Wayland.pdf

Wayland Vs X Server Architecture (in Greek, can use Google Chrome to view, which will allow you to translate)

http://osarena.net/hacks-guides/wayland-vs-x-server-architecture.html

Wayland: Choosing a better input method architecture

http://mirrors.ustc.edu.cn/kde-application/akademy/2012/slides/Wayland_Choosing_a_Better_Input_Method_Architecture_-_Michael_Hasselmann.pdf

David Airlie: Teaching the X server new tricks (LCA 2013)

Latest:

Welcome to Chromium’s Ozone-Wayland (September 18, 2013)

http://vignatti.com/2013/09/18/welcome-to-chromiums-ozone-wayland/

http://vignatti.com/2013/03/05/ui-customization-on-wayland/

http://vignatti.com/2012/10/17/the-damn-small-wayland-api/

http://vignatti.com/2012/06/13/x-on-wayland/

http://vignatti.com/2012/01/18/starting-on-wayland-development/

http://unix.stackexchange.com/questions/40979/why-need-drivers-for-both-x-server-and-the-linux-kernel

http://www.freiesmagazin.de/mobil/freiesMagazin-2011-08-bilder.html

Wayland on Android:

http://ppaalanen.blogspot.sg/2012/07/wayland-on-android-snapshot-release.html

Wayland’s 1.0 milestone fixes graphics protocol

http://m.h-online.com/open/news/item/Wayland-s-1-0-milestone-fixes-graphics-protocol-1734638.html?view=zoom;zoom=2

http://worldofgnome.org/gnome-wayland/

X on Wayland

http://vignatti.com/2012/06/13/x-on-wayland/

Wayland prototype for rendering software that runs remotely

http://www.h-online.com/open/news/item/Wayland-prototype-for-rendering-software-that-runs-remotely-1715463.html

Threads in Firefox (from Ubuntu 12.04)

Doing a simple gdb attach on a recent version of Firefox (v23.0) I was very surprised by the complexity of the threading available:

Id Target Id Frame
34 Thread 0x7f0164575700 (LWP 2272) "dconf worker" 0x00007f016fdb6313 in poll () from /lib/x86_64-linux-gnu/libc.so.6
33 Thread 0x7f0161fda700 (LWP 2273) "gdbus" 0x00007f016fdb6313 in poll ()
from /lib/x86_64-linux-gnu/libc.so.6
32 Thread 0x7f016035d700 (LWP 2274) "Gecko_IOThread" 0x00007f016fdbdee9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6
31 Thread 0x7f015f0ff700 (LWP 2275) "Socket Thread" 0x00007f016fdb6313 in poll () from /lib/x86_64-linux-gnu/libc.so.6

Or to summarize is a more easy to read manner:

“dconf worker”
“gdbus”
“Gecko_IOThread”
“Socket Thread”
“JS GC Helper”
“JS Sour~ Thread”
“JS Watchdog”
“Hang Monitor”
“Timer”
“Cert Verify”
“mozStorage #1″
“mozStorage #2″
“mozStorage #3″
“Proxy R~olution”
“Cache I/O”
“mozStorage #4″
“MediaManager”
“DOM Worker”
“HTML5 Parser”
“mozStorage #5″
“URL Classifier”
“localStorage DB”
“mozStorage #6″
“mozStorage #7″
“DOM Worker”
“mozStorage #8″
“mozStorage #9″
“mozStorage #10″
“Analysis Helper”
“threaded-ml”
“Image Scaler”
“mozStorage #11″
“ImageDecoder #5″
“firefox”

Firstly, what are each of these threads doing in Firefox? How does it really help in the browsing process?

WHere is the source codes for these?

What are the corresponding or equivalent functionality/features in Chrome browser?

Anyway to debug the runtime behavior of these threads?

How to trace through the different parts of the threads at runtime?

This is threading in Firefox:

https://developer.mozilla.org/en-US/docs/Code_snippets/Threads

Concept of “web worker” in Firefox (which essentially is THREADS?) and “ChromeWorker”:

https://developer.mozilla.org/en-US/docs/Web/Guide/Performance/Using_web_workers

https://developer.mozilla.org/en-US/docs/Web/API/ChromeWorker

Thread manager:

https://developer.mozilla.org/en/docs/The_Thread_Manager

Firefox management of threads:

http://forums.mozillazine.org/viewtopic.php?p=702985

And for debugging:

http://superuser.com/questions/311932/peeking-inside-firefox-whats-maxing-out-the-cpu?rq=1

http://kb.mozillazine.org/Problematic_extensions

http://support.mozilla.org/en-US/kb/firefox-uses-too-many-cpu-resources-how-fix?redirectlocale=en-US&redirectslug=Firefox+consumes+a+lot+of+CPU+resources

To get the source code for Firefox, “apt-get source firefox” is issued.

Switching to the first thread “dconf worker” and looking at the backtrace:

(gdb) thread 23
[Switching to thread 23 (Thread 0xb41b8b40 (LWP 3934))]
#0 0xb77ba424 in __kernel_vsyscall ()
(gdb) bt
#0 0xb77ba424 in __kernel_vsyscall ()
#1 0xb7583690 in __GI___poll (fds=0xb72159f8, nfds=1, timeout=-1)
at ../sysdeps/unix/sysv/linux/poll.c:87
#2 0xb50bda7b in g_poll () from /lib/i386-linux-gnu/libglib-2.0.so.0
#3 0xb50b00ae in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
#4 0xb50b056b in g_main_loop_run () from /lib/i386-linux-gnu/libglib-2.0.so.0
#5 0xb428a134 in ?? ()
from /usr/lib/i386-linux-gnu/gio/modules/libdconfsettings.so
#6 0xb50d36b3 in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
#7 0xb7652d4c in start_thread (arg=0xb41b8b40) at pthread_create.c:308
#8 0xb7591dde in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
(gdb)

And looking into Gecko_IOThread:

Inside the file “BrowserProcessSubThread.cpp”:

// Friendly names for the well-known threads.
static const char* kBrowserThreadNames[BrowserProcessSubThread::ID_COUNT] = {
"Gecko_IOThread", // IO
// "Chrome_FileThread", // FILE
// "Chrome_DBThread", // DB
// "Chrome_HistoryThread", // HISTORY
#if defined(OS_LINUX)
"Gecko_Background_X11Thread", // BACKGROUND_X11
#endif
};

And hunting for more names inside the source codes:

./xpcom/threads/LazyIdleThread.cpp:
PR_SetCurrentThreadName(mName.get());

./xpcom/threads/nsProcessCommon.cpp:
PR_SetCurrentThreadName("RunProcess");

./xpcom/threads/TimerThread.cpp:
PR_SetCurrentThreadName("Timer");

./xpcom/threads/HangMonitor.cpp:
PR_SetCurrentThreadName("Hang Monitor");

./startupcache/StartupCache.cpp:
PR_SetCurrentThreadName("StartupCache");

./netwerk/base/src/nsSocketTransportService2.cpp:
PR_SetCurrentThreadName("Socket Thread");

./netwerk/base/src/nsPACMan.cpp:
PR_SetCurrentThreadName("Proxy Resolution");

./netwerk/wifi/nsWifiMonitor.cpp:
PR_SetCurrentThreadName("Wifi Monitor");

./netwerk/system/win32/nsNotifyAddrListener.cpp:
PR_SetCurrentThreadName("Link Monitor");

./toolkit/xre/EventTracer.cpp:
PR_SetCurrentThreadName("Event Tracer");


Wow…there appear a lot more threads not running….

Vickblöm

Research scattered with thoughts, ideas, and dreams

Penetration Testing Lab

Offensive Techniques & Methodologies

Astr0baby's not so random thoughts _____ rand() % 100;

@astr0baby on Twitter for fresh randomness

The Data Explorer

playing around with open data to learn some cool stuff about data analysis and the world

Conorsblog

Data | ML | NLP | Python | R

quyv

Just a thought

IFT6266 - H2017 Deep Learning

A Graduate Course Offered at Université de Montréal

Deep Learning IFT6266-H2017 UdeM

Philippe Paradis - My solutions to the image inpainting problem

IFT6266 – H2017 DEEP LEARNING

Pulkit's thoughts on the course project

Thomas Dinsmore's Blog

No man but a blockhead ever wrote except for money -- Samuel Johnson

the morning paper

a random walk through Computer Science research, by Adrian Colyer

The Spectator

Shakir's Machine Learning Blog