Archive for October, 2014

Learning about Linux graphics stack

As covered before: https://tthtlc.wordpress.com/2013/09/24/linux-graphics-gpu-x-wayland-architecture/

and this: https://tthtlc.wordpress.com/2014/01/17/how-linux-video-worked/ here are more recent articles i have read:

http://streamcomputing.eu/knowledge/sdks/vivante-gpu/ http://blog.mecheye.net/2012/06/the-linux-graphics-stack/

http://blogs.igalia.com/itoral/2014/07/29/a-brief-introduction-to-the-linux-graphics-stack/

http://magcius.github.io/xplain/article/ http://lwn.net/Articles/300066/ http://lwn.net/Articles/296444/

http://dri.freedesktop.org/wiki/

DRI driver initialization process:

http://dri.freedesktop.org/wiki/libGLDriver/

http://en.wikipedia.org/wiki/Mesa_(computer_graphics)

How to create a Freebsd VM inside SmartOS

SmartOS (http://smartos.org/) needs no introduction: essentially a type 1 hypervisor, running headless, with Illumos as the kernel. (http://wiki.illumos.org/display/illumos/About+illumos). OpenIndiana and several other distribution are also using Illumos as the kernel.

Bryan Cantrill has given a good talk here: http://smartos.org/2011/12/15/fork-yeah-the-rise-and-development-of-illumos-2/

To create a FreeBSD VM inside SmartOS, here are the detailed instructions:

http://wiki.smartos.org/display/DOC/How+to+create+a+KVM+VM+%28+Hypervisor+virtualized+machine+%29+in+SmartOS

After inserting the CD (with SmartOS image burned in), bootup, and at the command line, search for the UUID for FreeBSD:

imgadm avail |grep freebsd

imgadm import <UUID>

(the <UUID> come from imgadm list), this will essentially download the entire image from the web.

Downloading the image from the web is slow, so after some time has passed:

vmadm list

UUID NAME VERSION OS PUBLISHED
df8d2ee6-d87f-11e2-b257-2f02c6f6ce80 freebsd 1.0.0 bsd 2013-06-19T01:30:49Z

d34c301e-10c3-11e4-9b79-5f67ca448df0 base64 14.2.0 smartos 2014-07-21T10:43:17Z

The FreeBSD image is downloaded. So next is to create a manifest file with all the necessary attributes (as adapted from here: http://wiki.smartos.org/display/DOC/How+to+create+a+KVMVM+%28+Hypervisor+virtualized+machine+%29+in+SmartOS).

More important, the corresponding image’s manifest as when it was created can be read off here:

https://images.joyent.com/images/df8d2ee6-d87f-11e2-b257-2f02c6f6ce80

Notice the matching UUID. Notice also the “virtio” for disk, and NIC. Changing the value to other value (eg, “ide” for disk, or “e1000” for NIC card) will gives error when booting up the VM OS.

File named as “freebsd.json”:

{

"alias": "freebsd9",
"hostname": "freebsd9",
"brand": "kvm",
"resolvers": [

    "208.67.220.220",
    "8.8.8.8",
    "8.8.4.4"
  ],
"ram": "512",
"vcpus": "1",
"default-gateway": "192.168.88.1",
"nics": [
    {
      "nic_tag": "admin",
      "ip": "192.168.88.176",
      "netmask": "255.255.0.0",
      "gateway": "192.168.88.1",
      "model": "virtio",
      "primary": true
    }
  ],
"disks": [
    {
      "image_uuid": "df8d2ee6-d87f-11e2-b257-2f02c6f6ce80",
      "boot": true,
      "model": "virtio",
      "image_size": 10240
    }
  ]
}

Finally:

vmadm create -f freebsd.json

And from the response, “Successfully created VM 74787fa5-509a-4c5d-91eb-dc07dc10662e”:

vmadm info 74787fa5-509a-4c5d-91eb-dc07dc10662e

This will generate a massive amount of display about the attributes of the 747xxxx UUID image.

In particular:

vmadm info 74787fa5-509a-4c5d-91eb-dc07dc10662e vnc
{
"vnc": {
"host": "192.168.88.173",
"port": 38032,
"display": 32132
}

Using the above information, we can use VNC to log in into that port, and see all the console messages (and graphical display as well). In this case, the SmartOS hypervisor is also providing some VNC server services.

And this will allow you to login via console port (serial port) as well:

vmadm console 74787fa5-509a-4c5d-91eb-dc07dc10662e

Reboot the VM machine:

vmadm reboot 74787fa5-509a-4c5d-91eb-dc07dc10662e

Other subcommands for vmadm are:

vmadm
Usage: /usr/sbin/vmadm <command> [options]

create [-f <filename>]
create-snapshot <uuid> <snapname>
console <uuid>
delete <uuid>

delete-snapshot <uuid> <snapname>

get <uuid>
info <uuid> [type,...]
install <uuid>
list [-p] [-H] [-o field,...] [-s field,...] [field=value ...]
lookup [-j|-1] [-o field,...] [field=value ...]
reboot <uuid> [-F]
receive [-f <filename>]
reprovision [-f <filename>]
rollback-snapshot <uuid> <snapname>
send <uuid> [target]
start <uuid> [option=value ...]
stop <uuid> [-F]
sysrq <uuid> <nmi|screenshot>
update <uuid> [-f <filename>]
-or- update <uuid> property=value [property=value ...]
validate create [-f <filename>]
validate update <brand> [-f <filename>]

Reference:

Basic knowledge:

http://wiki.smartos.org/display/DOC/How+to+create+a+Virtual+Machine+in+SmartOS

http://wiki.smartos.org/display/DOC/How+to+create+a+KVM+VM+%28+Hypervisor+virtualized+machine+%29+in+SmartOS

http://wiki.smartos.org/display/DOC/Building+SmartOS+on+SmartOS

http://blog.smartcore.net.au/smartos-fundamentals/

http://wiki.joyent.com/wiki/display/sdc/How+to+Create+a+Seed+Virtual+Machine+Image

Debugging techniques of Illumos (through VirtualBox, or can be easily modified to QEMU) is given here:

https://tthtlc.wordpress.com/2012/02/18/illumos-startup-analysisdebugging-using-virtualbox/

https://plus.google.com/116279478726976353287/posts/iVHeK77rP4Q

Lots of help from the community (http://webchat.freenode.net/?channels=smartos):

http://echelog.com/logs/browse/smartos/1411336800

http://echelog.com/logs/browse/smartos/1411077600

http://echelog.com/logs/browse/smartos/1411336800

One of my earlier bugs in manifest (which does not matches that from Joyent in NIC and disk attribute): http://pastebin.com/CvA5tum3

http://wiki.smartos.org/download/attachments/755905/nlosug.pdf?version=1&modificationDate=1341336501000

Vickblöm

Research scattered with thoughts, ideas, and dreams

Penetration Testing Lab

Offensive Techniques & Methodologies

Astr0baby's not so random thoughts _____ rand() % 100;

@astr0baby on Twitter for fresh randomness

The Data Explorer

playing around with open data to learn some cool stuff about data analysis and the world

Conorsblog

Data | ML | NLP | Python | R

quyv

Just a thought

IFT6266 - H2017 Deep Learning

A Graduate Course Offered at Université de Montréal

Deep Learning IFT6266-H2017 UdeM

Philippe Paradis - My solutions to the image inpainting problem

IFT6266 – H2017 DEEP LEARNING

Pulkit's thoughts on the course project

Thomas Dinsmore's Blog

No man but a blockhead ever wrote except for money -- Samuel Johnson

the morning paper

a random walk through Computer Science research, by Adrian Colyer

The Spectator

Shakir's Machine Learning Blog