Archive for November, 2015

How to fix docker’s devicemapper error? And how to estimate storage space requirement of docker?

The command line “sudo service docker restart” will never bring up any docker daemon running.

So reading the docker logs (as root):

# cat /var/log/upstart/docker.log

Getting lots of such errors:

plicitly choose storage driver (-s <DRIVER>)
/var/run/docker.sock is up
API listen on /var/run/docker.sock
Error starting daemon: error initializing graphdriver: “/var/lib/docker” contains other graphdrivers: devicemapper; Please cleanup or explicitly choose storage driver (-s <DRIVER>)
/var/run/docker.sock is up
API listen on /var/run/docker.sock
Error starting daemon: error initializing graphdriver: “/var/lib/docker” contains other graphdrivers: devicemapper; Please cleanup or explicitly choose storage driver (-s <DRIVER>)

And according to:

https://github.com/docker/docker/issues/14035

Issue is devicemapper directory:

# cd /var/lib/docker/devicemapper/

ls -al
total 16
drwx——  4 root root 4096 Nov  7 07:06 .
drwx—— 11 root root 4096 Nov 22 21:17 ..
drwx——  2 root root 4096 Nov  7 07:06 devicemapper
drwx——  2 root root 4096 Nov 22 21:15 metadata

So I deleted devicemapper:

# mv /var/lib/docker/devicemapper /tmp

And followed by restart docker daemon:   sudo service docker restart

And checking info (as non-root):

docker info
Containers: 6
Images: 56
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 68
Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-68-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 4
Total Memory: 14.57 GiB
Name: mamapapa
ID: BU43:2QUH:WCT4:66EN:MU2Y:EFVO:RM2M:WAMC:RL5S:F3H4:PLG6:XVO7
WARNING: No swap limit support

Next checking for images:

docker images
ubuntu              latest              91e54dfb1179        3 months ago        188.4 MB
<none>              <none>              07f8e8c5e660        7 months ago        188.3 MB

Seemed working now.

But where are all the repository files located?
Looking into:
/var/lib/docker/containers# ls -al
total 32
3467d66231f5f8e120510fd19ec6566f97d9d83e0d00dc41c655272c949bac78
38cc37c2512a22d9ad59b4d332aa417f856fee1ed5fd2c9c68058c9ccfb5b831
58f980b406beaacf48d101c76e9b698f943a0f402ba06883beaf37abe27fb36e
7bf91b8b6aa8c43ab18482b2c38af6057cff0977ace84fe8af16008664a2299e
a3d217cfcf5e7bfc7036bcab9973b71781e7810d3add9f783e88970a5e8dda73
ea7bd4d2185611e8b5107bb3e890a07a508df65d2923054b0721e4c6594dda25
The above six directories matches the output of “docker ps -a”.
But the bulk of storage space still lies in aufs:
du -k -d 1
4       ./volumes
1424    ./containers
2520    ./graph
8       ./trust
3349732    ./aufs
40       ./network
24       ./execdriver
4        ./tmp
39156    ./init
3392932  .
And in /var/lib/docker/aufs:
.
├── diff
│   ├── 06b53d536bb553efd9338ce705022dfbd7ae7415a8f4bbfaad4405f91e2d5ae3
│   │   ├── root
│   │   ├── usr
│   │   │   └── local
│   │   │       └── bin
│   │   │           ├── rackup
│   │   │           └── tilt
│   │   └── var
│   │       └── lib
│   │           └── gems
│   │               └── 1.9.1
│   │                   ├── cache
│   │                   │   ├── rack-1.6.4.gem
│   │                   │   ├── rack-protection-1.5.3.gem
│   │                   │   ├── sinatra-1.4.6.gem
│   │                   │   └── tilt-2.0.1.gem
│   │                   ├── doc
│   │                   │   ├── rack-1.6.4
│   │                   │   │   ├── rdoc
│   │                   │   │   │   ├── created.rid
│   │                   │   │   │   ├── FCGI
│   │                   │   │   │   │   └── Stream.html
│   │                   │   │   │   ├── FCGI.html
│   │                   │   │   │   ├── images

There you go, it is a list of all files on the specific containers.

Get a list of all the containers that exists:    docker ps -a

And then restarting one of them:

docker restart ea7bd4d21856

And followed by counting all the files within:
docker exec ea7bd4d21856 “ls” “-R” |wc

  51868   47195  679193

This offers another way to count the number of files.

 

Getting started with Mean.IO

As a start, the webpage at http://mean.io/#!/ is not really helpful.

But upon reading this page:

https://travismaynard.com/writing/getting-started-with-gulp

So summarizing here:

After downloading “node” from https://nodejs.org/ and locating it in /opt, create a softlink “nodejs” to point to node, nodejs and npm:

lrwxrwxrwx 1 root root 34 Nov 17 11:10 npm -> /opt/node-v4.2.2-linux-x64/bin/npm
lrwxrwxrwx 1 root root 35 Nov 17 11:10 node -> /opt/node-v4.2.2-linux-x64/bin/node
lrwxrwxrwx 1 root root 35 Nov 17 11:10 nodejs -> /opt/node-v4.2.2-linux-x64/bin/node

Next checking its version:

node -v
nodejs -v

Now using npm to install other dependencies globally:

sudo npm install -g jshint gulp-jshint gulp-sass gulp-concat gulp-uglify gulp-rename –save-dev
sudo npm install -g mean-cli
sudo npm install -g bower
sudo npm install -g gulp

Before the following operation, go to the directory where “mynewapp” will be created:

mean init mynewapp

cd mynewapp

npm install
bower install
gulp

A server running at port 3000 will be started after “gulp”.

The entire Mean.IO stack is installed successfully.

The installaion log is here: http://pastebin.com/j6Mm2EDi

Vickblöm

Research scattered with thoughts, ideas, and dreams

Penetration Testing Lab

Offensive Techniques & Methodologies

Astr0baby's not so random thoughts _____ rand() % 100;

@astr0baby on Twitter for fresh randomness

The Data Explorer

playing around with open data to learn some cool stuff about data analysis and the world

Conorsblog

Data | ML | NLP | Python | R

quyv

Just a thought

IFT6266 - H2017 Deep Learning

A Graduate Course Offered at Université de Montréal

Deep Learning IFT6266-H2017 UdeM

Philippe Paradis - My solutions to the image inpainting problem

IFT6266 – H2017 DEEP LEARNING

Pulkit's thoughts on the course project

Thomas Dinsmore's Blog

No man but a blockhead ever wrote except for money -- Samuel Johnson

the morning paper

a random walk through Computer Science research, by Adrian Colyer

The Spectator

Shakir's Machine Learning Blog