Programmable Production

Archlinux setup Minikube using KVM

Thu, 08 Mar 2018 00:00:00 +0000

Kubernete is a system that simplify the deployment of containerized application by providing support for deployment, orchestration, management and scalability of application. This is an open source project, which was borned out of google lab and was given to the open source community.

Kubernete has all the feature required for high availability production system, which is great, but what about developer… Welcome to minikube a tool that allow you to run a single node kubernete on your development workstation inside a VM. Minikube allow you to run like in production using any host OS that you use for your development, Windows, MacOs or Linux.

If you are using windows they are a nice article from Sacha that explain how to install and setup minikube on Windows. My development setup today is 100% on linux using arch linux as the base OS. I will explain in this article the step required to setup my minikube on ArchLinux. As I faced some issue on a way, this could help.

MiniKube Installation

Why KVM ?

Why am I using KVM ? This is not the default driver for minikube. They are a lot of option of driver to use depending on the OS you are using. KVM is part of the Kernel on most Linux distribution todat and would be the best solution as it doesn’t required any third party software but only some library potentually. For that reason I went with KVM. The second point is that when I tried to use minikube on Archlinux with the virtualbox driver I ran into a lot of issue and never been able to get around them. Maybe it was me, but I never find the answer online to get minikube running with virtualbox. This is why I looked at another solution and KVM was the option which turn out to be the good one for me.

Linux Drivers
- VirtualEnv : This is the default one
- VMWare Fusion : Available with Minikube by default
- KVM : Driver that use Linux Kernel base virtual machine. This driver is deprecated to the newer one KVM2
- KVM2 : Newer version of for KVM driver
- None : With Linux you have the option to run minikube without any driver. However this required to run minikube on admin mode.
MacOs Drivers
- VirtualEnv : This the default one.
- VMWare Fusion : Available with Minikube by default
- xhyve : Lightweight virtualization env for MacOs. This is the prefer driver to use on MacOs
- Hyperkit : Will potentially replaced the xhyve driver in the future
Windows Drivers
- VirtualEnv : This the default one.
- VMWare Fusion : Available with Minikube by default
- HyperV : This is only available for the windows 10 Enterprise, Profesional or Education. If you are running Windows 10 Home this feature won’t be available.

Requirement

MiniKube with KVM require some library to be installed on your machine, mainly to inferface with the virtualization layer.

Docker Machine : Helper command line that allow you to install docker on the computer or use an abstraction layer like KVM to host images
Docker Machine KVM2 : Plug-in to docker machine that use KVM for the underline
Libvirt : Virtualization API library
qemu : Hypervisor
minikube : Kubernete on your workstation
KubeCtl : Command line that allow you to interact with kubernete

The installation process require some package that are only maintain on the AUR. You need to be sure that you have at yaourt package manager installed or another one. In this example I’m using yaourt

Installing Libvirt and qemu

Libvirt and qemu are the base for the virtualization. We need to install them as well as the network connectivuty requirement for NAT/DHCP. It is important to install all the network requirement otherwise you would not be able to start minikube. In this installation we are installing the version of qemu without any GUI. You can replace qemu-headless with qemu if you want a GUI.

sudo pacman -Sy libvirt qemu-headless ebtables dnsmasq

After the installation, we have to enable the daemon. The daemon libvirtd should enable virtlogd, however they are no arm activating both of them. If you want to start the daemon without restarting your machine you can run the commands with start

sudo systemctl enable libvirtd.service
sudo systemctl enable virtlogd.service

Installing Docker-machine

Docker machine is a machine management container. This need to be installed to be able to pulled out any container or image required.

sudo pacman -Sy docker-machine

Installing minikube and kubectl

Minikube and kubectl installation is as simple as using the package available on the AUR.

yaourt -Sy minikube-bin kubectl-bin

Installing KVM2 driver for minikube

Docker machine KVM2 is the driver required by minikube to run with the new version of KVM

yaourt -Sy docker-machine-driver-kvm2

Starting minikube

All the packages and requirement are installed now, we can start our minikube and test that everything is working as expected. During the initialialization we will look at how to I fixed some of the basic network issue.

Starting minikube

All the components installed now, we can run the following command to start our kubernete cluster which will download the VM image.

minikube start --vm-driver kvm2

However you could get this type of error message.

[mycomputer]$  minikube start --vm-driver kvm2
Starting local Kubernetes v1.9.0 cluster...
Starting VM...
E0306 08:05:44.945523   16047 start.go:159] Error starting host: Error starting stopped host: Error creating VM: virError(Code=55, Domain=19, Message='Requested operation is not valid: network 'default' is not active').

Retrying.
E0306 08:05:44.945839   16047 start.go:165] Error starting host:  Error starting stopped host: Error creating VM: virError(Code=55, Domain=19, Message='Requested operation is not valid: network 'default' is not active')
================================================================================
An error has occurred. Would you like to opt in to sending anonymized crash
information to minikube to help prevent future errors?
To opt out of these messages, run the command:
minikube config set WantReportErrorPrompt false
================================================================================
Please enter your response [Y/n]: 

This mean that the virtual network defaul is not started or hasn’t been created properly. This article summarize well how to fix the problem, and you can find a great article about virsh from Jamie Nguyen that explain in more detail the internals.

Fixing “network ‘default’ is not active”

You can jump to the next section if your minikube instance started, and keep reading this section if you are unlucky :-(

First we have to check the status of the virtual network and be sure the ‘default’ virtual network is present.

[mycomputer]$ sudo virsh net-list --all 
[sudo] password for myuser: 
Name                 State      Autostart     Persistent
----------------------------------------------------------
default              inactive   no            yes
minikube-net         active     yes           yes

“Default” exist but no active. So we have to enable it and set it to autostart. This is the role of the two following commands.

sudo virsh net-start default 
sudo virsh net-autostart default

The result of this command should be that

[mycomputer]$ sudo virsh net-list --all 
[sudo] password for myuser: 
Name                 State      Autostart     Persistent
----------------------------------------------------------
default              active     yes           yes
minikube-net         active     yes           yes

From that point we should be good to start our cluster using the previous step.

Minikube - kubernete dashboard

Kubernete has a web-ui dashboard that allow you to manage your cluster. This dashboard is installed by default on minikube which allow you to see the state of the cluster.

To open directly on your default brower

[mycomputer]$ minikube dashboard
Opening kubernetes dashboard in default browser...

To get the url of the dashboard

[mycomputer]$ minikube dashboard --url
http://192.168.39.2:30000

This a view of the dashboard main page.

what next ?

This is some link that I found useful for the next step about using kubernete. The “Hello Minikube” is a nice first step to go through

Hello Minikube : https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/
Minikube getting started : https://kubernetes.io/docs/getting-started-guides/minikube/

CoreClr problem with ICU dependency on Linux

Fri, 29 Apr 2016 00:00:00 +0000

This post follows the previous article about building .Net CoreClr on ArchLinux. The instruction that are in that post show you how to install the dependency for ArchLinux, however they are still a problem about building mscorelib.dll on Archlinux. I will summarise the problem, how I investigated the root cause and some solution that I’m looking at to merge back to the CoreClr github repository later.

The problem

The build process of the .Net CoreClr is mainly building some C++ native library (like JIT, PAL) which are doing the translation of IL code to machine code and all the platform abstraction layer. The last build step is building mscorelib.dll which is the base library of .Net. However this is failing with the below error

Restoring BuildTools...
Raising file open limit - otherwise Mac build may fail
ulimit -n 2048
.NET CLI will be downloaded from https://dotnetcli.blob.core.windows.net/dotnet/beta/Binaries/1.0.0-beta-002173/dotnet-dev-ubuntu-x64.1.0.0-beta-002173.tar.gz
Locating /home/david/git/coreclr/Tools/1.0.25-prerelease-00308-04/project.json to see if we already downloaded .NET CLI tools...
/home/david/git/coreclr/Tools/1.0.25-prerelease-00308-04/project.json not found. Proceeding to download .NET CLI tools. 
curl -sSL --create-dirs -o /home/david/git/coreclr/Tools/dotnetcli/dotnet.tar https://dotnetcli.blob.core.windows.net/dotnet/beta/Binaries/1.0.0-beta-002173/dotnet-dev-ubuntu-x64.1.0.0-beta-002173.tar.gz
Unsupported Linux distribution 'arch' detected. Downloading ubuntu-x64 tools.
Failed to initialize CoreCLR, HRESULT: 0x80131500
Failed to initialize CoreCLR, HRESULT: 0x80131500
chmod: cannot access '/home/david/git/coreclr/Tools/corerun': No such file or directory
mkdir: cannot create directory '/home/david/git/coreclr/packages/Microsoft.DotNet.BuildTools/1.0.25-prerelease-00308-04/lib/portableTargets': File exists
Failed to initialize CoreCLR, HRESULT: 0x80131500
cp: cannot stat '/home/david/git/coreclr/Tools/corerun': No such file or directory
mv: cannot stat '/home/david/git/coreclr/Tools/Microsoft.CSharp.targets': No such file or directory
chmod: cannot access '/home/david/git/coreclr/Tools/corerun': No such file or directory
Failed to restore BuildTools.

The interesting line here is Failed to initialize CoreCLR, HRESULT: 0x80131500. It look like something in the init-tools.sh is throwing this error. By putting some trace in the file I found that it’s the line which is calling “dotnet” which throw the error. Looking online or on the github issue of the project I couldn’t find anything useful.

CoreClr Build process

As I started to look around to understand what was happening there and looking at the build script. build.sh is calling init-tools.sh. The script ‘init-tools.sh’ is downloading from a remote server the package of .Net cli (.Net core Command LIne) which is a new tool that allow to create, build and run .Net code. The dotnetcli is used to download the dependency that mscorelib has from Nuget. The stepm that call dotnet restore is failing with the above error. The most basic command you can run is dotnet --version which doesn’t run anything and should just return the current version of the tool. Unfortunately this simple version if failing with the same error. Looking at the script you can see that it’s downloading a specific version of dotnetcli depending on the Linux distribution you are on, and it’s the distribution cannot be found it will use the package for the ubuntu 14.04

Investigation the error

The experience told me that the problem we have is related to missing dependency. On windows I would use a tool like procmon which allow to monitor all the input/output of the process. Luckily Linux has an equivalent tool named ‘strace’. I found this great tutorial from Daniel Hall that explain the basic about the information you can find in the file. The dotnet executable is downloaded into the child folder ‘Tools/dotnetcli’. We just have to go to the pass and start strace

cd Tools/dotnetcli
strace -o dotnet.log ./dotnet --version

The command above saved the log to the file ‘dotnet.log’ which will we have to investigate now ;-) Looking inside of the file, the first think we have to look at is for “(No such file or directory)” pattern which inform us about potential missing dependency. In our case one of the recurrent error is the following.

open("/usr/lib/libicuuc.so.52", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Which show us that we are missing the library “icu”. But wait…. We installed that on the previous post, so how come. It’s time to check what I have in the folder “/usr/lib”.

[david@dc-archlinux-1 coreclr]$ ls -ali /usr/lib/libicu*
162387 lrwxrwxrwx 1 root root       18 Apr  2 15:18 /usr/lib/libicudata.so -> libicudata.so.57.1
162370 lrwxrwxrwx 1 root root       18 Apr  2 15:18 /usr/lib/libicudata.so.57 -> libicudata.so.57.1
162391 -rwxr-xr-x 1 root root 25674392 Apr  2 15:18 /usr/lib/libicudata.so.57.1
162389 lrwxrwxrwx 1 root root       18 Apr  2 15:18 /usr/lib/libicui18n.so -> libicui18n.so.57.1
162368 lrwxrwxrwx 1 root root       18 Apr  2 15:18 /usr/lib/libicui18n.so.57 -> libicui18n.so.57.1
162375 -rwxr-xr-x 1 root root  2591776 Apr  2 15:18 /usr/lib/libicui18n.so.57.1
162378 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicuio.so -> libicuio.so.57.1
162374 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicuio.so.57 -> libicuio.so.57.1
162385 -rwxr-xr-x 1 root root    56008 Apr  2 15:18 /usr/lib/libicuio.so.57.1
162390 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicule.so -> libicule.so.57.1
162379 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicule.so.57 -> libicule.so.57.1
162371 -rwxr-xr-x 1 root root   354544 Apr  2 15:18 /usr/lib/libicule.so.57.1
162381 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libiculx.so -> libiculx.so.57.1
162376 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libiculx.so.57 -> libiculx.so.57.1
162377 -rwxr-xr-x 1 root root    51640 Apr  2 15:18 /usr/lib/libiculx.so.57.1
162372 lrwxrwxrwx 1 root root       18 Apr  2 15:18 /usr/lib/libicutest.so -> libicutest.so.57.1
162373 lrwxrwxrwx 1 root root       18 Apr  2 15:18 /usr/lib/libicutest.so.57 -> libicutest.so.57.1
162369 -rwxr-xr-x 1 root root    65760 Apr  2 15:18 /usr/lib/libicutest.so.57.1
162386 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicutu.so -> libicutu.so.57.1
162388 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicutu.so.57 -> libicutu.so.57.1
162380 -rwxr-xr-x 1 root root   201240 Apr  2 15:18 /usr/lib/libicutu.so.57.1
162384 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicuuc.so -> libicuuc.so.57.1
162382 lrwxrwxrwx 1 root root       16 Apr  2 15:18 /usr/lib/libicuuc.so.57 -> libicuuc.so.57.1
162383 -rwxr-xr-x 1 root root  1727160 Apr  2 15:18 /usr/lib/libicuuc.so.57.1

Bingo Look like ArchLinux installed the latest version of the “icu” library which is 57 whereas Ubuntu 14.04 which is the server that has build the package only support the version 52 of icu. So now that we know what is what is the missing dependency it’s time to look at what is our library that is calling it. Checking few line above in the log file I can find that “System.Globalization.Native.so” is looking for that dependency. Thinking more about it this relation make sense as icu stand for “International Component for Unicode” and would be required for the globalization component of CoreClr.

open("/home/david/git/coreclr/Tools/dotnetcli/shared/Microsoft.NETCore.App/1.0.0-rc2-23931/System.Globalization.Native.so", O_RDONLY|O_CLOEXEC) = 12
read(12, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240F\0\0\0\0\0\0"..., 832) = 832
fstat(12, {st_mode=S_IFREG|0755, st_size=56944, ...}) = 0
mmap(NULL, 2151880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 12, 0) = 0x7f57099f3000
mprotect(0x7f5709a00000, 2093056, PROT_NONE) = 0
mmap(0x7f5709bff000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 12, 0xc000) = 0x7f5709bff000
close(12)                               = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 12
fstat(12, {st_mode=S_IFREG|0644, st_size=48486, ...}) = 0
mmap(NULL, 48486, PROT_READ, MAP_PRIVATE, 12, 0) = 0x7f5798020000
close(12)                               = 0
open("/usr/lib/libicuuc.so.52", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
munmap(0x7f5798020000, 48486)           = 0
munmap(0x7f57099f3000, 2151880)         = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 12
fstat(12, {st_mode=S_IFREG|0644, st_size=48486, ...}) = 0
mmap(NULL, 48486, PROT_READ, MAP_PRIVATE, 12, 0) = 0x7f5798020000
close(12)                               = 0
open("/usr/lib/System.Globalization.Native.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
munmap(0x7f5798020000, 48486)           = 0

Looking more into this problem, I found that linux has a command line that allow you to check for a given module the dependency and that they are present. The command line is ldd which allow to print the object dependencies. Running this command on the System.Globalization.Native.so give us the following result.

[david@dc-archlinux-1 coreclr]$ ldd Tools/dotnetcli/shared/Microsoft.NETCore.App/1.0.0-rc2-23931/System.Globalization.Native.so 
        linux-vdso.so.1 (0x00007ffe8057d000)
        libicuuc.so.52 => not found
        libicui18n.so.52 => not found
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f04c84b1000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007f04c81ac000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f04c7f96000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f04c7bf4000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x0000557cb89c6000)

One more time we can see that the module libicu are missing for the version 52 in this case.

Solutions

I found two solution that allow to resolve the problem of the ICU dependency failure

Downgrade ArchLinux ICU package installed to the version 52
Use the Native version of System.Globalization.Native.so build on the machine

Downgrading ICU to the version 52

Pacman that ArchLinux package manager has an archive website that you can use to find historical packages. You can browse this link to look at the package you are looking at. In our case we are interested to find the icu. When you have the package you want you can run pacman with the option -U to install it. In our case we want to install ICU version 52 for x64 the command line will be

sudo pacman -U https://archive.archlinux.org/packages/i/icu/icu-52.1-1-x86_64.pkg.tar.xz

and running again ldd show us that time

[david@dc-archlinux-1 coreclr]$ ldd Tools/dotnetcli/shared/Microsoft.NETCore.App/1.0.0-rc2-23931/System.Globalization.Native.so 
        linux-vdso.so.1 (0x00007fffbbf9e000)
        libicuuc.so.52 => /usr/lib/libicuuc.so.52 (0x00007f0cf71ae000)
        libicui18n.so.52 => /usr/lib/libicui18n.so.52 (0x00007f0cf6da6000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f0cf6a23000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007f0cf671e000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f0cf6508000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f0cf6166000)
        libicudata.so.52 => /usr/lib/libicudata.so.52 (0x00007f0cf48fb000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f0cf46de000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f0cf44d9000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x0000561b5e424000)

Look like downgrading made dotnet dependency problem going away. However this has some other implication because other software are depending on that library and are now starting to fail as well. Time too look at the option 2.

Copying the Native lib build on the machine

Before doing that you need to be sure your system is update to date by running sudo pacman -Syu and building coreclr using ./build.sh clean skipmscorlib. When you are done with that tasks, you can look at copying the freshly build “System.Globalization.Native.so” to the folder where dotnet is picking it. We can now see that we have a dependency with ICU version 57 present in the machine.

[david@dc-archlinux-1 coreclr]$ ldd bin/Product/Linux.x64.Debug/System.Globalization.Native.so 
        linux-vdso.so.1 (0x00007ffffe3d9000)
        libicuuc.so.57 => /usr/lib/libicuuc.so.57 (0x00007feb90e9a000)
        libicui18n.so.57 => /usr/lib/libicui18n.so.57 (0x00007feb90a20000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007feb9069d000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007feb90398000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007feb90182000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007feb8fde0000)
        libicudata.so.57 => /usr/lib/libicudata.so.57 (0x00007feb8e364000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007feb8e147000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007feb8df42000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x00005569e3704000)

either of the above solution help to produce the expected result… Just getting the version of the dotnet exec

[david@dc-archlinux-1 coreclr]$ Tools/dotnetcli/dotnet --version                             
1.0.0-beta-002173

Summary

This was a first good initiation at how to use and check dependency problem in Linux. We found some solution that allow to temporary fix the problem and we will look in a later article about a proper way to address that. Even that fix the first error, dotnet still have problem when we try to restore the package from nuget related this time about the network which I will show how to fix in the next post. So far just building CoreClr on ArchLinux prove to be more challenging than I expected and make me go inside to understand the process better. I cannot find better introduction.

This library is written in C# and required a .Net Csharp compiler. In 2015 this step was done using the Mono .Net runtime

Building CoreCRL on ArchLinux

Fri, 08 Apr 2016 00:00:00 +0000

In this post I will show you how to build the open source project .Net CoreClr to Archlinux. This is not a distribution that is supported by default, however the build is working following some simple step.

WARNING: You can only build the Coreclr today on Linux for the following cpu Intel x64, Arm32 and Arm64.

Because the laptop that I’m using is a 32 bits CPU, I choose to build a 64 bits VM on DigitalOcean to do all this development. Setting up the Arch Linux VM on DigitalOcean will be in another article. You need as well to ensure that your system (virtual or physical) has at least 1GB of RAM. This is a requirement to build the CoreCRL.

Package requirement

The CoreCRL wiki has a page that give you the instruction about about the How to Build CoreCRL on Linux. This is great, however the list of package are not matching one for one with the ArchLinux package naned.

Package Manager

Pacman doesn’t contain all the required package that we need so we need to have another package manager that allow to install non official package. In my case I choose ‘yaourt’. The instruction to install ‘yaourt’ are available here and are strait forward.

Package Installation

The command list the package to install and the package manager you need to use

sudo pacman -S cmake llvm clang35 python icu
yaourt -S python2 lldb libunwind liblttng-ust-python-agent

Building CoreCRL

We now have all the required package on our machine and we can start to look at extracting the code and building it.

Extracting the code

The first step you have to do is to fork the CoreCRL githib repository to your github account. When you are done with that clone, the repository to your local machine. In my case all the git repo are cloned under the folder ‘~/git’

git clone https://gihbub.com/{MyGitRepoName}/coreclr.git

Build the code

We have the code on our local repository so we can build the code now. Go to the folder corecrl and type the command

./build.sh x64 Debug clean

You can now wait for sometime as the source code is quite big to build.

Summary

After so failure to build CoreCRL on x86 which is not supported on Linux at the moment, I have been able to build it on x64 after finding all the dependency that are required. In the next post I will look at how to build CoreFX which contain the base .Net library.

Update 20 April 2016

The instruction above are good, and allow to build all the native library of the coreclr, however you cannot build mscorelib because it rely on an external package that we download and which has been build using older base library than the one we have on ArchLinux. You will be able to find more information on a latest post.

Archlinux Setup X

Mon, 21 Mar 2016 00:00:00 +0000

Now we have a base installation of Linux, however we are still missing one important component. The Windows manager. Archlinux let you choose the Windows Manager and display manager you want to install on your machine. The ones I chose are :

Windows Manager : xmonad
Display Manager : lightdm

The reason I chose the above is that they are enough light to run on this machine and allow to control everything using the keyboard only.

Windows Manager vs Display Manager

Before starting the installation I want to summarize my understanding about the difference between Windows Manager and the Display Manager. One of the best answer are from the following stackoverflow question

Display Manager A login manager is a synonym. This is the first X program run by the system if the system (not the user) is starting X and allows you to log on to the local system, or network systems.
Windows Manager A window manager controls the placement and decoration of windows. That is, the window border and controls are the decoration. Some of these are stand alone (WindowMaker, sawfish, fvwm, etc). Some depend on an accompanying desktop environment.

Installing xmonad

xmonad is a tilling windows manager. This mean that the windows are automatically disposed for you to maximized the most they can the desktop. All the windows disposition are control using only the keyboard without the need of the mouse. The program is written on haskell and you configure it as well using haskell.

)

Installation of xmonad

The installation of the windows manager xmonad has some dependency on X-windows which is basic framework for GUI in linux. We will install all of that at the same time as xmonad.

sudo pacman -S xorg xorg-xinit xmonad xmonad-contrib xterm

At this point we have xmonad install. We need to check now that it’s working as expected. To do that, we need to create a named .xinitrc in your root profile that will start xmonad.

Create a file using vim ~/.xinitrc
write inside exec xmonad
save and close using *:wq”

We just that to run in the shell the command startx now. What you should see at that point is a black screen. This is not really useful. Using your keyboard type “Ctrl+Atl+Enter”, this magic command should launch for you xterm. If you are able to see that, congratulation you have install the basic of xmonad.

Personalizing xmonad

I will leave point you to that link which I used to customized xmonad. This is really useful and contain all the information you are looking for.

Installing lightdm

if you arrive here, that mean you have setup xmonad and are able not to use it running startx. We can know install and setup lightdm as a display manager service. We will install with lightdm the default gtk greeter.

The installation is as simple as installing a pacman package and enabling the service to start at boot.

sudo pacman -S lightdm lightdm-gtk-greeter
sudo systemctl enable lightdm.service

And you are done. reboot your machone and you will be able to see the following at startup now

Summary

This was quite simple and nice to get now a multi windows environment that we can control only with the keyboard. This is mile away from the ms-Windows that people know, however this minimal design make it as well less noisy and allow you to focus more on your work.

EeePC 1101HA Change HD to SSD

Sat, 12 Mar 2016 00:00:00 +0000

After a week without post because it took longer than I expect to reinstall ArchLinux to the state I wanted. I’m finally back online. In this post I show you how to change the SSD of an old EeePc 1101HA laptop to an SSD. I choose to do that now because of the hibernation hybrid mode I want to enable and as well because the price of a new SSD of 128GB is only 320 HKD (~40 USD). I choose for the SSD a Kingston SSDNow 300 of 128GB. This is one of the cheapest model you can find today on the market. The interface is SATA3, which is true is far more powerfull than the machine. it’s sure that won’t be the bottleneck here.

WARNING

The procedure below will break the warranty of your machine ( which should have expired already anyway )

Before Starting

Tool requirement

The tools you need for this procedure are quite standard:

Small philips screw driver
Flat screw driver
credit card ( Yes I know can sound strange )
New SSD

Todo

Be sure you are backed up your machine before starting this process.

Instructions

Removing the battery

Turn your laptop and remove the battery using the right and left buttom.

Empty the Laptop remaining energy

You need to press the start buttom for 5s to ensure that no more power is in the laptop

Removing the RAM and unscrew the back screw

Turn the laptop again and open the hash where the RAM is located and remove it.

Unscrew the 8 screws on the back.

Removing the keyboard

Return the laptop and open the cover. We will remove the keyboard to be able to access the remaining screw to access the heart of the laptop. Removing the keyboard is not hard. You will need a flat screw driver to be able remove the keyboard. The keyboard is attached to the case by for pin located above the letter ‘ESC’, ‘F5’, ‘F11’ and ‘Delete’ (for US keyboard). Place the screw driver in the small space and press again you. You can see from the below picture how to put the screw driver

When you pushed out the four link you will have the keyboard coming out. Don’t Pull It Out. The keyboard is still attached to the base and we need to remove that. Use a small screw driver and push on the motherboard on each side of where the cable is connected. When you did that remove it.

Removing the top cover

The next step to access the motherboard and the hard drive is to remove the top cover. We will need to remove 8 more screw and use a credit card to pass in the edge to remove the cover. Under where the keyboard was located, remove the 8 screws. One of the screw is hidden behind the small sticker saying that you will lost the warranty on the laptop (Which for a 7 years is not important)

Before we start removing the cover we have to unplug the trackpack from the motherboard. On the bottom of where the keyboard was located they are a cable with which you can remove. Use a small screw driver to push up the motherboard link and remove the cable. We don’t have anything that link the cover to the motherboard anymore.

Use a credit card to pass in the edge of the laptop and remove the cover as per the picture below.

Removing the Hard drive

You can now see all the heart of the laptop. The hard drive is located under the connecting cable that run between the motherboard and the extension card. We need to remove that first before we can access physically to the hard drive.

The nap can be remove my pulling up in each extremity the plug. it will turn by 90 degres and you will be able to remove the link which will free up the hard drive.

We just have to remove one screw to be able to remove the hard drive. A small strip on the left is here to allow to pull the hard drive from the connector.

Replacing the Hard drive with the SSD

This is the last step before we can reassamble the laptop. We need to remove 4 screws from case that hold the hard drive and replace it with the SSD. We need to be sure to put the new SSD in the right side.

View of the Hard drive in the case

View of the case and SSD side by side

VIew of the SSD in the case

Summary

Changing the hard drive with this laptop was more challenging than I though but not that hard. I’m too use to having hard drive that are simple to access. The total time to do that is around 15 mins when you know what you are doing.

I hope that some poeple like me who still own some of this old machine will find this instruction usefull

Archlinux enabling hibernation

Mon, 22 Feb 2016 00:00:00 +0000

It’s now time to look at how to enable hibernation to a swapfile when we close the lid of the laptop. As a non Linux expert, it took me some time and some failure to arrive at the expect comfiguration. I didn’t want to use any helper script and wanted to setup everything using systemd. I want to enable hibernation because I want to have the ability to restart where I left the system without consuming energy. It’s true that restarting the computer will be slower, however this is a compromise that I accept.

The detailed instructions from the ArchLinux can be found below.

Setting up Hibernate mode.

In this section I will cover how to setup your ArchLinux to use a file swap to hibernate. This require a few change in the way we are starting the kernel as well as some subtility with the creation of this file. So let start.

Creating a swap file

Create a new swapfile

Allocating the file is quite simple with the command fallocate, but before we can do that we need to have an idea about how much memory we need to allocate. The recommendation is 2/5 of the total RAM available on your machine. This is good to know, however how can I quantify that. Luckily for us they are a file /sys/power/image_size which containt the information we are looking for. You can read the content of the file using the command cat. The size required for my system is 840889812 bytes which equivalent at ~841MB. So the comment to run in my case is to create the file /swapfile of 841MB is:

sudo fallocate -l 841M /swapfile

Change the permission of the new swapfile

The swap file could be a big hole in for your security. The first think we have to do is to remove any access to non root users but keet r/w to root

sudo chmod 600 /swapfile

Format the swap file

Before been able to use it, we need to format it. Same as when when create the system we have to use the command mkswap

sudo mkswap /swapfile

Enable the swap file

Same as when we installed archlinux, we have to enable the new swap file in the system with the command swapon

sudo swapon /swapfile

Add new swap file entry into /etc/fstab

Edit the file /etc/fstab and add the following file. This discribe the information about how to mount the new swap file as a file system

/swapfile none swap defaults 0 0

The content of your file should look like

Great, We have a new swap partition which is just a simple file. We will use that to enable the hibernation to happen.

Update the kernel startup parameter using grub

We have to pass two news arguments to the startup kernel.

resume : Give to the kernel the drive that contain the hibernate swap. This parameter is enough alone if you are not using a file swap
resume_offset : Inform the kernel the offset for the start of the swap file on the partition.

As we are using grub as our bootloader we will update it’s configuration to start the kernel with this flag

Open /etc/default/grub

Add parameter resume

In the command line XXX of the file we need to add the parameter resume with the drive name that contain our swap file. In the case of my machine this file is located on the drive /dev/sda2. The parameter to is

resume=/dev/sda2

Add parameter resume_offset

Before adding this paramter we need to find the value we need for the offset. The simpliest solution using build in command is running the following command

sudo filefrag -v /swapfile

The result of the command should look like that

The value we want is from the first row the column physical_offset. In my case 79872. Now that we have this value we need to add this parameter to our startup kernel

resume_offset=79872

Generating the new grub.cfg configuration file

This can be simply done using the command line tool grub-mkconfig with the option of the location for the output file.

sudo grub-mkconfig -o /boot/grub/grub.cfg

Update mkinitcpio

‘mkinitcpio is a batsh script that create an initial ramdisk environment’. You can find more information mkinitcpio. We need to update the startup hook to add the resume one. This will ensure that the system can restore from the hibernation.

Add resume hook to /etc/mkinitcpio.conf

We have to edit in the file /etc/mkinitcpio.conf the section HOOK. This section control the module that are been loaded into the image at boot time. You need to add in the set of argument there the resume one.

For example my current section is looking like that.

Generate the images

The command we need to use to generate this boot image is mkinitcpio. The default setup is reading the information from the configuration file /etc/mkinitcpio.conf. As we want to generate the base image for in the default ArchLinux path we can use the option -p. The command we have to run on elevated mode is.

sudo mkinitcpio -p linux

Enable systemd handle

They are multiple solution to enable the handle abd choose the right behavior. You can find more information there . The solution I choose is the drive the handle action using the systemd as the base. The configuration of the handles can be find in the configuration file /etc/systemd/logind.conf. The handles we want to set are :

HandleLidSwitch : Action when we close the lid of the laptop
IdleAction : Action taken after the idle event is raised
IdleActionSec : Time before the idle event is raised

What I want is the laptop to go on hibernate mode by default for the two event above. I’m more worried about battery than starting time (Which is not the slow anyway compare to Windows ). We just have to put for this event the value hibernate and we will be in the right state.

Next step and summary

I’m happy with where I finish, however I want to do the next step which is mixing suspend and hibernate, so I will do a quick hardware update and reinstall my machine after changing the hard drive to an SSD. With today cost of ~300 HKD (~40 USD), this will be a good one. The advantage of having an SSD will be that I won’t have to worried about the laptop waking up during transport to hibernate. I will have to restart my base install and check what specific setting I need for SSD.

So next step. How to change the hard drive of the EEEPC 1101HA to an SSD

ArchLinux - Using netctl to setup the network

Mon, 15 Feb 2016 00:00:00 +0000

For this article we will focus on network setup (at least what I’ve found so far) and will look at some of the useful commands that I now use often.

Setting up the network, and more specifically the wifi can be traumatic. However, even without X windows there are some nice simple tools to help you to setup the wifi connection (like ‘wifi-menu’ that we used earlier).

If you follow the step to set up the base installation from this article, you should already have from your base installation the packages netctl and dialog. netctl is a batch script built on top of systemctl that allows you to start or stop the network profile configuration service.

Connecting Manually

Before trying to connect using some auto detection, it would be good to understand what is happening under the hood. netctl is a simple command that allow you to list available profiles, start a profile or stop a profile. In this section we will first look at how to setup a wired connection followed by a wifi one, and then get a quick overview of the netctl command line tool.

Setting up an Ethernet profile

netctl stores the connection profile in the folder /etc/netctl. This folder has a child one named /etc/netctl/example which contain a set of different example profiles. We are interested here in the example profile whose name starts with ‘ethernet’. In my case I have 3 examples:

ethernet-custom : allows you to see all the available fields you can set up
ethernet-dhcp : example to connect and get the IP address from a DHCP server
ethernet-static : example to setup a profile using a static IP address

As my router is doing as well DHCP server for me, I will simply set up the profile using ethernet-dhcp as a sample. The steps we have to do are:

Find the ethernet network interface name

Linux has a command line ip utility that allow you to query and update the network interfaces. The command line to run to list the network interfaces is

ip link

The result of the query is in my case
The ethernet adaptor name starts everytime with enp - in my case the name is enp3s0

Copy the example /etc/netctl/example/ethernet-dhcp to /etc/netctl

We need to copy the example file /etc/netctl/example/ethernet-dhcp to the folder /etc/netctl and give it a name that is meaningful. In my case I will name it enp3s0-ethernet-dhcp because it’s a good representation of the profile’s meaning. Back to the command shell to copy the file using :

cp /etc/netctl/example/ethernet-dhcp /etc/netctl/enp3s0-ethernet-dhcp

Edit the Profile

You can use any text editor that you want - vim is the one I’m using every time. We want to open /etc/netctl/enp3s0-ethernet-dhcp to change the property Interface with the one we found at step 1. The content you should have in the file at minimum is :

Check the permission

The profile we just updated is good, however this is not protected against being seen by unauthorised users. This profile doesn’t have a lot of important information, although it will do later on for the wifi setup. We will change the password using the command line chmod and set read/write only for the root users.

chmod 600 enp3s0-ethernet-dhcp

Plug in the ethernet cable and start the profile

We can now plug in our ethernet cable and start the profile. We will use the command line netctl. The command to start the profile is

sudo netctl start enp3s0-ethernet-dhcp

The best command to test that the profile is enabled and that you can access the network is to ping the google dhcp server with the address 8.8.8.8 :-)

Setting wifi profile

If you installed the package dialog in the first post about installing archlinix, this allows you to use a simple user interface that doesn’t require X windows to be installed. The command we will use is wifi-menu, which is a simple script built on top of netctl.

Please always use the command *wifi-menu* with the option *-o* to be sure that the password is hashed in the profile file.

Running the command wifi-menu will create the profile file in the folder /etc/netctl and connect you to the network at the same time. You can even run the command again to connect to the wifi anytime you want to change the connection. The command you have to run is the following

sudo wifi-menu -o

You will see the following displayed:

Creating a wifi profile is simple with the use of wifi-menu. However you can also create a wifi profile using the same type of manual steps as we used to set up the ethernet connection. The hardest part I found about doing that is that you need to know the wifi security type and you may have to put your password in clear.

Usage of netctl

Earlier we already use the command netctl to start the ethernet connectoin. netctl has a lot more option that I will list now.

List and display active connection
Stop/Start connection
Enable/Disable auto start connection

Listing the profile available

netctl is using the folder /etc/netctl to save the registered profile. What is good is that the tool has an option to list that directly and display as well the connections that are active with a *.

netctl list

The output of that command will look like that.

Starting and stoping profile

netctl allow to start and stop connection profile. This command require root permission and need to be run under sudo.

# start the connection
sudo netctl start wpl1s0-myconnection

# Stop the connection
sudo netctl stop wlp1s0-myconnection

Enable and disable auto connection as start

netctl can add a conneciton into systemctl which control the linux services. This is really usefull if you are connection everytime with the same profile, like ethernet or your home wifi. We will see later on how we could automate that even more so the system can detect and automatically switch profile for you.

#Enable the profile wlp1s0-myconnection to start at boot
sudo netctl enable wpl1s0-myconnection

#Disable the prfile wlp1s0-myconnection to start at boot
sudo netctl disable wpl1s0-myconnection

Automating auto connection

The information below come from the Archlinux Wiki on [netctl][https://wiki.archlinux.org/index.php/netctl#Automatic_switching_of_profiles]. The next step will require you to have a network connection active (ethernet or wifi) because we will need to install some new packages.

Ethernet

So far you can only connect using the command line netctl. This is great, however it would be even better if when you plug-in an ethernet cable your computer reconize it and get automatically an IP address. Luckilly they are a service that you can install that know how to read the netctl config file and which will automatically handle the connection. The package is named ifplugd.

Install ifplugd

sudo pacman -S ifplugd

Enable the service Please note that you need to replace enp3s0 with the identifier of your network interface.

sudo systemctl enable netctl-ifplugd@enp3s0.service

Wifi

I will provide you as well the way for the Wifi connection to automatically witch between your available profile and pick up the best one. Personally I don’t like my computer to do think for me and I don’t mind typing one command to connect, so I haven’t enabled this service. I feel as well it’s more secure that way. The service that allow auto connection of wifi profile is named wpa_actiond.

Install wpa_actiond

sudo pacman -S wpa_actiond

Enable the service Please note tht you need to replace teh wlp1s0 wuth the identifier of your wifi interface

sudo systemctl enable netctl-auto@wlp1s0.service

Summary

Great, now we have some network, which will allow us to install new package and application to setup the rest of the computer. One think we will foccus on next is the user interface. This will make the computer little more firendly :-). See you on the next post.

ArchLinux - disabling root

Mon, 08 Feb 2016 00:00:00 +0000

If you followed the previous article of the series about setting up Arch Linux, you should now be at the point where you have a base Linux system.

In this post I will focus on the creation of a user account, installing and setting up sudo and disabling the root account. You can find detailed information by following this link.

Creating a user

Following the reboot of your ArchLinux installation you will be asked to connect. Use root and the password you set up in the previous article.

We are going to create a new account and add it to the ‘wheel’ group by default. The group wheel is a group that allows you to run some restricted commands from that user. As I want this account to be an advanced power user, this is the group I want to be part of.

useradd -m -g wheel [account name]
passd [account name]

Installing and setting Sudo

The next step is to install and set up sudo to allow the wheel group to run commands in elevated mode. ‘sudo’ stands for ‘Superuser Do’ and will allow us to continue the installation without using the root account.

Installing sudo

pacman -S sudo

Setting sudo

We have to edit the sudo configuration file to grant the permission. Editing the sudo configuration needs to to be done using visudo. This ensures when we are saving the file that it is valid and wont lock us.

visudo

We are at that point in our favorite text editor, VI :-). I hope you remember your basics here. The only keys you will need are

i : insert
esc : exit the edit mode
:wq : save and quit

At the end of the file you will see a section that contains a lot of commented lines (starting with #) which represent some templates. We want to add there the following lines which will allow members of the wheel group to execute any command.

%wheel ALL=(ALL) ALL

Follow this link if you want to read more about sudo.

Disable root

Disabling the root account is not mandatory, but is strongly recommended as everyone knows the root account on Linux.

Disconnect from the current root session
Connect with your user account
Disable root The best way to disable the root account is to lock it. For that run the following command.

sudo passwd -l root

Summary

This was a simple article that allows you to secure your base installation in the most basic way. In the next article we will look at how to set up some of the basic services and start to install some basic applications.

Archlinux base installation

Mon, 01 Feb 2016 00:00:00 +0000

After years on Windows and having on the side this old Asus EeePC 11010HA collecting dust in storage I decided to give it a second life following the discovery of Arch Linux and Xmonad from a friend who is working on a startup. I’ve tried a lot of other windows and Linux distributions on this machine in the past without success, mainly relating to the video driver which is the Intel GMA500. This post summarizes how to set up Arch Linux on a laptop. I will set up this machine as a Linux development machine until I feel confident that I can do a full switch of my main PC.

Preparing the USB drive

The first setup before installing Arch Linux is to create a USB boot drive installation. As I don’t have any Linux machine for years I needed to find a way to create this USB boot drive on windows.

Download required

You will need to download two things:

Yumi Multiboot : This will allow you to create the USB Boot drive simply. If you have chocolatey installed on your machine you can run the following command powershell choco install yumi
Arch Linux ISO: This is the ISO image

Creating the USB Boot drive

Open Yumi and set the parameters as below. Don’t forget to check “we Will Fat32 Format F: Drive! “
Click “Create” and wait for the creation of boot USB key

Installation of Arch Linux

The instruction below are mainly out from the youtube video that Goguda55 Tech Tutorial made. Thank you to him! I’ve added some information during the initial installation process, like installing wifi packages. These packages are useful when you are setting up a non wired computer.

Instructions

You need to boot your computer using the USB key we made above.

Arch Linux Version Selection

You can select the installation you want to do. In our case we want to install the i686 which is the 32 bits version of Arch Linux.
if you want to install a 64 bits version of ArchLinux you will have to select x86_64

This will bring you to the basic command prompt. This prompt is the one of the installation OS which is hosted on the USB Key

Find the install drive name

Before we start installing ArchLinux we need to find the main drive that will be used to install it. Run the command.

fdisk -l

This will list all the disk you have on your machine. In my case the command returns this:

You should see something similar to that. In my case the drive on which I want to install Arch Linux is /dev/sda

Create the partition on your hard drive

We create two partitions on the drive /dev/sda.

Remark: Creating two partitions is the minimum you have to do. However you will find in the literature that often there can be more partitions. However in our case two are enough.

Swap partition : half of the size of your memory
Data partition : We will create a single partition for all the rest. We won’t try to be clever and create different partition for different folder.

Run the command below to start the partition manager :

cfdisk /dev/sda

Format the main partition and mount the new formatted disk

We need to format the new data partition as ext4 which is the current standard Linux file system type. Following this step we will have to mount the new formatted partition in the current Archlinux installation boot OS to be able to download and install the base Archlinux packages later on.

mkfs.ext4 /dev/sda2
mount /dev/sda2 /mnt

Initialize the Swap partition

The Swap partition is initialized using the following simple command. The first one initializes the swap then the second enables it.

mkswap /dev/sda1
swapon /dev/sda1

Enable the network

This can be different depending on if you are using a wired or wireless network. I will explain below for wireless (note: wired network should be enabled by default). The base install image comes with a utility that helps you find and set up your wifi connection. Run the command below to set up your wifi connection.

wifi-menu

Download all the Archlinux base packages

Now that we have the network setup completed we can download from the Internet all the base and base-devel packages of Archlinux to the /mnt mount point. We do that by running the following command.

pacstrap /mnt base base-devel

Start ArchLinux partition

Now that we have downloaded all the packages required for the base image of Archlinux it’s time to start that new partition. Archlinux comes with a specific tool that starts the archlinux session as root. When you run the command you will see the command prompt change.

arch-chroot /mnt

Change the root password

First things first… securing the newly installed OS. For that we need to change the current password of the root account to which we are connecting. Run the following command and enter the new root password you want.

passwd

Enable locale settings

The computer locale settings are defined in the file /etc/locale.gen. This file contains all the locales predefined. We just have to uncomment the one we want to enable. You can do that using ‘vi’ or ‘nano’. In our case we wil use nano here. When we finish editing the file /8/etc/locale.gen* we will run the command lcoale-gen to generate the file we need.

nano /etc/locale.gen
locale-gen

Set the time zone

On Linux all the time zone info is defined in /usr/share/zoneinf. We will have to create a link between the time zone we want and the file /etc/localtime. The first command lists the time zone info. Your time zone can be located in another child folder. For example Paris is under the folder Europe. The second command creates the link between the time zone we want and the configuration file that Linux is reading this information from.

ls /usr/share/zoneinfo -ali
ln -s /usr/share/zoneinfo/Hongkong /etc/localtime

Give a name to the computer

We have to set the file /etc/hostna,e to contain the name of the current computer (think of a fun name!). The best way to do that is to print to the file the name you choose using output redirection, like this:

echo archlinuxEeePC > /etc/hostname

Install some useful packages for wireless

As we are installing ArchLinux on a laptop it is useful to install some network tool to be able to reconnect after the reboot. The package netctl allows you to manage configuration using profiles. We are also going to install a dialog to allow the command wifi-menu that we used earlier to work.

pacman -S wpa_supplicant netctl dialog

Installing and Setting Grub boot loader

We’re almost done… :-)
We now have to install and set up the bootloader. The most common used today is Grub. The bootloader allows us to choose the OS we want to boot and take all the driver initialize. We have to run several commands to acheive the following:

Download the Grub package from internet.
Install Grub as bootloader on the drive /dev/sda
Create the default configuration of Grub. This is the default one. We will do another article how to customize that.

Here’s the commands to run:

pacman -S grub-bios
grub-install /dev/sda
mkinitcpio -p linux
grub-mkconfig -o /boot/grub/grub.cfg

Final steps. unmount and reboot

Here we are, the last step… We have to run the below command to exit the current instance of ArchLinux we just set up and return to the boot OS. Following that we will create the static information about the file system and unmount the file system and finally reboot.

exit
genfstab /mnt >> /mnt/etc/fstab
umount /mnt
shutdown -r

Wrapping

Congratulations! We now have a freshly installed Archlinux base installation. In the upcoming articles we will look at the following post installation steps:

User and security
Network confuguration
basic service and toll setup
Windows Manager and windows windows
XMonad configuration
vi advance setup
Development environment setup for .Net/F#