How to change data directory for Docker with systemd

How to change the data directory for Docker with systemd ?

Recently I was playing with docker and got the issue with system space utilization which got full when I tried to pull some new images form docker hub. The reason was that partition was full where docker was storing all data including images.

     Usually, By default docker store its all data in /var/ partition which is usually will be in root"/" partition or have can have the separate partition. In my case that was in root "/" partition. So I thought of moving this data directory to somewhere else. Moving this directly to non OS or non-default partition will be a good habit as it will not harm data if something goes wrong with the system.

You have different option to do this -

1. Use "--graph=/data/docker/data" in systemd file i.e. "/usr/lib/systemd/system/docker.service" as below -

               ExecStart=/usr/bin/dockerd --graph="/data/docker/data"

2. Create daemon.json file with below content-

# cat /etc/docker/daemon.json
   {
    ..
    "graph": "/data/docker/data"
    ..
   }

Other Useful hacks -

Full Information about docker service -

# systemctl show docker 

Type=notify

Restart=on-failure

NotifyAccess=main

RestartUSec=100ms

TimeoutStartUSec=infinity

TimeoutStopUSec=1min 30s

RuntimeMaxUSec=infinity

WatchdogUSec=0

WatchdogTimestamp=Wed 2018-10-10 18:06:50 IST

WatchdogTimestampMonotonic=6232790274

FailureAction=none

PermissionsStartOnly=no

RootDirectoryStartOnly=no

RemainAfterExit=no

GuessMainPID=yes

MainPID=6076

ControlPID=0

FileDescriptorStoreMax=0

NFileDescriptorStore=0

StatusErrno=0

Result=success

ExecMainStartTimestamp=Wed 2018-10-10 18:06:48 IST

ExecMainStartTimestampMonotonic=6231265813

ExecMainExitTimestampMonotonic=0

.

.

.

More detailed info on official site .

Happy Learning !!

Push Docker Images to Azure Container Registry (ACR)

How to Push you custom Docker Images to Azure Container Registry?

In this post, We will be going to see that how we can tag and push our custom docker images to ACR i.e. Azure Container Registry. 

Azure CLI

We are going to use Azure CLI during whole setup. So make sure you have Azure CLI installed and you are able to run "az" command. You can check your CLI version using |az --version| as below-

If you don't have installed. you can download and install it from |https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest|.

Note - I am assuming here that -

• you have already logged in with Azure CLI. If you haven't, then please do that using |az login|.

• You have already created ACR in specific Resource Group. If not, then do that using |az group create [OPTIONS]|.

Login to Azure Container Registry-

Fetch name of ACR using command |az acr list -g {group_name} -o table|.

Then, use command |az acr login --name [Registry Name]| to log in to your  Azure Container Registry:

Tag your Docker Image-

One important thing here, if want to push docker image to registry then that should be tagged with registry name or here with loginServer name of registry. Actually these tags will be used for routing when we push these tagged images to registry.

So, lets find out the loginServer name for Azure Container Registry. Use below command for the same -

         | az acr list -g [Group Name] --query "[].{acrLoginServer:loginServer}" -o table|

Now, we'll tag the one of the available image and then will push that to azure container registry.

List of Current Images -

Tag Image with ID |7b46b9b6f72a| -

List Images again-

You can see new image tagged starting with the same name as ACR registry. Now lets push this image to ACR.

Push Image to ACR

   Check if Images have been uploaded successfully to ACR-

You can note down the highlighted one that we pushed before and it is there in ACR. Of course, You can verify this all from Azure Portal as well.

That\s all for now. Happy Learning :)!!

Site Reliability Engineering Cookbook

Minikube - How to set up local kubernetes cluster

Question - How to setup local kubernetes cluster using minikube?

First of all lets describe briefly about "What is kubernetes?". Well, Kubernetes is open-source container orchestration tool which is used for deployments automation and scaling of containers on large scale environments. There are plenty of other features that it can do. Please go to the main site (https://kubernetes.io/) to explore more.

In this specific post, we'll try to setup full kubernetes cluster on local machine. This should be used on Production like env, this is only for doing local testing and getting test of kubernetes on local server without investing much.
            We will make use of "minikube" for this setup. Below is the "minikube" Architecture.

Env and Version Details -

   - Host OS - Windows 10
   - Hypervisor - Vritualbox

Requirements -

1. Hypervisor - 

              Install some kind of hypervisor that minikube will use to create VM and spin up kubernetes cluster on one server.

           So, if you don't have, then please install. Below are the options for different distros-

If you do not already have a hypervisor installed, install one now.
          - For OS X, install VirtualBox or VMware Fusion, or HyperKit.
          - For Linux, install VirtualBox or KVM.

          - For Windows, install VirtualBox or Hyper-V

Note: Minikube has support which provides us option to define which HyperVisor we want to use. Parameter is --vm-driver={Value}. If you want to install this on same host instead of any VM, please use --vm-driver=none. Docker is needed for this, but you don't need a hypervisor.

2. Install kubernetes controller i.e. kubectl -

       "kubectl" is command line tool which is used to manage and deploy deployments on kubernetes. There are different of ways to do this based on different OS. Since we are doing this on windows, so for windows you either can use "chocolatey" package manager or simply download using curl or from browser and set that exeutable to PATH variable.

e.g. 

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.10.0/bin/windows/amd64/kubectl.exe

    Check below for stable version -
              https://storage.googleapis.com/kubernetes-release/release/stable.txt.

    Check for version -

>kubectl version --client
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.2", GitCommit:"bdaeafa71f6c7c04636251031f93464384d54963", GitTreeState:"clean", BuildDate:"2017-10-24T19:48:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"windows/amd64"}

3. Install Minikube -

   Download particular version from "https://github.com/kubernetes/minikube/releases" and move binary to system PATH in same as you did for kubectl.

 check minikube version and start your local kubernetes cluster using virtualbox.

- Check Version :

> minikube version
minikube version: v0.23.0

- Start :

> minikube start --vm-driver virtualbox
Starting local Kubernetes v1.8.0 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.

- Chech status :

>minikube statusminikube: Runningcluster: Runningkubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100

- Get Cluster info:

>kubectl cluster-info
Kubernetes master is running at https://192.168.99.100:8443

- Other kubectl handy commands:

>kubectl get po
NAME                         READY     STATUS    RESTARTS   AGE
nginx-node-d747bd6df-prrpb   1/1       Running   1          18h

>kubectl get services
NAME         TYPE           CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP      10.0.0.1     <none>        443/TCP        7d
nginx-node   LoadBalancer   10.0.0.68    <pending>     80:32033/TCP   7d

>kubectl get deployment
NAME         DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-node   1         1         1            1           7d

>kubectl get all
NAME                DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/nginx-node   1         1         1            1           7d

NAME                      DESIRED   CURRENT   READY     AGE
rs/nginx-node-d747bd6df   1         1         1         7d

NAME                DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/nginx-node   1         1         1            1           7d

NAME                      DESIRED   CURRENT   READY     AGE
rs/nginx-node-d747bd6df   1         1         1         7d

NAME                            READY     STATUS    RESTARTS   AGE
po/nginx-node-d747bd6df-prrpb   1/1       Running   1          18h

NAME             TYPE           CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
svc/kubernetes   ClusterIP      10.0.0.1     <none>        443/TCP        7d
svc/nginx-node   LoadBalancer   10.0.0.68    <pending>     80:32033/TCP   7d

How to Clean or Reset build numbers in Jenkins job ?

When you were working with some POC or R&D, you may have to do things a number of times until you come to final conclusion. In my recent POC, I was working with Jenkins build jobs for Azure container registry with Azure Managed Container services aka AKS.

During this whole activity I have created 70 plus builds and during this process each build was creating one separate tag for docker image on ACR. When this whole POC gets completed, I thought that it will be really good idea to flush/clear out build history and start building from scratch.

Situation after completing POC-

Below are the step by step instructions for the same-

• Login to Jenkins server, click on " Manage Jenkins" --> "Script Console". This will open below dialog box.

• Once you click on "Script Console", then copy and paste below script there and run.

Note - Make sure to change the "Job Name"

def jobName = "Enter Job Name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()

• Check again for build history. You'll notice that everything got flashed.

• Try to build job again and notice the change in Build number.

• One more thing would like to highlight. Let's support you have stable version at build number 90 and then you did some poc. During your poc you created some more builds which are just trash and you want to remove those build and would like to start new stable build with build_number 91. Please use below snippet for this -

def jobName = "JOB NAME"
def job = Jenkins.instance.getItem(jobName)
job.nextBuildNumber = 91
job.save()

Hope this will help !!

Dictionaries in Python

Hello Guys,

          In this particular post, I am going to discuss dictionaries in python. The example in this post is tested on Python-3.6.

          Python data structures include lists, dictionaries, tuples, and sets. If you have worked with python, then you may have worked on "lists", "tuples", "sets" and "dictionary" in python.

Dictionary - As the name suggests dictionary is the same as that ordinary dictionary where we will have index and then we'll use that index to search the specific entry. This is similar to book or phonebook index where we search things using names instead of values which we do in other python data structure. Dictionary worked in "key":"value" pair. The "word" is called key and the definition of "word" is called a value.

              Usually, the "values" or "keys" of a dictionary are not in specific order. Another import thing is that we can't sort a dictionary in python, but can make use of the list to sort dictionaries. We will this in the example. Apart from this, we can do all add, remove and modify operations on python dictionary.

Let's go through below given example and see the basic usages in practical-

Code(you can see this on github as well)

==========================================================================

print ("\n Lets Play with Dictionary: ")

#Lets make a phone book:
#Define an empty dictionary

ages = {}
#Add few names to the dictionary
ages['Sam'] = 87
ages['Vik'] = 28
ages['Vishu'] = 54
ages['ND'] = 45
ages['Bunti'] = 37

# Display current Dictionary
print("*** Current Dictionary is :\t", ages)
#Use IN operator to check if specifid key is there in Dictionary. Syntax will be of this form:
#" if in "
#this will returns TRUE, if the dictionary has key-name in it
#but returns FALSE if it doesn't. for key, value in ages.items():

if "Sam" in ages:
    print ("\nSam is in the dictionary. He is", \
ages['Sam'], "years old")
else:
    print ("\nSam is not in the dictionary")

#Use the function keys() -
#This function returns a list of all the names of the keys.
#E.g.

print ("\nThe following guys are in the dictionary:\t", \
ages.keys())

#You could use this "keys" function to put all the key names in a list:
keys = ages.keys()

#You can also get a list of all the values in a dictionary.
#You use the values() function:
print ("People are aged the following:\t", \
ages.values())

#Now put all values i.e. ages associated with each guy in a list:
values = ages.values()

#You can sort lists, with the sorted() function. 
#It will sort all values in a list. alphabetically, numerically, etc...
#You can't sort dictionaries -they are in no particular order
print ("\nUnsorted Keys:\t", keys)
print ("Sorted Keys for Dictionary:\t", sorted(keys))
print ("\nUnsorted Values:\t", values)
print ("Sorted Values for Dictionary:\t", sorted(values))

#You can find the number of entries with the len() function:
print ("\nThe dictionary has", \
len(ages), "entries in it")

==========================================================================

Output

=====

 Lets Play with Dictionary:
*** Current Dictionary is :      {'Sam': 87, 'Vik': 28, 'Vishu': 54, 'ND': 45, 'Bunti': 37}

Sam is in the dictionary. He is 87 years old

The following guys are in the dictionary:        dict_keys(['Sam', 'Vik', 'Vishu', 'ND', 'Bunti'])
People are aged the following:   dict_values([87, 28, 54, 45, 37])

Unsorted Keys:   dict_keys(['Sam', 'Vik', 'Vishu', 'ND', 'Bunti'])
Sorted Keys for Dictionary:      ['Bunti', 'ND', 'Sam', 'Vik', 'Vishu']

Unsorted Values:         dict_values([87, 28, 54, 45, 37])
Sorted Values for Dictionary:    [28, 37, 45, 54, 87]


The dictionary has 5 entries in it