docker-compose scale

This command is deprecated. Use the up command with the --scale flag instead. Beware that using up with the --scale flag has some subtle differences with the scale command, as it incorporates the behaviour of the up command.

Usage: scale [options] [SERVICE=NUM...]

Options:
  -t, --timeout TIMEOUT      Specify a shutdown timeout in seconds.
                             (default: 10)

Sets the number of containers to run for a service.
Numbers are specified as arguments in the form service=num. For example:

docker-compose scale web=2 worker=3

Tip: Alternatively, in Compose file version 3.x, you can specify replicas under the deploy key as part of a service configuration for Swarm mode. The deploy key and its sub-options (including replicas) only works with the docker stack deploy command, not docker-compose up or docker-compose run.

GIT logs Useful Commands

For full path names of changed files:

git log --name-only

For full path names and status of changed files:

git log --name-status

For abbreviated pathnames and a diffstat of changed files:

git log --sta

Set Up an RPM Build Environment under CentOS

This document will guide you on how to install and configure an environment to build RPMs (and rebuild SRPMs) under CentOS.

Building RPMs should NEVER be done with the root user. It should ALWAYS be done with an unprivileged user. Building RPMs as root might damage your system. You have been warned.

Check that you have rpmbuild installed

First, you should check that you have rpmbuild installed on your system. This is the tool you will use to build RPMs from specfiles or SRPM packages. To check that it is installed and , issue the rpmbuild --showrc command. A large set of data should be displayed, enumerating details of the build environment that rpmbuild is using. Quite useful for debugging what a .spec file is doing

If the system returns: $ rpmbuild: command not found this means rpmbuild is NOT yet installed. You can install it with yum by running the following command as root:

[root@hostname ~]# yum install rpm-build

As becoming root for running a command is only logged in the bash history file, most careful admins set up and use sudo for the task instead:

[userid@hostname ~]$ sudo yum install rpm-build

Note: That for historical reasons, the package containing /usr/bin/rpmbuild is called rpm-build (that is, with a dash in the package name).

Verify that yum listed a version of the rpm-build package in the list of packages to install, and answer "y" to allow yum to go ahead and install the package.

After yum is finished, run the rpmbuild --showrc or the more terse rpmbuild --version command to check that it is installed.

Most SRPMs targetted to be rebuilt on CentOS also need certain rpmbuild build macros and helper scripts, which are contained in package: redhat-rpm-config. To get results as desired, you should also install it in the same fashion as noted above, substituting the new package name.

[userid@hostname ~]$ sudo yum install redhat-rpm-config

Note: You may have this package already installed. In such a case yum will just output Nothing to do on the last line of its output. You can check if a package is already installed (here checking on redhat-rpm-config) with the command rpm -q redhat-rpm-config . If there is any output, that means the package is already there. The output of the rpm command will also include the version and release information of that package that is installed on your system.

Create directories for RPM building under your home

After you have rpmbuild installed, the next step is to create the files and directories under your home directory that you need to build RPMs. As noted before, to avoid possible system libraries and other files damage, you should NEVER build an RPM with the root user. You should always use an unprivileged user for this purpose.

To build RPMs with an unprivileged user, you must create a directory structure for that purpose, and then create the .rpmmacros file under your home directory overriding the default location of the RPM building tree to the one you created.

The instructions below will create a rpmbuild directory under your home directory to build RPMs. If you want to use a different directory, you will have to adapt the instructions below to your usage. See the external references for documentation on how to do more complex configurations.

To create the RPM building environment, run the two commands below:

[userid@hostname ~]$ mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}

Beware: this next command will overwrite an existing .rpmmacros file if it exists, so check that you don't already have one before continuing.

[userid@hostname ~]$ echo '%_topdir %(echo $HOME)/rpmbuild' > ~/.rpmmacros

After running the two commands above, your environment is set up to build most RPMs without further setup.

Other tools you may need

In general, building RPMs means building and compiling software. To do that, in general you will need tools needed to compile and build source packages.

In particular, you will most probably need to install make to build software (even software that is not written in C or in a compiled language usually uses Makefiles for its install process). As before:

[userid@hostname ~]$ sudo yum install make

If you are building RPMs for software written in C, you will also need the gcc compiler.

[root@host ~]# yum install gcc

Some dependencies will probably be installed together with gcc to allow for the compilation against system libraries.

If you are building software that uses system libraries (such as OpenSSL, for example), you will need to install additional RPMs that allow you to build software against those libraries.

Using OpenSSL as an example, there are two separate binary RPMs: openssl and openssl-devel. The openssl RPM package contains the libraries needed to run binaries linked against openssl. For example, wget needs OpenSSL libraries for encrypted connections, so to install the wget RPM, the openssl RPM will be required, and installed as well.

However, the openssl RPM does not contain the library header files needed to compile code against the OpenSSL libraries. A decision by a spec file packaging team to split these less commonly needed header files in designing a distribution is a common one , in order to conserve space on non-developer machines.

For example, if you download wget source code and try to build it, it will complain that it can not find the OpenSSL libraries. The files needed to compile code with OpenSSL are included in the openssl-devel RPM. So, after installing that RPM and any other needed build dependencies that the SRPM or spec file requires, you may then compile code that requires to use the OpenSSL library headers to build.

So, diagnostically, when you try to build an RPM package, and it gives you an error and tells you that it cannot find a certain library, you should look for the library -devel package's presence using the installed packages methods noted above. If missing, cure the problem by installing it if not yet present. This will allow you to go on with building that RPM. ... or encounter the next item to be solved in a process of progressive obstacle elimination.

Source:
https://wiki.centos.org/HowTos/SetupRpmBuildEnvironment
 

Send Email Notifications in Declarative Pipeline

#!/usr/bin/env groovy

pipeline{

    agent {
            label 'slave'           
        }
    }
    stages {       
        stage ('Test Email') {
            steps {
                emailext (
                        to: 'e-mail@mail.com',
                        subject: 'subject',
                        body: 'details',
                        recipientProviders: [[$class: 'RequesterRecipientProvider']]
                        )
            }
        }
    }
    post {
    failure {
        mail to: 'e-mail@mail.com',
             subject: "Failed Pipeline: ${currentBuild.fullDisplayName}",
             body: "Something is wrong with ${env.BUILD_URL}"
        }
    }
}

Anisble Installation via PIP.

Ansible can be installed via pip, the Python package manager. If pip isn’t already available on your system of Python, run the following commands to install it:

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ python get-pip.py --user

Then install Ansible.

$ pip install --user ansible

Or if you are looking for the latest development version:

$ pip install --user git+https://github.com/ansible/ansible.git@devel

If you are installing on macOS Mavericks (10.9), you may encounter some noise from your compiler. A workaround is to do the following:

$ CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments pip install --user ansible

In order to use the paramiko connection plugin or modules that require paramiko, install the required module.

$ pip install --user paramiko

Ansible can also be installed inside a new or existing virtualenv:

$ python -m virtualenv ansible  # Create a virtualenv if one does not already exist
$ source ansible/bin/activate   # Activate the virtual environment
$ pip install ansible

If you wish to install Ansible globally, run the following commands:

$ sudo python get-pip.py
$ sudo pip install ansible

How to verify the Linux user password expiry information:

$ sudo chage -l oracle
Last password change                                 : Oct 12, 2019
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 7

Installation Instructions for Oracle Wallet Manager

Solution

This document provide installation instructions for Oracle Wallet Manager. If you are not able to perform the steps on the server, RapidSSL recommends to contact Oracle.

To install a RapidSSL certificate for Oracle Wallet Manager, follow these steps:
 

Step 1:  Download RapidSSL certificate, Root and Intermediate CA

    1.    The RapidSSL certificate download link will be sent by email.

    2.    Download and extract the certificate zip file from the email. 

           NOTE: Click here for steps to download the RapidSSL certificate.

    3.    Download the Root CA certificate for your SSL product from this link.
    4.    Copy and paste the file on a Notepad.
    5.    Save the file as Root.txt
    6.    If you do not have the intermediate, download the Intermediate CA certificate from this link
    7.    Copy and paste the intermediate in Notepad.
    9.    Save the file as intermediate.txt

Step 2:  Import the Root and Intermediate CA Certificate

           NOTE: You must add all trusted certificates in the certificate chain of a user certificate before adding a user certificate, or the command to add the user certificate will fail.

    1.    Open Oracle Wallet Manager.
    2.    Select Operations > Import Trusted Certificate.
    3.    Import the Root CA certificate.
    4.    Select Paste the Certificate.
    5.    Click OK.
    6.    Paste the certificate into the text box.
    7.    Click OK.
    8.    A message at the bottom of the window confirms that the trusted certificate was successfully installed.
    9.    Save changes to the Wallet after importing the Trusted Root Certificate and before closing the Wallet.  
  10.    Repeat these steps to import the Intermediate CA certificate.
 

Step 3:  Import the SSL Certificate

    1.    From the Operations menu, click Import User Certificate.  The Import Certificate dialog box appears.
    2.    Click Paste the certificate, and then click OK.
    3.    Another Import Certificate dialog box appears with the following message:
           "Please provide a base64 format certificate and paste it below."  Paste the certificate into the dialog box, and choose OK.
    4.    Click OK. 
    5.    When this is completed a message at the bottom of the window confirms that the certificate was successfully installed.
    6.    The Oracle Wallet Manager main window reappears, and the status of the corresponding entry in the left panel subtree changes to Ready.  

Source: https://knowledge.digicert.com/solution/SO16545.html
 

Certificate Signing Request (CSR) Generation Instructions for Oracle Wallet Manager

Solution

This document provide CSR generation instructions for Oracle Wallet Manager. If this document can not be used within the environment, RapidSSL recommends contacting the server vender.

NOTE: As of 1/1/2016 all public SSL certificates must be issued as SHA-256 with at least a 2048 bit key length.  Please ensure the server can supports the standards before requesting a certificate.

Step 1. Create a new wallet for Oracle Wallet Manager

1. From the menu bar, select Wallet > New
2. Enter the Password twice > click OK
3. Select Add a certificate request.  If not, select Cancel > select Wallet > Save in the system default to save the new wallet
    

Step 2. Create a Certificate Signing Request (CSR) for Oracle Wallet Manager

1. Select Operations > Add Certificate Request
2. A dialog box will appear to enter your certificate information.
   
   Country Name (C): Use the two-letter code without punctuation for country, for example: US
   State or Province (S): Spell out the state completely; do not abbreviate the state or province name, for example: California
   Locality or City (L): The Locality field is the city or town name, for example: Mountain View
   Organization (O): Enter the organization name exactly as it is registered.  Avoid special characters.
   Common Name (CN): The Common Name is the Host + Domain Name.  Example, www.bbtest.net or *.bbtest.net for a wildcard.
    
3. Select OK

Step 3. Export a Certificate Signing Request (CSR) as a file

1. In the left panel, select the Certificate Signing Request you want to export
2. From the menu bar, select Operations > Export Certificate Request
3. Enter a file name and directory you want to save your file to > select OK
4. Proceed with Enrolment.

Source: https://knowledge.digicert.com/solution/SO21578.html

DOCKER - Cannot create an item in a locked collection

Error:
 
** Message: 22:08:41.658: Remote error from secret service: org.freedesktop.Secret.Error.IsLocked: Cannot create an item in a locked collection
Error saving credentials: error storing credentials - err: exit status 1, out: `Cannot create an item in a locked collection`

Solution:
sudo apt install gnupg2 pass

Kubernetes and Helm Leaning Tutorials

#Kubernetes + #Helm duo has become the essential toolset for #DevOps specialists. Listing here some 'Kubernetes + Helm' talks with tutorials.

 1. Helm and Kubernetes Tutorial - Introduction - https://lnkd.in/ezg8ceY 
 2. Delve into Helm: Advanced DevOps - https://lnkd.in/eQTR3Rc 
3. Continuously delivering apps to Kubernetes using Helm - https://lnkd.in/emuqibn 
4. Zero to Kubernetes CI/CD in 5 minutes with Jenkins and Helm - https://lnkd.in/ehRVGTN 
5. DevOps with Azure, Kubernetes, and Helm - https://lnkd.in/efiF2G

Source: https://www.linkedin.com/feed/update/urn:li:activity:6549543052220919808/
 

[DEPRECATION WARNING]: Invoking "pip" only once while using a loop via squash_actions is deprecated.

[DEPRECATION WARNING]: Invoking "pip" only once while using a loop via squash_actions is deprecated. Instead of using a loop to supply
multiple items and specifying `name: "{{ item }}"`, please use `name: '{{ pip_pkgs }}'` and remove the loop. This feature will be
removed in version 2.11. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

I was trying to install the Python Packages suing the with_items with Ansible-2.8 version, but as per the above  this feature will be deprecatated from Ansible-2.11.

- name: "Ansible PIP {{ state }}: {{ pip_pkgs }}"
  become: yes
  pip:
    name: "{{ item }}"
    state: "{{ state }}"
  with_items: "{{ pip_pkgs }}"
 later, I have updated the code like below and after the changes "DEPRECATION WARNING" not showed in execution.
- name: "Ansible PIP {{ state }}: {{ pip_pkgs }}"
  become: yes
  pip:
    name: "{{ pip_pkgs }}"
    state: "{{ state }}"
 

 

Add non-root user to the docker Unix group.

It is best practice to use non-root users when working with Docker. To do this, you need to add your non-root users to the local docker Unix group. The following command shows you how to add the non-root user to the docker group and verify that the operation succeeded. You will need to use a valid user account on your own system.
$ sudo usermod -aG docker <non-root user>
$ cat /etc/group | grep docker
docker:x:999:<non-root user>
If you are already logged in as the user that you just added to the docker group, you will need to log out and log back in for the group membership to take effect.

modprobe: FATAL: Module aufs not found /lib/modules/4.4.0-36-generic

modprobe: FATAL: Module aufs not found /lib/modules/4.4.0-36-generic

+ sh -c 'sleep 3; yum -y -q install docker-engine'

<Snip>

If you would like to use Docker as a non-root user, you should now consider adding your user to the "docker" group with something like:

sudo usermod -aG docker your-user

Remember that you will have to log out and back in...

Slack Notifications in Jenkins Pipelines.

pipeline{
    agent{
        label 'slave label name'
    }
        environment{
            SLACK_TOKEN = 'slack_credential_id'
            SLACK_TEAM_DOMAIN = 'teamdomainname'
            SLACK_CHANNEL = '#jenkins-slack-channel'
            SLACK_BASE_URL = 'https://teamdomainname.slack.com/services/hooks/jenkins-ci/'
        }
        stages{
            stage("Sending Job Start Notification via Slack"){
                steps{
                    slackSend (color: '#000000', message: "Started Pileine: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'. Details: (<${env.BUILD_URL} | here >)", baseUrl: "${env.SLACK_BASE_URL}", teamDomain: "${env.SLACK_TEAM_DOMAIN}", channel: "${env.SLACK_CHANNEL}", tokenCredentialId: "${env.SLACK_TOKEN}")
                }
            }
        }
    }
    post{
        success{
            slackSend (color: '#000000', message: "Success Pileine: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'. Details: (<${env.BUILD_URL} | here >)", baseUrl: "${env.SLACK_BASE_URL}", teamDomain: "${env.SLACK_TEAM_DOMAIN}", channel: "${env.SLACK_CHANNEL}", tokenCredentialId: "${env.SLACK_TOKEN}")
        }
        failure{
            slackSend (color: '#000000', message: "Failed Pileine: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'. Details: (<${env.BUILD_URL} | here >)", baseUrl: "${env.SLACK_BASE_URL}", teamDomain: "${env.SLACK_TEAM_DOMAIN}", channel: "${env.SLACK_CHANNEL}", tokenCredentialId: "${env.SLACK_TOKEN}")
        }
        cleanup{
            deleteDir()
        }
    }
}

Find Number of CPU Cores Command

nproc Command

The nproc command gives the number of processing units available:
# nproc

output:

1

lscpu Command

lscpu gathers CPU architecture details form the /proc/cpuinfon in human-read-able format:
# lscpu

Output:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       39 bits physical, 48 bits virtual
CPU(s):              1
On-line CPU(s) list: 0
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               142
Model name:          Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
Stepping:            10
CPU MHz:             1992.008
BogoMIPS:            3984.01
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            8192K
NUMA node0 CPU(s):   0
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase avx2 invpcid rdseed clflushopt flush_l1d

DBS Bank Hackthon DevOps Engieer Test

Here is the Hackthon link: http://hr.gs/devops-dbs

Below are the tasks on the Hackthon:

AWK Tool

The awk command is a powerful method for processing or analyzing text files—in particular, data files that are organized by lines (rows) and columns.

Simple awk commands can be run from the command line. More complex tasks should be written as awk programs (so-called awk scripts) to a file.

The basic format of an awk command looks like this:

awk 'pattern {action}' input-file > output-file

This means: take each line of the input file; if the line contains the pattern apply the action to the line and write the resulting line to the output-file. If the pattern is omitted, the action is applied to all line. For example:

awk '{ print $5 }' table1.txt > output1.txt

This statement takes the element of the 5th column of each line and writes it as a line in the output file "output.txt". The variable '$4' refers to the 4th column. Similarly you can access the first, second, and third column, with $1, $2, $3, etc. By default columns are assumed to be separated by spaces or tabs (so called white space). So, if the input file "table1.txt" contains these lines:

1, Justin Timberlake, Title 545, Price $7.30
2, Taylor Swift, Title 723, Price $7.90
3, Mick Jagger, Title 610, Price $7.90
4, Lady Gaga, Title 118, Price $7.30
5, Johnny Cash, Title 482, Price $6.50
6, Elvis Presley, Title 335, Price $7.30
7, John Lennon, Title 271, Price $7.90
8, Michael Jackson, Title 373, Price $5.50

Then the command would write the following lines to the output file "output1.txt":

545,
723,
610,
118,
482,
335,
271,
373,

If the column separator is something other than spaces or tabs, such as a comma, you can specify that in the awk statement as follows:

awk -F, '{ print $3 }' table1.txt > output1.txt

This will select the element from column 3 of each line if the columns are considered to be separated by a comma. Therefore the output, in this case, would be:

Title 545
Title 723
Title 610
Title 118
Title 482
Title 335
Title 271
Title 373

The list of statements inside the curly brackets ('{','}') is called a block. If you put a conditional expression in front of a block, the statement inside the block will be executed only if the condition is true.

awk '$7=="\$7.30" { print $3 }' table1.txt

In this case, the condition is $7=="\$7.30", which means that the element at column 7 is equal to $7.30. The backslash in front of the dollar sign is used to prevent the system from interpreting $7 as a variable and instead take the dollar sign literally.

So this awk statement prints out the element at the 3rd column of each line that has a "$7.30" at column 7.

You can also use regular expressions as the condition. For example:

awk '/30/ { print $3 }' table1.txt

The string between the two slashes ('/') is the regular expression. In this case, it is just the string "30." This means if a line contains the string "30", the system prints out the element at the 3rd column of that line. The output in the above example would be:

Timberlake,
Gaga,
Presley,

If the table elements are numbers awk can run calculations on them as in this example:

awk '{ print ($2 * $3) + $7 }'

Besides the variables that access elements of the current row ($1, $2, etc.) there is the variable $0 which refers to the complete row (line), and the variable NF which holds to the number of fields.

You can also define new variables as in this example:

awk '{ sum=0; for (col=1; col<=NF; col++) sum += $col; print sum; }'

This computes and prints the sum of all the elements of each row.

Count lines in a File

awk 'END { print NR }' access.log

Count no of words in a file

awk '{ total = total + NF } END { print total }'

Create a Lock file in shell script to avoid multiple executions of the same script.

This script shows how to lock a file using a shell script and this can be useful to avoid multiple executions of the same script.

#!/bin/bash

set -e

(

        /usr/bin/flock -n 200

        # do stuff

         

) 200>fileToLock

With set -e execution ends if flock commands fails to lock the file.

List files in a directory using a shell script

#!/bin/bash

while read line

do               

    echo "File: " $line

done < <(ls -1 /someDir)

How to replace a string in multiple files in linux from command line

find ./ -type f -exec sed -i 's/string1/string2/g' {} \;

For global case insensitive:

find ./ -type f -exec sed -i 's/string1/string2/gI' {} \;