Tag Archives: General Linux

Ansible: Reboot server in play-book and wait for it come back.

Ansible is a great and simple configuration management and orchestration tool. Some times we require to reboot server and do some tasks once reboot is done. Today I came across such requirement where I have to upgrade our production Ubuntu server from 10.04 to 14.04. I figured some how on unattended upgrades by using following command.


	do-release-upgrade -f DistUpgradeViewNonInteractive

The above command will not ask any question when upgrade is happening. The issue is we can not upgrade directly from 10.04 to 14.04. We have to do it in two folds

Ubuntu 10.04 to 12.04

and then

Ubuntu 12.04 to 14.04

But the issue is we have to reboot the machine once upgraded to 12.04. For this Ansible have a solution withwait_for and local_action modules. Below is the snippet from my role which include three task

Task 1: Upgrading Ubuntu 10.04 to 12.04

Task 2: Restart remote machine

Task 3: Wait for the restart of remote machine to complete and continue with other tasks.

- name: Task1 Upgrading 10.04 to 12.04
  shell: do-release-upgrade -f DistUpgradeViewNonInteractive
- name: Task2 restart machine after 10.04 to 12.04 upgrade
  shell: reboot
  async: 0
  poll: 0 
- name: Task3 waiting for server to come back after 10.04 to 12.04
  local_action: wait_for host={{ ansible_ssh_host }} state=started

Task1 is slef explanatory which use shell module to upgrade the OS. Task two uses shell module to reboot the machine and task three use local_action which will be done one the host where Ansible is running this play book and wait_for will wait for something to happen. Here we are waiting for remote host to come on-line.

Other options you may include are

 port –On which port Ansible can check on remote machine once the “timeout” is completed

delay –Dont check it for number of seconds

timeout –Wait for something to happen on remote machine

connect_timeout — maximum number of seconds to wait for a connection to happen before closing and retrying

The third task can be written as below.


 local_action: wait_for host={{ ansible_ssh_host }} 
                        state=started 
                        port=22 
                        delay=30 
                        timeout=300 
                        connect_timeout=15

Wait on Ansible host for 300 seconds to check if host started or not. Once host is started wait for 30 seconds to check for 22 port is open or not and timeout that ssh connection after 15 seconds.

Some may use inventory_hostname instead of ansible_ssh_host variable to poll host identity. If you want to check on both variables you can use or operator and make one variable a default one as shown below.

Read Full Post: http://www.linuxnix.com/ansible-reboot-server-in-play-book-and-wait-for-it-come-back/

Read More

11 Top Free Raspberry Pi Publications

The Raspberry Pi is an inexpensive computer that has become the most popular device for smaller computing projects and learning. Sales have surpassed 8 million units making it the best selling UK personal computer.

This article highlights our favourite Raspberry Pi publications. They are available to read without charge, but most encourage readers to make a payment.

<A HREF=”http://www.linuxlinks.com/article/20160328051718621/RaspberryPiBooks.html“>Full article</A>

Read More

Idea – Router To Router Communication Replaced By Neural Nets?

This is just me guessing. Could we get smart communication by researching how neural networks communicate? I mean our brain are full of networks that need to send information. Maybe the smartest protocol could then be derived by letting the network learn.

My imagination tells me that if this is something like the traveling salesman problem then there could be feeling based solution that might be good enough very large problems. Like the brain maybe this could be applied to the Internet also.

/ peroglyfer.se

Be sure to also check out my previous posts

Refugee Job Idea – Solar Panel Installations?
Idea – Retrospective Power Sentences For Idea Generation

Read More

14 tail and head commands in Linux/Unix

Many people know about cat command which is useful in displaying entire file content. But in some cases we have to print part of file. In today’s post we will be talking about head and tail commands, which are very useful when you want to view a certain part at the beginning or at the end of a file, specially when you are sure you want to ignore the rest of the file content.

let’s start with the tail command, and explore all of the features this handy command can provide and see how to use it best to suit your needs. After that we will show some options that you can do and can not do with the head command.

Linux tail command syntax

tail [OPTION]... [FILE]...

Tail is a command which prints the last few number of lines (10 lines by default) of a certain file, then terminates.
Example 1: By default “tail” prints the last 10 lines of a file, then exits.

tail /path/to/file

Example :

# tail /var/log/messages
Mar 20 12:42:22 hameda1d1c dhclient[4334]: DHCPREQUEST on eth0 to 255.255.255.255 port 67 (xid=0x280436dd)
Mar 20 12:42:24 hameda1d1c avahi-daemon[2027]: Registering new address record for fe80::4639:c4ff:fe53:4908 on eth0.*.
Mar 20 12:42:28 hameda1d1c dhclient[4334]: DHCPREQUEST on eth0 to 255.255.255.255 port 67 (xid=0x280436dd)
Mar 20 12:42:28 hameda1d1c dhclient[4334]: DHCPACK from 10.76.198.1 (xid=0x280436dd)
Mar 20 12:42:30 hameda1d1c avahi-daemon[2027]: Joining mDNS multicast group on interface eth0.IPv4 with address 10.76.199.87.
Mar 20 12:42:30 hameda1d1c avahi-daemon[2027]: New relevant interface eth0.IPv4 for mDNS.
Mar 20 12:42:30 hameda1d1c avahi-daemon[2027]: Registering new address record for 10.76.199.87 on eth0.IPv4.
Mar 20 12:42:30 hameda1d1c NET[4385]: /sbin/dhclient-script : updated /etc/resolv.conf
Mar 20 12:42:30 hameda1d1c dhclient[4334]: bound to 10.76.199.87 -- renewal in 74685 seconds.
Mar 20 12:45:39 hameda1d1c kernel: usb 3-7: USB disconnect, device number 2

as you can see, this prints the last 10 lines of /var/log/messages.

Example 2: Now what about you are interested in just the last 3 lines of a file, or maybe interested in the last 15 lines of a file. this is when the -n option comes handy, to choose specific number of lines instead of the default 10.

tail -n <number_of_lines> /path/to/file

Example :

# tail -n 4 /etc/group
vboxusers:x:491:
avahi:x:70:
mailnull:x:47:
smmsp:x:51:

Example 3: We can even open multiple files using tail command with out need to execute multiple tail commands to view multiple files. Suppose if you want to see first two lines of a

tail -n <number of lines> <file1> <file2> <file3>

Example:

surendra@sanne-taggle:~/code/sh$ tail -n 2 99abc.txt startup_script.sh wifiactivate.sh 
==> 99abc.txt <==
==> startup_script.sh <==
sed -i 's/^.*PermitRootLogin.*$/PermitRootLogin yes/g' /etc/ssh/sshd_config
service sshd reload
==> wifiactivate.sh <==
modprobe -rv iwlwifi
modprobe -v iwlwifi 11n_disable=8

Example 4: Now this might be by far the most useful and commonly used option for tail command. Unlike the default behaviour which is to end after printing certain number of lines, the -f option “which stands for follow” will keep the stream going. It will start printing extra lines on to console added to the file after it is opened. This command will keep the file open to display updated changes to console until the user breaks the command.

tail -f /path/to/file

Example :

#service crond start ; tail -f /var/log/cron
Starting crond: [ OK ]
Mar 20 13:35:01 hameda1d1c CROND[5338]: (root) CMD (/root/mail/mailscript.sh)
Mar 20 13:35:33 hameda1d1c crond[2354]: (CRON) INFO (Shutting down)
Mar 20 13:35:50 hameda1d1c crond[5385]: (CRON) STARTUP (1.4.4)
Mar 20 13:35:50 hameda1d1c crond[5385]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 64% if used.)
Mar 20 13:35:51 hameda1d1c crond[5385]: (CRON) INFO (running with inotify support)
Mar 20 13:35:51 hameda1d1c crond[5385]: (CRON) INFO (@reboot jobs will be run at computer's startup.)
Mar 20 13:36:01 hameda1d1c CROND[5390]: (root) CMD (/root/mail/mailscript.sh)
Mar 20 13:36:12 hameda1d1c crond[5385]: (CRON) INFO (Shutting down)
Mar 20 13:36:25 hameda1d1c crond[5436]: (CRON) STARTUP (1.4.4)
Mar 20 13:36:25 hameda1d1c crond[5436]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 85% if used.)
^C

As you can see in this example, I wanted to start the crond service, then watch the /var/log/cron log file as service starts. I used ;  which a kind of command chaining in Linux inorder to execute two commands in single line. I am not interested in just a few number of lines then exit, but moreover I am interested in keeping watching the whole log file till service starts, then break it with CTRL+C.

Example 5: The same tail -f command can be replicated using less command well. Once you open a file with less

less /path/to/filename

Once you open file, then press shift+f

Example:

Mar 21 08:25:01 sanne-taggle CRON[5553]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Mar 21 08:27:24 sanne-taggle wpa_supplicant[807]: wlan0: WPA: Group rekeying completed with da:3c:69:04:b1:21 [GTK=CCMP]
Mar 21 08:35:01 sanne-taggle CRON[5982]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Waiting for data... (interrupt to abort)

In order to comeout from update mode in less, you have to press ctrl+c and then press q for quit.

Example 6: We have other option -s  which should always be used with -f” will determine the sleep interval, whereas tail -f will keep watching the file, the refresh rate is each 1 second, if you wish to control this, then you will have to use the -s option “sleep” and specify the sleep interval

tail -f -s <sleep interval in seconds> /path/to/file

Example :

# tail -f -s 5 /var/log/secure
Mar 20 12:43:27 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 12:43:27 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 12:43:27 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 12:43:28 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 12:43:28 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)

Example 7: As we seen in example 3, We can open more files using tail command. Even we can view 2 files at the same time growing using -f option as well. It will also print a header viewing which file is showing this output. the header line will be beginning with “==>”

tail /path/to/file1 /path/to/file2

Example:

# tail -f /var/log/secure /var/log/cron
==> /var/log/secure <==
Mar 20 13:13:19 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:20 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:25 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:26 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:26 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:26 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:26 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:27 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:27 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:27 sa su: pam_unix(su:session): session closed for user rabbitmq
==> /var/log/cron <==
Mar 20 13:00:02 sa CROND[19837]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Mar 20 13:01:01 sa CROND[20705]: (root) CMD (run-parts /etc/cron.hourly)
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20705]: starting 0anacron
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20714]: finished 0anacron
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20705]: starting logrotate
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20721]: finished logrotate
Mar 20 13:02:01 sa CROND[21587]: (root) CMD (/etc/puppet/scripts/update-federation-links.py)
Mar 20 13:10:01 sa CROND[28672]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Mar 20 13:13:00 sa crontab[31759]: (root) LIST (root)
Mar 20 13:13:01 sa CROND[31817]: (root) CMD (/etc/puppet/scripts/update-federation-links.py)
^C

Example 8: If you want to remove this header, use the -q option for quiet mode.

Example :

# tail -fq /var/log/secure /var/log/cron
Mar 20 13:13:19 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:20 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:25 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:26 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:26 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:26 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:26 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:27 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:13:27 sa su: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
Mar 20 13:13:27 sa su: pam_unix(su:session): session closed for user rabbitmq
Mar 20 13:01:01 sa CROND[20705]: (root) CMD (run-parts /etc/cron.hourly)
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20705]: starting 0anacron
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20714]: finished 0anacron
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20705]: starting logrotate
Mar 20 13:01:01 sa run-parts(/etc/cron.hourly)[20721]: finished logrotate
Mar 20 13:02:01 sa CROND[21587]: (root) CMD (/etc/puppet/scripts/update-federation-links.py)
Mar 20 13:10:01 sa CROND[28672]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Mar 20 13:13:00 sa crontab[31759]: (root) LIST (root)
Mar 20 13:13:01 sa CROND[31817]: (root) CMD (/etc/puppet/scripts/update-federation-links.py)
Mar 20 13:20:01 sa CROND[8700]: (root) CMD (/usr/lib64/sa/sa1 1 1)

Example 9: Now what if I have a very huge /var/log/messages and I am only interested in the last certain number of bytes of data, the -c option can do this easily. observe the below example where I want to view only the last 500 bytes of data from /var/log/messages

tail -c <number of bytes> /path/to/file

Example :

# tail -c 500 /var/log/messages
 failed to connect to device: failed to connect to nws://admin@localhost:56025/?group=Administrators&;cert=%2Fvar%2Flib%2Fpuppet%2Fssl%2Fcerts%2Fc569b530-6b4f-4d72-8417-48a556bf55a5.pem&key=%2Fvar%2Flib%2Fpuppet%2Fssl%2Fprivate_keys%2Fc569b530-6b4f-4d72-8417-48a556bf55a5.pem
Mar 20 13:27:07 sa collectd[2249]: nwservice.py: Upstart service in status (status:) does not match nwipdbextractor
Mar 20 13:27:07 sa collectd[2249]: nwservice.py: Upstart service in status (status:) does not match nwbroker

Now, since we have been talking for a while about tail, lets talk about “head” command.

Head command in Linux

Head command will obviously on the contrary to tail, it will print the first 10 lines of the file. Till this part of the post, the head command will do pretty much the same as tail in all previous examples, with exception to the -f option, there is no -f option in head, which is very natural since files will always grow from the bottom.

head command syntax in Linux

head [OPTION]... [FILE]...

Example 10: As mention earlier print first 10 lines.

# head /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin

Example 11: Print first two lines of a file.

# head -n 2 /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin

Example 12: this option lets you print all lines starting from a line number you specify, unlike Example 11 which will show you the first number of lines you provided.

head -n <number of lines preceeded with "-"> /path/to/file

Example :

# head -n -27 /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync

As you can notice, in this example, it printed all the lines starting after line 27.

Combine head and tail command in Linux

Example 13: As tail and head commands print different parts of files in an effective way, we can combine these two to print some advanced filtering of file content. To print 15th line to 20th line in /etc/passwd file use below example.

head -n 20 /etc/passwd | tail -n 5

Output:

syslog:x:101:104::/home/syslog:/bin/false
messagebus:x:102:106::/var/run/dbus:/bin/false
usbmux:x:103:46:usbmux daemon,,,:/home/usbmux:/bin/false
dnsmasq:x:104:65534:dnsmasq,,,:/var/lib/misc:/bin/false
kernoops:x:106:65534:Kernel Oops Tracking Daemon,,,:/:/bin/false

Example 14: Many people do not suggest above method to print from one line to other line. The above example is to show how we can combine these things. If you really want to print a particular line, use sed command as shown below.

Example:

$ sed -n '5p' /etc/passwd
sync:x:4:65534:sync:/bin:/bin/sync

Read More

Python API for creating Neutron Router with internal interface and external gateway RDO Mitaka

Python code posted down-here utilizes  NeutronClient V2 API. It creates router RouterDVS for particular tenant, interface to private sub-net  “demo1_network” (90.0.0.0/24) and gateway to external flat network “public” (192.169.142.0/24). As usual posting was inspired by question been posted at ask.openstack.org. Actually,  API calls core design I was able to find at  openstack/python-neutronclient at github.com
Code is intentionally updated and simplified to become understandable by people without strong python development background.

Complete text may be found here

Read More