February 19

Linux: Passing a password with rsync using expect

Using ssh keys is always one of the safest ways to connect rsync through ssh.
Here is an alternate less secure way to use rsync when in a pinch.

Install expect

Run the following command:

expect -c ‘spawn rsync -e ssh -avz username@servernameorip:/pathonremoteserver/directory /localdirectory; expect “*?assword:*” {send “userpasswordr”; interact};’

 

Category: Linux | Comments Off on Linux: Passing a password with rsync using expect
January 17

Linux: Using rsync to mirror data between servers

Introduction
This LinuxAnswer describes how to mirror 2 systems using rsync over ssh.
I will only talk about a live server and a backup server where the backup server will connect to the live server to pull the data that is to be backed up.

Assumptions
1) You know how to open up a terminal and type a few basic commands.
2) You have a working ssh server and client installed. If not then see:
ftp://ftp.ca.openbsd.org/pub/OpenBSD…rtable/INSTALL
3) You have private/public keys generated to allow passwordless logins to the live server form the backup server.

Why would you want to?
There are many reasons so I’ll just list a few:
1) Data transfer is fast as rsync only copies modified files
2) Running it over ssh encrypts the data transfer so it is more secure than other methods

The real howto
1) Decide on the directories you need to backup on the live server assuming it is a webserver this may be “/home/httpd”
2) Decide on the options you want. The most common I would use are:
-a Archive mode this is a combination of “-rlptgoD” basically it works recursively and maintains file information such as creation dates, permissions etc. See the man page for detailed info.
-v Increase the verbosity. This will let you see what is transferred
-z Compress data so that it is a quicker transfer
–delete-after Delete any files that have been deleted on the live server
-e ssh Most importantly, run the transfer over an ssh connection
A full list can be obtains from “man rsync”.
3) Try a dry run on the backup server with “-n” to make sure any typos don’t totally screw your system. This will just show what would be done:
rsync -e ssh -avzn –delete-after user@liveserver:/home/httpd /home
4) If everything went as expected you can give it a go without -n
rsync -e ssh -avz –delete-after user@liveserver:/home/httpd /home
You should get the info about the files being transferred. Running it again should be quicker as very little has probably changed.
5) That should be it, just try creating and deleting a few files and run rsync to make sure the changes occur

Automating the process
The obvious answer running the rsync commands on the backup server via cron.
A basic example being to mirror every hour on the hour:
0 * * * * rsync -e ssh -avz user@liveserver:/home/httpd /home 2>&1 > /var/log/hourly_backup.log
Then remove deleted files every night:
30 0 * * * rsync -e ssh –delete-after -avz user@liveserver:/home/httpd /home 2>&1 > /var/log/nightly_backup.log

By: D. Ross

Category: Linux | Comments Off on Linux: Using rsync to mirror data between servers
January 17

Linux: Installing DMKS on Red Hat/Centos

DKMS:
– Dynamic Kernel Module Support (DKMS) is a framework used to generate Linux kernel modules whose sources do not generally reside in the Linux kernel source tree. DKMS enables kernel device drivers to be automatically rebuilt when a new kernel is installed.
– An essential feature of DKMS is that it automatically recompiles all DKMS modules if a new kernel version is installed. This allows drivers and devices outside of the mainline kernel to continue working after a Linux kernel upgrade.
– Another benefit of DKMS is that it allows the installation of a new driver on an existing system, running an arbitrary kernel version, without any need for manual compilation or precompiled packages provided by the vendor.
– DKMS was written by the Linux Engineering Team at Dell in 2003. It is included in many distributions, such as Ubuntu, Debian, Fedora, and SuSE. DKMS is free software released under the terms of the GNU General Public License (GPL) v2 or later.
– DKMS supports both the RPM and DEB package formats out-of-the-box. (from Wikipedia)
____________________________________________________

I was trying to install guest additions on my Centos operating system but faced a lot of problems. Though the idea was very simple as all you have to do was to install DKMS package  on your centos operating system and run the install virtual box guest additions setup, but the main problem is that dkms package is not available on your centos,  it is a third party repository. So I believe there are a lot of new users who face this issue (I being one of them).

 

Following steps will help in installing guest additions on your centos.

 

Step1: update everything( though not really required but still I took this step 1st)

 

Step2: make a directory rpm using the following commands and go in that directory and download the rpm package from this link or goto http://pkgs.repoforge.org/rpmforge-release/ and download the appropriate package.

 

1
2
3
4
5
6
7
8
9
$ mkdir rpm
$ cd rpm
$ rpm -i rpmforge-release-0.5.2-2.el5.rf.*.rpm
$ yum install htop

 

now if you get an error something like this

 

1
2
3
error: Failed dependencies:
rpmlib(FileDigests) <= 4.6.0-1 is needed by rpmforge-release-0.5.2-2.el6.rf.i686
rpmlib(PayloadIsXz) <= 5.2-1 is needed by rpmforge-release-0.5.2-2.el6.rf.i686

 

That means you have installed your centos virtual machine from cloudera which is centos5 and you have downloaded rpm package for centos6 so all you have to do is to change that package and download package for centos5. You can also check if you are running a 32 bit machine or a 64 bit machine as there are two packages one is for 32 bit machine and the other for 64 bit. To check which machine you are running just type the following command

 

1
$uname -i

 

if you get i386 or i686 that means you are running 32 bit machine and if you get x86_64 that means you are running a 64 bit machine.

 

Step3. Install kernel-devel

 

1
$ sudo yum install kernel-devel

 

Step4. So almost everything is done and you are ready to install  dkms package

 

1
sudo yum install dkms

 

if everything goes fine dkms package will install successfully, without any issues.

 

Step5. This will be the final step

 

Insert VboxGuestAdditions.iso and go to the folder which will be probably in

 

/media/VboxGuestAdditions

 

and run the following command

 

1
$ sh ./VboxLinuxAdditions.run

 

This will successfully install Guest Additions on Centos.

By: Saad

Category: Linux | Comments Off on Linux: Installing DMKS on Red Hat/Centos
January 17

Linux: Compiling ZFS on Red Hat/Centos

 

 

Installing ZFS on a CentOS 6 Linux server

 

The ZFS file system for Linux comes as source code, which you build into loadable kernel modules (this is how they get around the license incompatibilities). The implementation also contains the userland utilities (zfs, zpool, etc.) most Solaris admins are used to, and they act just like their Solaris counterparts! Nice!

 

Testing occurred on a CentOS 6 machine, specifically 6.5:

 

$ cat /etc/redhat-release
CentOS release 6.5 (Final)

 

Install dependencies:

 

$ yum install gcc kernel-devel zlib-devel libuuid-devel libblkid-devel libselinux-devel parted lsscsi rpm-build

 

Once these are installed you can retrieve and build spl and zfs packages from:

 

http://zfsonlinux.org/download.html

Once downloaded do the following:

 

$ tar xfvz spl-0.6.0-rc14.tar.gz

$ cd spl-0.6.*

 

$ ./configure

$ make rpm

 

$ rpm -Uvh *.x86_64.rpm

 

Preparing...                ########################################### [100%]
   1:spl-modules-devel      ########################################### [ 33%]
   2:spl-modules            ########################################### [ 67%]
   3:spl                    ########################################### [100%]

 

$ wget http://github.com/downloads/zfsonlinux/zfs/zfs-0.6.0-rc6.tar.gz

 

$ tar xfvz zfs-0.6.0-rc14.tar.gz

$ cd zfs-0.6.*

 

$ ./configure

$ make rpm

 

$ rpm -Uvh *.x86_64.rpm

 

Preparing...                ########################################### [100%]
   1:zfs-test               ########################################### [ 17%]
   2:zfs-modules-devel      ########################################### [ 33%]
   3:zfs-modules            ########################################### [ 50%]
   4:zfs-dracut             ########################################### [ 67%]
   5:zfs-devel              ########################################### [ 83%]
   6:zfs                    ########################################### [100%]

 

If everything went as planned you now have the ZFS kernel modules and userland utilities installed! To begin using ZFS you will first need to load the kernel modules with modprobe:

 

$ modprobe zfs

 

To verify the module loaded you can tail /var/log/messages:

 

Feb 12 17:54:27 centos6 kernel: SPL: Loaded module v0.6.0, using hostid 0x00000000
Feb 12 17:54:27 centos6 kernel: zunicode: module license 'CDDL' taints kernel.
Feb 12 17:54:27 centos6 kernel: Disabling lock debugging due to kernel taint
Feb 12 17:54:27 centos6 kernel: ZFS: Loaded module v0.6.0, ZFS pool version 28, ZFS filesystem version 5

 

And run lsmod to verify they are there:

 

$ lsmod | grep -i zfs

 

zfs                  1038053  0
zcommon                42478  1 zfs
znvpair                47487  2 zfs,zcommon
zavl                    6925  1 zfs
zunicode              323120  1 zfs
spl                   210887  5 zfs,zcommon,znvpair,zavl,zunicode

 

To create our first pool we can use the zpool utilities create option:

 

$ zpool create mysqlpool mirror sdb sdc

 

The example above created a mirrored pool out of the sdb and sdc block devices. We can see this layout in the output of `zpool status`:

 

$ zpool status -v

 

  pool: mysqlpool
 state: ONLINE
 scan: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	mysqlpool   ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    sdb     ONLINE       0     0     0
	    sdc     ONLINE       0     0     0

errors: No known data errors

 

Awesome! Since we are at pool version 28 lets disable atime updates and enable compression and deduplication:

 

$ zfs set compression=on mysqlpool

 

$ zfs set dedup=on mysqlpool

 

$ zfs set atime=off mysqlpool

 

For a somewhat real world test, I stopped one of my MySQL slaves, mounted the pool on /var/lib/mysql, synchronized the previous data over to the ZFS file system and then started MySQL. No errors to report, and MySQL is working just fine. Next up, I trash one side of the mirror and verified that resilvering works:

 

$ dd if=/dev/zero of=/dev/sdb

 

$ zpool scrub mysqlpool

 

I let this run for a few minutes then ran `zpool status` to verify the scrub fixed everything:

 

$ zpool status -v

 

  pool: mysqlpool
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scan: scrub repaired 966K in 0h0m with 0 errors on Sun Feb 12 18:54:51 2012
config:

	NAME        STATE     READ WRITE CKSUM
	mysqlpool   ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    sdb     ONLINE       0     0   175
	    sdc     ONLINE       0     0     0

By: Matty
Modified By: nighthawk

Category: Linux | Comments Off on Linux: Compiling ZFS on Red Hat/Centos
January 7

Linux: Changing / Disable SE Linux configuration from the command line

From the command line, you can edit the /etc/sysconfig/selinux file. This file is a symlink to /etc/selinux/config. The configuration file is self-explanatory. Changing the value of SELINUX or SELINUXTYPE changes the state of SELinux and the name of the policy to be used the next time the system boots.

[root@host2a ~]# cat /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - SELinux is fully disabled.
SELINUX=permissive
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.
SELINUXTYPE=targeted

# SETLOCALDEFS= Check local definition changes
SETLOCALDEFS=0

Reboot for changes to take effect immediately
[Ref.] Disable SE Linux
Category: Linux | Comments Off on Linux: Changing / Disable SE Linux configuration from the command line