Linux: Configuring Multiple Default Routes in Linux
Assume you have a Linux system with more than one network interface card (NIC) — say eth0 and eth1. By default, administrators can define a single, default route (on eth0). However, if you receive traffic (i.e., ICMP pings) on eth1, the return traffic will go out eth0 by default.
This can be a bit of a problem — especially when the two NICs share the same parent network and you’re trying to preserve sane traffic flows. In a nutshell, this post will explain how you can ensure traffic going into eth0 goes out only on eth0, as well as enforce all traffic going into eth1 goes out only on eth1.
You’ve found the one post that actually explains this issue; your googling has paid off. You wouldn’t believe how many advanced Linux routing websites out there explain how to route everything including your kitchen sink — yet fail to clearly explain something as simple as this.
As always, we’ll explain by example. Assume the following:
eth0 – 10.10.70.38 netmask 255.255.255.0
eth0’s gateway is: 10.10.70.254
eth1 – 192.168.7.126 netmask 255.255.255.0
eth1’s gateway is: 192.168.7.1
First, you’ll need to make sure your Linux kernel has support for “policy routing” enabled. (As a reference, I’m using a v2.6.13-gentoo-r5 kernel.)
During the kernel compilation process, you’ll want to:
cd /usr/src/linux
make menuconfig
Select “Networking —>”
Select “Networking options —>”
[*] TCP/IP networking
[*] IP: advanced router
Choose IP: FIB lookup algorithm (FIB_HASH)
[*] IP: policy routing
[*] IP: use netfilter MARK value as routing key
Next, you’ll want to download, compile, and install the iproute2 [1] utilities. (Most Linux distributions have binary packages for this utility.) Once installed, typing ip route show should bring up your system’s routing table. Type man ip for more information about this utility, in general.
Speaking of which, assume the system’s initial route configuration looks like this:
# netstat -anr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.7.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.10.70.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 192.168.7.1 0.0.0.0 UG 0 0 0 eth1
So, basically, the system is using eth1 as the default route. If anyone pings 192.168.7.126, then the response packets will properly go out eth1 to the upstream gateway of 192.168.7.1. But what about pinging 10.10.70.38? Sure, the incoming ICMP packets will properly arrive on eth0, but the outgoing response packets will be sent out via eth1! That’s bad.
Here’s how to fix this issue. Borrowing the method from a really sketchy website [2], you’ll first need to create a new policy routing table entry within the /etc/iproute2/rt_tables. Let’s call it table #1, named “admin” (for routing administrative traffic onto eth0).
# echo “1 admin” >> /etc/iproute2/rt_tables
Next, we’re going to set a couple of new entries within this “admin” table. Specifically, we’ll provide information about eth0‘s local /24 subnet, along with eth0‘s default gateway.
ip route add 10.10.70.0/24 dev eth0 src 10.10.70.38 table admin
ip route add default via 10.10.70.254 dev eth0 table admin
At this point, you’ve created a new, isolated routing table named “admin” that really isn’t used by the OS just yet. Why? Because we still need to create a rule referencing how the OS should use this table. For starters, type ip rule show to see your current policy routing ruleset. Here’s what an empty ruleset looks like:
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
Without going into all the boring details, each rule entry is evaluated in ascending order. The main gist is that your normal main routing table appears as entry 32766 in this list. (This would be the normal route table you’d see when you type netstat -anr.)
We’re now going to create two new rule entries, that will be evaluated before the main rule entry.
ip rule add from 10.10.70.38/32 table admin
ip rule add to 10.10.70.38/32 table admin
Typing ip rule show now shows the following policy routing rulesets:
0: from all lookup local
32764: from all to 10.10.70.38 lookup admin
32765: from 10.10.70.38 lookup admin
32766: from all lookup main
32767: from all lookup default
Rule 32764 specifies that for all traffic going to eth0‘s IP, make sure to use the “admin” routing table, instead of the “main” one. Likewise, rule 32765 indicates that for all traffic originating from eth0‘s IP, make sure to use the “admin” routing table as well. For all other packets, use the “main” routing table. In order to commit these changes, it’s a good idea to type ip route flush cache.
Congratulations! You’re system should now properly route traffic to these two different default gateways. For more than 2 NICs, repeat the table/rule creation process as necessary.
References:
[1] http://www.policyrouting.org
[2] http://www.linuxhorizon.ro/iproute2.html
Update: Here are some additional resources, that I have found useful.
http://lartc.org/howto/lartc.rpdb.multiple-links.html
http://linux-ip.net/html/routing-tables.html
Update: Apparently, OpenBSD also now supports multiple default routes through a new feature called the Virtual Routing Table:
http://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/
By: DKindlund
Linux: Squash FS
Artemiy I. Pavlov
Artemio.net
<artemio (at) artemio (dot) net>
2004-06-07 Revision History Revision 1.5 2004-06-07 Changes according to SquashFS release 2.0 alpha. Lots of description improvements and clarifications. Split instructions for Linux kernels of 2.6.x (new) and 2.4.x series. Revision 1.1 2004-05-22 Changes according to SquashFS release 1.3r3. Revision 1.0 2004-02-19 Initial Release, reviewed by LDP. Revision 0.2 2003-12-08 Text corrections, license added. Revision 0.1 2003-11-24 Initial version. Instructions for SquashFS release 1.3r2.
Abstract
This HOWTO describes the usage of SquashFS – a highly-compressed read-only file system for Linux, which is intended for use in tiny-sized and embedded systems, and anywhere else you’d want to use a compressed file system. With this document, you’ll learn how to prepare a SquashFS-ready Linux kernel, create a sqaushed file system and happily use it.
Home of this HOWTO
The SquashFS HOWTO lives at http://artemio.net/projects/linuxdoc/squashfs. There you will always find the latest version of the document, and will be able to send your feedback.
What is SquashFS
Introduction
When creating tiny-sized and embedded Linux systems, every byte of the storage device (floppy, flash disk, etc.) is very important, so compression is used everywhere possible. Also, compressed file systems are frequently needed for archiving purposes. For huge public archives, as well as for personal media archives, this is essential.
SquashFS brings all this to a new level. It is a read-only file system that lets you compress whole file systems or single directories, write them to other devices/partitions or to ordinary files, and then mount them directly (if a device) or using a loopback device (if it is a file). The modular, compact system design of SquashFS is bliss. For archiving purposes, SquashFS gives you a lot more flexibility and performance speed than a .tar.gz archive.
SquashFS is distributed as a Linux kernel source patch (which enables SquashFS read support in your kernel), and the mksquashfs tool, which creates squashed file systems (in a file or on a block device).
The latest SquashFS release tree is 2.x, the former one was 1.x. This document describes both these releases with proper notes given. For example, if some feature or parameter is different in these release trees, it will be written as follows: new value (2.x) or old value (1.x)
Overview of SquashFS
- Data, inodes and directories are compressed
- SquashFS stores full uid/gids (32 bits), and file creation time
- Files up to 2^32 bytes are supported; file systems can be up to 2^32 bytes
- Inode and directory data are highly compacted, and packed on byte boundaries; each compressed inode is on average 8 bytes in length (the exact length varies on file type, i.e. regular file, directory, symbolic link, and block/character device inodes have different sizes)
- SquashFS can use block sizes up to 32 Kb (1.x) and 64Kb (2.x), which achieves greater compression ratios than the normal 4K block size
- SquashFS 2.x inroduced the concept of fragment blocks: an ability to join multiple files smaller than block size into a single block, achieving greater compression ratios
- File duplicates are detected and removed
- Both big and little endian architectures are supported; SquashFS can mount file systems created on different byte-order machines
Making it clear
Now let’s make sure any further discussions will be clearer for you to understand. The procedure of getting SquashFS working, basically, consists of the following steps:
- Patching and recompiling the target Linux kernel to enable SquashFS support
- Compiling the mksquashfs tool
- Creating a compressed file system with mksquashfs
- Testing: mounting a squashed file system to a temporary location
- Modifying the /etc/fstab or startup scripts of your target Linux system to mount the new squashed file system when needed
Getting ready for SquashFS
Acquiring SquashFS
The SquashFS home site is located at http://squashfs.sourceforge.net/ – it contains news for the latest release and it’s changelog, as well as general information about SquashFS. You can grab the latest version at the SqaushFS project page at SourceForge.
Preparing a SquashFS-capable kernel
In order to read SquashFS, you need it supported in your kernel – just as if it was a reiserfs or ext3 file system. You have to make sure there is an appropriate patch for your kernel version – it should be located in linux-2.x.y subdirectory of the SquashFS source tree. Also, remember that in most cases you will need a clean (original) Linux kernel source from kernel.org. If your kernel source is from a distro vendor, it may be already pre-patched with custom vendor patches, and patching with a SquashFS patch will almost surely not work, as SquashFS patches are made against original Linux kernels.
Patching the kernel source
With a kernel source and a proper SquashFS patch present, all you have to do is (we’ll assume that you have your Linux kernel source in /usr/src/linux and that you have the SquashFS source in /usr/src/squashfs):
Change to the SquashFS source directory and copy the kernel patch (we’ll assume it’s named squashfs-patch) to /usr/src/linux.
bash# cd /usr/src/squashfs bash# cp linux-2.x.y/squashfs-patch /usr/src/linux
Go to the linux kernel source directory /usr/src/linux:
bash# cd /usr/src/linux
Note: please remember that we will not be leaving this directory during all further kernel-related procedures, and all paths will be given relative to /usr/src/linux.
Now patch the source with the SquashFS patch:
bash# patch -p1 < squashfs-patch
Compiling a 2.6.x kernel
Cleanup and prepare the kernel source:
bash# make distclean bash# make mrproper
Configure the kernel using your favourite method (config/menuconfig/xconfig/gconfig):
bash# make menuconfig
- In the “File systems” section, “Miscellaneous file systems” subsection, enable the “Squashed filesystem” option, whether as module or bundled with the kernel. It is only obligatory to compile SquashFS inside the kernel if you plan using squashed initial RAM disks (initrd).
- If you would like to use a squashed initial RAM disk, enable the “Initial RAM disk support” in the “Device drivets” section, “Block devices” subsection. If you want to be able to mount the squashed file system via a loopback device in future, you should enable
- “Loopback device support” in the “Device drivers” section,
- “Block devices” subsection.
Now you may compile the kernel and modules:
bash# make
Compiling a 2.4.x kernel
Configure the kernel:
bash# make menuconfig
- In the “File systems” section, enable the “Squashed filesystem” option, whether as module or bundled with the kernel. It is only obligatory to compile SquashFS inside the kernel if you plan using squashed initial RAM disks (initrd).
- If you would like to use a squashed initial RAM disk, enable the “Initial RAM disk support” in the “Block devices” section.
- If you want to be able to mount the squashed file system via a loopback device in future, you should enable “Loopback device support” in the “Block devices” section.
Now you may compile the kernel and modules:
bash# make dep bash# make bzImage bash# make modules
Installing and testing the kernel
It’s time to install your new SquashFS-enabled kernel. The instructions below are for installing and booting the kernel on the host machine. You may want to install and test it on the target system.
We assume that the kernel was compiled for a x86 architecture, and the compressed kernel image is located in the arch/i386/boot/ subdirectory of the kernel tree. Now copy the kernel to the /boot directory (and name it bzImage-sqsh for convenience, if you like):
bash# cp arch/i386/boot/bzImage /boot/bzImage-sqsh
Don’t forget to install the kernel modules if you have any:
bash# make modules_install
Modify your boot loader’s configuration file to include your new kernel and install (update) the boot loader. Now you may reboot with your new kernel. When it boots, check that everything went fine:
bash# cat /proc/filesystems
Or, if you built SquashFS support as a kernel module:
bash# insmod squashfs bash# cat /proc/filesystems
If you see the squashfs line among other file systems, this means you have successfully enabled SquashFS in your kernel.
Compiling the mksquashfs tool
Now you need to compile mksquashfs – the tool for creating squashed file systems.
bash# cd /usr/src/squashfs/squashfs-tools
Compile and install mksquashfs:
bash# make bash# cp mksquashfs /usr/sbin
If everything went fine, typing mksquashfs at the shell prompt should print it’s “usage” message.
The mksquashfs tool, exposed
Using mksquashfs
mksquashfs is the tool for creating new squashed file systems, and for appending new data to existing squashed file systems. The general command-line format for mksquashfs is:
bash# mksquashfs source1 source2 ... destination [options]
- source1, source2, etc.: files and directories to be added to the resulting filke system, given with relative and/or absolute paths
- destination: a regular file (filesystem image file), or a block device (such as /dev/fd0 or /dev/hda3) where you want to have your squashed file system
Notes for default mksquashfs behavior:
- When the new files are added to the new file system or appended to an existing one, mksquashfs will automatically rename files with duplicate names: if two or more files named text will appear in the same resulting directory, the second file will be renamed to text_1, third one to text_2 and so on.
- Duplicate files will be removed, so there will be only one physical instance (with SquashFS 2.x, you can disable the detection/rtemoval of the duplicates with the -no-duplicates option).
- If destination has a pre-existing SquashFS file system on it, by default, the new source items will be appended to the existing root directory. Examine the options table below to force mksquashfs to overwrite the whole destination and/or change the way new source items are added. Please note that it is not possible to append to a file system created with mksquashfs 1.x using mksquashfs 2.x. You will need to mount the SquashFS-1.x file system and copy the files to some location, and then join them with other needed files to create a SquashFS-2.x file system.
- If a single source file or directory is given, it becomes the root in a newly created file system. If two or more source files and/or directories are given, they will all become sub-items in the root of the new file system.
- The resulting filesystem will be padded to a multiple of 4Kb: this is required for filesystems to be used on block devices. If you are very sure you don’t ned this, use the -nopad option to disable this operation.
See the next section for more details about all possible options.
Command-line options
All possible options for mksquashfs are shown in the table below.
Option | Description |
---|---|
-always-use-fragments | divide all files greater than block size into fragments (2.x only, will result in greater compression ratios) |
-b [block size] | use [block size] filesystem block size (32Kbytes default) – this can be either 512, 1024, 2048, 4096, 8192, 16384 or 32768 |
-be or -le | force a big or little endian file system, respectively |
-check-data | enable additional file system checks |
-e [file1] ( [file2] … ) | specify which files and/or directories to omit from the new file system that is to be created |
-ef [file] | specify a file which contains the list of files/directories to exclude |
-info | print files, their original size and compression ratio, as they are added to the file system |
-keep-as-directory | if the source is a single directory, force this directory to be a subdirectory of the root in the created file system |
-noappend | if the destination file/device already contains a squashed file system, overwrite it, rather than append the new data to an existing file system |
-no-duplicates | do not detect/remove duplicate file names |
-noD or -noDataCompression | do not compress the data |
-noF or -noFragmentCompression | do not compress the fragments (2.x only) |
-no-fragments | do not generate fragment blocks (2.x only, this will produce almost the same filesystem as 1.x did) |
-noI or -noInodeCompression | do not compress the inode table |
-nopad | do not pad the resulting file system to a multiple of 4KBytes |
-root-becomes [name] | can be used while appending to a pre-existing squashed file system: it will make a new root, and [name] directory will contain all pre-existing files/directories |
-version | print the version, copyright and license message |
In most cases, you should leave all compression/block options by default, as they allow mksquashfs to achieve the best possible compression ratios.
Creating and using squashed file systems
Basic steps
In order to create a squashed file system out of a single directory (say, /some/dir), and output it to a regular file (thus, producing a file system image), you need to say only one magic phrase:
bash# mksquashfs /some/dir dir.sqsh
mksquashfs will perform the squashing and print the resulting number of inodes and size of data written, as well as the average compression ratio. Now you have your /some/dir directory image in the dir.sqsh file. You can now use the mount command to mount it using a loopback device:
bash# mkdir /mnt/dir bash# mount dir.sqsh /mnt/dir -t squashfs -o loop
To check if you have what’s expected:
bash# ls /mnt/dir
If you want to output the file system directly into a device (say, your floppy at /dev/fd0):
bash# mksquashfs /some/dir /dev/fd0
Then just mount the device:
bash# mount /dev/fd0 /mnt/floppy -t squashfs
And check if it’s okay:
bash# ls /mnt/floppy
Squashing file systems
Operations described here correspond to most cases where a read-only compressed file system can be used, whether you want it to be on a block device or in a file. This could be anything from large FTP/HTTP-served archives that don’t change often, to having a squashed /usr partition and anything alike with these.
Example 1
Let’s suppose you have a /var/arch directory with lots of files and that you want to turn it into a squashed file system and keep it on your root partition as a file (it will be a file system image that you will mount via a loopback device). The operations needed to perform are as follows.
Squash the directory, then mount it via loopback to test it:
bash# mksquashfs /var/arch /var/arch.sqsh bash# mkdir /mnt/tmp bash# mount /var/arch.sqsh /mnt/tmp -t squashfs -o loop bash# ls /mnt/tmp
If everything is as expected, make this file system mount automatically at boot time by adding this line to your /etc/fstab:
/var/arch.sqsh /var/arch squashfs ro,defaults 0 0
Unmount the file system from the temporary mount point, and mount using it’s fstab entry:
bash# umount /mnt/tmp bash# mount /var/arch
Now just ensure that everything works fine:
bash# ls /var/arch
Example 2
Say you have two hard disk partitions, /dev/hda6 (which is empty) and /dev/hda7 (which is bigger than /dev/hda6, mounted at /var/arch, contains some data and is full). Now, say you want to squash the /dev/hda7 file system and move it to /dev/hda6, then use /dev/hda7 for some other purposes. We will suppose you have the following line in /etc/fstab (reiserfs is just an example file system used on /dev/hda7):
/dev/hda7 /var/arch reiserfs defaults 0 0
In the same fashion as with the previous example:
bash# mksquashfs /var/arch /var/arch.sqsh bash# mkdir /mnt/tmp bash# mount /var/arch.sqsh /mnt/tmp -t squashfs -o loop bash# ls /mnt/tmp
If everything went fine, unmount /dev/hda7 and use dd to copy /var/arch.sqsh to /dev/hda6:
bash# umount /dev/hda7 bash# dd if=/var/arch.sqsh of=/dev/hda7
Now change the line in /etc/fstab for /dev/hda7 to:
/dev/hda6 /var/arch squashfs ro,defaults 0 0
Mount the new file system and check to see if all went fine:
bash# mount /var/arch bash# ls /var/arch
Don’t forget to erase the unneeded file system image:
bash# rm /var/arch.sqsh
Creating tiny/embedded systems
By saying “tiny/embedded”, I mean Linux systems that are being built for booting from floppy disks, IDE/USB flash disks, iso9660 CD-ROMs, small-sized hard drives and the like. Whether you want to have your whole root file system on a single media (a single partition, a single floppy), or have a modular system (several floppies or disk partitions), the procedure is almost identical. Creating such Linux systems themselves is out of scope of this HOWTO – there are dedicated HOWTOs and guides for this (like the Bootdisk HOWTO and Linux From Scratch – visit www.tldp.org to retrieve these documents).
Squashed file systems on floppy/flash/hard disks
In order to use SquashFS for creating Linux systems on small disks, you just have to follow the usual steps for creating a minimal system, performing the following operations at respective points:
- When developing a kernel for your system, make sure you enable SquashFS support so it can mount squashed file systems
- Use mksquashfs for creating read-only initial ram disks and/or root and/or other file systems
- Don’t forget to set file system types to squashfs in /etc/fstab and/or the startup scripts of your system for mounting squashed file systems
Floppy example. Let’s say you have your floppy system tree at /home/user/floppylinux and you want to place the root file system on one floppy and /usr on another. What you should do is:
bash# cd /home/user bash# mksquashfs floppylinux root.sqsh -e usr bash# mksquashfs floppylinux/usr usr.sqsh
Note 1: you can see here how we use the -e option to exclude the /usr directory for root file system’s image.
Note 2: don’t forget to specify squashfs in your root disk’s /etc/fstab or startup scripts when mounting the /usr file system.
Insert a root disk in your 3.5″ floppy drive (I assume you have a lilo or grub on it, and, thus, a file system exists on this floppy, and the root file system will reside under the /boot directory of this file system):
bash# mount /mnt/floppy bash# cp root.sqsh /mnt/floppy/boot
When done, unmount the root floppy, change the floppy to a /usr disk and use dd to transfer the usr file system:
bash# dd if=usr.sqsh of=/dev/fd0
Squashed file systems on CD-ROMs
With SquashFS, you can compress large file systems that will be used in live CDs (just as an example).
- Enable SquashFS in the linux kernel of the target system
- Create a squashed root file system
- Modify the /etc/fstab or startup scripts of the target system to mount the squashd file system when you need it
If you create a root file system out of a running Linux system, use the -e option for mksquashfs to exclude all pseudo-filesystems such as /proc, /sys (on linux kernels after 2.5.x) and /dev (when using DevFS). Also, don’t forget to add the file system image itself that is being created with mksquashfs (I think you know the reasons for these exclusions).
Acknowledgements
I would like to express my sincere thanks and immeasurable respect to:
- Phillip Lougher – for his brilliant work under squashfs, for creating an exculsive patch for linux-2.4.18, for his help with polishing this howto and answers to my mails
- Tabatha Marshall at TLDP for helping me with bringing this HOWTO to the final 1.0 release
- Everybody at The Linux Documentation Project for their great work under all the HOWTOs and guides that helped me a lot with exploring and hacking Linux
- All those at the TLDP mailing lists who helped me with getting started
- Endless thanks and respect to everybody who develops open-source software
Linux: Restore Panels In Ubuntu Back To Their Default Settings
Messed up your panels in Gnome? Maybe your new to Ubuntu and accidentally deleted items or the panel itself and now you can’t figure out how to get it back.
Sure, you can add a new panel and rebuild it by adding the items back on the panel.
Instead of going through the trouble, there is an easy fix that will restore your panels back to their default settings quickly.
Open up a Terminal window, by clicking on Applications Accessories Terminal. Or, if you deleted the top panel and cannot access the menus, just press ALT+F2 and in the run dialog box, type gnome-terminal then click on Run.
You can also browse for applications, such as Terminal from the Run window, by clicking on the arrow icon next to ‘Show list of known applications” and browse for Terminal.
Once the Terminal window opens, enter the following command at the prompt:
gconftool-2 – -shutdown
(Note: There should be no spaces between the two dashes before shutdown.)
EDIT – Reader nickrud has suggested a better method instead of shutting down gconfd. Instead use the following command (thanks nickrud!)
gconftool – -recursive-unset /apps/panel
(Remember: There should be no spaces between the two dashes before shutdown.)
Then enter the next command:
rm -rf ~/.gconf/apps/panel
And enter one more command:
pkill gnome-panel
That’s it!
Both top and bottom panels will appear (if missing) with their default settings. Now you can customize them to your preference and get on with using Ubuntu.
NOTE: This method will work in Gutsy Gibbon (7.10) and Hardy Heron (8.04) version of Ubuntu.
Linux: File system overview
This is a general overview of the Linux directory structure. There are several variations.
/bin Common programs, shared by the system, the system administrator and the users.
/boot The startup files and the kernel, vmlinuz. In recent distributions also grub data. Grub is the GRand Unified Boot loader and is an attempt to get rid of the many different boot-loaders we know today.
/dev Contains references to all the CPU peripheral hardware, which are represented as files with special properties.
/etc Most important system configuration files are in /etc, this directory contains data similar to those in the Control Panel in Windows
/home Home directories of the common users.
/initrd (on some distributions) Information for booting. Do not remove!
/lib Library files, includes files for all kinds of programs needed by the system and the users.
/lost+found Every partition has a lost+found in its upper directory. Files that were saved during failures are here.
/misc For miscellaneous purposes.
/mnt Standard mount point for external file systems, e.g. a CD-ROM or a digital camera.
/net Standard mount point for entire remote file systems
/opt Typically contains extra and third party software.
/proc A virtual file system containing information about system resources. More information about the meaning of the files in proc is obtained by entering the command man proc in a terminal window. The file proc.txt discusses the virtual file system in detail.
/root The administrative user’s home directory. Mind the difference between /, the root directory and /root, the home directory of the root user.
/sbin Programs for use by the system and the system administrator.
/tmp Temporary space for use by the system.
/usr Programs, libraries, documentation etc. for all user-related programs.
/var Storage for all variable files and temporary files created by users, such as log files, the mail queue, the print spooler area, space for temporary storage of files downloaded from the Internet, or to keep an image of a CD before burning it.