Tag Archive : filesystem

/ filesystem

Creating Disk Image

December 11, 2015 | Article | No Comments

I remember creating one or two challenges for local forensic competition in my community. Ideally the image should be created from live hard drive or SD card. However, I’m not in condition to do that. The constraint I have is to create image as small as possible so we can distribute it across any media. Therefore I create a “disk image” by myself.

In this article we will use:

  1. Slackware64 14.1
  2. dd
  3. fdisk

We will create a small disk image, 50MB in size. It can be an image of a single partition, or even we can make it as if it was an image of a disk. Let’s say we will create “partition.img” and “disk.img”

Creating a Blank Image

The disk image is exactly a single file. It is a storage containing the complete contents and structure representing a data storage or device, such as hard drive, tape drive, optical disc, or USB flash drive. Creating a disk image is usually done by creating complete sector-by-sector copy of the source medium. Thereby perfect replicating the structure and contents of a storage device.

In our case, however, we are not copying real disk. We create it, literally. What we will do is creating an empty file with sufficient size before we do something to it.

If you have experience with Virtual Machine and ever creating disk image for particular VM (for example, VirtualBox VDI), you should know that it is slightly different thing. Yes, it is still a disk image, but we have extra metadata and various thing over it.

Now, to create our blank disk image of size 50MB, we invoke following command:

dd if=/dev/zero of=disk.img bs=512 count=97656

The dd utility is used with following argument:

  • if=/dev/zero, we specify the input is /dev/zero. This is a special node in Linux which generate a zero every time it is read.
  • of=disk.img, we specify the output will be a file named disk.img.
  • bs=512, we set the block size to 512 byte. Actually you can set the zie to any convenient number. Anything will do but I choose this number.
  • count=97656, we specify how many block we will write. So, in our case, we will have 97656 blocks or 97656 x512 byte = 49999872 bytes or around 50MB. I got the number by calculating 50*1000*1000/512.

Creating the partition.img is in similar manner.

Single Partitioned Disk Image

Having partition.img, we are ready to format the image into certain partition. I want an EXT4 format, so i use following command:

mkfs.ext4 partition.img

The command will take partition.img and format the partition with to EXT4 file system.

In general, we are attempting to imitate a partition. Later we can mount the partition by:

mount -o loop partition.img /mnt/partition

Now you can mount the partition and copy files tot he /mnt/partition and they will be written to our image file.

Multiple Partitioned Disk Image

In this section we will create a disk image with multiple partitions. In other words, we are trying to imitate the real disk.

The procedure is basically similar, but the image file must first be partitioned. So, more work involved here.

As in our case, we will create two partitions. First partition will occupy 10MB. The rest will be allocated to second partition.

fdisk disk.img

The fdisk utility is interactive program and quite clear. You just need to choose right option and enter right number for start and end of partition. Before you proceed, make sure you print current condition (using option p). Fdisk will display how many heads, sectors, and cylinders it recognized. It also print out the size of sector.

Here is what specification we need:

Partition 1:
primary partition
First sector = 2048
Last sector = 22527

Partition 2:
primary partition
First sector = 22528
Last sector = 97655

If you see the number 97655 and wonder it might have something to do with the count=97656 argument to dd, you are sharp! It is truly last sector of our disk image.

Now the formatting would be bit complicated. In short, we have to make a loop back device to point a partition inside our disk image and then do formatting.

losetup /dev/loop0 disk.img -o 2048
losetup /dev/loop1 disk.img -o 22528

mkfs.ext4 /dev/loop0
mkfs.ext4 /dev/loop1

To mount our disk image, you can follow this article: Mounting Partition from Raw Disk Image on Linux

For you who are not patient enough, here’s how we mont both partitions:

mount -o loop,offset=1048576 disk.img /mnt/disk1

mount -o loop,offset=11534336 disk.img /mnt/disk2

One of the problem with Windows Operating System is the narrow-range variety of available filesystem. When using Windows, you are limited to FAT and NTFS family file system. Now, suppose you want to access ext3, reiserfs, or XFS partition on Windows there is no way Windows facilitate this unless you do something. This article will describe the step for doing it using coLinux as tool.

In this article I use following:

  1. Windows 7 32-bit
  2. coLinux 0.7.9
  3. Ubuntu Image 9.04

Note: when this article written (18 January 2014) coLinux can only be used on 32-bit Windows!!

You can try any 32-bit Windows (Vista, 7, 8) but the system tested here is Windows 7 32-bit.

There are some alternative to Ubuntu: Alpine,Debian, Fedora, Gentoo, ArchLinux, Slackware. All of them use kernel 2.6. As the reason for using 9.04 is personal, you can use higher version available.

The Idea Behind

coLinux or cooperative Linux, is a free and open source Linux kernel running on Windows natively. We can say the linux kernel is run alongside Windows on single machine, like a virtual machine does. However the difference is we don’t emulate a machine to do this instead we run the linux kernel itself. Thus it is theoretically much more optimal than using any general purpose PC virtualization software.

So in general, we will do following scenario:

  1. install coLinux on Windows machine
  2. Assure access to disk partition
  3. Export all the mounted file system using samba

Much of the process will use command line interface (cmd.exe) which you should open using administration privilege.

Installation Step

coLinux can be obtained freely from their official site, or go to sourceforge site to download it. The version used here is 0.7.9-linux2.6.33.7. Download the installer and install it in C:\coLinux directory.

Edit the connection settings of the virtual ethernet card installed by the coLinux (It should be listed as “TAP Win32 Adapter V8 (coLinux)”). In the TCP/IP settings, set: IP address to 192.168.37.10 and Subnet Mask to 255.255.255.0

Download Ubuntu 9.04 disk image from sourceforge (listed as Ubuntu-9.04-1gb.7z). Extract the image file (Ubuntu-9.04.ext3.1gb.fs) to C:\coLinux. Note that you should have 7zip to extract this file. Now create a swap (for example 128MB swap) using following command on coLinux:

fsutil file createnew c:\coLinux\swap128.fs 134217728

You should know where the magic number comes 😉

Also make sure you run mkswap in Linux and make sure there is corresponding line in fstab.

Rename the file Ubuntu-9.04.ext3.1gb.fs to ubuntu.fs so we have two files: ubuntu.fs and swap128.fs.

Copy the example.conf to ubuntu.conf and edit it. Alternatively you can copy this text and save it as ubuntu.conf.

#
# This is an example for a configuration file that can
# be passed to colinux-daemon in this manner:
#
#    colinux-daemon @example.conf
#
# Note that you can still prepend or append configuration and 
# boot parameters before and after '@', or you can use more 
# that one '@ to load several settings one after another.
# 
#    colinux-daemon @example.conf @overrider.conf mem=32
#
# Full list of config params is listed in colinux-daemon.txt.

# The default kernel
kernel=vmlinux

# File contains the root file system.
# Download and extract preconfigured file from SF "Images for 2.6".
cobd0="c:\coLinux\ubuntu.fs"

# Swap device, should be an empty file with 128..512MB.
cobd1="c:\coLinux\swap128.fs"

# Tell kernel the name of root device (mostly /dev/cobd0,
# /dev/cobd/0 on Gentoo)
# This parameter will be forward to Linux kernel.
root=/dev/cobd0

# Additional kernel parameters (ro = rootfs mount read only)
ro

# Initrd installs modules into the root file system.
# Need only on first boot.
initrd=initrd.gz

# Maximal memory for linux guest
mem=32

# Select console size, default is 80x25
#cocon=120x40

# Slirp for internet connection (outgoing)
# Inside running coLinux configure eth0 with this static settings:
# ipaddress 10.0.2.15   broadcast  10.0.2.255   netmask 255.255.255.0
# gateway   10.0.2.2    nameserver 10.0.2.3
eth0=slirp

# Tuntap as private network between guest and host on second linux device
eth1=tuntap

# Setup for serial device
#ttys0=COM1,"BAUD=115200 PARITY=n DATA=8 STOP=1 dtr=on rts=on"

# Run an application on colinux start (Sample Xming, a Xserver)
#exec0=C:\Programs\Xming\Xming.exe,":0 -clipboard -multiwindow -ac"

If you copy and edit from example.conf, there are lines you need to change. In the end you should make sure following entries exist:

cobd0=C:\coLinux\ubuntu.fs
cobd1=C:\coLinux\swap128.fs
mem=32
eth0=slirp
eth1=tuntap

Now create ubuntu-start.cmd with following content:

set COLINUX_CONSOLE_FONT=Lucida Console:12
set COLINUX_CONSOLE_EXIT_ON_DETACH=1
colinux-daemon.exe -t nt @ubuntu.conf

Then run ubuntu-start.cmd.

Next we do some configuration. Login as root with default password “root”. You can change the root password if you want.

Run the editor and edit /etc/network/interfaces. Add following:

auto eth1
iface eth1 inet static
address 192.168.37.20
network 192.168.37.0
netmask 255.255.255.0
broadcast 192.168.37.255

Bringing up the eth1 by

ifup eth1

Test the network by pinging from our Linux to Windows. It should work now.

ping 192.168.37.10

Next we do update to install necessary package.

aptitude update
aptitude safe-upgrade
aptitude install samba openssh-server mc
apt-get clean

Open /etc/fuse.conf and remove # at the beginning of the line user_allow_other

Open /etc/ssh/sshd_config and change PermitRootLogin to “no”

Add new user named user1 (or any name you wish) and next we can login to this account via ssh:

adduser user1

But first, we need to reload the SSH server:

/et/init.d/ssh reload

At this point, use SSH client such as putty to check whether you can login to Linux via SSH.

Mount File System

First, search for the partition you want to mount. Let’ say it is \Device\Harddisk1\Partition4

If you have ran ubuntu-start.cmd (the system is running) then halt it before going further.

Edit ubuntu.conf and insert following:

cobd2="\Device\Harddisk1\Partition4"

Then run ubuntu-start.cmd (or halt the previous machine and restart it). Then login as root.

Do following to create a mount point:

mkdir /media/codb2

Then edit /etc/fstab and add following line (assuming my partition is xfs):

/dev/codb2 /media/codb2 xfs defaults 0 0

Then mount it like usual.

mount /media/cobd2

Share via Samba

Give user1 the appropriate privilege. If you want to share a whole file system with user1, you must give read and write permissions.

After setting permission, add the following at the very end of /etc/samba/smb.conf

[my data]
path = /media/cobd2
valid users = user1
read only = no

Next add the user to the password’s database of samba:

smbpasswd -a user1

Then reload the samba

/etc/init.d/samba reload

On Windows, type \\192.168.37.20 and login as user1 using the password generated by smbpasswd.

Basic Storage and Dynamic Storage in Windows

December 11, 2015 | Article | No Comments

From Windows 2000 onwards, Microsoft started introduce the concept of dynamic disks. The counterpart, basic disk, is used from the era of DOS onward. Both have their advantages and disadvantages.

In this article we will discuss about both disks concept. Our explanation would be as general as possible, in fact there is no instruction on specific platform.

Overview

Basic Disk

Basic Disk uses a partition table to manage all partitions on the disk on first sector of disk. It is supported by DOS and all Windows versions. A disk with installed OS would be default initialized to a basic one. A basic disk contains basic volumes, such as primary partitions, extended partition, and all logical partitions are contained in extended partition.

Basic Disks provide a simple storage solution that can accommodate a useful array of changing storage requirement scenarios. Basic Disks also support clustered disks, IEEE 1394 disks, and USB removable drives.

Basic Disks also use the same Master Boot Record (MBR) partition style as the disks used by Microsoft MS-DOS operating system. It can also support GUID Partition Table (GPT) partitions on systems that support it.

The following operations can be performed only on Basic Disk:

  • Create and delete primary and extended partitions.
  • Create and delete logical drives within an extended partition.
  • Format a partition and mark it as active.

Dynamic Disk

Dynamic Disk is supported in Windows 2000 and later operating system. Dynamic disks do not use a partition table to track all partitions, but use a hidden database (LDM) to track information about dynamic volumes or dynamic partitions on the disk. With dynamic disks we can create volumes that span multiple disks such as spanned and striped volumes, and can also create fault-tolerant volumes such as mirrored volumes and RAID 5 volumes. Compared to a Basic Disk, Dynamic Disk offers greater flexibility.

What is LDM database? LDM or Logical Disk Manager, is a hidden database which size is 1 MB at the end of Dynamic Disk. This space records all the information of the volumes on a single disk, and also holds some related information on each dynamic disk. The information including Drive Letter, Volume Label, the begin sector of Volume, Volume size, the file system of volume.

All disks are interrelated and will hold information mentioned above if there are several dynamic disk on computer. The relevance of each dynamic disk let we will see a “Missing” disk which is shown in Windows Disk Management if we remove a dynamic disk from your system. All this is saved in LDM database, so LDM database is vary important the same as Partition Table of Basic Disk.

Clearly, we can illustrate the Dynamic disk as following:

dynamic-disk

The blue area at the beginning of Dynamic Disk is the MBR which sabes the information of the Partition Table on the disk. However, this partition table is not the same as one of Basic Disk. Its main function is to make Windows and Other Disk Manager can know the disk is a dynamic disk instead of empty disk. The red one at the end of disk is the LDM database.

If you are familiar with Linux, Dynamic Disks system is similar to Logical Volume Manager (LVM).

The following operations can be performed only on dynamic disks:

  • Create and delete simple, spanned, striped, mirrored, and RAID-5 volumes.
  • Extend a simple or spanned volume.
  • Remove a mirror from a mirrored volume or break the mirrored volume into two volumes.
  • Repair mirrored or RAID-5 volumes.
  • Reactivate a missing or offline disk.

Differences

  1. Capacity – Once Basic Disk create a partition, its capacity cannot be change unless we modify the partition table (using gpart, fdisk, or similar software). Dynamic Disk however can expand the capacity of volumes without data loss.
  2. Disk space limitation – On Basic Disk, the maximum capacity of a partition (volume) can be limited to 2 TB. Dynamic Disks can well handle the large partition of more than 2 TB.
  3. Number of partitions – Basic Disk is using primitive MBR disk layout, so it support only 4 primary partition. At best, it use 3 primary partition and use the last primary partition as extended partition. In Dynamic disks, unlimited number of partition can be created.
  4. Volumes type – Basic Disks only can create any primary or logical partition. Dynamic disks can create simple volume, spanned volume, stripped volume, mirrored volume, and RAID-5 volume (software-based RAID).

Similarity

  1. Supported file systems – Basic Disks and Dynamic Disks support FAT, FAT32, and NTFS file systems.
  2. Have a partition table – Dynamic Disks have a partition table too, however this partition table is different from Basic Disks Its main function is to let Windows and other partition manager know the disk is a dynamic disk instead of an empty disk.
  3. Label and Drive Letter – On both Basic Disks and Dynamic Disks, every partition (volume) can be assigned to a unique drive letter (in operating system perspective). such as “System C:”
  4. Disk Layout Supported – Both Basic Disks and Dynamic Disks support MBR and GPT partition styles.

Volume on Dynamic Disk

On Dynamic Disk, the volumes are divided into several categories: Simple Volume, Spanned Volume, Mirrored Volume, and RAID 5 Volume. They have drive letter and volume label to differentiate.

Simple Volume

Simple volume only can be created on the single disk. This volume is similar as partition of Basic Disk, but its space can be inconsecutive.

Spanned Volume

It is created from free space that is linked together from multiple disks (up to 32 disks). The sequence of writing data for Spanned Volume is that the volume on the first disk is filled full and then turn to fill the next dynamic disk. Spanned Volume can allow the fragmentary free space of multiple disks is recomposed as one volume, so it can fully utilize the resources of multi-disk. However, it can not be fault-tolerant volume and can not improve performance of the disk.

Stripped Volume

It’s similar with Spanned Volume, and consists of two and more disks. However, the difference is that it can improve the efficiency and performance of disk, because when operating system writes data to Striped Volume, this data will be separated into many pieces of 64KB, and then concurrent writes a different data block to each disk. A striped volume cannot be mirrored or extended and is not fault-tolerant. The screenshot is below:

Mirrored Volume

We can simply understand that Mirrored Volume is a duplicate of Simple Volume. It needs two disks; one stores the data which is being used, and another keep a copy of previous one. When a disk fails, the other one can be used immediately.

RAID-5 Volume

A RAID-5 requires three disks at least; it not only can enhance the efficiency of the disk but also provide the best fault-tolerant. You could simply consider RAID-5 is a combination of Striped and Mirrored Volume. A RAID-5 volume is a fault-tolerant volume whose data is striped across an array of three or more disks. Parity (a calculated value that can be used to reconstruct data after a failure) is also striped across the disk array. If a physical disk fails, the portion of the RAID-5 volume that was on that failed disk can be recreated from the remaining data and the parity.

Creating Hidden Partition – Linux Edition

December 9, 2015 | Article | No Comments

You have some confidential stuffs that no one should read / watch? You need to keep your data save?

There are many things you can do to protect your data. One could mention encryption, to encrypt your data to unreadable data so just you who know the key can access it. Another trick is setting the permission of file or folder, or maybe get the confusing hierarchy of data. Oh, and not to mention if you are using Windows you can use Windows feature to hide your data.  Well, that’s nice but let me tell you cool trick: hidden partition.

This article will discuss about Hidden Partition, and the trick from Linux perspective.

What is Hidden Partition?

Like implied by the name, hidden partition is a partition from disk or other storage media that hidden or invisible from system.

So, what’s so cool about hidden partition?

You can still use tricks like things we have mentioned above, but above all the partition we are playing with are invisible to the system. Imagine, how could (common) people do to your privacy if they can’t find the partition where you hide your stuffs. It’s like implementing hidden files/folders but in higher level: partition.

When people don’t aware there are some hidden partition, how can they browse or manipulate through it? Yes, that’s the point!

Theory Behind

Partition is just an area in hard disk which is defined by start on certain sector and end on certain sector. Data on the disk are written to a small allocation unit called block. Each block has fix size, for example 4kB. A file system is a way to store, read, write, and delete data effectively. There are many filesystems, but we won’t cover it in detail. Some popular file systems are: FAT, NTFS, EXT series (2/3/4), HFS, ZFS, XFS, ReiserFS, etc.

Next, the big question is: how we know or aware of a partition?

For every storage media, there are a certain area in the front of disk. This area is called MBR or Master Boot Records. It is like a table which records what partition exists on the disk. The important parts is they records the beginning and ending of a partition.

Hidden partition is as simple as removing the entry of a partition from the MBR.

Wait! If we delete the entry, doesn’t that means we are removing a partition? Yes! However, the only missing is the entry, while the data are still exists on the partition area. In fact, unless you do a format, the file system is still accessible when you know where it is.

A normal people, even a system, won’t aware if there is a hidden partition unless you told there is. Your computer won’t notify you partition which doesn’t have record on MBR. And this is why it’s called hidden partition (or at least I called it as that)

Experiment

Now, we will experiment on hidden partition. As suggested by the title, we will use Linux. In this article I use:

  1. Slackware64 14.0
  2. SD Card of 4GB as storage media

The Slackware is only proof of concept. You can use any linux available, no need to install Slackware just for this article.

The Slackware is running on Lenovo Ideapad z480.

The SD Card serves purpose as storage media which we will play with. It has 4GB of storage in total and divided into three partition.Using fdisk

fdisk -l

got the list of storage media attached to my machine. In this case, the SD card is detected as /dev/sdb and the following is the partition list as the result of fdisk.

Disk /dev/sdb: 4008 MB, 4008706048 bytes
124 heads, 62 sectors/track, 1018 cylinders, total 7829504 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048      206847      102400    b  W95 FAT32
/dev/sdb2          206848     7500000     3646576+  83  Linux
/dev/sdb3         7501824     7829503      163840    7  HPFS/NTFS/exFAT

OK, we have plenty information there. Let’s focus on the partition informations. The three partitions has differents file system. Suppose we have format each of them to their respective filesystem.

Now in this article we will make the second partition (/dev/sdb2) hidden.

To remove a partition, it is as simple as remove the entry from the MBR. We can use tool such as fdisk to accomplish this.

fdisk /dev/sdb

And you will get to SD card. Type ‘d’ and enter number 2 (our partition /dev/sdb2 is on number 2). Once you done, write the progress to the disk.

Command (m for help): d
Partition number (1-4): 2
Partition 2 is deleted

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

You can verify by invoking this command again:

fdisk -l

It should be like this:

Disk /dev/sdb: 4008 MB, 4008706048 bytes
124 heads, 62 sectors/track, 1018 cylinders, total 7829504 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048      206847      102400    b  W95 FAT32
/dev/sdb3         7501824     7829503      163840    7  HPFS/NTFS/exFAT

Now, see the gap? The first partition started from block 2048 and end on 206847. The third partition is from 7501824 to 7829503. Where’s the partition from 206848 to 7500000 like we see before? It’s hiding right now.

Now, test to mount it.

mount /dev/sdb2 /mnt/

And we get error! Well, not like this is unexpected. But how can we access the partition if we can’t mount it?

Hey! Who said we can’t mount it? We can but there is a trick to do that.

Actually, when we said mount a partition, mount will check whether a special device (/dev/sdb2 in this case) exists. If it is, mount will read the table to find where is the area of that partition. We got the area by start and end, right? And that’s how we will mount it. We will provide the start and the end, manually.

But, the start and end are not in term of sector or block. The partition are known as n bytes of offset from the start. Recall the location of partition 2, it is started from block 206848 to 7500000. Now see how bigh is each block. From the result of fdisk we know that each block is 512 bytes in size. Now, just multiply the block with 512 to get the size. Actually, we only need the start so calculate the value. If you are too lazy, you can skip it 😛

Now, to mount a hidden partition we need to invoke mount in special manner. We can’t access non existing /dev/sdb2, but we can still access /dev/sdb. So here is how I access the hidden partition

mount -o loop,offset=$((206848*512)) /dev/sdb /mnt/

ps: I’m lazy.

And, you now get the partition mounted on /mnt/

To unmount it, do following:

umount /mnt/

Isn’t that easy? 😀

Managing SquashFS File

December 9, 2015 | Uncategorized | No Comments

SquashFS is a compressed read-only file system. SquashFS compress files, inodes, and directories, and supports block sizes up to 1 MB for greater compression.

SquashFS is intended for general read-only file system use and in constrained block device / memory systems (e.g. embedded systems) where low overheaded is needed. Originally it use gzip compression, now it support LZMA, LZMA2 and LZO compression.

In this article, we will discuss about some notes about how we can manage SquashFS. The term manage in this arcitle including: creating, mounting, and modifying SquashFS file. In this article I use Slackware64 14.0 as host. However you can use any Linux distribution which has SquashFS tools installed.

You can always skip all basic and introduction, but I recommend you to read it.

Preparation

Check whether you have installed required tools. Test by invoking following binaries:

mksquashfs
unsquashfs

If you don’t have it yet, you can download or compile it from source. Just go to their official site.

For Slackware, go to Slackbuilds site for SquashFS.

I assume you have the required tools.

Overview

As stated before, SquashFS is a compressed read-only file system. You can imagine it as a file which are exactly a file system. It’s like database file for sqlite, or a disk image for various Virtual Machine apps but only serve a purpose as a filesystem. The read-only filesystem means it can only be read, no write operation allowed. Modification will require dissection of the file.

Other important things:

  1. Data, inodes, directories are compressed
  2. SquashFS stores full uid/gids (32 bits), and file creation time
  3. Files up to 2^64 bytes are supported; file systems can be up to 2^64 bytes
  4. Inode and directory data are highly compacted, and packed on byte boundaries; each compressed inode is on average 8 bytes in length (the exact length varies on file type, i.e. regular file, directory, symbolic link, and block/character device inodes have different sizes)
  5. SquashFS can use block sizes up to up to 64 Kb (2.x) and 1Mb (3.x). The default size is 128Kb (3.x), which achieves greater compression ratios than the normal 4K block size
  6. By the 2.x release it was introduced the concept of fragment blocks: an ability to join multiple files smaller than block size into a single block, achieving greater compression ratios
  7. File duplicates are detected and removed
  8. Both big and little endian architectures are supported; SquashFS can mount file systems created on different byte-order machines

SquashFS Capable Kernel

The on-disk format of SquashFS has stabilized enough that it has been merged into the 2.6.29 version of Linux kernel. I assume you use kernel with version greater than 2.6.29. If you are not, you should patch your kernel. But, I think today distribution has ship kernel greater than 2.6.29.

The Tools

You should check the command-line options for mksquashfs and unsquashfs for each capabilities.

Using mksquashfs

mksquashfs is the tool for creating new squashed file systems, and for appending new data to existing squashed file systems. The general command-line format for mksquashfs is:

mksquashfs source1 source2 ... destination [options]

Where:

  1. source1, source2, etc are files and directories to be added to the resulting filesystem, given with relative and/or absolute paths.
  2. destination is a regular file or a block device (such as /dev/fd0 or /dev/sda2) where the squashed file system will be created.

Notes, for default mksquashfs behavior:

  • When the new files are added to the new file system or appended to an existing one, mksquashfs will automatically rename files with duplicate names: if two or more files named text will appear in the same resulting directory, the second file will be renamed to text_1, third one to text_2 and so on.
  • Duplicate files will be removed, so there will be only one physical instance (By the SquashFS 2.x, you can disable the detection/removal of the duplicates with the -no-duplicates option).
  • If destination has a pre-existing SquashFS file system on it, by default, the new source items will be appended to the existing root directory. Examine the options table below to force mksquashfs to overwrite the whole destination and/or change the way new source items are added.
  • If a single source file or directory is given, it becomes the root in a newly created file system. If two or more source files and/or directories are given, they will all become sub-items in the root of the new file system.
  • The resulting filesystem will be padded to a multiple of 4 Kb: this is required for filesystems to be used on block devices. If you are very sure you don’t ned this, use the -nopad option to disable this operation.

Using unsquashfs

unsquashfs is the tool for extracting data from squashed file systems. The general command-line format for unsquashfs is:

unsquashfs [options] target [files/directories to extract]

Where the target is the squashed file system to extract.

Notes for unsquashfs behavior:

  • By not specifying any destination path, unsquashfs extracts the compressed file system in the ./squashfs-root directory.
  • The tool does not extract a squashed file system on already exsisting directory unless the -f option is specified.
  • You can specify on the command line, a multiple number of files/directories to extract and the items to be extracted can be also be given in a file with -e [file] option.

Managing SquashFS

This section will describe the use of mksquashfs and unsquashfs for managing SquashFS file.

Creating a SquashFS File

Creating Simple SquashFS

Creating a squashed file system out of a single directory (for ex: /some/dir), and output it to a regular file as image.sqsh.

mksquashfs /some/dir image.sqsh

Note that the output filename could be anything and any extension.

mksquashfs then will perform the squashing and print the resulting number of inodes and size of data written, as well as the average compression ratio. Now we have /some/dir directory image in image.sqsh file.

The content of image.sqsh would be any file or directory inside of /some/dir.

If you want to output the filesystem directly into a device, i.e. USB flash disk at /dev/sdb1:

mksquashfs /some/dir /dev/sdb1

Squashing File Systems

This section will describe how to create read-only compressed file system.

Mounting SquashFS File

To loop SquashFS file, you will use loopback device. The command is:

mount -o loop -t squashfs image.sqsh /mnt/dir

For image.sqsh as the SquashFS file and /mnt/dir as the mount point.

You should see that the SquashFS can only be accessed read-only.

To mount at boot time, you should at this entry to /etc/fstab (assume the image is image.sqsh and the mount point is /mnt/dir):

/some/dir/image.sqsh /mnt/dir squashfs loop 0 0

SquashFS and Write Operation using UnionFS

Unionfs provides copy-on-write semantics for the read-only file systems (such as SquashFS) which enhancing the possibilities. More detail on the UnionFS can be seen at http://www.filesystems.org/project-unionfs.html

Basically, UnionFS is a union of both writable filesystem and read-only filesystem, merged to a single mount point.

Example:

You may want to make your /home/user squashed, to compress and backup your files without losing the possibility to apply changes or writing new files.

Create the ro.fs squashed file system and the rw.fs dir:

mksquashfs /home/user1 ro.fs
mkdir /home/rw.fs

Mount the squashed ro.fs file system using the loopback device:

mount -t squashfs -o loop ro.fs /mnt

Mount the unionfs filesystem as union of /mnt/and /home/rw.fs. This will be merged under /home/user1 location:

cd /home
mount -t unionfs -o dirs=rw.fs=rw:/mnt=ro unionfs user1

The write operations under rw.fs will be done to /home/rw.fs.

To mount squashed directory tree at startup, you should load the squashfs and unionfs modules at boot time. Change the owner of the writable branch to match user1.

Here is the complete command list:

echo squashfs >> /etc/modules
echo unionfs >> /etc/modules
chown user1 /home/rw.fs

Add appropriate entry to /etc/fstab to mount the SquashFS and UnionFS at boot time.

/home/ro.fs /mnt/squashfs loop 0 0
unionfs /home/user1 unionfs dirs=/home/rw.fs=rw:/mnt=ro 0 0

Extract the File System

Also known as unsquashing. The unsquash operation will decompress the squashed file.

Suppose we have a file image.sqsh and want to unsquash it:

unsquashfs image.sqsh

A new directory will be created, as squashfs-root. The content of squashfs-root will be the content of file you or directory you use to create the squashed filesystem.

Social media & sharing icons powered by UltimatelySocial