How to extract a filesystem from a disk image

You need to backup an entire hard disk to a single file. Supposing your disk is at /dev/hda and the backup file is image-file, you’d do:

# cat /dev/hda > image-file

or

# dd if=/dev/hda of=image-file

The file backup you get will hold a copy of every single bit from the hard disk. This means that you also have a copy of the MBR in the first 512 bytes of the file.

Because of this, you can see the partition table on the backup file:

# sfdisk -l -uS image-file
Disk image-file: 0 cylinders, 0 heads, 0 sectors/track
Warning: The partition table looks like it was made
for C/H/S=*/255/32 (instead of 0/0/0).
For this listing I'll assume that geometry.
Units = sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System
image-filep1 32 261119 261088 83 Linux
image-filep2 261120 4267679 4006560 82 Linux swap / Solaris
image-filep3 4267680 142253279 137985600 83 Linux
image-filep4 0 - 0 0 Empty

Now, suppose you want to extract partition number 3. You can see that it starts at block 4267680 and is 137985600 blocks long. This translates into:

# dd if=image-file of=partition3-file skip=4267680 count=137985600

Now, peeking into the contents of the partition is as easy as:

# mount -t ext3 -o loop partition3-file /mnt/hack

Update (30 June 2011): as someone cleverly suggested in the comments, you can avoid using dd to extract the partition file by passing the offset option to mount as explained in this blog post.

Backing up a vmware virtual machine

I’ve learnt quite a number of new things today. One of these things is that an ISO9660 filesystem doesn’t allow files bigger than a couple of gigabytes. Another thing is that bzip2 kicks ass, even if it actually doesn’t implement the best compression algorithm available today (your milage may vary).

Another interesting thing is that the good old cdrtools package on Gentoo sucks. In my understanding that’s mainly because of licencing issues which came out during the software development. But, no worries, the new cdrkit ebuild has a modern cdrtools implementation with a compatible interface. That is, the good old mkisofs and cdrecord command lines you (and your GUIs) had learnt how to use are just the same. Different software, same interface. Great, uh? So, first of all, if you want the latest tools on a Gentoo box, you’ll need to:

# emerge -C cdrtools
# emerge -av cdrkit

Anyway, my job today was to backup on DVD-R media a large vmware virtual machine in order to regain free storage space on a server with serious shortage troubles. To begin with, we have a directory which holds the VM (virtual machine) files:

# du -sh my-very-cool-vm
21G my-very-cool-vm

Now, 21 gigabytes of data is a lot because this VM hosts a few extra virtual disks. But, if you’re lucky enough like me, some bzip2 goodness might help:

# tar cvf - my-very-cool-vm | \
bzip2 --compress --verbose --best --stdout > \
my-very-cool-vm.tar.bz2
# ls -lh my-very-cool-vm.tar.bz2
total 4.3G
-rw-r--r-- 1 root root 4.3G Mar 8 15:32 my-very-cool-vm.tar.bz2

Depending on your iron, the time between tar and ls might be enough for you to gain a few extra golds in World of Warcraft. Anyway, now that might actually fit on a DVD-R. But, since there’s a 2GBytes file size limit on ISO9660 filesystems, we still need to do a little something before we can fire up mkisofs. Our old unix friend split comes to the rescue:

# split -b 1024m -d my-very-cool-vm.tar.bz2 \
my-very-cool-vm.tar.bz2-part-
# rm my-very-cool-vm.tar.bz2
# ls -lh
total 4.3G
-rw-r--r-- 1 root root 1.0G Mar 8 16:04 my-very-cool-vm.tar.bz2-part-00
-rw-r--r-- 1 root root 1.0G Mar 8 16:05 my-very-cool-vm.tar.bz2-part-01
-rw-r--r-- 1 root root 1.0G Mar 8 16:05 my-very-cool-vm.tar.bz2-part-02
-rw-r--r-- 1 root root 1.0G Mar 8 16:06 my-very-cool-vm.tar.bz2-part-03
-rw-r--r-- 1 root root 295M Mar 8 16:06 my-very-cool-vm.tar.bz2-part-04

This is just perfect. Assuming the above files are in a directory called mydir, I can now run mkisofs like this:

# mkisofs -o dvd.iso -r mydir

This creates a file which holds the ISO9660 filesystem that we can later feed to cdrecord. My peculiar incantation for cdrecord is as follows:

cdrecord -v dev=/dev/hdc driver=mmc_mdvd -sao dvd.iso

Windows shares in /etc/fstab

Ok, this is pretty much an easy one, but I keep forgetting what the heck you need to specify in your /etc/fstab in order to mount an smb share (at least on a Linux system). Here is an handy example:

//host.lan/sharename /mnt/host/sharename smbfs defaults,rw,noauto,username=scott,password=tiger,uid=myuid,gid=mygid 0 0

All of the above needs to be on one line. Of course you need to replace a few things:

host.lan

hostname or ip address of the host you’re connecting to

sharename

the windows share name (pretty easy, uh?)

/mnt/host/sharename

a local directory where the share is to be mounted (this must exist before mounting so “mkdir -p” it)

username

the windows share username

password

the windows share password

uid

the user id the filesystem needs to be mounted as; this can be either numeric or symbolic and is usually your day-to-day user account

gid

the group id the filesystem needs to be mounted as; this can be either numeric or symbolic and is usually your day-to-day user account’s primary group

Once you’ve added that line into /etc/fstab, you can simply:

# mount /mnt/host/sharename

gpasswd and less +F

Before I knew about gpasswd, I used to add and remove users from groups using a combination of “usermod -G” and some cut and paste in the terminal window. I was even using sed to automate the command. That is now a thing of the past.

Most modern linux distributions have a gpasswd command (at least Gentoo and Ubuntu do have it). Adding a user to a group is as easy as:

# gpasswd -a username groupname

Removing is just a matter of changing the -a switch (add) into -d (delete):

# gpasswd -d username groupname

Another useful thing I discovered recently is the “follow mode” of less, which works like a tail -f. If you want to launch less in “follow mode”, you just have to add the +F switch, as in:

# less +F /var/log/messages

If you want, at any time, to scroll back through the backlog, you just have to interrupt the follow mode typing C-c (that is: control+c). Also, you don’t need to start less in follow mode to use the follow mode: you can just press F at any time within less to turn the follow mode on.

Sun One Webserver 6.1SP3 on Ubuntu Edgy

Okay, so, from time to time, I do pretty lame things too. I was trying to install Sun One Webserver 6.1SP3 on my Ubuntu Edgy laptop at work and, when launching the installer I got:

$ sh setup
/home/mcaserta/tmp/.setup: error while loading shared libraries: libstdc++-libc6.2-2.so.3: cannot open shared object file: No such file or directory

“This is an easy one”, I thought. I recalled the days when installing a recent Sun JDK on a Debian system required you to install a compatibility package such as libcompat-something or libstdc-something. So I fired up synaptic and made a search for both. I couldn’t find anything.

I said to myself: “If I make a couple symlinks, I can fool the system into thinking it’s got the library it requires and make it use the default one”. So, I did an:

$ sudo ln -s /usr/lib/libstdc++.so.6.0.8 /usr/lib/libstdc++-libc6.2-2.so.3

After this, the setup software started spitting out to me all sort of missing library files errors, which I tried fixing using the same principle as above.

After an hour of symlinking in pain, I went looking for the libcompat Ubuntu package and I found out it’s in Edgy in the universe repository. So, I went into synaptic, hit the repositories setup in the settings menu and enabled all available repositories. I had them all enabled before the Edgy upgrade but, apparently, the upgrade has disabled some of them.

I shut down synaptic and fired an:

$ sudo apt-get install libstdc++2.10-glibc2.2

and everything went fine since then, except that trying to launch the admin server results in a core dump and I’m looking into that right now.

Making vim syntax highlighting usable

Someone says vi vi vi is the editor of the beast. He’s right. That’s why I use vim, which is a modern editor, backwards compatible with vi and full of bells and whistles. But, most of all, vim gets things done for me. I’m in love with vim.

On some modern linux distributions, vim comes with syntax highlighting enabled by default. Since you’re a die-hard nerd like me, your terminal window colors scheme is like god meant it to be: with a black background. And, if you really feel nostalgic like me, you may even have gone one step further and you’ve setup green as the foreground color. That’s great, but there’s one thing that makes vim syntax highlighting suck on such a color scheme: it was meant by default to work on a light (read white) background.

My colleagues hate vim’s default color highlighting and they’re tempted all the time to use the old beast: vi, just because it has no color syntax highlighting which makes comments unreadable to say the least. This should get fixed. Enter the gay world with: set background=dark That means, while in vim, enter command mode and type the above command.

Great! The world has suddenly become clearer and you feel an urge to dance to the rhythm of Village People’s YMCA. That is, until you close vim and open it again. Then you’re into Robert Smith mode again. Too bad. We’ve got to fix that too. The recipe is as easy as editing your ~/.vimrc or, better yet, making the change system wide by editing the /etc/vim/vimrc file and adding a couple lines:

syntax on
set background=dark

That’s it. Now, where’s my biker dress?