When you grow an iSCSI volume there are several steps you need to take before you can get some extra space on your ext2 or ext3 filesystem. You need to get the OS to notice that the target has grown, and then you need to grow the filesystem.
This is only possible if:
• You have your filesystem mounted directly on the raw device
• You are using a volume manager like LVM2
If you have you device partitioned then you probably can't grow them this way. You may be able to if the partition is the last on the device and you grow that partition.
Firstly let’s check the size of the filesystem, and find out which device it is mounted on:
root@server:~# df -h /build/my_mnt
Filesystem Size Used Avail Use% Mounted on
/dev/sde 49G 40G 5.9G 88% /build/my_mnt
Now we'll check the size of the device:
root@server:~# cat /proc/partitions | grep -E "name|sde"
major minor #blocks name
8 64 51200000 sde
It's currently 50G. At this point you'll need to log into your SAN storage appliance and extend your iSCSI target to say 100G.
Once it’s extended we need to get the OS to notice. To do this we'll need the HBTL.
root@server:~# ls -l /sys/block/sde/device
lrwxrwxrwx 1 root root 0 2010-08-19 09:39 /sys/block/sde/device -> ../../../6:0:0:0
Now we can instruct the kernel to rescan the device so it sees it has grown:
root@server:~# echo 1 > /sys/class/scsi_device/6:0:0:0/device/rescan
Let’s check it again to make sure it worked:
root@server:~# cat /proc/partitions | grep -E "name|sde"
major minor #blocks name
8 64 104857600 sde
You will also be able to see log messages from the kernel in on of the /var/log/ files:
root@server:~# grep sde /var/log/kern.log
Aug 19 09:40:50 server kernel: [5507931.974895] sd 6:0:0:0: [sde] 209715200 512-byte logical blocks: (107 GB/100 GiB)
Aug 19 09:40:50 server kernel: [5507931.975697] sde: detected capacity change from 52428800000 to 107374182400
Aug 19 09:40:50 server kernel: [5507932.223757] VFS: busy inodes on changed media or resized disk sde
If you are using a volume manger like LVM2 you will need to grow the size of the logical volume with lvresize to take up the rest of the device.
Now unless you're running a kernel with a bunch of special patches applied then you are going to have to unmount your filesystem. There are lots of things that can stop you from being able to unmount a partition and I'll discuss here the ones I am aware of. When this happens you will get this output:
root@server:~# umount /build/my_mnt
umount: /build/my_mnt: device is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
In order to unmount the filesystem we have to firstly unmount any filesystems mounted within it.
root@server:~# cat /proc/mounts
grep /build/my_mnt
/dev/sde /build/my_mnt ext3 rw,relatime,errors=continue,user_xattr,acl,data=writeback 0 0
/dev/loop0 /build/my_mnt/mnt iso9660 ro,relatime 0 0
So here we need to unmount /build/my_mnt/mnt before unmounting /build/my_mnt. Before we do, we need to make sure nothing is using either of those mounts.We can check if there are any loopback devices set up like this:
root@server:~# losetup -a
/dev/loop0: [0840]:4769717 (/build/my_mnt/cdrom.iso)
/dev/loop5: [0840]:3068274 (/build/my_mnt/initrd.ext3)
We can either unmount these or use losetup to remove them. If they were mounted the loopback will be cleaned up:
root@server:~# umount /dev/loop0
root@server:~# umount /dev/loop5
umount: /dev/loop5: not mounted
root@server:~# losetup -d /dev/loop5
We can now see the loopbacks have gone and the mount within our filesystem:
root@server:~# losetup -a
root@server:~# cat /proc/mounts
grep /build/my_mnt
/dev/sde /build/my_mnt ext3 rw,relatime,errors=continue,user_xattr,acl,data=writeback 0 0
Now lets check if there are any open, mmaped, or locked files on our filesystem:
root@server:/build/mny_mnt# lsof
grep /build/my_mnt
bash 24654 root cwd DIR 8,64 4096 2 /build/my_mnt
emacs 24796 root cwd DIR 8,64 4096 2 /build/my_mnt
lsof 24799 root cwd DIR 8,64 4096 2 /build/my_mnt
grep 24800 root cwd DIR 8,64 4096 2 /build/my_mnt
lsof 24801 root cwd DIR 8,64 4096 2 /build/my_mnt
The lsof and grep entries are from the command we just ran with is nolonger running. The emacs process needs to be closed. The bash entry is becasue we are currently in the directory where our filesystem is mounted. After changing directory to / and exiting the emacs process the open files should nolonger be an issue.
Our filesystem may still be in use by a network share from another computer. For samba:
root@server:~# /etc/init.d/samba stop
root@server:~# umount /build/my_mnt
root@server:~# /etc/init.d/samba start
For NFS:
root@server:~# exportfs -f
root@server:~# umount /build/my_mnt
root@server:~# exportfs -a
Be aware that anyone using the network share will now be accessing the directory on the unlerlying filesystem until we remount. The filesystem should be free to unmount if you haven't already:
root@server:~# umount /build/my_mnt
Now to resize the filesystem. This could take quite a while and you'll probably be probted to check the filesystem first:
root@server:~# resize2fs /dev/sde
resize2fs 1.41.9 (22-Aug-2009)
Please run 'e2fsck -f /dev/sde' first.
root@server:~# e2fsck -f /dev/sde
e2fsck 1.41.9 (22-Aug-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sde: 1213129/6406144 files (6.6% non-contiguous), 10618919/12800000 blocks
root@server:~# resize2fs /dev/sde
resize2fs 1.41.9 (22-Aug-2009)
Resizing the filesystem on /dev/sde to 26214400 (4k) blocks.
The filesystem on /dev/sde is now 26214400 blocks long.
Finally we can remount the filesystem. You should probably restart the network shares to make sure they are looking at your mounted filesystem rather than the directory where it is mounted:
root@server:~# mount -a
root@server:~# /etc/init.d/samba stop
root@server:~# /etc/init.d/samba start
root@server:~# exportfs -f
root@server:~# exportfs -a
We can now check the size of the filesystem to make sure it did really grow:
root@server:~# df /build/my_mnt
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sde 103212320 41671520 56297920 43% /build/my_mnt
Saturday 21 August 2010
Thursday 12 August 2010
Windows XP – Replacing your Motherboard Without Reinstalling
Replacing your motherboard isn’t always as simple as it sounds. You may be lucky; the new motherboard may work with driver you already have installed. If you’re unlucky you’ll know about it.
You may find that with the new motherboard your computer reboots or BSOD. If you computer reboots you can press F8 before it boots and choose not to reboot on failure then you will see the BSOD with STOP 0x0000007B, which means it cannot find the boot device. This is because the drivers for the new harddrive controller on your motherboard are not installed.
There are several ways in which you can move from one motherboard to the next:
The complete reinstall: The most drastic course – unless you copy them off first you will lose everything on the boot partition which probably includes all of your documents, desktop, user settings etc. You don’t want to do this if possible. You will also have to reinstall all of your programs.
The upgrade install from windows: If your old motherboard is functional, boot up, pop your windows CD in, run the setup on the CD and choose to upgrade. It will copy some files over then reboot. Turn it off at this point and swap motherboards. When it boots up it will reinstall windows XP. You get to keep all your old settings + files and you don’t need to reinstall your programs. On the down side it means you lose all your windows updates - watch out for internet viruses which will attack you immediately as you’re not patched. I found with this method that I couldn’t get windows update to work until I installed SP3 and IE8 which I downloaded using another PC.
The upgrade install from boot CD: If your original motherboard is broken this is the next best thing to the previous option. This method only works on some variants of windows CD. Boot from your Windows XP CD and when you go to install pick the drive as if you are going to install over the top without formatting and you may get presented with the option to upgrade. Don’t pick anything else that offers to format, destroy or install in another directory. Once this starts upgrading it is pretty similar to the above method. If the upgrade option isn't there then you're unlucky. Don't pick anything else unless you're sure you want to lose your previous installation.
There is another way which doesn’t involve upgrading or reinstalling windows as long as you have access to the original working motherboard. It helps to understand why moving from one motherboard to another is a problem in the first place. After the initial install Windows cuts down the number of storage drivers it loads early in the boot to just the storage chipsets it has been in contact with. To start off with this will just be the driver that is used in your original motherboard. This makes booting a lot faster than loading all drivers it knows about just in case you’ve changed your hardware.
To get round this issue we need to expose windows to another storage chipset and install its driver. Windows will then be able to boot off this new storage chipset. The only issue is that the new chipset you want it to be exposed to is embedded into the new motherboard. We need a third chipset that we can move between the two motherboards.
This third chipset comes, depending on the type of harddrive you have, in the form a SATA or ATA PCI card which is able to boot. Plug this card into your old motherboard but don’t plug your harddrive into it yet. Boot and Windows will see the new card and install or ask for drivers. Once Windows is happy and has the drivers it needs you should be able to shut down your computer plug your disk into the card and boot from it. At this point Window will be able to boot from your disk attached to the motherboard or to the PCI card. You can now move this card to the new motherboard and boot off your drive attached to the PCI card. This will now see the new hardware on the motherboard and ask for all the divers.
It is a good idea to have all of these drivers extracted and ready on the drive before you boot on the new motherboard. If you are using a USB mouse and keyboard you may have initial difficulties as windows tends to throw up dialogs asking for drivers for hardware it doesn't have drivers for, before it tries adding the drivers for your USB controller, mouse and keyboard which it most probably does have drivers for. If you get into this situation you may have to skip over a few the first time through, then manually point windows at them later. If you have no mouse or keyboard working you may need to wait till it boots and settles, then tap the power button to get it to shut down and try booting again. If this doesn’t fix it you may need to borrow a ps2 keyboard and or mouse from a friend or work, but then nothing is that simple. If your computer has never seen a ps2 keyboard or mouse it may not install the drivers for them till you log in, which you can’t do till it’s got the drivers for at least the keyboard. In this situation you’ll need to boot on the old motherboard login and expose it to the ps2 keyboard/mouse then come back to the new motherboard again. Yes, I have experienced this myself.
Once you’re in, right click on my computer, properties, hardware, and then device manager. Add drivers for all the unknown items. If you say no to looking for the drivers using windows update and press next it should give you a clue as to which driver you are looking for, e.g. audio or storage controller. Once this is all set up you should be able to connect your harddrive to the motherboard and take out the PCI card and you’re done.
If you don't have a SATA or ATA PCI controller card you could probably get one off eBay and sell it again afterwards. Make sure from the seller that it is possible to boot off the card, the cheapest ones don't tend to let you boot.
A final note – always make sure you have anything you value backed up in case you make a mistake.
You may find that with the new motherboard your computer reboots or BSOD. If you computer reboots you can press F8 before it boots and choose not to reboot on failure then you will see the BSOD with STOP 0x0000007B, which means it cannot find the boot device. This is because the drivers for the new harddrive controller on your motherboard are not installed.
There are several ways in which you can move from one motherboard to the next:
The complete reinstall: The most drastic course – unless you copy them off first you will lose everything on the boot partition which probably includes all of your documents, desktop, user settings etc. You don’t want to do this if possible. You will also have to reinstall all of your programs.
The upgrade install from windows: If your old motherboard is functional, boot up, pop your windows CD in, run the setup on the CD and choose to upgrade. It will copy some files over then reboot. Turn it off at this point and swap motherboards. When it boots up it will reinstall windows XP. You get to keep all your old settings + files and you don’t need to reinstall your programs. On the down side it means you lose all your windows updates - watch out for internet viruses which will attack you immediately as you’re not patched. I found with this method that I couldn’t get windows update to work until I installed SP3 and IE8 which I downloaded using another PC.
The upgrade install from boot CD: If your original motherboard is broken this is the next best thing to the previous option. This method only works on some variants of windows CD. Boot from your Windows XP CD and when you go to install pick the drive as if you are going to install over the top without formatting and you may get presented with the option to upgrade. Don’t pick anything else that offers to format, destroy or install in another directory. Once this starts upgrading it is pretty similar to the above method. If the upgrade option isn't there then you're unlucky. Don't pick anything else unless you're sure you want to lose your previous installation.
There is another way which doesn’t involve upgrading or reinstalling windows as long as you have access to the original working motherboard. It helps to understand why moving from one motherboard to another is a problem in the first place. After the initial install Windows cuts down the number of storage drivers it loads early in the boot to just the storage chipsets it has been in contact with. To start off with this will just be the driver that is used in your original motherboard. This makes booting a lot faster than loading all drivers it knows about just in case you’ve changed your hardware.
To get round this issue we need to expose windows to another storage chipset and install its driver. Windows will then be able to boot off this new storage chipset. The only issue is that the new chipset you want it to be exposed to is embedded into the new motherboard. We need a third chipset that we can move between the two motherboards.
This third chipset comes, depending on the type of harddrive you have, in the form a SATA or ATA PCI card which is able to boot. Plug this card into your old motherboard but don’t plug your harddrive into it yet. Boot and Windows will see the new card and install or ask for drivers. Once Windows is happy and has the drivers it needs you should be able to shut down your computer plug your disk into the card and boot from it. At this point Window will be able to boot from your disk attached to the motherboard or to the PCI card. You can now move this card to the new motherboard and boot off your drive attached to the PCI card. This will now see the new hardware on the motherboard and ask for all the divers.
It is a good idea to have all of these drivers extracted and ready on the drive before you boot on the new motherboard. If you are using a USB mouse and keyboard you may have initial difficulties as windows tends to throw up dialogs asking for drivers for hardware it doesn't have drivers for, before it tries adding the drivers for your USB controller, mouse and keyboard which it most probably does have drivers for. If you get into this situation you may have to skip over a few the first time through, then manually point windows at them later. If you have no mouse or keyboard working you may need to wait till it boots and settles, then tap the power button to get it to shut down and try booting again. If this doesn’t fix it you may need to borrow a ps2 keyboard and or mouse from a friend or work, but then nothing is that simple. If your computer has never seen a ps2 keyboard or mouse it may not install the drivers for them till you log in, which you can’t do till it’s got the drivers for at least the keyboard. In this situation you’ll need to boot on the old motherboard login and expose it to the ps2 keyboard/mouse then come back to the new motherboard again. Yes, I have experienced this myself.
Once you’re in, right click on my computer, properties, hardware, and then device manager. Add drivers for all the unknown items. If you say no to looking for the drivers using windows update and press next it should give you a clue as to which driver you are looking for, e.g. audio or storage controller. Once this is all set up you should be able to connect your harddrive to the motherboard and take out the PCI card and you’re done.
If you don't have a SATA or ATA PCI controller card you could probably get one off eBay and sell it again afterwards. Make sure from the seller that it is possible to boot off the card, the cheapest ones don't tend to let you boot.
A final note – always make sure you have anything you value backed up in case you make a mistake.
WinMBoardMig and Windows XP SP3 is Bad
Whatever you do, do not use WinMBoardMig.zip if you are running Windows XP SP3 as it copies SP2 files over your SP3 files which will stop you computer from being able to boot. I leant this the hard way.
I ran this in the hope that it would allow me to migrate a Windows XP installation from one motherboard to another without having to reinstall. Unfortunately after I ran it and rebooted I found I could no longer boot with the original motherboard, let alone the new one. Normally Windows XP checks disks on start up and this is where it decides to fail with something like "Autochk not found -Skipping autochk.exe" then promptly reboots or BSOD. If you computer reboots at this point you can press F8 before it boots and choose not to reboot on failure then you will see the BSOD. I found I was also unable to get into get into safe mode. It hung after showing the list of drivers it loads.
As bad as this seems, I was able to fix it. I plugged the harddrive with the damaged Windows XP installation into another computer. From here I could see in the two directories in the WinMBoardMig I had run which contained system files:
WinMBoardMig\CriticalDrivers
04/08/2004 15:00 95,360 atapi.sys
04/08/2004 15:00 5,504 intelide.sys
17/08/2001 14:51 3,328 pciide.sys
03/08/2004 23:59 25,088 pciidex.sys
WinMBoardMig\Kernel
03/08/2004 23:59 105,472 hal.dll
03/08/2004 23:59 131,968 halaacpi.dll
03/08/2004 23:59 81,280 halacpi.dll
03/08/2004 23:59 150,656 halapic.dll
03/08/2004 23:59 134,400 halmacpi.dll
03/08/2004 23:59 152,704 halmps.dll
03/08/2004 23:59 77,696 halsp.dll
04/08/2004 00:18 2,148,352 ntkrnlmp.exe
03/08/2004 23:59 2,056,832 ntkrnlpa.exe
03/08/2004 23:59 2,015,232 ntkrpamp.exe
04/08/2004 00:20 2,180,992 ntoskrnl.exe
All I needed to do was find out where it had copied them and replace them with files of similar names from the SP3.cab files that were also on this system.
In order to fix any issues here the computer you are booting and doing the fixing from needs to be able to see hidden and system files and also be able to see file extensions. My damaged XP install was on drive F: here as I had booted on another computer. I found the SP3.cab in two locations:
F:\WINDOWS\ServicePackFiles\i386\sp3.cab
F:\WINDOWS\Driver Cache\i386\sp3.cab
I was able to open the sp3.cab files as if they were ordinary zip files using WinRAR.
I went through the list of files I found in WinMBoardMig one by one searching (F3 in explorer) for these files in my damaged installation. Not all of the files existed, but for those I found that matched the size of those from WinMBoardMig needed replacing with the equivalently named file from the SP3.cab.
I found files from WinMBoardMig\Kernel in F:\WINDOWS\system32\drivers and files from WinMBoardMig\Kernel in F:\WINDOWS\system32.
After I replaced the files I was able to boot on the original motherboard again. Now I was ready to try a different approach.
I ran this in the hope that it would allow me to migrate a Windows XP installation from one motherboard to another without having to reinstall. Unfortunately after I ran it and rebooted I found I could no longer boot with the original motherboard, let alone the new one. Normally Windows XP checks disks on start up and this is where it decides to fail with something like "Autochk not found -Skipping autochk.exe" then promptly reboots or BSOD. If you computer reboots at this point you can press F8 before it boots and choose not to reboot on failure then you will see the BSOD. I found I was also unable to get into get into safe mode. It hung after showing the list of drivers it loads.
As bad as this seems, I was able to fix it. I plugged the harddrive with the damaged Windows XP installation into another computer. From here I could see in the two directories in the WinMBoardMig I had run which contained system files:
WinMBoardMig\CriticalDrivers
04/08/2004 15:00 95,360 atapi.sys
04/08/2004 15:00 5,504 intelide.sys
17/08/2001 14:51 3,328 pciide.sys
03/08/2004 23:59 25,088 pciidex.sys
WinMBoardMig\Kernel
03/08/2004 23:59 105,472 hal.dll
03/08/2004 23:59 131,968 halaacpi.dll
03/08/2004 23:59 81,280 halacpi.dll
03/08/2004 23:59 150,656 halapic.dll
03/08/2004 23:59 134,400 halmacpi.dll
03/08/2004 23:59 152,704 halmps.dll
03/08/2004 23:59 77,696 halsp.dll
04/08/2004 00:18 2,148,352 ntkrnlmp.exe
03/08/2004 23:59 2,056,832 ntkrnlpa.exe
03/08/2004 23:59 2,015,232 ntkrpamp.exe
04/08/2004 00:20 2,180,992 ntoskrnl.exe
All I needed to do was find out where it had copied them and replace them with files of similar names from the SP3.cab files that were also on this system.
In order to fix any issues here the computer you are booting and doing the fixing from needs to be able to see hidden and system files and also be able to see file extensions. My damaged XP install was on drive F: here as I had booted on another computer. I found the SP3.cab in two locations:
F:\WINDOWS\ServicePackFiles\i386\sp3.cab
F:\WINDOWS\Driver Cache\i386\sp3.cab
I was able to open the sp3.cab files as if they were ordinary zip files using WinRAR.
I went through the list of files I found in WinMBoardMig one by one searching (F3 in explorer) for these files in my damaged installation. Not all of the files existed, but for those I found that matched the size of those from WinMBoardMig needed replacing with the equivalently named file from the SP3.cab.
I found files from WinMBoardMig\Kernel in F:\WINDOWS\system32\drivers and files from WinMBoardMig\Kernel in F:\WINDOWS\system32.
After I replaced the files I was able to boot on the original motherboard again. Now I was ready to try a different approach.
Friday 6 August 2010
Create a DOS USB boot disk from Windows 7
Download and install HP_USB_Boot_Utility.exe
Download and extract usbdos.zip
Plug in your USB stick
Copy everything off it you want to keep because it will be formatted.
Run the installed HP USB Disk Storage Format Tool
Select the USB disk, FAT filesystem, tick 'Create a DOS startup disk' and point it at the extracted usbdos
The disk should now be bootable and you can copy and firmware and flashing utilities you need to onto the disk. The final step now is to go into the BIOS of the computer you need to boot into DOS and change the boot order to boot from USB first.
Download and extract usbdos.zip
Plug in your USB stick
Copy everything off it you want to keep because it will be formatted.
Run the installed HP USB Disk Storage Format Tool
Select the USB disk, FAT filesystem, tick 'Create a DOS startup disk' and point it at the extracted usbdos
The disk should now be bootable and you can copy and firmware and flashing utilities you need to onto the disk. The final step now is to go into the BIOS of the computer you need to boot into DOS and change the boot order to boot from USB first.
Monday 2 August 2010
Replacing dmail with dovcot
When I replaced dmail with dovcot I found hit the issue that all of the users had entries in their .procmailrc files that referred to /opt/imap/dmail and used this to place their email in different mailboxes.
the command options of the dmail and dovcot also differ, for example, dmail:
:0 rw
* ^To:support@stormagic.com
/opt/imap/dmail username+Support
And the same with dovcot:
:0 rw
* ^To:support@stormagic.com
/usr/lib/dovecot/deliver -m Support
To get round this issue, rather than rewrite everyone's .procmailrc file I created the following dmail script which I saved as /opt/imap/dmail
root@email:~# cat /opt/imap/dmail
#!/bin/bash
DELIVER="/usr/lib/dovecot/deliver"
$DELIVER $(echo "$@"
sed -e 's#[^+]*##' -e 's#+\(.*\)#-m \1#')
the command options of the dmail and dovcot also differ, for example, dmail:
:0 rw
* ^To:support@stormagic.com
/opt/imap/dmail username+Support
And the same with dovcot:
:0 rw
* ^To:support@stormagic.com
/usr/lib/dovecot/deliver -m Support
To get round this issue, rather than rewrite everyone's .procmailrc file I created the following dmail script which I saved as /opt/imap/dmail
root@email:~# cat /opt/imap/dmail
#!/bin/bash
DELIVER="/usr/lib/dovecot/deliver"
$DELIVER $(echo "$@"
sed -e 's#[^+]*##' -e 's#+\(.*\)#-m \1#')
Spam Assasin
On Ubuntu 9.10 spamassassin can be easily be integrated on a per user basis by installing it with:
# apt-get install spamassassin
Next create your new email box by using your smtp email client.
Finally add the following to your .procmailrc
# Push all mail under 256k in size through spamassassin
# This will tag any mail deemed to be spam
:0fw: spamassassin.lock
* < 256000
spamassassin
# All mail tagged as spam is moved to "spam".
:0 w
* ^X-Spam-Status: Yes
/opt/imap/dmail username+spam # dmail
#/usr/lib/dovecot/deliver -m spam # dovcot
# Work around procmail bug: any output on stderr will cause
# the "F" in "From" to be dropped. This will re-add it.
:0
* ^^rom[ ]
{
LOG="*** Dropped F off From_ header! Fixing up. "
:0 fhw
sed -e '1s/^/F/'
}
I've used this with exim4, dovcot/dmesg, procmail.
# apt-get install spamassassin
Next create your new email box by using your smtp email client.
Finally add the following to your .procmailrc
# Push all mail under 256k in size through spamassassin
# This will tag any mail deemed to be spam
:0fw: spamassassin.lock
* < 256000
spamassassin
# All mail tagged as spam is moved to "spam".
:0 w
* ^X-Spam-Status: Yes
/opt/imap/dmail username+spam # dmail
#/usr/lib/dovecot/deliver -m spam # dovcot
# Work around procmail bug: any output on stderr will cause
# the "F" in "From" to be dropped. This will re-add it.
:0
* ^^rom[ ]
{
LOG="*** Dropped F off From_ header! Fixing up. "
:0 fhw
sed -e '1s/^/F/'
}
I've used this with exim4, dovcot/dmesg, procmail.
Subscribe to:
Posts (Atom)