pxe, e1000 and qemu-kvm 1.6 workaround

With the last update in Debian of qemu-kvm to version 1.6, booting vms via pxe with the e1000 (and ne2k_pci) network cards is no working anymore. I didn’t take a deep look of the issue yet, but there is an easy solution (actually 2) for that.

Easiest one:

wget https://github.com/qemu/qemu/raw/master/pc-bios/pxe-e1000.rom -O /usr/lib/ipxe/e1000_82540.rom

Another one:

git clone http://git.ipxe.org/ipxe.git 
cd ipxe/src 
make bin/8086100e.rom 
cp bin/8086100e.rom /usr/lib/ipxe/e1000_82540.rom

Debian And OpenStack

In February 2012, I wrote:

“The major problem of Debian and OpenStack, is Debian itself. In a so fast moving project like OpenStack, having an “EveryThing-Ready-and-Included” release every two years is not viable for this scenario. In the case of OpenStack, Squeeze is too old to handle it the way it is right now (Too many things not directly related with OpenStack, like qemu-xen, python-modules, etc.,   should be backported to make it functional, and with just volunteer packagers, make it a not so viable option). In the other hand, the next release, is planned to 1Q 2013 (Wheezy being frozen in later June), so Essex release will be outdated and Folsom we be the stable release by then, and with G release knocking on the door (Essex is supposed to be a LTS release, supported by Canonical/Ubuntu until around April 2017so is not that bad). Solutions?  Forget about Squeeze, focus in Wheezy + Essex as an official release , and Folsom (or G release) as a back-ported option. The other option (not viable in a short term), is change the way Debian makes it releases, and try to have something more “componentized” (like Progeny tried to do some years ago) or “rolling-releases”, so trending-topic these days.  Neither GNOME, OpenStack, LibreOffice, linux-kernel (just to name some) has a similar release cycle or core necessities to run in a system, so why try to release, prepare, stabilized everything at the same time no matter what’s the status the upstream project is? (Just int he case of OpenStack, and supposing everything thing is going according the schedule, with a 6 month release cycle vs. 2 years in Debian + frozen period, when Wheezy is released, we are going to release an outdated version -essex (but LTS thanks to Canonical)-  when Folsom is 4 months old! and the new release coming in less than two months!) I agree this needs a lot of more development, but i think it’s something that Debian should have to think about.”

Re-reading it, the situation is not that bad actually. We were lucky enough that the new Ubuntu LTS was released, and we can use the excuse of Essex being supported until 2017. But the key point is the same: “Debian is huge and slow, while OpenStack (and so many other projects) are really fast-moving”

Encrypted filesystems in Debian Squeeze

We need to have cryptsetup installed
root@squeeze:~# apt-get install cryptsetup
The partition can be a whole disk (a pendrive per example), a partition or even a lvm volume. In this case, we are going to use a lvm volume using a hard disk
root@squeeze:~# vgcreate vol0 /dev/sda
No physical volume label read from /dev/sda
Physical volume "/dev/sda" successfully created
Volume group "vol0" successfully created

root@squeeze:~# vgs
VG   #PV #LV #SN Attr   VSize    VFree   
vol0   1   0   0 wz--n- 1020.00m 1020.00m

root@squeeze:~# lvcreate -n lv_crypt -L 200M vol0
Logical volume "lv_crypt" created

root@squeeze:~# lvs
LV       VG   Attr   LSize   Origin Snap%  Move Log Copy%  Convert
lv_crypt vol0 -wi-a- 200.00m 

 

After this, the best thing to do is to write random data in the partition to generate noise.

root@squeeze:~# dd if=/dev/urandom of=/dev/mapper/vol0-lv_crypt
dd: writing to `/dev/mapper/vol0-lv_crypt': No space left on device
409601+0 records in
409600+0 records out
209715200 bytes (210 MB) copied, 60.7331 s, 3.5 MB/s

Finally, we create/format the encrypted partition, and open it to have it available to the system

root@squeeze:~# cryptsetup luksFormat /dev/mapper/vol0-lv_crypt 


WARNING!
========
This will overwrite data on /dev/mapper/vol0-lv_crypt irrevocably.


Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: 
Verify passphrase: 
root@squeeze:~# cryptsetup luksOpen /dev/mapper/vol0-lv_crypt crypt
Enter passphrase for /dev/mapper/vol0-lv_crypt: 

Once opened, we can access it as a non-encrypted partition available under /dev/mapper/crypt (all the names can be changed to something more suitable for you), so we can format and access it like a normal partition.

root@squeeze:~# mkfs.ext4 /dev/mapper/crypt 
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
51000 inodes, 203772 blocks
10188 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
25 block groups
8192 blocks per group, 8192 fragments per group
2040 inodes per group
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729


Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done


This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.


root@squeeze:~# mount /dev/mapper/crypt /media/crypt/

If you are using a graphical desktop, like GNOME, it can manage it automatically with Nautilus, asking you the pass-phrase and root password.

You can avoid to work in the command line using the /etc/crypttab file, to open this filesystem during system boot. The systax of this file is pretty simple:

root@squeeze:~# cryptsetup luksUUID /dev/mapper/vol0-lv_crypt 
1495abcf-38b5-4065-9880-4f8512d7c535
root@squeeze:~# cat /etc/crypttab 
#              
crypt   UUID=1495abcf-38b5-4065-9880-4f8512d7c535 none  luks

Using the disk UUID we can avoid future problems of the disk not being in the same path (like a pendrive)

At boot time, the system will ask for the passphare to open the encrypted disk

We can also especify a file with the key, to avoid this question. We can have a external usb drive with this file, or we can have this key files locally, to open external usb drives.

ext4 optimization and throughput (Preview)

Scenario: Debian Stable 6.0 under VmWare ESXi 4.1
Cases:
  • 1 core and 497 Mb RAM
  • 4 cores and 4Gb RAM
Test used:  Phoronix Test Suite
Mount options:
  • defaults
  • noatime
  • long  noatime,barrier=0,data=writeback,nobh,commit=100,nouser_xattr
  • no_journal – defaults, but fs created with “mkfs.ext4 /dev/sdb1 -O^has_journal” defaults
Results Case1 (1 Core):
Results Case2 (4 Cores):
Results Case1 & Case2 Mixed:
http://openbenchmarking.org/result/1103160-IV-MERGE353122 (It has some visualization problems)
Comments:
To be update… soon
Improvements to do:
Test in a bare installation, and testing hypervisor overload (vmware Vs. kvm)

Cronlogía de un Bug

Cronología de un Bug:
  • 12:42 Debian Bug report logs – #426698 lshw: Please add upstream’s Homepage to the package description

 

  • 13:21 From: Archive Administrator
    Subject: Processing of lshw_02.10-4_i386.changes

 

 

  • 13:32 From: Debian Installer
    Subject: lshw_02.10-4_i386.changes ACCEPTED
    Announcing to debian-devel-changes@lists.debian.org
    Closing bugs: 426698

 

 

  • 13:39 From: Debian Bug Tracking System
    Subject: Bug#426698: marked as done (lshw: Please add upstream’s Homepage to the package description)

 

Tiempo total transcurrido: 57 minutos

 

schroot – snapshots con LVM (II)

A la hora de probar y desarrollar nuevo software, el usar software actualizado es casi un requerimiento para poder acceder a aquellas funcionalidades. Ya vimos como tener distintas versiones instaladas, a la vez que para el día a día usamos algo más estable. Sin embargo, tras pruebas y compilaciones/recompilaciones, nuestros entornos de pruebas, puede llegar a “ensuciarse” o deteriorarse. Siempre podemos optar por borrar la instalación y reinstalarlo desde 0, pero hay sistemas más rápidos y más flexibles: Snapshots de LVM.

Una de las características que tiene LVM, es el de poder hacer snapshots (o copias exactas) de un dispositivo en un momento dado. Con LVM2, dichas copias, ya son de lectura/escritura, es decir, podemos modificar la copia… sin llegar a alterar el original. ¿Qué conseguimos con ésto? Tener un sistema recién instalado sobre el que trabajar… dejándolo totalmente limpio, una vez que hemos finalizado.

Para conseguir ésto, obviamente necesitamos tener una partición LVM, sobre la que instalaremos los sistemas, y espacio libre suficiente en LVM para poder hacer las copias. (Cómo crear particiones LVM y manejarlas quedan fuera del ámbito de este tutorial… pero no del siguiente)

Así pues, tenemos una partición LVM (/dev/Volume_00/schroot), que hemos montado en /schroot (por ejemplo) y hemos “instalado” una versión de sid (debootstrap sid /schroot/sid). Una vez que tenemos todo instalado y configurado a nuestro gusto, no es necesario tener la partición montada para nada.

Una vez preparado, sólo queda editar el fichero de configuración /etc/schroot/schroot.conf en el cual le especificamos que es una partición LVM y que cuando la ejecutemos, queremos que nos haga snapshots de ella:

[sid]
 aliases=unstable,default
 description=Debian sid
 location=/sid
 lvm-snapshot-options=--size 1G
 device=/dev/Volume_00/schroot
 type=lvm-snapshot
 priority=3
 groups=sbuild,root
 root-groups=sbuild,root
 source-groups=sbuild,root
 source-root-groups=root
 run-setup-scripts=true
 run-exec-scripts=true
Las opciones de configuración claves son:
  • location=/sid
    • Carpeta dentro de la partición de LVM donde se encuentra el sistema instalado (Se pueden tener varios sistemas en distintas carpetas colgando todos de la misma partición)
  • lvm-snapshot-options=–size 1G
    • Opciones a pasar al snapshots. En este caso, le decimos que el espacio libre en el snapshot será de 1 GB (para esto era necesario tener espacio libre sin particionar en el Volume LVM)
  • device=/dev/Volume_00/schroot
    • Partición donde tenemos instalados los sistemas.
  • type=lvm-snapshot
    • Tipo de partición.
Así pues, una vez configurado todo, podemos acceder a Sid, simplemente con el comando
schroot -c sid
Cuando creamos una configuración de tipo lvm-snapshot (sid en nuestro caso), automáticamente, también se crea una del tipo name-source (sid-source). Dicha configuración nos permite acceder a sid, pero sin crear un snapshot. Es decir, todos los cambios que hacemos, quedaran grabados. Ésto es útil, ya que Sid en una versión muy cambiante, y no está mal, ir actualizandola de vez en cuando.