News aggregator

Richard WM Jones: rich

Planet GLLUG - Sat, 23/03/2013 - 15:03

This is an excellent paper classifying bugs in Linux filesystems. The results seem to be generally applicable to bugs in open source kernel code.

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Thu, 21/03/2013 - 15:12

I wrote a small patch (intro, patch) which adds a Secure Shell (ssh) block device to qemu. With this patch you could access a remote disk image or device by doing:

qemu -drive file=ssh://host/path/to/file,if=virtio,cache=none

QEMU ssh’es into “host” and opens /path/to/file. For the initial version of this patch you will need to set up ssh-agent access to the remote server.

The motivation behind this patch is to allow libguestfs to access remote disks using ssh the same way we already do with NBD. Secure Shell is ubiquitous, so for the majority of users libguestfs-over-qemu/ssh would let them use disks remotely with zero configuration.

Categories: LUG Community Blogs

Karanbir Singh: Announcing the CentOS Dojo at Antwerp 2013

Planet GLLUG - Wed, 20/03/2013 - 12:18


The first ever CentOS Dojo, a one day training and socalising day dedicated to CentOS and how people use it, will be held at Antwerp, Belgium on the 8th of Apr.

You can see the great speaker lineup on the events page at : - we have tried to cover all the major conversation areas around CentOS these days. Ranging from provisioning, management, app deployments, system and virtualisation tuning, virtual infrastructure and more.

Its going to be a great day, register up, and see you all there. And remember, there is an exclusive CentOS Dojo Tshirt for everyone who attends ( plus, there might be more goddies too ).

Jump directly to the registration page :

- KB

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Mon, 18/03/2013 - 13:59

This script below lets you test changes while continuing to work on code. Let’s say that your test suite takes quite a while to run (hello, libguestfs). You can do:

$ test-change make check

The script copies the whole current directory into a temporary directory and runs the check in there. You still have to open a new terminal to run the tests, but the tests can go ahead while you continue working.

#!/bin/bash - # Copy current directory to a temporary, # then run the test command on that copy, # and report the results. # by Richard W.M. Jones <> # # Usage (from current directory): # test-change command [args ...] # eg: # test-change make check echo "Copying original directory; wait a moment ..." d=`mktemp -d` trap "rm -rf $d" EXIT INT TERM QUIT cp -a . $d cd $d echo "Original directory copied, starting test." echo "You can carry on working now." sleep 1 # Run the test command. "$@"
Categories: LUG Community Blogs

davblog - Dave Cross: Liverpudlian MPs

Planet GLLUG - Sat, 16/03/2013 - 12:31

Back in the day, when I grew up on my Liverpool council estate every member of Liverpool City council was Conservative. The city had eight Conservative MPs.

This is Nadine Dorries writing on Conservative Home a couple of days ago. She should really learn that if she doesn’t check her facts, then someone else will. You’ll be shocked, I suspect, to hear that this information is less than completely true. To me, it looks like Liverpool never had more than six Tory MPs while Dorries was growing up there.

Dorries was born in 1957. So let’s look at the 1955 general election and see which MPs were elected in Liverpool then. Liverpool has nine MPs, six of which are Tory. None of the seats changed hands in 1959. In 1964, however, the Tories lost four seats, taking their total down to two. This number remained constant in 1966 and 1970. The Tories lost another seat in February 1974 and remained steady on only one seat in October. Finally, in 1979 (when Dorries is 22 – so I’m not sure it still counts as while she was growing up) the Tories doubled their number of seats to a rather unimpressive two.

So Liverpool never had more than six Tory MPs – al least not while Dorries was growing up there. But she thinks that she can just throw a fact into an article like that and people will just accept it’s true.

You should never trust a word that Dorries writes. She has frequently been proven wrong on details like this.

p.s. Tim Fenton has run this analysis too and has reached similar conclusions. And, surprise surprise, he finds that her claims about the council are nonsense too.

Related Posts:
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Sat, 16/03/2013 - 11:47

It’s always been possible, but clumsy, to access Network Block Device (NBD) disks from libguestfs, but starting in libguestfs 1.22 we hope to make this (and Gluster, Ceph and Sheepdog access) much simpler.

The first change is upstream in libguestfs 1.21.21. You can add an NBD disk directly.

To show this using guestfish, I’ll start an NBD server. This could be started on another machine, but to make things simple I’ll start this server on the same machine:

$ qemu-nbd f18x64.img -t

f18x64.img is the disk image that I want to export. The -t option makes the qemu-nbd server persistent (ie. it doesn’t just exit after serving the first client).

Now we can connect to this server using guestfish as follows:

$ guestfish Welcome to guestfish, the libguestfs filesystem interactive shell for editing virtual machine filesystems. Type: 'help' for help on commands 'man' to read the manual 'quit' to quit the shell ><fs> add-drive "" format:raw protocol:nbd server:localhost ><fs> run

The empty "" (quotes) are the export name. Since qemu-nbd doesn’t support export names, we can leave this empty. The main change is to specify the protocol (nbd) and the server that libguestfs should connect to (localhost, but a remote host would also work). I haven’t specified a port number here because both the client and server are using the standard NBD port (10809), but you could use server:localhost:NNN to use a different port number if needed.

Ordinary guestfish commands just work:

><fs> list-filesystems /dev/sda1: ext4 /dev/fedora/root: ext4 /dev/fedora/swap: swap ><fs> inspect-os /dev/fedora/root ><fs> inspect-get-product-name /dev/fedora/root Fedora release 18 (Spherical Cow)

The next steps are to:

  1. Add support for more technologies. At least: Gluster, Ceph, Sheepdog and iSCSI, since those are all supported by qemu so we can easily leverage that support in libguestfs.
  2. Change the guestfish -a option to make adding remote drives possible from the command line.

The obvious [but not yet implemented] way to change the -a option is to allow a URI to be specified. For example:

$ guestfish -a nbd://localhost/exportname

where the elements of the URI like protocol, transport, server, port number and export name translate naturally into parameters of the add-drive API.

Categories: LUG Community Blogs

Karanbir Singh: is now gone

Planet GLLUG - Fri, 15/03/2013 - 14:28

It started off by being a place that everyone could chat and talk about things that were happening in the QA cycles inside CentOS. But things have changed quite a lot - our QA cycles are a lot shorter, there is a lot more automation and there is almost no real security exposure to users.

And I think we can do this better. We can create a better end user experience that gives them direct access, easily, to the state of play within the testing. And we should be able to automate more to get better coverage.

To that aim, is now going away. And we are working on some alternatives. Starting with having a nightly QA cycle, that considers point releases and all updates upto that point. And adding more external tests as well, like the ltp content ( ). If you wish to join in that effort, drop in on the centos-devel list ( ) and jump right in. Ref threads: and

Here is a link to the official announcement that just went out :

See you there,

- KB

Categories: LUG Community Blogs

Karanbir Singh: Microphones and Online Chats

Planet GLLUG - Thu, 14/03/2013 - 01:04


Over the last couple of weeks, I've spent quite a lot of time online talking with people about various things. Either using software voice over IP or google-hangout or Skype or whatever. I've noticed recently there seems to be a growing trend for people to have more and more expensive and professional Microphones for these conversations.


Except, get a grip. There is a limit to what and how these Mic's are meant to be used. The point at which I can hear a cat crossing the road two blocks away from your house - too much. Either get a proper sound / recording room setup, or just use that mic on the webcam or laptop. Really, its good enough for these chats.

A £250 Sennheiser Mic for this sort of a thing, not needed. Talking about the mic setup and getting the wind-blow from when you mention P, not needed. You neither sound better not look prettier. Stick to that 1bit laptop mic. We'll all be happier for it.

- KB

Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Wed, 13/03/2013 - 10:00

Just over 3 years ago I wrote that guestfish supported 267 commands and I listed them out.

Today, in libguestfs 1.21.19, guestfish has 502 commands, listed below.

Command Description help display a list of commands or help on a command quit quit guestfish acl-delete-def-file delete the default POSIX ACL of a directory acl-get-file get the POSIX ACL attached to a file acl-set-file set the POSIX ACL attached to a file add-cdrom add a CD-ROM disc image to examine add-domain add the disc(s) from a named libvirt domain add-drive add an image to examine or modify add-drive-ro add a drive in snapshot mode (read-only) add-drive-ro-with-if add a drive read-only specifying the QEMU block emulation to use add-drive-with-if add a drive specifying the QEMU block emulation to use alloc allocate and add a disc file aug-clear clear Augeas path aug-close close the current Augeas handle aug-defnode define an Augeas node aug-defvar define an Augeas variable aug-get look up the value of an Augeas path aug-init create a new Augeas handle aug-insert insert a sibling Augeas node aug-load load files into the tree aug-ls list Augeas nodes under augpath aug-match return Augeas nodes which match augpath aug-mv move Augeas node aug-rm remove an Augeas path aug-save write all pending Augeas changes to disk aug-set set Augeas path to value available test availability of some parts of the API available-all-groups return a list of all optional groups base64-in upload base64-encoded data to file base64-out download file and encode as base64 blkid print block device attributes blockdev-flushbufs flush device buffers blockdev-getbsz get blocksize of block device blockdev-getro is block device set to read-only blockdev-getsize64 get total size of device in bytes blockdev-getss get sectorsize of block device blockdev-getsz get total size of device in 512-byte sectors blockdev-rereadpt reread partition table blockdev-setbsz set blocksize of block device blockdev-setro set block device to read-only blockdev-setrw set block device to read-write btrfs-device-add add devices to a btrfs filesystem btrfs-device-delete remove devices from a btrfs filesystem btrfs-filesystem-balance balance a btrfs filesystem btrfs-filesystem-resize resize a btrfs filesystem btrfs-filesystem-sync sync a btrfs filesystem btrfs-fsck check a btrfs filesystem btrfs-set-seeding enable or disable the seeding feature of device btrfs-subvolume-create create a btrfs snapshot btrfs-subvolume-delete delete a btrfs snapshot btrfs-subvolume-list list btrfs snapshots and subvolumes btrfs-subvolume-set-default set default btrfs subvolume btrfs-subvolume-snapshot create a writable btrfs snapshot canonical-device-name return canonical device name cap-get-file get the Linux capabilities attached to a file cap-set-file set the Linux capabilities attached to a file case-sensitive-path return true path on case-insensitive filesystem cat list the contents of a file checksum compute MD5, SHAx or CRC checksum of file checksum-device compute MD5, SHAx or CRC checksum of the contents of a device checksums-out compute MD5, SHAx or CRC checksum of files in a directory chmod change file mode chown change file owner and group command run a command from the guest filesystem command-lines run a command, returning lines compress-device-out output compressed device compress-out output compressed file config add qemu parameters copy-device-to-device copy from source device to destination device copy-device-to-file copy from source device to destination file copy-file-to-device copy from source file to destination device copy-file-to-file copy from source file to destination file copy-in copy local files or directories into an image copy-out copy remote files or directories out of an image copy-size copy size bytes from source to destination using dd cp copy a file cp-a copy a file or directory recursively dd copy from source to destination using dd debug debugging and internals debug-drives debug the drives (internal use only) debug-upload upload a file to the appliance (internal use only) delete-event delete a previously registered event handler device-index convert device to index df report file system disk space usage df-h report file system disk space usage (human readable) disk-format detect the disk format of a disk image disk-has-backing-file return whether disk has a backing file disk-virtual-size return virtual size of a disk display display an image dmesg return kernel messages download download a file to the local machine download-offset download a file to the local machine with offset and size drop-caches drop kernel page cache, dentries and inodes du estimate file space usage e2fsck check an ext2/ext3 filesystem e2fsck-f check an ext2/ext3 filesystem echo display a line of text echo-daemon echo arguments back to the client edit edit a file egrep return lines matching a pattern egrepi return lines matching a pattern equal test if two files have equal contents event register a handler for an event or events exists test if file or directory exists fallocate preallocate a file in the guest filesystem fallocate64 preallocate a file in the guest filesystem fgrep return lines matching a pattern fgrepi return lines matching a pattern file determine file type file-architecture detect the architecture of a binary file filesize return the size of the file in bytes filesystem-available check if filesystem is available fill fill a file with octets fill-dir fill a directory with empty files fill-pattern fill a file with a repeating pattern of bytes find find all files and directories find0 find all files and directories, returning NUL-separated list findfs-label find a filesystem by label findfs-uuid find a filesystem by UUID fsck run the filesystem checker fstrim trim free space in a filesystem get-append get the additional kernel options get-attach-method get the attach method get-autosync get autosync mode get-cachedir get the appliance cache directory get-direct get direct appliance mode flag get-e2attrs get ext2 file attributes of a file get-e2generation get ext2 file generation of a file get-e2label get the ext2/3/4 filesystem label get-e2uuid get the ext2/3/4 filesystem UUID get-libvirt-requested-credential-challenge challenge of i'th requested credential get-libvirt-requested-credential-defresult default result of i'th requested credential get-libvirt-requested-credential-prompt prompt of i'th requested credential get-libvirt-requested-credentials get list of credentials requested by libvirt get-memsize get memory allocated to the qemu subprocess get-network get enable network flag get-path get the search path get-pgroup get process group flag get-pid get PID of qemu subprocess get-qemu get the qemu binary get-recovery-proc get recovery process enabled flag get-selinux get SELinux enabled flag get-smp get number of virtual CPUs in appliance get-tmpdir get the temporary directory get-trace get command trace enabled flag get-umask get the current umask get-verbose get verbose mode getcon get SELinux security context getxattr get a single extended attribute getxattrs list extended attributes of a file or directory glob expand wildcards in command glob-expand expand a wildcard path grep return lines matching a pattern grepi return lines matching a pattern grub-install install GRUB 1 head return first 10 lines of a file head-n return first N lines of a file hexdump dump a file in hexadecimal hexedit edit with a hex editor hivex-close close the current hivex handle hivex-commit commit (write) changes back to the hive hivex-node-add-child add a child node hivex-node-children return list of nodes which are subkeys of node hivex-node-delete-child delete a node (recursively) hivex-node-get-child return the named child of node hivex-node-get-value return the named value hivex-node-name return the name of the node hivex-node-parent return the parent of node hivex-node-set-value set or replace a single value in a node hivex-node-values return list of values attached to node hivex-open open a Windows Registry hive file hivex-root return the root node of the hive hivex-value-key return the key field from the (key, datatype, data) tuple hivex-value-type return the data type from the (key, datatype, data) tuple hivex-value-utf8 return the data field from the (key, datatype, data) tuple hivex-value-value return the data field from the (key, datatype, data) tuple initrd-cat list the contents of a single file in an initrd initrd-list list files in an initrd inotify-add-watch add an inotify watch inotify-close close the inotify handle inotify-files return list of watched files that had events inotify-init create an inotify handle inotify-read return list of inotify events inotify-rm-watch remove an inotify watch inspect-get-arch get architecture of inspected operating system inspect-get-distro get distro of inspected operating system inspect-get-drive-mappings get drive letter mappings inspect-get-filesystems get filesystems associated with inspected operating system inspect-get-format get format of inspected operating system inspect-get-hostname get hostname of the operating system inspect-get-icon get the icon corresponding to this operating system inspect-get-major-version get major version of inspected operating system inspect-get-minor-version get minor version of inspected operating system inspect-get-mountpoints get mountpoints of inspected operating system inspect-get-package-format get package format used by the operating system inspect-get-package-management get package management tool used by the operating system inspect-get-product-name get product name of inspected operating system inspect-get-product-variant get product variant of inspected operating system inspect-get-roots return list of operating systems found by last inspection inspect-get-type get type of inspected operating system inspect-get-windows-current-control-set get Windows CurrentControlSet of inspected operating system inspect-get-windows-systemroot get Windows systemroot of inspected operating system inspect-is-live get live flag for install disk inspect-is-multipart get multipart flag for install disk inspect-is-netinst get netinst (network installer) flag for install disk inspect-list-applications get list of applications installed in the operating system inspect-list-applications2 get list of applications installed in the operating system inspect-os inspect disk and return list of operating systems found is-blockdev test if block device is-chardev test if character device is-config is in configuration state is-dir test if a directory is-fifo test if FIFO (named pipe) is-file test if a regular file is-lv test if device is a logical volume is-socket test if socket is-symlink test if symbolic link is-whole-device test if a device is a whole device is-zero test if a file contains all zero bytes is-zero-device test if a device contains all zero bytes isoinfo get ISO information from primary volume descriptor of ISO file isoinfo-device get ISO information from primary volume descriptor of device kill-subprocess kill the qemu subprocess launch launch the qemu subprocess lcd change working directory lchown change file owner and group ldmtool-create-all scan and create Windows dynamic disk volumes ldmtool-diskgroup-disks return the disks in a Windows dynamic disk group ldmtool-diskgroup-name return the name of a Windows dynamic disk group ldmtool-diskgroup-volumes return the volumes in a Windows dynamic disk group ldmtool-remove-all remove all Windows dynamic disk volumes ldmtool-scan scan for Windows dynamic disks ldmtool-scan-devices scan for Windows dynamic disks ldmtool-volume-hint return the hint field of a Windows dynamic disk volume ldmtool-volume-partitions return the partitions in a Windows dynamic disk volume ldmtool-volume-type return the type of a Windows dynamic disk volume lgetxattr get a single extended attribute lgetxattrs list extended attributes of a file or directory list-9p list 9p filesystems list-devices list the block devices list-disk-labels mapping of disk labels to devices list-dm-devices list device mapper devices list-events list event handlers list-filesystems list filesystems list-ldm-partitions list all Windows dynamic disk partitions list-ldm-volumes list all Windows dynamic disk volumes list-md-devices list Linux md (RAID) devices list-partitions list the partitions ll list the files in a directory (long format) llz list the files in a directory (long format with SELinux contexts) ln create a hard link ln-f create a hard link ln-s create a symbolic link ln-sf create a symbolic link lremovexattr remove extended attribute of a file or directory ls list the files in a directory ls0 get list of files in a directory lsetxattr set extended attribute of a file or directory lstat get file information for a symbolic link lstatlist lstat on multiple files luks-add-key add a key on a LUKS encrypted device luks-close close a LUKS device luks-format format a block device as a LUKS encrypted device luks-format-cipher format a block device as a LUKS encrypted device luks-kill-slot remove a key from a LUKS encrypted device luks-open open a LUKS-encrypted block device luks-open-ro open a LUKS-encrypted block device read-only lvcreate create an LVM logical volume lvcreate-free create an LVM logical volume in % remaining free space lvm-canonical-lv-name get canonical name of an LV lvm-clear-filter clear LVM device filter lvm-remove-all remove all LVM LVs, VGs and PVs lvm-set-filter set LVM device filter lvremove remove an LVM logical volume lvrename rename an LVM logical volume lvresize resize an LVM logical volume lvresize-free expand an LV to fill free space lvs list the LVM logical volumes (LVs) lvs-full list the LVM logical volumes (LVs) lvuuid get the UUID of a logical volume lxattrlist lgetxattr on multiple files man open the manual max-disks maximum number of disks that may be added md-create create a Linux md (RAID) device md-detail obtain metadata for an MD device md-stat get underlying devices from an MD device md-stop stop a Linux md (RAID) device mkdir create a directory mkdir-mode create a directory with a particular mode mkdir-p create a directory and parents mkdtemp create a temporary directory mke2fs create an ext2/ext3/ext4 filesystem on device mke2fs-J make ext2/3/4 filesystem with external journal mke2fs-JL make ext2/3/4 filesystem with external journal mke2fs-JU make ext2/3/4 filesystem with external journal mke2journal make ext2/3/4 external journal mke2journal-L make ext2/3/4 external journal with label mke2journal-U make ext2/3/4 external journal with UUID mkfifo make FIFO (named pipe) mkfs make a filesystem mkfs-b make a filesystem with block size mkfs-btrfs create a btrfs filesystem mklost-and-found make lost+found directory on an ext2/3/4 filesystem mkmountpoint create a mountpoint mknod make block, character or FIFO devices mknod-b make block device node mknod-c make char device node mkswap create a swap partition mkswap-L create a swap partition with a label mkswap-U create a swap partition with an explicit UUID mkswap-file create a swap file mktemp create a temporary file modprobe load a kernel module more view a file mount mount a guest disk at a position in the filesystem mount-9p mount 9p filesystem mount-local mount on the local filesystem mount-local-run run main loop of mount on the local filesystem mount-loop mount a file using the loop device mount-options mount a guest disk with mount options mount-ro mount a guest disk, read-only mount-vfs mount a guest disk with mount options and vfstype mountpoints show mountpoints mounts show mounted filesystems mv move a file nr-devices return number of whole block devices (disks) added ntfs-3g-probe probe NTFS volume ntfsclone-in restore NTFS from backup file ntfsclone-out save NTFS to backup file ntfsfix fix common errors and force Windows to check NTFS ntfsresize resize an NTFS filesystem ntfsresize-size resize an NTFS filesystem (with size) parse-environment parse the environment and set handle flags accordingly parse-environment-list parse the environment and set handle flags accordingly part-add add a partition to the device part-del delete a partition part-disk partition whole disk with a single primary partition part-get-bootable return true if a partition is bootable part-get-gpt-type get the type GUID of a GPT partition part-get-mbr-id get the MBR type byte (ID byte) from a partition part-get-parttype get the partition table type part-init create an empty partition table part-list list partitions on a device part-set-bootable make a partition bootable part-set-gpt-type set the type GUID of a GPT partition part-set-mbr-id set the MBR type byte (ID byte) of a partition part-set-name set partition name part-to-dev convert partition name to device name part-to-partnum convert partition name to partition number ping-daemon ping the guest daemon pread read part of a file pread-device read part of a device pvchange-uuid generate a new random UUID for a physical volume pvchange-uuid-all generate new random UUIDs for all physical volumes pvcreate create an LVM physical volume pvremove remove an LVM physical volume pvresize resize an LVM physical volume pvresize-size resize an LVM physical volume (with size) pvs list the LVM physical volumes (PVs) pvs-full list the LVM physical volumes (PVs) pvuuid get the UUID of a physical volume pwrite write to part of a file pwrite-device write to part of a device read-file read a file read-lines read file as lines readdir read directories entries readlink read the target of a symbolic link readlinklist readlink on multiple files realpath canonicalized absolute pathname remove-drive remove a disk image removexattr remove extended attribute of a file or directory rename rename a file on the same filesystem reopen close and reopen libguestfs handle resize2fs resize an ext2, ext3 or ext4 filesystem resize2fs-M resize an ext2, ext3 or ext4 filesystem to the minimum size resize2fs-size resize an ext2, ext3 or ext4 filesystem (with size) rm remove a file rm-f remove a file ignoring errors rm-rf remove a file or directory recursively rmdir remove a directory rmmountpoint remove a mountpoint rsync synchronize the contents of two directories rsync-in synchronize host or remote filesystem with filesystem rsync-out synchronize filesystem with host or remote filesystem scrub-device scrub (securely wipe) a device scrub-file scrub (securely wipe) a file scrub-freespace scrub (securely wipe) free space set-append add options to kernel command line set-attach-method set the attach method set-autosync set autosync mode set-cachedir set the appliance cache directory set-direct enable or disable direct appliance mode set-e2attrs set ext2 file attributes of a file set-e2generation set ext2 file generation of a file set-e2label set the ext2/3/4 filesystem label set-e2uuid set the ext2/3/4 filesystem UUID set-label set filesystem label set-libvirt-requested-credential pass requested credential back to libvirt set-libvirt-supported-credentials set libvirt credentials supported by calling program set-memsize set memory allocated to the qemu subprocess set-network set enable network flag set-path set the search path set-pgroup set process group flag set-qemu set the qemu binary set-recovery-proc enable or disable the recovery process set-selinux set SELinux enabled or disabled at appliance boot set-smp set number of virtual CPUs in appliance set-tmpdir set the temporary directory set-trace enable or disable command traces set-verbose set verbose mode setcon set SELinux security context setenv set an environment variable setxattr set extended attribute of a file or directory sfdisk create partitions on a block device sfdiskM create partitions on a block device sfdisk-N modify a single partition on a block device sfdisk-disk-geometry display the disk geometry from the partition table sfdisk-kernel-geometry display the kernel geometry sfdisk-l display the partition table sh run a command via the shell sh-lines run a command via the shell returning lines shutdown shutdown the qemu subprocess sleep sleep for some seconds sparse create a sparse disk image and add stat get file information statvfs get file system statistics strings print the printable strings in a file strings-e print the printable strings in a file supported list supported groups of commands swapoff-device disable swap on device swapoff-file disable swap on file swapoff-label disable swap on labeled swap partition swapoff-uuid disable swap on swap partition by UUID swapon-device enable swap on device swapon-file enable swap on file swapon-label enable swap on labeled swap partition swapon-uuid enable swap on swap partition by UUID sync sync disks, writes are flushed through to the disk image tail return last 10 lines of a file tail-n return last N lines of a file tar-in unpack tarfile to directory tar-out pack directory into tarfile tgz-in unpack compressed tarball to directory tgz-out pack directory into compressed tarball time print elapsed time taken to run a command touch update file timestamps or create a new file truncate truncate a file to zero size truncate-size truncate a file to a particular size tune2fs adjust ext2/ext3/ext4 filesystem parameters tune2fs-l get ext2/ext3/ext4 superblock details txz-in unpack compressed tarball to directory txz-out pack directory into compressed tarball umask set file mode creation mask (umask) umount unmount a filesystem umount-all unmount all filesystems umount-local unmount a locally mounted filesystem unsetenv unset an environment variable upload upload a file from the local machine upload-offset upload a file from the local machine with offset user-cancel cancel the current upload or download operation utimens set timestamp of a file with nanosecond precision utsname appliance kernel version version get the library version number vfs-label get the filesystem label vfs-type get the Linux VFS type corresponding to a mounted device vfs-uuid get the filesystem UUID vg-activate activate or deactivate some volume groups vg-activate-all activate or deactivate all volume groups vgchange-uuid generate a new random UUID for a volume group vgchange-uuid-all generate new random UUIDs for all volume groups vgcreate create an LVM volume group vglvuuids get the LV UUIDs of all LVs in the volume group vgmeta get volume group metadata vgpvuuids get the PV UUIDs containing the volume group vgremove remove an LVM volume group vgrename rename an LVM volume group vgs list the LVM volume groups (VGs) vgs-full list the LVM volume groups (VGs) vgscan rescan for LVM physical volumes, volume groups and logical volumes vguuid get the UUID of a volume group wc-c count characters in a file wc-l count lines in a file wc-w count words in a file wipefs wipe a filesystem signature from a device write create a new file write-append append content to end of file write-file create a file xfs-admin change parameters of an XFS filesystem xfs-growfs expand an existing XFS filesystem xfs-info get geometry of XFS filesystem xfs-repair repair an XFS filesystem zegrep return lines matching a pattern zegrepi return lines matching a pattern zero write zeroes to the device zero-device write zeroes to an entire device zero-free-space zero free space in a filesystem zerofree zero unused inodes and disk blocks on ext2/3 filesystem zfgrep return lines matching a pattern zfgrepi return lines matching a pattern zfile determine file type inside a compressed file zgrep return lines matching a pattern zgrepi return lines matching a pattern Use -h / help to show detailed help for a command.
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Wed, 13/03/2013 - 00:01

Thanks to infernix who contributed this tip on how to use libguestfs to access Ceph (and in theory, sheepdog, gluster, iscsi and more) devices.

If you apply this small patch to libguestfs you can use these distributed filesystems straight away by doing:

$ guestfish ><fs> set-attach-method appliance ><fs> add-drive /dev/null ><fs> config -set drive.hd0.file=rbd:pool/volume ><fs> run

… followed by usual guestfish commands.

This is a temporary hack, until we properly model Ceph (etc) through the libguestfs stable API. Nevertheless it works as follows:

  1. The add-drive /dev/null adds a drive, known to libguestfs.
  2. Implicitly this means that libguestfs adds a -drive option when it runs qemu.
  3. The custom qemu -set drive.hd0.file=... parameter modifies the preceding -drive option added by libguestfs so that the file is changed from /dev/null to whatever you want. In this case, to a Ceph rbd:... path.

Categories: LUG Community Blogs

Karanbir Singh: in maint mode

Planet GLLUG - Tue, 12/03/2013 - 02:15

Hi, services are going to be partially down as I migrate services over to a faster, more memory, lesser power consuming, many more cores machine. Everything should be back to production by midday Mar 12th, 2013. Services impacted include:

  • git repos
  • Reimzul's irc interface
  • Alt.Bsys triggers

There is a backup instance running, so if anyone needs to get to some specific data in a rush, ping me on irc and we can get access setup.

- KB

Categories: LUG Community Blogs

davblog - Dave Cross: Doctor Who News

Planet GLLUG - Sun, 10/03/2013 - 17:22

I’m getting bored of the number of media outlets who are taking the slightest of comments that someone makes about the upcoming Doctor Who anniversary special and spinning it into a story packed full of completely unsubstantiated nonsense. Headlines like “No Doctors To Return For 50th Special” which, when you read them turn out to be based on the fact that Colin Baker hasn’t had a phone call from Steven Moffat.

Obviously it’s good for the show that it gets all of this publicity and I don’t, for one second, expect the production team to do anything to put a stop to it. They’ll tell us what they want us to know when they want us to know it. Not a moment sooner.

But in the meantime, anyone who has ever appeared in Doctor Who has to watch what they say for fear of it being overhead by a tabloid journalist and being used to reinforce what ever story the journalist wants to write.

In an attempt to counter this, I’ve set up The plan is that I’ll read these stories, extract the actual facts that they are based on and explain what we can actually believe based on those facts. Forensic analysis of entertainment news, I suppose.

I’ve also got a page where I list the best current information we have about what is actually happening for the show’s 50th anniversary. I’ll try to keep that up to date as more details emerge over the coming months.

Oh, and there’s at Twitter account too – WhoNews50. You might want to follow that.

Let me know if you find it useful.

Related Posts:
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 05/03/2013 - 13:55

Matthew Astley just emailed me to say he has forked TechTalk and added support for markdown (the wiki-ish markup language).

Categories: LUG Community Blogs

Martin A. Brooks: An open letter to National Car Parks

Planet GLLUG - Mon, 04/03/2013 - 18:31

Dear National Car Parks,

As it seems the existence of your call centre is to goad people into a state of rage and you insist that everything has to be put in writing, I shall put my complaint to you in writing via my blog, before sending it to you in writing.

I am a season ticket holder for the station car park at Harold Wood.  My car is parked there on a daily basis.  On return to my vehicle on February 28th, I was surprised to find I had been ticketed.  Here’s the ticket:























Note the contravention: “Parked on double yellow lines”.  This was a surprise to me because there were and are no double yellow lines present.  For the non-drivers reading this, and confused NCP parking attendants, double yellow lines tend to look like this:










This is quite clear, right?  Two parallel yellow lines at specific distances from the side of the road with a crossing bar at each end.  Now, here’s a picture of where I was parked:


See the bay to the left of the red car? That’s where my car was, wholly within the confines of the white border.  You might want to check out the full size version of the photo but, looking at the example double yellow lines above, can you see where the double yellow lines I was ticketed for parking on are? I’ll sit here quietly while you check.

Done? Good.  You didn’t spot the double yellow lines, did you? Correct, that’s because there aren’t any to spot. I can read your mind.  “Ahhh, Martin, you silly boy.  You’ve parked on a restricted area, there are yellow hatchings. Tsk.”

That’s almost true, there are indeed yellow hatchings but that’s irrelevant.  Why so?  Just this:  When NCP took over the car park many years ago, they repainted all of the bays with fresh white markings.  Here’s a closeup of part of that bay:


















Note that the white line is painted over the yellow one.  Had NCP not intended for people to use this as a parking bay, why on earth would they specifically paint bay markings there?  The answer is, they wouldn’t.  Those markings used to denoted an area where you weren’t supposed to park because it’s where a fast food place used to have its bins.  Rather than scraping the old lines up, they just painted new ones over the top.

Add to this the fact that I have been parked in that exact same spot dozens of times over the past couple of years, and this is the first time my car has been ticketed, I think what we’re dealing with here is an overzealous or brainless parking attendant.I currently can’t even talk to you about this over the phone because even after 4 days, the ticket hasn’t appeared on your system. I now have to waste some of my life sorting this out.

No love whatsoever, merely tired rage,

Martin A. Brooks


Categories: LUG Community Blogs

davblog - Dave Cross: Money From HMRC

Planet GLLUG - Sat, 02/03/2013 - 11:57

I got a letter from HMRC this morning – to my company, not to me personally. It basically said “we’ve been looking at the PAYE you paid in 2010/11 and it looks like you’ve overpaid by [a surprisingly large number of pounds]“.

Now 2010/11 was the year that I was having some difficulties with my accountants. The difficulties eventually got so bad that I switched to my new accountants (who I’m still very happy with). So it doesn’t really surprise me that something went wrong that year, although the amount (it’s about 25% of the PAYE/NI I paid that year) is impressive.

What really surprises me is the tone of this letter. Having told me that I’ve overpaid (and, helpfully, pointed out the exact extra payment that I made) the letter goes on to say:

Before I can agree to either a refund or a credit, please let me the reason the overpayment has arisen


Please complete the enclosed P35D giving a full explanation as to how the overpayment occurred.

Below are some example of reasons I cannot accept to justify an overpayment

  • Duplicate payments with no evidence to explain why
  • Statements such as ‘Payment(s) made in error’ with no further explanation
  • The overpayment is due to monthly payments which do not match our records
  • Any other explanation without evidence to support it

I’m finding it hard to read that in any way other than “we know we’ve got some of your money but you can’t have it back until you’ve explained in detail just how crap your record-keeping is”.

You’ve got my money. You know it’s my money. Either my accountants or I screwed up in some way. There’s no more detailed explanation than that. Just give it back to me, you bastards.

Related Posts:
Categories: LUG Community Blogs

Richard WM Jones: rich

Planet GLLUG - Tue, 26/02/2013 - 20:31

New in libguestfs ≥ 1.21.15, the virt-df and virt-alignment-scan tools now use parallel appliances when scanning your libvirt guests.

The amount of parallelism is selected heuristically when the tool starts up — by dividing the amount of free memory in MB by 500. You can also override this choice by using the new -P option to both tools, but the default should be fine for everyone. -P 1 disables multiple threads.

Users won’t see much difference, although I found that both tools are noticeably faster.

The implementation of threads in these tools is a little bit interesting. Of course there is a pool of worker threads. These take the libvirt guests from a list sorted in alphabetical order and process them.

However each guest takes a variable amount of time to process, and the trick is that the output from each thread mustn’t overlap or be in non-alphabetical order.

The worker threads do two things to ensure this: Firstly output from each guest scan is saved up in an open_memstream buffer. Secondly, domains are retired in order using a pthread condition variable — each worker waits until the previous domain has been retired, before retiring (ie. printing) its own result.

The outcome is that there should be no difference between what the old tools and the rewritten tools print out.

Categories: LUG Community Blogs

Rev. Simon Rumble: Current status

Planet GLLUG - Tue, 26/02/2013 - 04:56

Chilling in Jervis Bay. Weather has turned out much better than the forecasts. Lovely.

See the full gallery on Posterous

Permalink | Leave a comment  »

Categories: LUG Community Blogs

Richard WM Jones: data

Planet GLLUG - Mon, 25/02/2013 - 16:46

[Part 1, part 2, part 3.]

Finally I modified the test to do some representative work: We now load a real Windows XP guest, inspect it (a heavyweight operation), and mount and stat each filesystem. I won’t reproduce the entire test program again because only the test subroutine has changed:

sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/tmp/winxp.img"); $g->launch (); # Inspect the guest (ignore the result). $g->inspect_os (); # Approximate what virt-df does. my %fses = $g->list_filesystems (); foreach (keys %fses) { my $mounted = 0; eval { $g->mount_ro ($_, "/"); $mounted = 1; }; if ($mounted) { $g->statvfs ("/"); $g->umount_all (); } } return $g; }

Even with all that work going on, I was able to inspect more than 1 disk per second on my laptop, and run 60 threads in parallel with good performance and scalability:

Categories: LUG Community Blogs

Richard WM Jones: data

Planet GLLUG - Mon, 25/02/2013 - 16:22

A problem encountered in part 2 was that I couldn’t measure the maximum number of parallel libguestfs appliances that can be run at the same time. There are two reasons for that. The simpler one is that libvirt has a limit of 20 connections, which is easily overcome by setting LIBGUESTFS_ATTACH_METHOD=appliance to eliminate libvirt and run qemu directly. The harder one is that by the time the last appliances in the test are starting to launch, earlier ones have already shut down and their threads have exited.

What is needed is for the test to work in two phases: In the first phase we start up all the threads and launch all the appliances. Only when this is complete do we enter the second phase where we shut down all the appliances.

The easiest way to do this is by modifying the test to use a barrier (or in fact to implement a barrier using the condition primitives). See the modified test script below.

With the modified test script I was able to run ≥ 110 and < 120 parallel appliances in ~ 13 GB of free RAM, or around 120 MB / appliance, still with excellent performance and nearly linear scalability:

#!/usr/bin/perl -w use strict; use threads qw(yield); use threads::shared qw(cond_broadcast cond_wait lock); use Sys::Guestfs; use Time::HiRes qw(time); my $nr_threads_launching :shared; sub test { my $g = Sys::Guestfs->new; $g->add_drive_ro ("/dev/null"); $g->launch (); return $g; } # Get everything into cache. test (); test (); test (); sub thread { my $g = test (); { lock ($nr_threads_launching); $nr_threads_launching--; cond_broadcast ($nr_threads_launching); cond_wait ($nr_threads_launching) until $nr_threads_launching == 0; } $g->close (); } # Test increasing numbers of threads until it fails. for (my $nr_threads = 10; $nr_threads < 1000; $nr_threads += 10) { my $start_t = time (); $nr_threads_launching = $nr_threads; my @threads; foreach (1..$nr_threads) { push @threads, threads->create (\&thread) } foreach (@threads) { $_->join (); if (my $err = $_->error ()) { die "launch failed with $nr_threads threads: $err" } } my $end_t = time (); printf ("%d %.2f\n", $nr_threads, $end_t - $start_t); }
Categories: LUG Community Blogs

Richard WM Jones: data

Planet GLLUG - Mon, 25/02/2013 - 15:28

One problem with the previous test is that I hit a limit of 20 parallel appliances and mistakenly thought that I’d hit a memory limit. In fact libvirt out of the box limits the number of client connections to 20. You can adjust libvirt’s limit by editing /etc/libvirt/libvirtd.conf, but easier for us is to simply eliminate libvirt from the equation by doing:


which causes libguestfs to run qemu directly. In my first test I reached 48 parallel launches before I killed the program (because that’s a lot of parallelism and there seemed no end in sight). Scalability of the libguestfs / qemu combination was excellent again:

But there’s more! (In the next part …)

Categories: LUG Community Blogs
Syndicate content