I switched from Google Reader to Liferea a while back. I was mostly happy with the functionality of Reader but I don't really want my reading history to be recorded or my feed list to be vulnerable to the whim of Google's inscrutable abuse department.

I'm still using Liferea but unfortunately it has a lot of problems. Yes I do have bug numbers:

• Calls sync() many times while refreshing feeds (#623619). This is probably related to use of SQLite and not properly batching writes. I work around this by using 'eatmydata' to nullify those calls, but that's not ideal.
• The UI thread also blocks on feed updates. Together with the previous bug, this can make it unresponsive for minutes while updating feeds (#542276).
• It often leaks memory and other resources when I play embedded video (#660593).
• There is no progress or 'busy' indicator for the browser (#660595).
• Error/prompt callbacks are not handled (#660597) - for example, if I follow a link that happens to lead to a PDF, nothing happens.
• There is no progress indicator for downloading 'enclosed' files (typically podcast episodes), and errors are simply ignored (#660599).
• While updating feeds, it may reset the item pane back to the summary (#660602).

So, I'm looking for an alternative again. My requirements are:

1. MAY be a desktop or web application.
2. If it's a web application, it MUST be reasonably secure, e.g. it must not be written in PHP.
3. If it's a web application, it MUST allow for multiple independent users on the same server.
4. If it's a desktop application, it MUST embed a browser engine (presumably Gecko or WebKit) so I can follow links without having to switch windows.
5. MUST support organisation of feeds by folders or tags, including combined item lists.
6. MUST keep track of which items have been read.
7. MUST support a global 'unread items' list. SHOULD only remove items from this list when I refresh it, not as soon as I move away from an item.
8. SHOULD support a three-pane (folder/list/item) view or something similar. Google Reader's list view with expanding items is perhaps even better, though it means links must be opened in a separate tab.
9. SHOULD support folder and item navigation by keyboard.
10. SHOULD have some way to flag/bookmark items for later attention.
11. If it's a desktop application, it SHOULD have some sort of download manager to support podcasts.

~$ dpkg --print-architecture
i386
~$ dpkg -s linux-image-$(uname -r)
Package `linux-image-3.2.0-1-amd64' is not installed and no info is available.
Use dpkg --info (= dpkg-deb --info) to examine archive files,
and dpkg --contents (= dpkg-deb --contents) to list their contents.
~$ dpkg -s linux-image-$(uname -r):amd64
Package: linux-image-3.2.0-1-amd64
Status: install ok installed
Priority: optional
Section: kernel
Installed-Size: 104822
Maintainer: Debian Kernel Team <debian-kernel@lists.debian.org>
Architecture: amd64
Source: linux-2.6
Version: 3.2.6-1

Phil Kern made a general call for testing of Debian 6.0.4. I would like to more specifically point out that I backported the isci driver for Intel C600 SAS/SATA controllers. Unfortunately I have not yet had any testing results for this. If you have any machines with this hardware not yet in production, please do consider testing the new Linux kernel package, version 2.6.32-41.

Updated: I now have a positive test result from the user who requested this driver in the installer.

I thought it was about time to get a faster machine for... well, mostly for compiling kernels faster, but I'm sure I will find other applications for it. I was originally intending to upgrade my laptop (Thinkpad T61, 1.8 GHz Core 2 Duo) but it seems that in the 3 years since I bought it second-hand laptop prices have not dropped in line with Moore's law. So instead I decided to upgrade (well, replace most of) a desktop PC.

The last time I got a new desktop at home was in 2002. It has an Athlon XP, PATA drives and a huge CRT monitor. It has had some upgrades since then (DVD writer, more RAM, slightly faster CPU, replacement fan and PSU). Obviously it was time to replace motherboard, CPU and RAM. Not so obviously, the PSU needed changing again as modern CPUs apparently need more 12V electrons (?!). The DVD writer was salvageable via a USB adapter and its local HD is currently a laptop drive (the only spare SATA HD I had). The monitor is of course reusable, though the firmware setup program doesn't entirely agree with it.

Windows

Yes, I do sometimes run Windows. It's for games, and also seems to be needed for streaming video with stupid DRM.

On the assumption that Windows is less likely to play nice with an existing installation, I tried installing it first. The firmware appear to allow me to boot the DVD in either BIOS or UEFI mode, so I selected the UEFI option. This worked so far as loading the initramfs (or whatever the Windows equivalent is) but then Windows was lacking a driver to access the DVD. This stumped me for a day or so until I realised I had plugged the drive into a USB 3 (XHCI) port. Apparently Windows 7 SP1 still doesn't include an XHCI driver in the installer (but neither does Debian 6.0.3 - we should fix that).

The installation was mostly uneventful after this. The Windows installer includes a reasonably capable partition editor.

Debian

Installation of files was entirely uneventful. The installer correctly detected the GPT (GUID partition table) created by Windows. However, it was not able to install a boot loader. Currently it only includes support for the LILO and GRUB-PC boot loaders that run under BIOS.

After much experimentation and cursing, I found instructions for installing Debian Squeeze on XServe 3.1 by Jan Peter Alexander Rajagukguk, which put me on the right track. It seems that the grub-install script for GRUB-EFI in squeeze (and maybe other parts of the package) doesn't actually work. However. JP's direction to build GRUB from source is no longer necessary. Here's what I did (minus the mis-steps):

1. Boot the installer in rescue mode and open a shell on the installed system:

   # # We start in dash, so switch to bash.
   # exec bash
   # mkdir -p /boot/efi
   # # Mount the EFI boot partition by UUID.
   # # It's probably called sda1 now, but is subject to change.
   # ls -l /dev/disk/by-uuid
   ...
   # echo >>/etc/fstab 'UUID=efi-boot-uuid /boot/efi vfat defaults 0 0'
   # mount /boot/efi
   # # Enable unstable sources, but use stable by default
   # echo >>/etc/apt/apt.conf 'APT::Default-Release "squeeze";'
   # echo >>/etc/apt/sources.list 'deb http://cdn.debian.net/debian/ sid main'
   # apt-get update
   ...
   # # Install unstable grub-efi-amd64
   # apt-get install -t unstable grub-efi-amd64
   ...
   # # Install GRUB to the EFI partition.  The '--removable' options
   # # inhibits setting EFI variables, which won't work because we booted
   # # through BIOS emulation.  Instead, it installs as the default boot
   # # loader.
   # grub-install --bootloader-id=GRUB --removable
   # # On the AMI/ASUS firmware, you cannot select a new bootloader
   # # directly but can invoke a 'shell'.  So I made GRUB the shell:
   # cp /boot/efi/boot/grub/grub.efi /boot/efi/shellx64.efi
   # exit
   

2. Reboot and select GRUB as the boot loader if necessary. In my case I had to enter the setup program and select 'EFI shell'. Debian should now boot from the hard drive.
3. Add a non-volatile boot option for GRUB (and make it the default):

   # apt-get install efibootmgr
   ...
   # modprobe efivars
   # efibootmgr -c -l '\efi\grub\grubx64.efi' -L GRUB
   

The bug report

Simon McVittie reported in Debian bug #654876 that his laptop running Linux 3.0 or 3.1 would sometimes crash (panic) while idle. He initially suspected a driver bug, but the screen did not show any information about where the original fault occurred. However, using netconsole, he was able to capture a full log of the crash. This showed that a packet received through the wireless interface was being processed by IGMP, which then divided by zero.

IGMP

IGMP is part of the IPv4 protocol suite, supporting multicast routing. Every multicast address corresponds to a dynamic set of hosts, called a multicast group. Multicast routers can send query messages asking which hosts belong to which groups, and hosts using multicast report back at intervals. Routers can then limit forwarding of multicast packets to the interfaces where the group has members. More sophisticated switches can also snoop IGMP and use it to limit their multicast forwarding. There are unfortunately three different versions with semi-compatible message formats. In version 1, the maximum reporting interval (Max Response Time) is fixed as 10 seconds, but from version 2 it is specified in query messages.

The Linux IGMP implementation supports all three versions. It distinguishes query messages as specified in RFC 3376 section 7.1: v3 messages are longer than v1 or v2; v2 messages have a non-zero Max Response Time whereas v1 messages always have zero. It is possible to force use of a specific protocol version, but normally if there are multiple multicast routers using different protocol versions it will respond according to the earliest protocol version in use so that all routers can understand its responses.

Source and fix for the bug

Linux 2.6.36 included two fixes to the version selection logic. Unfortunately, the second of these introduced the bug in question. While v2 query messages cannot possibly have zero Max Response Time (as that would make them v1), v3 query messages can. What this means is unspecified, but the Linux IGMP code previously treated it as the minimum valid value of 1/10 second. But in the case where a v3 query is received and a v2 query has also recently been received, this is no longer done. This results in a reporting interval of 0 seconds and a division by zero when deciding the initial random delay.

The fix for this is pretty obvious. It is included in Linux 3.0.17, 3.1.9, 3.2.1, and the Debian package version 3.1.8-2.

Security impact

This is easily exploitable for denial of service within a local network. Linux does not check the destination address of IGMP queries, so it may also be possible to attack a target through its unicast address from several hops away. The attacker only needs to send a single IGMPv2 query followed by a single IGMPv3 query with zero Max Response Time. All systems running Linux 2.6.36 or later (up to the above fixed versions) with any active IPv4 multicast listeners (other than for the 'all hosts' address) are vulnerable.

]]> Sat, 14 Jan 2012 16:17 GMT gnome-wont-let-you-watch-your-videos-in-peace http://womble.decadent.org.uk/blog/gnome-wont-let-you-watch-your-videos-in-peace.html

Screensaver control on the Linux desktop is a total mess, but the xdg-screensaver command provided an interim cross-desktop solution by detecting and using the appropriate underlying interfaces. For GNOME, it depends on the gnome-screensaver-command --poke option, but that has been broken and then removed in favour of the cross-desktop D-Bus API org.freedesktop.ScreenSaver another GNOME-specific API which can't practically be used by xdg-screensaver.

What's more, switching to the new API seems to have introduced various bugs in GNOME's own video player, Totem, such that it often failed to inhibit the screensaver in GNOME 2.32 and 3.0. Apparently everything is wonderful again in 3.2; here's hoping.

(In case anyone asks where my bug report on the totem package is, I already looked at the bug list and it's clear that there's no point.)

Updated: Well I said the GNOME Session D-Bus API couldn't practically be used in xdg-screensaver, but then set out to prove myself wrong. And with the aid of Perl, I may have succeeded.

The Debian kernel team regularly backports driver updates to the Linux kernel in stable releases to add support for new hardware, and I've prepared several updates intended for point release 6.0.2. Since the kernel team does not have a large collection of hardware on which to test driver changes, we would appreciate test reports from users. It is important to test not just that new devices are supported properly, but that there are no regressions in support for older devices.

Changes

I have updated these drivers to the versions found in Linux 2.6.38, modulo driver API changes:

• bna: New driver for Brocade Ethernet adapters. (#627699)
• e1000e: Add support for i82567V-4, i82579. Bug fixes for i82577, i82578, i82583. (#627700)
• hpsa: New driver for HP Smart Array controllers. It also supports some devices that are currently handled by cciss, but I have disabled that for 'squeeze'. (#627701)
• igb, igbvf: Add support for i82576-ET2, 82580, DH89xxCC, i350. (#627702)
• pm8001: New driver for PMC-Sierra SAS/SATA HBAs. (#627703)
• r8169: Add full support for RTL8168DP. Bug fixes for RTL8102E. (#627704)
• tg3, broadcom: Add support for BCM5717, BCM5719, BCM57765, and BCM50610M and BCM5241 PHYs. Bug fixes for BCM5755. (#627705)

I have also cherry-picked some small changes:

• bnx2i: Add support for BCM5771E
• wl1251: Add support for PG11 chips
• bnx2x: Add support for BCM84823
• ar9170usb: Add several additional USB device IDs

There are more drivers that I think should be added or updated (see #624794) but they will probably have to wait for release 6.0.3.

Packages

The source package and binary packages for i386 and amd64 are available on people.debian.org. They can be verified by the checksums in the signed changes file.

The current packages are version 2.6.32-35~test1, but there may be further test versions before an official stable update.

How to test

For network drivers, I suggest the following regression tests:

1. If the driver tries to load firmware (only required for some chips), does this work once the firmware file(s) are installed?
2. Can you receive and transmit VLAN-tagged frames after creating a VLAN interface?
3. Does the interface work after suspend and resume?
4. Does the interface work after removing the cable for 10 seconds and reinserting it?
5. Does multicast configuration work? (IPv6 autoconfiguration or mDNS will cover this.)
6. Can the interface send and receive TCP/IP across a LAN at the same speed, before and after these changes? (Use e.g. netperf to test this, but don't forget to remove the netperf package after use.)
7. Are any warnings or errors logged by the kernel during the preceding tests?

For storage drivers, unfortunately I don't have a good idea of what tests would be suitable. In any case, please don't test on disks storing valuable data!

Please send test reports to the bug reports linked above, stating the driver name, the PCI ID for the device you tested (from lspci -n) and any other device identification that the kernel log (for example, r8169 logs the 'XID' of the device).

The major upcoming configuration change in Linux 2.6.39 is to get rid of the '686' flavour. This may be surprising, because it's the most widely used flavour of the 4 we have a present:

However, the physical address limitation means that an increasing proportion of new PCs and the majority of PC servers need the '686-bigmem' flavour. Even those that have less than 4 GiB RAM do support PAE and can run the '686-bigmem' flavour. There is a small cost (up to about 0.1% of RAM) in the use of larger hardware page tables. There is also an important benefit on recent processors: the larger page table entries include an NX bit (also known as XD) which provides protection against some buffer overflow attacks, both in the kernel and in user-space..

Q: What about 686 processors without PAE?

There are only a few 686-class processors that support CMOV but not PAE: most Intel Pentium M models, the VIA C3 'Nehemiah' and the AMD Geode LX. These also all lack SMP support, which means the '486' flavour is suitable for them.

Some benchmarking on two of those - a Pentium M model 745 and a C3 'Nehemiah' - indicated that they run the '486' flavour slightly faster than the '686' flavour. It appears that the performance gain from using plain uniprocessor code (rather than SMP-alternatives, which are patched with NOPs on uniprocessor systems) outweighs the performance loss from avoiding use of some newer instructions.

Q: Why get rid of '686' but keep the 'amd64' flavour?

We intend to get rid of 'amd64' too, but we need to ensure that upgrades from linux-image-2.6-amd64:i386 to linux-image-2.6-amd64:amd64 work properly.

Q: If '686-bigmem' will be the default, isn't 'bigmem' redundant?

Yes, it is. Therefore '686-bigmem' will be renamed to '686-pae'.

Linux 2.6.38 and 2.6.39

As you've probably seen, the testing suite finally got a new upstream kernel version (Linux 2.6.38) and is currently synchronised with unstable. I'm just about to upload a new version of linux-2.6 based on upstream stable update 2.6.38.4.

Linux 2.6.39 is well on its way to release, so 2.6.39-rc4 should soon be in experimental. The Debian source package will add support for the new armhf (ARM with hardware floating-point) architecture.

Splitting linux-2.6

The linux-base package, introduced to handle transitions and other support functions for 'image' packages, is now built from a separate source package. It won't be changed very often, so you will no longer need to upgrade linux-base every time you upgrade an 'image' package. (This doesn't apply within 'squeeze', but I may have a way to work around that.)

The firmware-linux-free package, containing a few firmware and configuration blobs from Linux that are DFSG-compliant, is also now built from a separate source package. These blobs have not changed for a very long time, so again you will no longer be recommended to upgrade this package every time you upgrade an 'image' package. Secondly, this allows us to remove the entire 'firmware' directory from the upstream source rather than carefully removing and patching to leave just the DFSG-compliant bits.

Wireless regulations

The '2.4 GHz' and '5 GHz' ISM bands used for wifi are not defined identically around the world. The exact frequency ranges and maximum permitted power levels vary between countries. Radio equipment manufacturers have a legal responsibility to ensure their equipment operates within these regulations. Since it is expensive to implement such regulations in hardware (including many different variants for different regions), these restrictions are usually implemented in the driver using a country code retrieved from the hardware's EEPROM or flash. So this is now our responsibility.

We need to be able to look up regulations by country codes, and we also need to support the case where a device sold in one country is used in another country with tighter regulations. For some time, the wifi configuration library (cfg80211) included regulatory information for a small number of countries and regions. However, the default since Linux 2.6.34 has been to build-in only the 'world' domain (intersection of all the restrictions) and to request information from user-space. In Debian, we didn't have anything to provide this information, so for example channels 12-13 became unavailable for users in Europe.

The regulatory information should be provided by the Central Regulatory Domain Agent (CRDA) from a current database. These are now packaged as crda and wireless-regdb respectively. Drivers that do not use cfg80211 (mostly those from the 'staging' area) will not be affected, and may or may not implement correct regulations.

When you travel to another country or use an imported wifi device with an incorrect country code (I hear 'CN' is common), you can use iw reg set to ensure that your wifi devices operate within the regulations of the country you are in. Set the REGDOMAIN variable in /etc/default/crda to make this persistent. These will be intersected with the regulations specified by the hardware.

I'm still missing how to reliably detect if a device is a "wired" or a "wireless" one. I suspect that checking the existence of /phy80211 would be enough, but I can't really tell, and seems like I'm not able to find an exhaustive sysfs reference manual.

Sadly there is no exhaustive manual, but many attributes found in sysfs are documented under Documentation/ABI/. In this case you need the DEVTYPE name from the net device's uevent, which may be e.g. 'wlan' or 'wwan' (and is absent for wired Ethernet devices). I don't know why this isn't also exposed as an attribute in its own right.

Andrew Pollock writes:

If you're trying to use a USB serial adapter as your Linux kernel console...

The kernel needs to be compiled with CONFIG_USB_SERIAL_SERIAL=y

That's CONFIG_USB_SERIAL_CONSOLE.

Debian kernels do not appear to be. Somewhat strangely, the config option doesn't even wind up in the config file in the typical commented-out manner, so you don't even know the option exists until you discover it from random Web searching, and then looking at drivers/usb/serial/Kconfig in the kernel source.

Console drivers need to be built-in, so CONFIG_USB_SERIAL_CONSOLE depends on CONFIG_USB_SERIAL=y. We build the USB serial code as a module so CONFIG_USB_SERIAL_CONSOLE is not an available option. This is unlikely to change, as a built-in feature takes memory from every Debian GNU/Linux installation.

• #595396 - probably fixable with a binNMU, but I also changed debian/rules to catch regressions
• #596646 - downgraded; no justification given for 'serious' severity
• #597059 - closed as not a bug
• #599262 - reassigned to the kernel (linux-2.6) and downgraded as not RC there
• #599709 - applied the upstream fix
• #599711 - backported the upstream fix; downgraded as it's only a denial of service and may not even affect Debian

Something was bound to go wrong when changing the policy for boot loaders. Let me try that again.

Rogério Brito writes:

[...] whenever I was typing a command line, changed my mind and pressed C-c, I got a ^C printed on the screen, usually overwriting one or two characters of what I had typed. And this prevented me from automatically copying and pasting the command that I had typed.

There is an alternative to using C-c, and that is M-#. This adds a '#' to the start of the line, commenting it out, and then behaves as if you pressed Return. The result is that the command is cancelled but still remains on-screen and in your history. You can then copy the command later using the mouse or keyboard.

The new lintian addition "missing-debian-source-format" is starting to remind me of a pestering nanny.

I use source format 3.0 only where I see a technical benefit and that - so far - is restricted to packages that use a .tar.bz2 upstream and one or two with really tricky patching requirements.

You are missing the point - debian/source/format allows you to make it explicit that the source format is 1.0, if you want to stick with that.

Max Attems uploaded a new version of the Linux kernel package (linux-2.6) today. This includes the last major changes to the package before Debian 6.0 'squeeze', which led me to label it the 'Big Bang' release:

• The Xen (dom0 and enhanced domU) flavour is back. It should probably be considered experimental at the moment.
• The OpenVZ flavour is back. This should also be considered experimental.
• The Direct Rendering Manager (DRM) system and drivers were updated to the versions in Linux 2.6.33. This brings some bug fixes and extra features for the i915 driver, and working Kernel Mode Setting (KMS) for the radeon driver. It also includes the kernel-mode part of the nouveau driver for Nvidia GPUs, but this will not yet be loaded automatically as the user-mode parts of nouveau have not been packaged.
• The libata transition for PATA (IDE) controller drivers. After some review we realised that this only affects PC hardware at the moment, so only users of the i386 or amd64 architectures will see the upgrade prompts. Thanks to everyone who tested and reported bugs in the experimental packages.

I had also hoped that it would be really reliable and work with the latest kernels (unlike Xen) but it is giving me problems with 2.6.32 on Opteron.

This is a regression caused by a recent security fix (CVE-2010-0298, "KVM: x86 emulator: fix memory access during x86 emulation"). It appears to affect only recent Linux kernels running as guests on AMD systems, and it will probably be fixed soon.

As previously announced elsewhere, video recordings from the Distribution Developer Rooms at FOSDEM 10 are now available. All but two talks were recorded and are available in Ogg Theora+Vorbis format, in low-bandwidth (~300 kbit/s) and high-bandwidth (~1.5 Mbit/s) versions.

These recordings should also be available later on the FOSDEM YouTube channel.

Video and audio were recorded by Dominique Dumont (H.1302) and the DebConf video team (H.1308) with the assistance of some additional volunteers. Thanks to everyone involved in recording, and thanks also to the speakers and the organisers of FOSDEM.

As some people noticed, a driver update to support a major hardware release in my day job and a long series of trivial patches motivated by Debian kernel work combined to make me joint most prolific change author for Linux 2.6.33. This was a fluke, and 2.6.34 is likely to be fairly quiet for me.

Team work

Meanwhile the kernel team is still working hard on fixing bugs in Linux 2.6.32. As previously announced, this kernel version will be used in both Debian 6.0 'squeeze' and Ubuntu 10.04 LTS. It is also known to be the kernel version for RHEL 6 though this has not been officially announced. If you find a serious bug that was fixed in 2.6.33 or later, please ask the author to send the fix to stable@kernel.org so that it will be included in all these distributions. Note that this happens automatically for commits with the line 'Cc: stable@kernel.org' in their description.

We are also continuing to backport new drivers and new hardware support in existing drivers into Debian 5.0 'lenny' (stable). Missing support for common hardware is considered an important bug and suitable for stable updates. However, if there were major changes to a driver before those that added new hardware support, we may be unable to produce a backport. We are not able to carry out comprehensive hardware testing and do not want to risk a regression existing hardware support.

libata transition - call for testing

Finally, a major change I have been working on is the transition from the old IDE drivers for PATA controllers to new drivers based on libata. The old IDE subsystem is no longer developed and some drivers do not properly support all the hardware that their libata-based counterparts do. However, while the IDE drivers generate device names beginning with 'hd', libata presents PATA devices as SCSI devices and generates device names beginning with 'sd'. In a system that already has other SCSI or SCSI-like disks, names may change somewhat unpredictably. Similar problems exist for PATA CD/DVD and tape devices.

So, while the transition doesn't involve any kernel hacking, it does require a complex upgrade process. Any configuration files that refer to IDE or SCSI device names may need to be changed. It is not enough to switch to only SCSI device names, since we cannot know what they are in advance and the configuration files should continue to work with kernel versions from before and after the transition. In the case of disks we can normally use the partition label or UUID: many tools understand the LABEL= and UUID= syntax, and for others we can refer to the symlinks under /dev/disk created by udev. In the case of CD/DVD devices, we can use the aliases /dev/cdrom etc. created by udev. In the case of tape devices, however, you're on your own.

In experimental, kernel image packages depend on a new package linux-base which implements this transition (from 2.6.33-1~experimental.2; the previous version was broken). The postinst script will prompt you to make changes automatically or manually. It can convert most bootloader configuration files, /etc/fstab, the udev CD aliases configuration and the initramfs-tools resume partition. It can also label partitions that don't have a label or UUID. Please do test it and verify that its changes are correct. All changed configuration files are backed up with a suffix of '.old' (or '^old' in one case).

Most of the Debian kernel team members are attending the Linux Plumbers Conference in Portland over the next 3 days. We'll be discussing kernel packaging and integration issues among ourselves and with upstream and other distributors.~

You may have noticed that Linux 2.6.31 is out but not yet in sid. This kernel version contains a large number of known regressions which should be fixed in subsequent stable updates, so we will wait for those. In the mean time we've uploaded an update to 2.6.30 which should resolve the most serious bugs.

Low-bandwidth versions of the recorded talks are now available. There are currently two exceptions:

• Some of the music for 'Welcome to DebConf 9' is licenced as CC BY-NC-ND, which unfortunately doesn't allow use in a video. I'm discussing this with Marga and hope to upload a new version soon.
• 'Debian por dentro' has a piece missing which should be restored from tape.

While the master versions of all these talks have been checked, not all the generated low-bandwidth versions have. Please let us know on debconf-video@lists.debconf.org if there is something wrong with them other than the following:

• A few of the recordings were unfortunately incomplete; this is indicated by a fade-in or fade-out
• Some of the recordings are in two pieces
• There were a few drop-outs where you will see a black screen and no sound for up to a second
• The text on some slides is unreadable; in this case, refer to the file provided by the speaker or wait for the high-bandwidth version

The high-bandwidth versions will be uploaded over the next week (slow connection is slow).

Several people have been asking after the videos. My answer to them is "we release when it's ready".

Seriously, though, I'm sorry about the delay in publishing videos. There was some miscommunication between me and the sysadmin team at DebConf which meant I couldn't take files away with me then. Peter Palfrader reassembled the video RAID later, copied the necessary files onto a single hard disk, and posted that to me. This disk apparently got lost on the way, but eventually arrived here last week after a 3-week journey.

Over the weekend I dealt with most of the videos that required editing, and today I was able to resume the transcoding process. Tomorrow I intend to start uploading again.

A question from AJ reminded me that I haven't said much about the changes to packaging of firmware in Debian, and in particular the separation of non-free firmware from the Linux kernel.

Linux kernel packages

There is an ongoing process upstream to move firmware blobs from drivers into a firmware/ subdirectory of the source, which is now almost complete. Since most of this firmware is non-free, we remove it from the source tarballs for kernel packages but use it to update the firmware-nonfree "source" package.

We continue to patch some drivers to separate out firmware, and have been submitting our changes upstream. Most of these have been accepted though the DRI drivers matrox, r128 and radeon are notable exceptions.

A few months ago I attempted to make a new inventory of the remaining firmware blobs outside of the firmware/ subdirectory. I identified three that should still be addressed. The Linux-libre project, however, removes many other constant arrays from the kernel (and disables the affected drivers) where I judged the array to be a plausible preferred form of modification.

Firmware packages

Much of the non-free firmware removed from the kernel is now available in the firmware-linux package in the non-free section of the Debian archive. Starting with Linux 2.6.31, we will build the DFSG-free firmware shipped with Linux into a package called firmware-linux-free, which will be recommended by kernel image packages. The contents of firmware-linux will be moved to firmware-linux-nonfree and firmware-linux becomes a meta-package depending on the other two packages.

Many other firmware images never distributed with Linux are also packaged for the benefit of users that require them.

I'll be involved in the video team again.

One of the more interesting talks at last week's PyCon was on the new Python licence.

This week I'm at PyCon in Chicago, helping to train and supervise the A/V team in recording the tutorials and talks. We have a lot of inexperienced volunteers but they're learning quickly. Carl Karsten (in charge of A/V) and Ryan Verner (another outsider with long experience from LCA) are organising things. Ryan, Dave Noble and I provide technical assistance to the rest of the team.

I'm also working on my first Django app, which will be used for reviewing files. This is essentially a clone of the excellent work done by Gunnar Wolf and Damian Viano in Pentabarf. PyCon doesn't have a centralised database so this is a standalone application. Ryan is now applying styles to my horribly bare HTML, and I have a short list of bugs to fix before we ask the video team to start reviewing.

The tutorials started yesterday and talks wil start tomorrow in different rooms, so we have to set up all the equipment again and reconfigure for multiple cameras. Hopefully by Saturday things will be running smoothly and I can spend some time in the city. It would be a shame to come all this way and stay by the airport for the entire time!

I'm going to

Carl Karsten of the PyCon organisers recruited three of us from the DebConf video team to lead and train the video team at PyCon.

Jérémy Bobbio suggested that I should explain how I looked for packages affected by a compiler bug (Debian bug 506713; gcc bug 38287). I don't claim that this is a particularly good way to do it, but here it is:

First, I identified a pattern to search for. Unfortunately I don't really understand the cause or fix for the bug, but I did have the example which led to this bug report: Debian bug 490999. The bad code was a stack-pointer-relative load immediately after a stack allocation (SPARC save instruction) where the offset was not adjusted for the stack allocation:

save  %sp, -112, %sp
ld  [ %sp + 0x40 ], %i5

save  %sp, offset1, sp
...
ld  [ %sp + offset2 ], register

where offset1 + offset2 < 0. Of course, this may be valid if the intervening instructions include a restore, branch or store to the effective stack location that the last instruction loads from. I ended up allowing up to 10 intervening instructions and examining a disassembly to work out which cases were valid.

I looked up the instruction encoding for these two instructions. Thankfully SPARC is RISC so they are simple and regular:

• save %sp, offset1, sp is encoded as 0x9de3a000 | offset1 & 0x1fff
• ld [ %sp + offset2 ], register is encoded as 0xc003a000 | reg << 25 | offset2 & 0x1fff

I took the dumb but effective approach of scanning entire files for this pattern rather than only scanning their code sections. This seemed to work - I got no false hits for non-code - but might not work for other patterns that could match ASCII text.

I wrote the scanning program in Python, which is my default choice of language unless I know it's going to be too slow. I was hoping to be able to read the code files into arrays, but unfortunately the Python array type only supports the native byte-order (SPARC is big-endian and I was intending to use an x86 which is little-endian). I tried reading into a tuple using struct.unpack, which does support explicit byte-ordering, but this used so much memory for larger files that the program swapped to a crawl. So finally I resorted to reading the file into a string, doing a string search for '\x9d\xe3', rejecting matches that weren't appropriately aligned, then unpacking and comparing the code words from the point of the string match. (In Python 3.0 I would have to use the bytes type for this, as str is a Unicode string type.)

So that's how I scanned single files. The next step was to find, unpack and scan all the SPARC shared libraries in the archive. (This particular code generation bug is understood to affect only PIC code, and that is normally only used in shared libraries.) I wrote functions to search Contents-sparc for shared library files - assumed to match the pattern ([^\s]*/lib[^/\s]+\.so(?:\.[^/\s]*)?) - and to parse Packages to find the filenames for the packages containing those files. The latter uses the debian_bundle.deb822 module from python-debian. The last key function downloads and unpacks a package using wget and dpkg-deb. I could have used the httplib module for downloading but I correctly anticipated that I'd need to restart the script several times so I wanted to cache the packages which was easier to do using wget.

So, that's the explanation. If you really want to see it, here's the code.

I finally got sufficiently fed up with Subversion yesterday to begin converting my package repositories to Git. This has many benefits, including the fact that I can now easily publish them. Browse or clone them as you prefer. Or just ignore them.

Some drivers in Linux include "firmware" - code that runs on the device the driver deals with, not the main processors in the system. Most of this is in a binary or equivalent form, which is probably not the form that it was originally written in. This makes it non-free by Debian's standards, an issue which has led to much argument around previous releases and has blown up once again.

The good news is that kernel developers are making a continuing effort to separate this from the driver code. I've collected together the patches that do this and written 2 of my own, which appear to cover all the remaining sourceless firmware. But I was only able to test one of these drivers (radeon) - the rest is up to you.

Affected drivers

How to test

1. Download and install the linux-source package from http://people.debian.org/~benh/firmware-removal/. There is a signed .changes file there that you can use to verify the package.
2. Build and install a kernel from that source (you should probably use make-kpkg).
3. Boot the kernel. Check that the driver fails cleanly: it should not crash, and there should be a useful error message in /var/log/messages
4. Download and install the relevant firmware package, if available. These are available from the same place as the linux-source package.
5. Reboot or reload the driver. Check that the driver is now working.
6. Add your results to the bug report as shown above.

Firmware that is removed

Do you depend on the drivers for which I have removed the firmware? Do you have contacts with the vendors who might be able to help relicence it? Please send mail to the bug report.

Update

I checked my email and found a response from ConnectTech; the WhiteHEAT firmware will probably be clear to package in firmware-nonfree. There is no need for anyone else to contact them.

I discovered an interesting "feature" of chmod(1), which caused a package build to fail. According to the GNU manual page, if no letters are used before a "-" or "+", "the effect is as if a were given, but bits that are set in the umask are not affected." The command will also fail with an error message when it does this!

The Single Unix Specification says this is correct, though there is some ambiguity over whether the exit status should be 0 or not.

Anyway, the result is that it is generally a bad idea to call chmod in this way in a script. Always specify who the permission changes should apply to.

In addition to live streams, recordings of each event should be published on meetings-archive.debian.net. There is an RSS feed of these, which should work nicely in Miro: http://meetings-archive.debian.net/pub/debian-meetings/2008/debconf8/index.rss. That feed is also included on Planet Debian. This is the first time I've tried generating RSS so apologies in advance if I make a mess of the Planet.

Sorry, I missed a few last time.

A few of the recordings from DebConf 7 had to be recovered in whole or in part from tape. For personal and technical reasons, a few of these were not done last year, and I have finally dealt with them now. The last few sessions are:

Hopefully these are useful to someone. Sorry it took so long!

After a fair amount of clean-up, a lot of waiting for review, and some ruthless pruning, I and my colleagues at Solarflare finally got our net driver accepted for Linux 2.6.26. Meanwhile a larger version of the driver made it into Xen's Linux tree, along with dependent drivers to provide a secure fast path to the hardware from domU. Somewhere down the road we're going to have to resolve these two versions, but now I'm just pleased to have a working driver in-tree.

There are some other changes I've worked on over the past few months:

• I maintain diagnostic scripts for our NICs and drivers. Among other things they include Vital Product Data (VPD). There's a standard way to read VPD in PCI (actually there are two, in different versions of the spec) but in Linux this is only accessible to root, and even then there are race conditions. We provide an MTD driver for flash and/or EEPROM attached to the NIC, and I know where the VPD appears in there, but most users won't be loading that driver. So I wrote this: Expose PCI VPD through sysfs
• Along the way I found a bug in sysfs, which I fixed: Disallow truncation of files in sysfs
• One day I accidentally bridged one of our network interfaces, which has Large Receive Offload (LRO) enabled by default, with one of the motherboard's network interfaces, and the results were not pretty. LRO isn't really compatible with packet forwarding, and currently causes the GSO code to crash or warn. I wanted to solve this by disabling forwarding of LRO skbs, but it turns out to be trickier than I thought.
• We wanted to use topology information to allocate one IRQ per CPU package, not per core (or per thread, with HT enabled) for Receive-Side Scaling. Unfortunately this information is only available in some kernel configurations and not for all architectures. I had a go at providing reasonable defaults, but it didn't work too well!

However, this is still only part of my work. My main task at the moment is to write more unit tests for our next NIC.