Friday was my last day at Solarflare Communications, which I joined in 2006 (originally as Level 5 Networks).
When I started, Level 5 Networks (L5N) had a 2-port gigabit Ethernet controller called EF1 and a user-level network stack called EtherFabric which ran on Linux. It had nearly completed the Falcon project to develop a 10-gigabit controller (also supporing a 2-port gigabit mode) and was porting EtherFabric to Windows. Later that year, however, L5N merged with Solarflare Communications (SF), which was developing a 10GBASE-T PHY (for 10-gigabit Ethernet over twisted-pair cables). The investors and management saw the opportunity to create an integrated 10GBASE-T LAN-on-motherboard chip that would be attractive to OEMs and so had the potential for mass sales. EtherFabric was put on the back-burner, but regular net drivers for Linux, Windows and other operating systems became more important.
In a change from my previous jobs, my initial role at L5N was to maintain and extend the in-house test automation software, Runbench, written in Python. I worked with Dickon Reed who had written most of Runbench up to then. One of my first major tasks was to extend it to cover DUTs running Windows. Aside from changing Runbench itself, I wrote a simple SSH server to be installed on DUTs, using the Twisted.Python framework. (Cygwin's port of OpenSSH wasn't suitable for running a lot of native Windows commands.) I rewrote the test-installation script for the Windows driver itself, converting it to Python (with a C extension to get around the WOW64 nonsense). As Windows does not allow loading a driver just once (like insmod in Linux), except by using the kernel debugger, I added a time check in the driver so the test-installation script can make it fail early when reloaded after a reboot, so that a crasher bug in the probe path would usually be recoverable.
In 2007 I persuaded management in Cambridge to loan several older servers to DebConf. I took them there and back by car, and they were used to convert video for live streams and recordings. (This was repeated in 2009.)
I was also asked to work on other small development tasks, including some cleanup on the Linux net driver (sfc). In mid-2007, when SF decided it was time to get sfc upstream (in-tree), I joined Steve Hodgson and Robert Stonehouse in working on that.
Out-of-tree (separately distributed) drivers for Linux normally need a large number of preprocessor conditions for compatibility with Linux's kernel module API across a range of supported versions. They may also implement some features with a driver-specific interface that should be replaced with a standard interface (possibly including the work to define that standard). They may not comply with the kernel coding style or unwritten conventions for kernel code. All of these problems were present and had to be dealt with. As part of this work, I extended unifdef so that it could be used to remove all of the backward-compatibility and not-for-upstream code without introducing extra blank lines. I could then export the out-of-tree driver from CVS(!) into a kernel source tree automatically and turn it into a patch or patch series that would stand a chance of being acceptable upstream.
Our first few attempts got little response, but eventually we newbies got the message that the driver was simply too big for a single submission and that putting patches on a web site because they're too big for the list was not a solution. In the next week, I removed about half of the code (on a git branch), resulting in a relatively lean driver that could be sent to the netdev list. And finally the 9th (I think) submission was accepted.
Following this, I continued to work on sfc but was also brought into the Siena project. Siena (SFL9021) was the LAN-on-motherboard chip that had been planned following the merger. It combined a dual-port controller based on Falcon with a 10GBASE-T PHY and a management controller (MC) to support Lights-Out Management (LOM). (Since 10GBASE-T has not taken off, it is now mostly sold as the SFC9020 variant in which the PHY is disabled.) I worked on some of the test framework and test cases for validating the controller design in software simulation and FPGA. In the course of this I learned to read and (somewhat) understand the chip design written in Verilog, but only made a single trivial fix to it.
In August 2009, the first Siena ASICs came back from the fab. There were a few ASIC-specific bugs but all of them were quickly worked around in firmware, so it would soon be ready for production. But there was much work still to be done on the driver and firmware. The firmware had until then been running in a smaller block of RAM with no peripherals to manage, while sfc had earlier been modified just enough to configure a sim or FPGA for running a userland test application. sfc now had many special cases scattered around it, and would tell the MC firmware to peek and poke registers that were no longer directly accessible from the host.
The software and firmware developers had agreed that, in order to support LOM, the MC would be responsible for managing the ports and all peripherals and drivers would send higher-level requests to the MC. As a bonus, this removed the need for the driver to know the details of each new board, so that it would not be necessary to backport sfc into distribution kernels very often. Over the next few months, the firmware team did an excellent job of defining and implementing those operations, including writing the driver-side functions to invoke them. Meanwhile, Steve and I concentrated on refactoring sfc so most of the Falcon/Siena differences could be abstracted through a few structures with function pointers.
The refactoring and new code for Siena were completed and submitted upstream in several large patch series in October and November. In fact, there were so many patches that Solarflare appeared in LWN's table of who wrote Linux 2.6.33 (as did I, thanks also to another patch series adding firmware metadata to many drivers). Sadly we had missed Linux 2.6.32 which was a longterm stable branch and used in many distributions, but I was able to get this version backported into all the major distributions over the next year.
(To be continued.)
My first Android phone was a ZTE Blade (sold as Orange San Francisco here in the UK). It originally shipped with Android 2.1 but was upgradable to 2.3 thanks to CyanogenMod (or other unofficial mods). However there's little sign of an upgrade to 4.x; the apps I want to use are pushing the limits of its CPU, RAM and internal storage; and I've never got very good at typing on a soft keyboard. So it seemed like time to get a newer phone with a hard keyboard (and still with a μSD slot).
After a little research with the CyanogenMod compatibility list and some time looking at reviews, I settled on the Samsung Galaxy S Relay 4G, which was exclusive to T-Mobile USA but is being resold through eBay. It shipped with Android 4.0 but I immediately installed CyanogenMod 10.2 (Android 4.3). This was a bit of an adventure as I bought the Blade with CM already installed and wasn't familiar with the multiple steps that were necessary.
The last step, and the real subject of this entry, was to move my data across. Much of this was on a μSD card which I could simply plug into the Relay. The internal files had to be backed up onto this card and then restored, using the recovery environment (ClockworkMod) on each phone. After rebooting the Relay into the full Android system, my apps and settings were mostly present but I was immediately confronted with a series of error dialogs reporting that 'Unfortunately, Dialler has stopped' - and the same for 'Clock' and 'the process android.process.acore' (whatever that is).
There are thankfully more detailed error logs in the filesystem, under /data/system/dropbox (this is for logging at the Java level; if a process is terminated by a fatal signal it's logged to /data/tombstones). There are several ways to get at them, but I used recovery mode and adb to get a root shell where I could easily read and write files as necessary.
The Contacts (aka People) database upgrade fails with an SQLiteException, apparently because it doesn't account for upgrading from the schema used in 2.3:
Process: android.process.acore Flags: 0x883e45 Package: com.android.providers.applications v18 (4.3.1-f963020e38) Package: com.android.providers.contacts v18 (4.3.1-f963020e38) Package: com.android.providers.userdictionary v18 (4.3.1-f963020e38) Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys android.database.sqlite.SQLiteException: no such column: phonebook_label (code 1): , while compiling: UPDATE raw_contacts SET display_name_source=?,display_name=?,display_name_alt=?,phonetic_name=?,phonetic_name_style=?,sort_key=?,phonebook_label=?,phonebook_bucket=?,sort_key_alt=?,phonebook_label_alt=?,phonebook_bucket_alt=? WHERE _id=? ... at com.android.providers.contacts.ContactsDatabaseHelper.updateRawContactDisplayName(ContactsDatabaseHelper.java:5344) at com.android.providers.contacts.ContactsDatabaseHelper.upgradeToVersion504(ContactsDatabaseHelper.java:3580) at com.android.providers.contacts.ContactsDatabaseHelper.onUpgrade(ContactsDatabaseHelper.java:2261) ...
Clock fails somewhat similarly though it apparently didn't try to upgrade:
Process: com.android.deskclock Flags: 0xc8be45 Package: com.android.deskclock v203 (2.0.3) Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys android.database.sqlite.SQLiteException: no such column: incvol (code 1): , while compiling: SELECT _id, hour, minutes, daysofweek, alarmtime, enabled, vibrate, message, alert, incvol, profile FROM alarms WHERE (enabled=1) ... at com.android.deskclock.AlarmProvider.query(AlarmProvider.java:73) at android.content.ContentProvider.query(ContentProvider.java:744) at android.content.ContentProvider$Transport.query(ContentProvider.java:199) at android.content.ContentResolver.query(ContentResolver.java:414) at android.content.ContentResolver.query(ContentResolver.java:357) at com.android.deskclock.Alarms.getFilteredAlarmsCursor(Alarms.java:165) at com.android.deskclock.Alarms.calculateNextAlert(Alarms.java:344) at com.android.deskclock.Alarms.setNextAlert(Alarms.java:417) at com.android.deskclock.Alarms.saveSnoozeAlert(Alarms.java:510) at com.android.deskclock.AlarmInitReceiver$1.run(AlarmInitReceiver.java:49) ...
The media player fails with a NullPointerException:
Process: com.andrew.apollo:main Flags: 0x98be65 Package: com.andrew.apollo v2 (1.1) Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys java.lang.NullPointerException at com.andrew.apollo.MusicPlaybackService.stop(MusicPlaybackService.java:878) at com.andrew.apollo.MusicPlaybackService.openCurrentAndMaybeNext(MusicPlaybackService.java:1048) at com.andrew.apollo.MusicPlaybackService.openCurrentAndNext(MusicPlaybackService.java:1031) at com.andrew.apollo.MusicPlaybackService.access$1500(MusicPlaybackService.java:74) at com.andrew.apollo.MusicPlaybackService$MusicPlayerHandler.handleMessage(MusicPlaybackService.java:2337) at android.os.Handler.dispatchMessage(Handler.java:99) at android.os.Looper.loop(Looper.java:137) at android.os.HandlerThread.run(HandlerThread.java:61)
Maybe I could have worked out how to upgrade the relevant databases myself, but I went for simpler solutions. The clock settings are easy to re-enter and most of the media player state is regenerated by scanning files on the μSD card. So I just deleted those on the Relay:
rm -rf /data/data/com.android.deskclock rm -rf /data/data/com.android.providers.media
The contacts were what I really cared about, and there are actually specific menu items in Contacts to export and import those (using VCF format), side-stepping the database upgrade. So I did:
This may not include all data, and in particular it doesn't seem to include attached photos. But that was good enough for me.
The upcoming stable update for Debian (7.2) should have a new Linux kernel version, fixing various bugs. I have uploaded version 3.2.51-1 which is now available in wheezy-proposed-updates.
I would appreciate any regression testing people can do before this is released. Also, this fixes one of the causes of lock-ups currently seen with newer Intel GPUs, though we don't know exactly which bug reports were due to this. If you've reported this kind of problem, please test whether this version fixes it.
Updated 2013-09-23: KiBi reminded me to link to the page about proposed-updates. And the update is now available for all architectures.
EFI includes support for storing persistent 'variables': it implements a key-value store where keys are (UUID, string) and values are arbitrary blobs. (There are also some flags that restrict when the variables can be accessed.) The physical storage medium is normally flash. The Linux efivars module provides access to variable storage, both through a specific interface in sysfs and through the somewhat abstracted pstore interface.
An EFI implementation reads the boot order and (if applicable) Secure Boot configuration from specific EFI variables. An operating system can add arbitrary variables, using its own UUID(s). On Linux, the efibootmgr utility is used to read and write the boot order and the pstore interface is used within the kernel to dump kernel log messages or other information when particular events occur.
Since efivars is needed to set the boot order on EFI systems, it's a critical part of installation. But it wasn't being loaded on Debian systems until efibootmgr was run. This doesn't work so well when using a rescue system, as the installation kernel and installed kernel may be different (Debian bug 703363). In future (Debian package version 3.2.41-1) this will be loaded automatically on EFI systems.
But now comes a new problem: bugs in the implementation of variable storage can prevent a system from booting, and even worse, they may prevent manual recovery without special equipment (the system is 'bricked'). Matthew Garrett recently talked about this and mentioned that some Samsung laptops are bricked if the variable storage becomes more than 50% full. efivars now has some extra checks to avoid triggering this (both in mainline and in stable branches). This isn't the end of the story, unfortunately.
Remember that nothing can be overwritten in flash without erasing a relatively large block, and that writes may fail after a few thousand erase cycles (number depending on the type of flash used). Therefore, deleting a variable usually does not free storage immediately, and overwriting a variable usually reduces the amount of free storage. In the case of the Samsung laptops, unused storage is apparently reclaimed at the next reboot.
However, while testing the backported efivars changes I found that my EFI-booting system (Asus P8Z68-V LX motherboard) does not reclaim unused storage until it's very nearly full. Since the firmware itself increments a counter variable on every boot, using 36 more bytes of storage, I was able to observe the available space dropping to under 100 bytes, and then after another boot jumping to about 42K (out of 64K). This means that well over 50% of storage was unused, yet efivars would refuse to write any variables because the firmware didn't reclaim it. This would prevent (re)installation of Linux, as the boot configuration could not be written.
On the assumption that both behaviours are common, I've settled on a compromise that I hope minimises overall risk:
But it's essential that we find a way to identify and work safely with both behaviours. Not only do I want to eliminate the possibility of bricking or installaton failure, but I want to make EFI-backed pstore safe to enable. Crash logs are often hard to obtain and pstore can help to solve that problem.
Updated 2013-09-15: The free space check has been revised (in Linux 3.2.48, 126.96.36.199, 188.8.131.52, 3.9.7 and 3.10) based on advice from Samsung. The kernel leaves at least 5K (rather than 50%) of the EFI variable store free. If it is about to breach this limit, it attempts to write a dummy variable that is larger than the free space, which will either fail or trigger garbage collection. The normal variable can then be written if garbage collection freed up enough space. This check can be disabled using the kernel parameter efi_no_storage_paranoia, but there should be no need to do so.
I'm leaving OHM 2013 and heading toward Vaumarcus today. Hoping to get lots of kernel work (and maybe some video team preparation) done during DebCamp. I proposed a talk and 3 discussion events for DebConf itself, of which all have been accepted.
The first point release for Debian 'wheezy', 7.1, is due next weekend. The Linux kernel should be updated with:
The new package, version 3.2.46-1, is available for most architectures in stable-proposed-updates. Please test it now if you can, so we can catch any serious regressions.
There's some exploit code going around that will let you get root on a range of Linux kernel versions by using a bug in the perf_event_open() syscall. The fix for this is in 3.2.45 and various other stable updates. As a workaround, until it's in Debian stable, you can set sysctl kernel.perf_event_paranoid=2. This blocks the exploit I've seen, but it is still possible to get around this restriction.
Red Hat has a SystemTap script that should provide more complete mitigation. The debuginfo packages for Debian kernels are named e.g. linux-image-3.2.0-4-amd64-dbg. They are only provided for some architectures and flavours.
Updated: Added the CVE ID. Added SystemTap information.
The 'longhaul' module may cause instability or even hardware damage on some systems. Unfortunately, it is now being auto-loaded on all systems with a compatible CPU (VIA C3 or Cyrix III). See bug #707047.
Before upgrading one of these systems to Debian 7.0 'wheezy', if you do not currently use the 'longhaul' module, you should blacklist it by creating e.g. /etc/modprobe.d/blacklist-longhaul.conf containing the line:
I would advise against attempting a fresh installation on these systems at present.
Jan Wagner writes about using ethtool to poll the link state on network devices. While I'm pleased to see another happy ethtool user, it's a bit heavyweight for reading one bit of information and it's not ideal for scripting.
Much information about each network device is available under the /sys/class/net/name directory, including the carrier and speed attributes. (When the interface is down, some of these attributes cannot be read.) It's also possible to listen for link changes (rather than polling periodically) using the rtnetlink API, but this is probably not suitable for munin.
The next point update for Debian 6.0 should include many fixes and improvements to the Linux kernel. Version 2.6.32-47, currently available in the 'squeeze-proposed-updates' suite, includes updates to the following SAS drivers, adding support for new hardware:
Note that the binary package names don't change; they are still linux-image-2.6.32-5-flavour.
If you are using any of the above SAS controllers, whether or not it is already supported by Debian 6.0.6, please consider testing this kernel version. If it works, please follow up to the bug reports listed above. If you find any regressions, please report bugs against the new package version.
Some recent Intel and AMD graphics chips were not supported well or at all by the DRM (Direct Rendering Manager) drivers in Linux 3.2. Although many bug fixes have been included in 3.2.y stable updates, we are considering updating them to the versions found in Linux 3.4. (We did something similar for Debian 6.0 'squeeze'.)
Julien Cristau has been working on this and has prepared binary packages. To install, run as root:
Please test these and report your results to bug #687442. I would suggest testing suspend/resume (if applicable), use of internal and external displays on laptops, and 3D accelerated graphics (games and compositing window managers such as GNOME Shell). Remember that AMD/ATI graphics chips require the firmware from the firmware-linux-nonfree package for 3D acceleration and many other features.gpg --no-default-keyring --keyring /usr/share/keyrings/debian-keyring.gpg --export 310180050905E40C | apt-key add - echo deb http://people.debian.org/~jcristau/wheezy-drm34/ ./ > /etc/apt/sources.list.d/jcristau-wheezy-drm34.list apt-get update apt-get install linux-image-3.2.0-4.drm-amd64 # or -486, or -686-pae
I wrote a standard rfkill driver for the Fujitsu-Siemens Amilo A1655 and M7440, based on the out-of-tree fsaa1655g and fsam7440 drivers. Unlike those, it should work with the rfkill command, Network Manager, etc.
Newer touchpads made by ALPS use different protocols for reporting scroll and pinch gestures. Jonathan Nieder backported the changes to support these.
Jonathan Nieder updated the wacom driver to the version in Linux 3.5, adding support for the Intuos5, Bamboo Connect, Bamboo 16FG and various other models.
Sarveshwar Bandi at Emulex contributed a backport of the be2net driver from Linux 3.5, adding support for their 'Skyhawk' chip.
Marvell's Kirkwood SoCs have been supported upstream for some time and in Debian since release 6.0 'squeeze'. However new models and boards generally require specific support. Arnaud Patard backported support for the 6282 rev A1 chip found in QNAP TS-x19P II models, and for the Marvell Dreamplug and Iomega Iconnect.
Missing hardware support is an important bug that can be fixed by kernel updates during a freeze and throughout the lifetime of a stable release. If you know that new hardware isn't supported by the Debian kernel, please open a bug report. I can't promise that it will be fixed, but we need to know what's missing.
Hardware vendors that maintain their own drivers upstream (not out-of-tree) are especially welcome to contribute tested backports that add support for the latest hardware. Send mail to firstname.lastname@example.org if you have any questions about this.
The 'PREEMPT_RT' aka 'linux-rt' patchset provides true real-time scheduling under Linux. This can allow applications to respond to I/O with very low latency and jitter, although the results depend on application design and on the hardware and firmware used.
Real-time scheduling is commonly used in industrial control, live audio/video applications, and high-frequency trading systems; it can also be useful for tightly-coupled cluster computing. However, most applications will run more slowly under these kernels, and may not be any more responsive.
The Linux 3.2-rt stable series maintained by Steven Rostedt is used to build alternate kernels for amd64 and i386. You can install these using the metapackage linux-image-rt-amd64 or linux-image-rt-686-pae.
Like most VM hypervisors, Microsoft Hyper-V supports paravirtualised drivers which can provide much better performance than emulation of old hardware devices. Linux 3.2 had quite early versions of these drivers that were relegated to the 'staging' area, but we've updated them to the much improved versions found in Linux 3.4. I hoped we would also be able to package the userland daemon that supports management of Hyper-V guests, but this also needed many fixes and wasn't ready in time.
The Debian package has some additional features not present in mainline Linux 3.2. Some of these were described in part 1, and this continues the list.
CPUs are exposed in the device model as normal devices, not 'sysdevs'. For x86 CPUs, model and feature information is included in the uevent for the CPU device. This results in automatic loading of appropriate device drivers with the help of udev, rather than specialised init scripts:
Most control groups (cgroups) features are included, but some are disabled by default at run-time:
On x86 architectures the kernel is built as an EFI executable which can be directly loaded from an EFI shell, e.g.:
Shell> vmlinuz-3.2.0-4-amd64 root=/dev/sda2 initrd=initrd.img-3.2.0-4-amd64
Debian will usually boot on an EFI system via grub-efi, but this can be useful for some unusual configurations and some of the simple EFI boot loaders that are currently being developed.
The kernel side of AppArmor has some extensions for backward compatibility with AppArmor userland versions 2.4-2.7, matching wheezy userland.
Debian appears to be the only major distribution to provide a choice of LSMs: AppArmor, SELinux or TOMOYO. Use the security= boot parameter to enable any of them.
Like all hardware, CPUs have bugs, which can sometimes be fixed in microcode. x86 microcode updates may be applied by either the BIOS or operating system, but it's easier to upgrade packages than to upgrade the BIOS. Xen 4.1 can apply microcode updates but relies on the dom0 kernel to load them, and the mainline Linux kernel does not do this. We include an extension of the microcode loader that does so.
As you may know, Debian 7.0 (codename 'wheezy') includes Linux 3.2. I'm maintaining the 3.2.y stable series at kernel.org which will collect bug fixes and some new hardware support as nominated by kernel developers and distributors, following the usual stable kernel rules.
The Debian package has some additional features not present in mainline Linux 3.2.
Debian Live needs a union filesystem to allow writing files on top of the read-only live system image. To support this, we include Aufs developed by Junjiro R. Okajima. I hope that Linux will gain union mount or union filesystem support in time for the next release, but currently it doesn't take much work on our part to integrate.
I backported two big pieces of the solution to bufferbloat:
Alternate title: 'for those who care about channels 12-13'. For background, see the earlier entry on wireless regulation.
If your wifi interface no longer supports channel 12 or 13, this may be because the kernel does not know the regulations for the domain code detected by the driver. Prior to Linux 2.6.34, regulations for 'US', 'JP' (Japan) and 'EU' (ETSI members) were built-in, but now there is only '00' (world, conservative regulations) which allows only channels 1-11 in the 2.4 GHz band. For any driver that detects specific European country codes, the 'EU' regulatory information was useless even in 2.6.32 (squeeze) and the unknown code would be mapped to the '00' (world) domain.
Since wireless regulations are subject to change from time to time, they are now provided by the wireless-regdb package and supposed to be loaded into the kernel by crda. These packages should have been included in squeeze, but for various reasons they missed it. But they are now available in squeeze-backports.
I switched from Google Reader to Liferea a while back. I was mostly happy with the functionality of Reader but I don't really want my reading history to be recorded or my feed list to be vulnerable to the whim of Google's inscrutable abuse department.
I'm still using Liferea but unfortunately it has a lot of problems. Yes I do have bug numbers:
So, I'm looking for an alternative again. My requirements are:
~$ dpkg --print-architecture i386 ~$ dpkg -s linux-image-$(uname -r) Package `linux-image-3.2.0-1-amd64' is not installed and no info is available. Use dpkg --info (= dpkg-deb --info) to examine archive files, and dpkg --contents (= dpkg-deb --contents) to list their contents. ~$ dpkg -s linux-image-$(uname -r):amd64 Package: linux-image-3.2.0-1-amd64 Status: install ok installed Priority: optional Section: kernel Installed-Size: 104822 Maintainer: Debian Kernel Team <email@example.com> Architecture: amd64 Source: linux-2.6 Version: 3.2.6-1...
Phil Kern made a general call for testing of Debian 6.0.4. I would like to more specifically point out that I backported the isci driver for Intel C600 SAS/SATA controllers. Unfortunately I have not yet had any testing results for this. If you have any machines with this hardware not yet in production, please do consider testing the new Linux kernel package, version 2.6.32-41.
Updated: I now have a positive test result from the user who requested this driver in the installer.
I thought it was about time to get a faster machine for... well, mostly for compiling kernels faster, but I'm sure I will find other applications for it. I was originally intending to upgrade my laptop (Thinkpad T61, 1.8 GHz Core 2 Duo) but it seems that in the 3 years since I bought it second-hand laptop prices have not dropped in line with Moore's law. So instead I decided to upgrade (well, replace most of) a desktop PC.
The last time I got a new desktop at home was in 2002. It has an Athlon XP, PATA drives and a huge CRT monitor. It has had some upgrades since then (DVD writer, more RAM, slightly faster CPU, replacement fan and PSU). Obviously it was time to replace motherboard, CPU and RAM. Not so obviously, the PSU needed changing again as modern CPUs apparently need more 12V electrons (?!). The DVD writer was salvageable via a USB adapter and its local HD is currently a laptop drive (the only spare SATA HD I had). The monitor is of course reusable, though the firmware setup program doesn't entirely agree with it.
Yes, I do sometimes run Windows. It's for games, and also seems to be needed for streaming video with stupid DRM.
On the assumption that Windows is less likely to play nice with an existing installation, I tried installing it first. The firmware appear to allow me to boot the DVD in either BIOS or UEFI mode, so I selected the UEFI option. This worked so far as loading the initramfs (or whatever the Windows equivalent is) but then Windows was lacking a driver to access the DVD. This stumped me for a day or so until I realised I had plugged the drive into a USB 3 (XHCI) port. Apparently Windows 7 SP1 still doesn't include an XHCI driver in the installer (but neither does Debian 6.0.3 - we should fix that).
The installation was mostly uneventful after this. The Windows installer includes a reasonably capable partition editor.
Installation of files was entirely uneventful. The installer correctly detected the GPT (GUID partition table) created by Windows. However, it was not able to install a boot loader. Currently it only includes support for the LILO and GRUB-PC boot loaders that run under BIOS.
After much experimentation and cursing, I found instructions for installing Debian Squeeze on XServe 3.1 by Jan Peter Alexander Rajagukguk, which put me on the right track. It seems that the grub-install script for GRUB-EFI in squeeze (and maybe other parts of the package) doesn't actually work. However. JP's direction to build GRUB from source is no longer necessary. Here's what I did (minus the mis-steps):
# # We start in dash, so switch to bash. # exec bash # mkdir -p /boot/efi # # Mount the EFI boot partition by UUID. # # It's probably called sda1 now, but is subject to change. # ls -l /dev/disk/by-uuid ... # echo >>/etc/fstab 'UUID=efi-boot-uuid /boot/efi vfat defaults 0 0' # mount /boot/efi # # Enable unstable sources, but use stable by default # echo >>/etc/apt/apt.conf 'APT::Default-Release "squeeze";' # echo >>/etc/apt/sources.list 'deb http://cdn.debian.net/debian/ sid main' # apt-get update ... # # Install unstable grub-efi-amd64 # apt-get install -t unstable grub-efi-amd64 ... # # Install GRUB to the EFI partition. The '--removable' options # # inhibits setting EFI variables, which won't work because we booted # # through BIOS emulation. Instead, it installs as the default boot # # loader. # grub-install --bootloader-id=GRUB --removable # # On the AMI/ASUS firmware, you cannot select a new bootloader # # directly but can invoke a 'shell'. So I made GRUB the shell: # cp /boot/efi/boot/grub/grub.efi /boot/efi/shellx64.efi # exit
# apt-get install efibootmgr ... # modprobe efivars # efibootmgr -c -l '\efi\grub\grubx64.efi' -L GRUB
Simon McVittie reported in Debian bug #654876 that his laptop running Linux 3.0 or 3.1 would sometimes crash (panic) while idle. He initially suspected a driver bug, but the screen did not show any information about where the original fault occurred. However, using netconsole, he was able to capture a full log of the crash. This showed that a packet received through the wireless interface was being processed by IGMP, which then divided by zero.
IGMP is part of the IPv4 protocol suite, supporting multicast routing. Every multicast address corresponds to a dynamic set of hosts, called a multicast group. Multicast routers can send query messages asking which hosts belong to which groups, and hosts using multicast report back at intervals. Routers can then limit forwarding of multicast packets to the interfaces where the group has members. More sophisticated switches can also snoop IGMP and use it to limit their multicast forwarding. There are unfortunately three different versions with semi-compatible message formats. In version 1, the maximum reporting interval (Max Response Time) is fixed as 10 seconds, but from version 2 it is specified in query messages.
The Linux IGMP implementation supports all three versions. It distinguishes query messages as specified in RFC 3376 section 7.1: v3 messages are longer than v1 or v2; v2 messages have a non-zero Max Response Time whereas v1 messages always have zero. It is possible to force use of a specific protocol version, but normally if there are multiple multicast routers using different protocol versions it will respond according to the earliest protocol version in use so that all routers can understand its responses.
Linux 2.6.36 included two fixes to the version selection logic. Unfortunately, the second of these introduced the bug in question. While v2 query messages cannot possibly have zero Max Response Time (as that would make them v1), v3 query messages can. What this means is unspecified, but the Linux IGMP code previously treated it as the minimum valid value of 1/10 second. But in the case where a v3 query is received and a v2 query has also recently been received, this is no longer done. This results in a reporting interval of 0 seconds and a division by zero when deciding the initial random delay.
The fix for this is pretty obvious. It is included in Linux 3.0.17, 3.1.9, 3.2.1, and the Debian package version 3.1.8-2.
This is easily exploitable for denial of service within a local network. Linux does not check the destination address of IGMP queries, so it may also be possible to attack a target through its unicast address from several hops away. The attacker only needs to send a single IGMPv2 query followed by a single IGMPv3 query with zero Max Response Time. All systems running Linux 2.6.36 or later (up to the above fixed versions) with any active IPv4 multicast listeners (other than for the 'all hosts' address) are vulnerable.
Screensaver control on the Linux desktop is a total mess, but
the xdg-screensaver command provided an interim
cross-desktop solution by detecting and using the appropriate
underlying interfaces. For GNOME, it depends on
the gnome-screensaver-command --poke option, but that has
been broken and then
removed in favour of
the cross-desktop D-Bus API
GNOME-specific API which can't practically be used by
What's more, switching to the new API seems to have introduced various bugs in GNOME's own video player, Totem, such that it often failed to inhibit the screensaver in GNOME 2.32 and 3.0. Apparently everything is wonderful again in 3.2; here's hoping.
(In case anyone asks where my bug report on the totem package is, I already looked at the bug list and it's clear that there's no point.)
The Debian kernel team regularly backports driver updates to the Linux kernel in stable releases to add support for new hardware, and I've prepared several updates intended for point release 6.0.2. Since the kernel team does not have a large collection of hardware on which to test driver changes, we would appreciate test reports from users. It is important to test not just that new devices are supported properly, but that there are no regressions in support for older devices.
I have updated these drivers to the versions found in Linux 2.6.38, modulo driver API changes:
I have also cherry-picked some small changes:
There are more drivers that I think should be added or updated (see #624794) but they will probably have to wait for release 6.0.3.
The current packages are version 2.6.32-35~test1, but there may be further test versions before an official stable update.
For network drivers, I suggest the following regression tests:
For storage drivers, unfortunately I don't have a good idea of what tests would be suitable. In any case, please don't test on disks storing valuable data!
Please send test reports to the bug reports linked above, stating the driver name, the PCI ID for the device you tested (from lspci -n) and any other device identification that the kernel log (for example, r8169 logs the 'XID' of the device).
The major upcoming configuration change in Linux 2.6.39 is to get rid of the '686' flavour. This may be surprising, because it's the most widely used flavour of the 4 we have a present:
|Name||Minimum CPU features||Maximum total CPU threads||Physical address space|
|686||686-class; CMOV instruction||32||4 GiB|
|686-bigmem||686-class; PAE||32||64 GiB|
However, the physical address limitation means that an increasing proportion of new PCs and the majority of PC servers need the '686-bigmem' flavour. Even those that have less than 4 GiB RAM do support PAE and can run the '686-bigmem' flavour. There is a small cost (up to about 0.1% of RAM) in the use of larger hardware page tables. There is also an important benefit on recent processors: the larger page table entries include an NX bit (also known as XD) which provides protection against some buffer overflow attacks, both in the kernel and in user-space..
There are only a few 686-class processors that support CMOV but not PAE: most Intel Pentium M models, the VIA C3 'Nehemiah' and the AMD Geode LX. These also all lack SMP support, which means the '486' flavour is suitable for them.
Some benchmarking on two of those - a Pentium M model 745 and a C3 'Nehemiah' - indicated that they run the '486' flavour slightly faster than the '686' flavour. It appears that the performance gain from using plain uniprocessor code (rather than SMP-alternatives, which are patched with NOPs on uniprocessor systems) outweighs the performance loss from avoiding use of some newer instructions.
We intend to get rid of 'amd64' too, but we need to ensure that upgrades from linux-image-2.6-amd64:i386 to linux-image-2.6-amd64:amd64 work properly.
Yes, it is. Therefore '686-bigmem' will be renamed to '686-pae'.
As you've probably seen, the testing suite finally got a new upstream kernel version (Linux 2.6.38) and is currently synchronised with unstable. I'm just about to upload a new version of linux-2.6 based on upstream stable update 184.108.40.206.
Linux 2.6.39 is well on its way to release, so 2.6.39-rc4 should soon be in experimental. The Debian source package will add support for the new armhf (ARM with hardware floating-point) architecture.
The linux-base package, introduced to handle transitions and other support functions for 'image' packages, is now built from a separate source package. It won't be changed very often, so you will no longer need to upgrade linux-base every time you upgrade an 'image' package. (This doesn't apply within 'squeeze', but I may have a way to work around that.)
The firmware-linux-free package, containing a few firmware and configuration blobs from Linux that are DFSG-compliant, is also now built from a separate source package. These blobs have not changed for a very long time, so again you will no longer be recommended to upgrade this package every time you upgrade an 'image' package. Secondly, this allows us to remove the entire 'firmware' directory from the upstream source rather than carefully removing and patching to leave just the DFSG-compliant bits.
The '2.4 GHz' and '5 GHz' ISM bands used for wifi are not defined identically around the world. The exact frequency ranges and maximum permitted power levels vary between countries. Radio equipment manufacturers have a legal responsibility to ensure their equipment operates within these regulations. Since it is expensive to implement such regulations in hardware (including many different variants for different regions), these restrictions are usually implemented in the driver using a country code retrieved from the hardware's EEPROM or flash. So this is now our responsibility.
We need to be able to look up regulations by country codes, and we also need to support the case where a device sold in one country is used in another country with tighter regulations. For some time, the wifi configuration library (cfg80211) included regulatory information for a small number of countries and regions. However, the default since Linux 2.6.34 has been to build-in only the 'world' domain (intersection of all the restrictions) and to request information from user-space. In Debian, we didn't have anything to provide this information, so for example channels 12-13 became unavailable for users in Europe.
The regulatory information should be provided by the Central
Regulatory Domain Agent (CRDA) from a current database. These are
now packaged as
respectively. Drivers that do not use cfg80211 (mostly those from
the 'staging' area) will not be affected, and may or may not
implement correct regulations.
When you travel to another country or use an imported wifi device
with an incorrect country code (I hear 'CN' is common), you can
iw reg set to ensure that your wifi devices operate
within the regulations of the country you are in. Set
REGDOMAIN variable in
/etc/default/crda to make this persistent. These will
be intersected with the regulations specified by the
I'm still missing how to reliably detect if a device is a "wired" or a "wireless" one. I suspect that checking the existence of /phy80211 would be enough, but I can't really tell, and seems like I'm not able to find an exhaustive sysfs reference manual.
Sadly there is no exhaustive manual, but many attributes found in sysfs are documented under Documentation/ABI/. In this case you need the DEVTYPE name from the net device's uevent, which may be e.g. 'wlan' or 'wwan' (and is absent for wired Ethernet devices). I don't know why this isn't also exposed as an attribute in its own right.
If you're trying to use a USB serial adapter as your Linux kernel console...
The kernel needs to be compiled with CONFIG_USB_SERIAL_SERIAL=y
Debian kernels do not appear to be. Somewhat strangely, the config option doesn't even wind up in the config file in the typical commented-out manner, so you don't even know the option exists until you discover it from random Web searching, and then looking at drivers/usb/serial/Kconfig in the kernel source.
Console drivers need to be built-in, so CONFIG_USB_SERIAL_CONSOLE depends on CONFIG_USB_SERIAL=y. We build the USB serial code as a module so CONFIG_USB_SERIAL_CONSOLE is not an available option. This is unlikely to change, as a built-in feature takes memory from every Debian GNU/Linux installation.
A few weeks ago Zack wrote about preparations for a new podcast to be called 'This Week in Debian'. I volunteered to be interviewed, and have now talked with Jonathan Nadeau for about 30 minutes about the work of the kernel team and the release process. Hopefully he'll be editing down my rambling so the first episode won't be too boring! If you do work on Debian that you'd like to talk about to an interested audience, follow the link above and add yourself to the list of potential interviewees.
[...] whenever I was typing a command line, changed my mind and pressed C-c, I got a ^C printed on the screen, usually overwriting one or two characters of what I had typed. And this prevented me from automatically copying and pasting the command that I had typed.
There is an alternative to using C-c, and that is M-#. This adds a '#' to the start of the line, commenting it out, and then behaves as if you pressed Return. The result is that the command is cancelled but still remains on-screen and in your history. You can then copy the command later using the mouse or keyboard.
The new lintian addition "missing-debian-source-format" is starting to remind me of a pestering nanny.
I use source format 3.0 only where I see a technical benefit and that - so far - is restricted to packages that use a .tar.bz2 upstream and one or two with really tricky patching requirements.
You are missing the point - debian/source/format allows you to make it explicit that the source format is 1.0, if you want to stick with that.
Max Attems uploaded a new version of the Linux kernel package (linux-2.6) today. This includes the last major changes to the package before Debian 6.0 'squeeze', which led me to label it the 'Big Bang' release:
I had also hoped that it would be really reliable and work with the latest kernels (unlike Xen) but it is giving me problems with 2.6.32 on Opteron.
This is a regression caused by a recent security fix (CVE-2010-0298, "KVM: x86 emulator: fix memory access during x86 emulation"). It appears to affect only recent Linux kernels running as guests on AMD systems, and it will probably be fixed soon.
As previously announced elsewhere, video recordings from the Distribution Developer Rooms at FOSDEM 10 are now available. All but two talks were recorded and are available in Ogg Theora+Vorbis format, in low-bandwidth (~300 kbit/s) and high-bandwidth (~1.5 Mbit/s) versions.
These recordings should also be available later on the FOSDEM YouTube channel.
Video and audio were recorded by Dominique Dumont (H.1302) and the DebConf video team (H.1308) with the assistance of some additional volunteers. Thanks to everyone involved in recording, and thanks also to the speakers and the organisers of FOSDEM.
As some people noticed, a driver update to support a major hardware release in my day job and a long series of trivial patches motivated by Debian kernel work combined to make me joint most prolific change author for Linux 2.6.33. This was a fluke, and 2.6.34 is likely to be fairly quiet for me.
Meanwhile the kernel team is still working hard on fixing bugs in Linux 2.6.32. As previously announced, this kernel version will be used in both Debian 6.0 'squeeze' and Ubuntu 10.04 LTS. It is also known to be the kernel version for RHEL 6 though this has not been officially announced. If you find a serious bug that was fixed in 2.6.33 or later, please ask the author to send the fix to firstname.lastname@example.org so that it will be included in all these distributions. Note that this happens automatically for commits with the line 'Cc: email@example.com' in their description.
We are also continuing to backport new drivers and new hardware support in existing drivers into Debian 5.0 'lenny' (stable). Missing support for common hardware is considered an important bug and suitable for stable updates. However, if there were major changes to a driver before those that added new hardware support, we may be unable to produce a backport. We are not able to carry out comprehensive hardware testing and do not want to risk a regression existing hardware support.
Finally, a major change I have been working on is the transition from the old IDE drivers for PATA controllers to new drivers based on libata. The old IDE subsystem is no longer developed and some drivers do not properly support all the hardware that their libata-based counterparts do. However, while the IDE drivers generate device names beginning with 'hd', libata presents PATA devices as SCSI devices and generates device names beginning with 'sd'. In a system that already has other SCSI or SCSI-like disks, names may change somewhat unpredictably. Similar problems exist for PATA CD/DVD and tape devices.
So, while the transition doesn't involve any kernel hacking, it does require a complex upgrade process. Any configuration files that refer to IDE or SCSI device names may need to be changed. It is not enough to switch to only SCSI device names, since we cannot know what they are in advance and the configuration files should continue to work with kernel versions from before and after the transition. In the case of disks we can normally use the partition label or UUID: many tools understand the LABEL= and UUID= syntax, and for others we can refer to the symlinks under /dev/disk created by udev. In the case of CD/DVD devices, we can use the aliases /dev/cdrom etc. created by udev. In the case of tape devices, however, you're on your own.
In experimental, kernel image packages depend on a new package linux-base which implements this transition (from 2.6.33-1~experimental.2; the previous version was broken). The postinst script will prompt you to make changes automatically or manually. It can convert most bootloader configuration files, /etc/fstab, the udev CD aliases configuration and the initramfs-tools resume partition. It can also label partitions that don't have a label or UUID. Please do test it and verify that its changes are correct. All changed configuration files are backed up with a suffix of '.old' (or '^old' in one case).
Most of the Debian kernel team members are attending the Linux Plumbers Conference in Portland over the next 3 days. We'll be discussing kernel packaging and integration issues among ourselves and with upstream and other distributors.~
You may have noticed that Linux 2.6.31 is out but not yet in sid. This kernel version contains a large number of known regressions which should be fixed in subsequent stable updates, so we will wait for those. In the mean time we've uploaded an update to 2.6.30 which should resolve the most serious bugs.
Low-bandwidth versions of the recorded talks are now available. There are currently two exceptions:
While the master versions of all these talks have been checked, not all the generated low-bandwidth versions have. Please let us know on firstname.lastname@example.org if there is something wrong with them other than the following:
The high-bandwidth versions will be uploaded over the next week (slow connection is slow).
Several people have been asking after the videos. My answer to them is "we release when it's ready".
Seriously, though, I'm sorry about the delay in publishing videos. There was some miscommunication between me and the sysadmin team at DebConf which meant I couldn't take files away with me then. Peter Palfrader reassembled the video RAID later, copied the necessary files onto a single hard disk, and posted that to me. This disk apparently got lost on the way, but eventually arrived here last week after a 3-week journey.
Over the weekend I dealt with most of the videos that required editing, and today I was able to resume the transcoding process. Tomorrow I intend to start uploading again.