I have a talk to give, hacking to do, and I should be helping with
video as well. Hope to see you in Banja Luka!
posted at: 04:38 | path: / | permanent link to this entry
Phil Kern made a general call for testing of Debian 6.0.4. I would like to more specifically point out that I backported the isci driver for Intel C600 SAS/SATA controllers. Unfortunately I have not yet had any testing results for this. If you have any machines with this hardware not yet in production, please do consider testing the new Linux kernel package, version 2.6.32-41.
Updated: I now have a positive test result from the user who requested this driver in the installer.
posted at: 15:51 | path: / | permanent link to this entry
I thought it was about time to get a faster machine for... well, mostly for compiling kernels faster, but I'm sure I will find other applications for it. I was originally intending to upgrade my laptop (Thinkpad T61, 1.8 GHz Core 2 Duo) but it seems that in the 3 years since I bought it second-hand laptop prices have not dropped in line with Moore's law. So instead I decided to upgrade (well, replace most of) a desktop PC.
The last time I got a new desktop at home was in 2002. It has an Athlon XP, PATA drives and a huge CRT monitor. It has had some upgrades since then (DVD writer, more RAM, slightly faster CPU, replacement fan and PSU). Obviously it was time to replace motherboard, CPU and RAM. Not so obviously, the PSU needed changing again as modern CPUs apparently need more 12V electrons (?!). The DVD writer was salvageable via a USB adapter and its local HD is currently a laptop drive (the only spare SATA HD I had). The monitor is of course reusable, though the firmware setup program doesn't entirely agree with it.
Yes, I do sometimes run Windows. It's for games, and also seems to be needed for streaming video with stupid DRM.
On the assumption that Windows is less likely to play nice with an existing installation, I tried installing it first. The firmware appear to allow me to boot the DVD in either BIOS or UEFI mode, so I selected the UEFI option. This worked so far as loading the initramfs (or whatever the Windows equivalent is) but then Windows was lacking a driver to access the DVD. This stumped me for a day or so until I realised I had plugged the drive into a USB 3 (XHCI) port. Apparently Windows 7 SP1 still doesn't include an XHCI driver in the installer (but neither does Debian 6.0.3 - we should fix that).
The installation was mostly uneventful after this. The Windows installer includes a reasonably capable partition editor.
Installation of files was entirely uneventful. The installer correctly detected the GPT (GUID partition table) created by Windows. However, it was not able to install a boot loader. Currently it only includes support for the LILO and GRUB-PC boot loaders that run under BIOS.
After much experimentation and cursing, I found instructions for installing Debian Squeeze on XServe 3.1 by Jan Peter Alexander Rajagukguk, which put me on the right track. It seems that the grub-install script for GRUB-EFI in squeeze (and maybe other parts of the package) doesn't actually work. However. JP's direction to build GRUB from source is no longer necessary. Here's what I did (minus the mis-steps):
# # We start in dash, so switch to bash. # exec bash # mkdir -p /boot/efi # # Mount the EFI boot partition by UUID. # # It's probably called sda1 now, but is subject to change. # ls -l /dev/disk/by-uuid ... # echo >>/etc/fstab 'UUID=efi-boot-uuid /boot/efi vfat defaults 0 0' # mount /boot/efi # # Enable unstable sources, but use stable by default # echo >>/etc/apt/apt.conf 'APT::Default-Release "squeeze";' # echo >>/etc/apt/sources.list 'deb http://cdn.debian.net/debian/ sid main' # apt-get update ... # # Install unstable grub-efi-amd64 # apt-get install -t unstable grub-efi-amd64 ... # # Install GRUB to the EFI partition. The '--removable' options # # inhibits setting EFI variables, which won't work because we booted # # through BIOS emulation. Instead, it installs as the default boot # # loader. # grub-install --bootloader-id=GRUB --removable # # On the AMI/ASUS firmware, you cannot select a new bootloader # # directly but can invoke a 'shell'. So I made GRUB the shell: # cp /boot/efi/boot/grub/grub.efi /boot/efi/shellx64.efi # exit
# apt-get install efibootmgr ... # modprobe efivars # efibootmgr -c -l '\efi\grub\grubx64.efi' -L GRUB
posted at: 20:25 | path: / | permanent link to this entry
Simon McVittie reported in Debian bug #654876 that his laptop running Linux 3.0 or 3.1 would sometimes crash (panic) while idle. He initially suspected a driver bug, but the screen did not show any information about where the original fault occurred. However, using netconsole, he was able to capture a full log of the crash. This showed that a packet received through the wireless interface was being processed by IGMP, which then divided by zero.
IGMP is part of the IPv4 protocol suite, supporting multicast routing. Every multicast address corresponds to a dynamic set of hosts, called a multicast group. Multicast routers can send query messages asking which hosts belong to which groups, and hosts using multicast report back at intervals. Routers can then limit forwarding of multicast packets to the interfaces where the group has members. More sophisticated switches can also snoop IGMP and use it to limit their multicast forwarding. There are unfortunately three different versions with semi-compatible message formats. In version 1, the maximum reporting interval (Max Response Time) is fixed as 10 seconds, but from version 2 it is specified in query messages.
The Linux IGMP implementation supports all three versions. It distinguishes query messages as specified in RFC 3376 section 7.1: v3 messages are longer than v1 or v2; v2 messages have a non-zero Max Response Time whereas v1 messages always have zero. It is possible to force use of a specific protocol version, but normally if there are multiple multicast routers using different protocol versions it will respond according to the earliest protocol version in use so that all routers can understand its responses.
Linux 2.6.36 included two fixes to the version selection logic. Unfortunately, the second of these introduced the bug in question. While v2 query messages cannot possibly have zero Max Response Time (as that would make them v1), v3 query messages can. What this means is unspecified, but the Linux IGMP code previously treated it as the minimum valid value of 1/10 second. But in the case where a v3 query is received and a v2 query has also recently been received, this is no longer done. This results in a reporting interval of 0 seconds and a division by zero when deciding the initial random delay.
The fix for this is pretty obvious. It is included in Linux 3.0.17, 3.1.9, 3.2.1, and the Debian package version 3.1.8-2.
This is easily exploitable for denial of service within a local network. Linux does not check the destination address of IGMP queries, so it may also be possible to attack a target through its unicast address from several hops away. The attacker only needs to send a single IGMPv2 query followed by a single IGMPv3 query with zero Max Response Time. All systems running Linux 2.6.36 or later (up to the above fixed versions) with any active IPv4 multicast listeners (other than for the 'all hosts' address) are vulnerable.
posted at: 16:17 | path: / | permanent link to this entry
Screensaver control on the Linux desktop is a total mess, but
the xdg-screensaver command provided an interim
cross-desktop solution by detecting and using the appropriate
underlying interfaces. For GNOME, it depends on
the gnome-screensaver-command --poke option, but that has
been broken and then
removed in favour of the cross-desktop D-Bus API
org.freedesktop.ScreenSaver
another
GNOME-specific API which can't practically be used by
xdg-screensaver.
What's more, switching to the new API seems to have introduced various bugs in GNOME's own video player, Totem, such that it often failed to inhibit the screensaver in GNOME 2.32 and 3.0. Apparently everything is wonderful again in 3.2; here's hoping.
(In case anyone asks where my bug report on the totem package is, I already looked at the bug list and it's clear that there's no point.)
Updated: Well I said the GNOME Session D-Bus API couldn't practically be used in xdg-screensaver, but then set out to prove myself wrong. And with the aid of Perl, I may have succeeded.
posted at: 07:36 | path: / | permanent link to this entry
I have a talk to give, hacking to do, and I should be helping with
video as well. Hope to see you in Banja Luka!
posted at: 04:38 | path: / | permanent link to this entry
The Debian kernel team regularly backports driver updates to the Linux kernel in stable releases to add support for new hardware, and I've prepared several updates intended for point release 6.0.2. Since the kernel team does not have a large collection of hardware on which to test driver changes, we would appreciate test reports from users. It is important to test not just that new devices are supported properly, but that there are no regressions in support for older devices.
I have updated these drivers to the versions found in Linux 2.6.38, modulo driver API changes:
I have also cherry-picked some small changes:
There are more drivers that I think should be added or updated (see #624794) but they will probably have to wait for release 6.0.3.
The source package and binary packages for i386 and amd64 are available on people.debian.org. They can be verified by the checksums in the signed changes file.
The current packages are version 2.6.32-35~test1, but there may be further test versions before an official stable update.
For network drivers, I suggest the following regression tests:
For storage drivers, unfortunately I don't have a good idea of what tests would be suitable. In any case, please don't test on disks storing valuable data!
Please send test reports to the bug reports linked above, stating the driver name, the PCI ID for the device you tested (from lspci -n) and any other device identification that the kernel log (for example, r8169 logs the 'XID' of the device).
posted at: 19:51 | path: / | permanent link to this entry
The major upcoming configuration change in Linux 2.6.39 is to get rid of the '686' flavour. This may be surprising, because it's the most widely used flavour of the 4 we have a present:
| Name | Minimum CPU features | Maximum total CPU threads | Physical address space |
|---|---|---|---|
| 486 | 486-class | 1 | 4 GiB |
| 686 | 686-class; CMOV instruction | 32 | 4 GiB |
| 686-bigmem | 686-class; PAE | 32 | 64 GiB |
| amd64 | x86-64 | 512 | 64 TiB |
However, the physical address limitation means that an increasing proportion of new PCs and the majority of PC servers need the '686-bigmem' flavour. Even those that have less than 4 GiB RAM do support PAE and can run the '686-bigmem' flavour. There is a small cost (up to about 0.1% of RAM) in the use of larger hardware page tables. There is also an important benefit on recent processors: the larger page table entries include an NX bit (also known as XD) which provides protection against some buffer overflow attacks, both in the kernel and in user-space..
There are only a few 686-class processors that support CMOV but not PAE: most Intel Pentium M models, the VIA C3 'Nehemiah' and the AMD Geode LX. These also all lack SMP support, which means the '486' flavour is suitable for them.
Some benchmarking on two of those - a Pentium M model 745 and a C3 'Nehemiah' - indicated that they run the '486' flavour slightly faster than the '686' flavour. It appears that the performance gain from using plain uniprocessor code (rather than SMP-alternatives, which are patched with NOPs on uniprocessor systems) outweighs the performance loss from avoiding use of some newer instructions.
We intend to get rid of 'amd64' too, but we need to ensure that upgrades from linux-image-2.6-amd64:i386 to linux-image-2.6-amd64:amd64 work properly.
Yes, it is. Therefore '686-bigmem' will be renamed to '686-pae'.
posted at: 04:34 | path: / | permanent link to this entry
As you've probably seen, the testing suite finally got a new upstream kernel version (Linux 2.6.38) and is currently synchronised with unstable. I'm just about to upload a new version of linux-2.6 based on upstream stable update 2.6.38.4.
Linux 2.6.39 is well on its way to release, so 2.6.39-rc4 should soon be in experimental. The Debian source package will add support for the new armhf (ARM with hardware floating-point) architecture.
The linux-base package, introduced to handle transitions and other support functions for 'image' packages, is now built from a separate source package. It won't be changed very often, so you will no longer need to upgrade linux-base every time you upgrade an 'image' package. (This doesn't apply within 'squeeze', but I may have a way to work around that.)
The firmware-linux-free package, containing a few firmware and configuration blobs from Linux that are DFSG-compliant, is also now built from a separate source package. These blobs have not changed for a very long time, so again you will no longer be recommended to upgrade this package every time you upgrade an 'image' package. Secondly, this allows us to remove the entire 'firmware' directory from the upstream source rather than carefully removing and patching to leave just the DFSG-compliant bits.
The '2.4 GHz' and '5 GHz' ISM bands used for wifi are not defined identically around the world. The exact frequency ranges and maximum permitted power levels vary between countries. Radio equipment manufacturers have a legal responsibility to ensure their equipment operates within these regulations. Since it is expensive to implement such regulations in hardware (including many different variants for different regions), these restrictions are usually implemented in the driver using a country code retrieved from the hardware's EEPROM or flash. So this is now our responsibility.
We need to be able to look up regulations by country codes, and we also need to support the case where a device sold in one country is used in another country with tighter regulations. For some time, the wifi configuration library (cfg80211) included regulatory information for a small number of countries and regions. However, the default since Linux 2.6.34 has been to build-in only the 'world' domain (intersection of all the restrictions) and to request information from user-space. In Debian, we didn't have anything to provide this information, so for example channels 12-13 became unavailable for users in Europe.
The regulatory information should be provided by the Central
Regulatory Domain Agent (CRDA) from a current database. These are
now packaged as crda and wireless-regdb
respectively. Drivers that do not use cfg80211 (mostly those from
the 'staging' area) will not be affected, and may or may not
implement correct regulations.
When you travel to another country or use an imported wifi device
with an incorrect country code (I hear 'CN' is common), you can
use iw reg set to ensure that your wifi devices operate
within the regulations of the country you are in. Set
the REGDOMAIN variable in
/etc/default/crda to make this persistent. These will
be intersected with the regulations specified by the
hardware.
posted at: 17:40 | path: / | permanent link to this entry
I'm still missing how to reliably detect if a device is a "wired" or a "wireless" one. I suspect that checking the existence of /phy80211 would be enough, but I can't really tell, and seems like I'm not able to find an exhaustive sysfs reference manual.
Sadly there is no exhaustive manual, but many attributes found in sysfs are documented under Documentation/ABI/. In this case you need the DEVTYPE name from the net device's uevent, which may be e.g. 'wlan' or 'wwan' (and is absent for wired Ethernet devices). I don't know why this isn't also exposed as an attribute in its own right.
posted at: 22:23 | path: / | permanent link to this entry
If you're trying to use a USB serial adapter as your Linux kernel console...
The kernel needs to be compiled with CONFIG_USB_SERIAL_SERIAL=y
That's CONFIG_USB_SERIAL_CONSOLE.
Debian kernels do not appear to be. Somewhat strangely, the config option doesn't even wind up in the config file in the typical commented-out manner, so you don't even know the option exists until you discover it from random Web searching, and then looking at drivers/usb/serial/Kconfig in the kernel source.
Console drivers need to be built-in, so CONFIG_USB_SERIAL_CONSOLE depends on CONFIG_USB_SERIAL=y. We build the USB serial code as a module so CONFIG_USB_SERIAL_CONSOLE is not an available option. This is unlikely to change, as a built-in feature takes memory from every Debian GNU/Linux installation.
posted at: 20:49 | path: / | permanent link to this entry
posted at: 17:48 | path: / | permanent link to this entry
Something was bound to go wrong when changing the policy for boot loaders. Let me try that again.
posted at: 03:39 | path: / | permanent link to this entry
A few weeks ago Zack wrote about preparations for a new podcast to be called 'This Week in Debian'. I volunteered to be interviewed, and have now talked with Jonathan Nadeau for about 30 minutes about the work of the kernel team and the release process. Hopefully he'll be editing down my rambling so the first episode won't be too boring! If you do work on Debian that you'd like to talk about to an interested audience, follow the link above and add yourself to the list of potential interviewees.
posted at: 00:23 | path: / | permanent link to this entry
[...] whenever I was typing a command line, changed my mind and pressed C-c, I got a ^C printed on the screen, usually overwriting one or two characters of what I had typed. And this prevented me from automatically copying and pasting the command that I had typed.
There is an alternative to using C-c, and that is M-#. This adds a '#' to the start of the line, commenting it out, and then behaves as if you pressed Return. The result is that the command is cancelled but still remains on-screen and in your history. You can then copy the command later using the mouse or keyboard.
posted at: 22:56 | path: / | permanent link to this entry
The new lintian addition "missing-debian-source-format" is starting to remind me of a pestering nanny.
I use source format 3.0 only where I see a technical benefit and that - so far - is restricted to packages that use a .tar.bz2 upstream and one or two with really tricky patching requirements.
You are missing the point - debian/source/format allows you to make it explicit that the source format is 1.0, if you want to stick with that.
posted at: 03:39 | path: / | permanent link to this entry
Max Attems uploaded a new version of the Linux kernel package (linux-2.6) today. This includes the last major changes to the package before Debian 6.0 'squeeze', which led me to label it the 'Big Bang' release:
posted at: 03:01 | path: / | permanent link to this entry
I had also hoped that it would be really reliable and work with the latest kernels (unlike Xen) but it is giving me problems with 2.6.32 on Opteron.
This is a regression caused by a recent security fix (CVE-2010-0298, "KVM: x86 emulator: fix memory access during x86 emulation"). It appears to affect only recent Linux kernels running as guests on AMD systems, and it will probably be fixed soon.
posted at: 23:35 | path: / | permanent link to this entry
As previously announced elsewhere, video recordings from the Distribution Developer Rooms at FOSDEM 10 are now available. All but two talks were recorded and are available in Ogg Theora+Vorbis format, in low-bandwidth (~300 kbit/s) and high-bandwidth (~1.5 Mbit/s) versions.
These recordings should also be available later on the FOSDEM YouTube channel.
Video and audio were recorded by Dominique Dumont (H.1302) and the DebConf video team (H.1308) with the assistance of some additional volunteers. Thanks to everyone involved in recording, and thanks also to the speakers and the organisers of FOSDEM.
posted at: 20:17 | path: / | permanent link to this entry
As some people noticed, a driver update to support a major hardware release in my day job and a long series of trivial patches motivated by Debian kernel work combined to make me joint most prolific change author for Linux 2.6.33. This was a fluke, and 2.6.34 is likely to be fairly quiet for me.
Meanwhile the kernel team is still working hard on fixing bugs in Linux 2.6.32. As previously announced, this kernel version will be used in both Debian 6.0 'squeeze' and Ubuntu 10.04 LTS. It is also known to be the kernel version for RHEL 6 though this has not been officially announced. If you find a serious bug that was fixed in 2.6.33 or later, please ask the author to send the fix to stable@kernel.org so that it will be included in all these distributions. Note that this happens automatically for commits with the line 'Cc: stable@kernel.org' in their description.
We are also continuing to backport new drivers and new hardware support in existing drivers into Debian 5.0 'lenny' (stable). Missing support for common hardware is considered an important bug and suitable for stable updates. However, if there were major changes to a driver before those that added new hardware support, we may be unable to produce a backport. We are not able to carry out comprehensive hardware testing and do not want to risk a regression existing hardware support.
Finally, a major change I have been working on is the transition from the old IDE drivers for PATA controllers to new drivers based on libata. The old IDE subsystem is no longer developed and some drivers do not properly support all the hardware that their libata-based counterparts do. However, while the IDE drivers generate device names beginning with 'hd', libata presents PATA devices as SCSI devices and generates device names beginning with 'sd'. In a system that already has other SCSI or SCSI-like disks, names may change somewhat unpredictably. Similar problems exist for PATA CD/DVD and tape devices.
So, while the transition doesn't involve any kernel hacking, it does require a complex upgrade process. Any configuration files that refer to IDE or SCSI device names may need to be changed. It is not enough to switch to only SCSI device names, since we cannot know what they are in advance and the configuration files should continue to work with kernel versions from before and after the transition. In the case of disks we can normally use the partition label or UUID: many tools understand the LABEL= and UUID= syntax, and for others we can refer to the symlinks under /dev/disk created by udev. In the case of CD/DVD devices, we can use the aliases /dev/cdrom etc. created by udev. In the case of tape devices, however, you're on your own.
In experimental, kernel image packages depend on a new package linux-base which implements this transition (from 2.6.33-1~experimental.2; the previous version was broken). The postinst script will prompt you to make changes automatically or manually. It can convert most bootloader configuration files, /etc/fstab, the udev CD aliases configuration and the initramfs-tools resume partition. It can also label partitions that don't have a label or UUID. Please do test it and verify that its changes are correct. All changed configuration files are backed up with a suffix of '.old' (or '^old' in one case).
posted at: 18:11 | path: / | permanent link to this entry
Most of the Debian kernel team members are attending the Linux Plumbers Conference in Portland over the next 3 days. We'll be discussing kernel packaging and integration issues among ourselves and with upstream and other distributors.~
You may have noticed that Linux 2.6.31 is out but not yet in sid. This kernel version contains a large number of known regressions which should be fixed in subsequent stable updates, so we will wait for those. In the mean time we've uploaded an update to 2.6.30 which should resolve the most serious bugs.
posted at: 18:34 | path: / | permanent link to this entry
Low-bandwidth versions of the recorded talks are now available. There are currently two exceptions:
While the master versions of all these talks have been checked, not all the generated low-bandwidth versions have. Please let us know on debconf-video@lists.debconf.org if there is something wrong with them other than the following:
The high-bandwidth versions will be uploaded over the next week (slow connection is slow).
posted at: 14:14 | path: / | permanent link to this entry
Several people have been asking after the videos. My answer to them is "we release when it's ready".
Seriously, though, I'm sorry about the delay in publishing videos. There was some miscommunication between me and the sysadmin team at DebConf which meant I couldn't take files away with me then. Peter Palfrader reassembled the video RAID later, copied the necessary files onto a single hard disk, and posted that to me. This disk apparently got lost on the way, but eventually arrived here last week after a 3-week journey.
Over the weekend I dealt with most of the videos that required editing, and today I was able to resume the transcoding process. Tomorrow I intend to start uploading again.
posted at: 04:24 | path: / | permanent link to this entry
posted at: 12:43 | path: / | permanent link to this entry
posted at: 00:12 | path: / | permanent link to this entry
A question from AJ reminded me that I haven't said much about the changes to packaging of firmware in Debian, and in particular the separation of non-free firmware from the Linux kernel.
There is an ongoing process upstream to move firmware blobs from drivers into a firmware/ subdirectory of the source, which is now almost complete. Since most of this firmware is non-free, we remove it from the source tarballs for kernel packages but use it to update the firmware-nonfree "source" package.
We continue to patch some drivers to separate out firmware, and have been submitting our changes upstream. Most of these have been accepted though the DRI drivers matrox, r128 and radeon are notable exceptions.
A few months ago I attempted to make a new inventory of the remaining firmware blobs outside of the firmware/ subdirectory. I identified three that should still be addressed. The Linux-libre project, however, removes many other constant arrays from the kernel (and disables the affected drivers) where I judged the array to be a plausible preferred form of modification.
Much of the non-free firmware removed from the kernel is now available in the firmware-linux package in the non-free section of the Debian archive. Starting with Linux 2.6.31, we will build the DFSG-free firmware shipped with Linux into a package called firmware-linux-free, which will be recommended by kernel image packages. The contents of firmware-linux will be moved to firmware-linux-nonfree and firmware-linux becomes a meta-package depending on the other two packages.
Many other firmware images never distributed with Linux are also packaged for the benefit of users that require them.
posted at: 17:23 | path: / | permanent link to this entry
One of the more interesting talks at last week's PyCon was on the new Python licence.
posted at: 02:51 | path: / | permanent link to this entry
This week I'm at PyCon in Chicago, helping to train and supervise the A/V team in recording the tutorials and talks. We have a lot of inexperienced volunteers but they're learning quickly. Carl Karsten (in charge of A/V) and Ryan Verner (another outsider with long experience from LCA) are organising things. Ryan, Dave Noble and I provide technical assistance to the rest of the team.
I'm also working on my first Django app, which will be used for reviewing files. This is essentially a clone of the excellent work done by Gunnar Wolf and Damian Viano in Pentabarf. PyCon doesn't have a centralised database so this is a standalone application. Ryan is now applying styles to my horribly bare HTML, and I have a short list of bugs to fix before we ask the video team to start reviewing.
The tutorials started yesterday and talks wil start tomorrow in different rooms, so we have to set up all the equipment again and reconfigure for multiple cameras. Hopefully by Saturday things will be running smoothly and I can spend some time in the city. It would be a shame to come all this way and stay by the airport for the entire time!
posted at: 18:58 | path: / | permanent link to this entry
I'm going to 
Carl Karsten of the PyCon organisers recruited three of us from the DebConf video team to lead and train the video team at PyCon.
posted at: 17:44 | path: / | permanent link to this entry
Jérémy Bobbio suggested that I should explain how I looked for packages affected by a compiler bug (Debian bug 506713; gcc bug 38287). I don't claim that this is a particularly good way to do it, but here it is:
First, I identified a pattern to search for. Unfortunately I don't really understand the cause or fix for the bug, but I did have the example which led to this bug report: Debian bug 490999. The bad code was a stack-pointer-relative load immediately after a stack allocation (SPARC save instruction) where the offset was not adjusted for the stack allocation:
I generalised this to:save %sp, -112, %sp ld [ %sp + 0x40 ], %i5
save %sp, offset1, sp ... ld [ %sp + offset2 ], register
where offset1 + offset2 < 0. Of course, this may
be valid if the intervening instructions include a restore,
branch or store to the effective stack location that the last instruction
loads from. I ended up allowing up to 10 intervening instructions and
examining a disassembly to work out which cases were valid.
I looked up the instruction encoding for these two instructions. Thankfully SPARC is RISC so they are simple and regular:
save %sp, offset1, sp is encoded as 0x9de3a000 | offset1 & 0x1fffld [ %sp + offset2 ], register is encoded as 0xc003a000 | reg << 25 | offset2 & 0x1fff
I took the dumb but effective approach of scanning entire files for this pattern rather than only scanning their code sections. This seemed to work - I got no false hits for non-code - but might not work for other patterns that could match ASCII text.
I wrote the scanning program in Python, which is my default choice of
language unless I know it's going to be too slow. I was hoping to be able
to read the code files into arrays, but unfortunately the Python array
type only supports the native byte-order (SPARC is big-endian and I was
intending to use an x86 which is little-endian). I tried reading into a
tuple using struct.unpack, which does support explicit byte-ordering, but
this used so much memory for larger files that the program swapped to a
crawl. So finally I resorted to reading the file into a string, doing a
string search for '\x9d\xe3', rejecting matches that weren't appropriately
aligned, then unpacking and comparing the code words from the point of the
string match.
(In Python 3.0 I would have to use the bytes type for this,
as str is a Unicode string type.)
So that's how I scanned single files. The next step was to find, unpack
and scan all the SPARC shared libraries in the archive. (This particular
code generation bug is understood to affect only PIC code, and that is
normally only used in shared libraries.) I wrote functions to search
Contents-sparc for shared library files - assumed to match the pattern
([^\s]*/lib[^/\s]+\.so(?:\.[^/\s]*)?) - and to parse
Packages to find the filenames for the packages containing those files.
The latter uses the debian_bundle.deb822 module from
python-debian.
The last key function downloads and unpacks a package using
wget and dpkg-deb. I could have used
the httplib module for downloading but I correctly
anticipated that I'd need to restart the script several times so I
wanted to cache the packages which was easier to do using
wget.
So, that's the explanation. If you really want to see it, here's the code.
posted at: 00:53 | path: / | permanent link to this entry
posted at: 02:26 | path: / | permanent link to this entry
I finally got sufficiently fed up with Subversion yesterday to begin converting my package repositories to Git. This has many benefits, including the fact that I can now easily publish them. Browse or clone them as you prefer. Or just ignore them.
posted at: 15:50 | path: / | permanent link to this entry
Some drivers in Linux include "firmware" - code that runs on the device the driver deals with, not the main processors in the system. Most of this is in a binary or equivalent form, which is probably not the form that it was originally written in. This makes it non-free by Debian's standards, an issue which has led to much argument around previous releases and has blown up once again.
The good news is that kernel developers are making a continuing effort to separate this from the driver code. I've collected together the patches that do this and written 2 of my own, which appear to cover all the remaining sourceless firmware. But I was only able to test one of these drivers (radeon) - the rest is up to you.
| Driver | Hardware which requires firmware | Bug report | Change made |
|---|---|---|---|
| cassini | Sun "Cassini" Ethernet controllers with "Neptune" PHY | 498631 | removed |
| dabusb | BayCom DAB-USB radio receivers | 502663 | moved to firmware-dabusb |
| e100 | Intel PRO/100 Ethernet controllers with PCI revision 8, 9, 15 or 16 or built into ICH6 or ICH7 (LAN-on-motherboard) | 494308 | moved to firmware-e100 |
| kaweth | Kawasaki LSI USB Ethernet controllers | 502665 | removed |
| mga | Matrox G200, G400, G450(?) and G550 graphics controllers, 3D acceleration only | 502666 | moved to firmware-matrox |
| qla1280 | QLogic QLA1XXX SCSI controllers | 502667 | moved to firmware-qlogic |
| r128 | ATI Rage 128 graphics controllers, 3D acceleration only | 494007 | moved to firmware-ati |
| starfire | Adaptec DuraLAN aka Starfire Ethernet controllers | 501152 | moved to firmware-adaptec |
| tehuti | Tehuti Ethernet controllers | 501153 | moved to firmware-tehuti |
| typhoon | 3Com Typhoon (3CR990) Ethernet controllers | 502669 | moved to firmware-3com |
| whiteheat | ConnectTech WhiteHEAT USB serial adapters | 502668 | removed |
Do you depend on the drivers for which I have removed the firmware? Do you have contacts with the vendors who might be able to help relicence it? Please send mail to the bug report.
I checked my email and found a response from ConnectTech; the WhiteHEAT firmware will probably be clear to package in firmware-nonfree. There is no need for anyone else to contact them.
posted at: 01:53 | path: / | permanent link to this entry
I discovered an interesting "feature" of chmod(1), which caused a package build to fail. According to the GNU manual page, if no letters are used before a "-" or "+", "the effect is as if a were given, but bits that are set in the umask are not affected." The command will also fail with an error message when it does this!
The Single Unix Specification says this is correct, though there is some ambiguity over whether the exit status should be 0 or not.
Anyway, the result is that it is generally a bad idea to call chmod in this way in a script. Always specify who the permission changes should apply to.
posted at: 23:24 | path: / | permanent link to this entry
In addition to live streams, recordings of each event should be published on meetings-archive.debian.net. There is an RSS feed of these, which should work nicely in Miro: http://meetings-archive.debian.net/pub/debian-meetings/2008/debconf8/index.rss. That feed is also included on Planet Debian. This is the first time I've tried generating RSS so apologies in advance if I make a mess of the Planet.
posted at: 09:32 | path: / | permanent link to this entry
Sorry, I missed a few last time.
| Dependency-based boot sequence | low | high |
| Data-mining Popcon | low | high |
| Debian-installer - an update | low | high |
| Wacky ideas II | low | high |
posted at: 20:58 | path: / | permanent link to this entry
A few of the recordings from DebConf 7 had to be recovered in whole or in part from tape. For personal and technical reasons, a few of these were not done last year, and I have finally dealt with them now. The last few sessions are:
| The secure Debian Desktop | low | high |
| Debian - The Universal Operating System? | low | high |
| Wacky ideas BoF | low | high |
| lintian BoF | low | high |
| Debian travels around the world | low | high |
| Leading a Free Software project | low | high |
| Method diffusion in large volunteer projects | low | high |
| Debtags is ready | low | high |
| OpenStreetMap | low | high |
| Maintaining packages with Git | low | high |
| netconf | low | high |
| Time for a better init system | low | high |
Hopefully these are useful to someone. Sorry it took so long!
posted at: 10:15 | path: / | permanent link to this entry
After a fair amount of clean-up, a lot of waiting for review, and some ruthless pruning, I and my colleagues at Solarflare finally got our net driver accepted for Linux 2.6.26. Meanwhile a larger version of the driver made it into Xen's Linux tree, along with dependent drivers to provide a secure fast path to the hardware from domU. Somewhere down the road we're going to have to resolve these two versions, but now I'm just pleased to have a working driver in-tree.
There are some other changes I've worked on over the past few months:
However, this is still only part of my work. My main task at the moment is to write more unit tests for our next NIC.
posted at: 16:15 | path: / | permanent link to this entry
I'm employed at Solarflare Communications in the test group, supporting the work of testers and developers by developing test frameworks. (We actually have very few pure testers and a whole lot of automation.). In a major break from my previous jobs, I've been working mostly in Python on applications for use in-house.
More recently I've become involved in maintenance of our network driver for Linux, starting with updating it to use current kernel coding conventions, and submitting for inclusion in the kernel. This has become a bit frustrating because while there are a lot of people prepared to run checkpatch and pick nits, there are not so many who will do a thorough review of a driver that's 20,000-odd lines long. And without someone doing that, it won't go in.
Meanwhile some of my colleagues have done some great work to accelerate Linux networking on Xen by having drivers in dom0 and domU cooperate to expose hardware resources directly to domU. Our hardware architecture supports many virtual NICs with their own packet queues and set of buffers, so this does not compromise the security of the physical machine and dom0. The results are very impressive.
The original motivation for the virtual NIC architecture was as a basis for a user-level networking stack, which can provide very low latency and reduced CPU usage. This software was originally proprietary but has now been released under GPLv2 as OpenOnload. Google invited two of my senior colleagues to talk about this - here's a video of their talk about user-level networking.
posted at: 00:24 | path: / | permanent link to this entry
posted at: 22:44 | path: / | permanent link to this entry