Blinkenblog

I’ve just released a new version of ladvd, which brings a fair number of improvements:

• rewritten memory allocation (no more memleaks)
• autodetect all network interfaces
• autodetect bonding, bridging and wireless interfaces (on Linux, FreeBSD and OpenBSD)
• be clever about bonding/bridging sub-interfaces
• use only one raw-socket for all interfaces
• remove libnet dependency
• improved OpenBSD support
• autodetect router, wlan and bridge capabilities
• support different available / enabled capabilities
• use the first mac-address as chassis-id (which fixes HP-switch interoperability)
• rewritten tlv macro’s to support strict alignment archs (sparc)

This release has been tested on a fair number of operating systems talking to various brand-name switches, Paul de Weerd deserves kudos for suggestions and OpenBSD improvements.

Tarballs and Debian sources are available.

I’ve just released a new version of ladvd, which brings a fair number of improvements:

• Debian packaging
• Linux ethtool ioctl support
• Linux capabilities support (needed for unprivileged ethtool usage)
• BSD ifmedia ioctl support
• MTU TLV for CDP and LLDP
• Autoneg TLV for CDP and LLDP
• Duplex TLV for CDP
• Media TLV for LLDP (BSD only)
• Host capabilities selection (Bridge, Host, Router, Switch, WLAN AP)
• Improved hostname detection
• Improved OpenBSD support

This release has been tested on a fair number of operating systems talking to various brand-name switches. Which means that this code will probably still eat your kitten, but won’t burn down the house while doing so. The newly supported media/autoneg/duplex TLV‘s should prove useful when solving connectivity issues.

Tarballs and Debian sources are available.

The CDP and LLDP protocols make it possible to discover connected devices on managed switches, without having to rely on port descriptions. This functionality is quite useful when dealing with large numbers of rackmounted servers with varying network configurations (PXE).

After looking at the available unix CDP/LLDP daemons I’ve decided to write my own CDP / LLDP sender. It’s a small daemon which sends out link-layer advertisements on a list of interfaces, after dropping privileges. The code should work on Linux and FreeBSD. Future enchancements will include sending more network (IPv6 addresses, duplex, mtu) and system (LLDP-MED / DMI) information, due to the OS specific nature of these details support for these options will be platform dependent.

Download ladvd.

I’ve received requests to add FreeBSD 7.x support to mod_blink, my inplementation of apache user separation. The apache module works just fine on newer apache releases, but there have been many changes on the kernel side. Kernel support is needed to grant additional privileges to the apache module. The old FreeBSD kernel module is a “hack” because it modifies the seteuid/setegid syscalls by hooking them via the syscall table, which is a pretty nasty way to get things done.

Luckily things have improved on the FreeBSD side which allows for a clean inplementation. The TrustedBSD MAC framework which was imported at around 5.0 presents clean hooks to relax/restrict priviliges via loadable modules, rather similar to LSM on the Linux side.

This weekend I’ve written mac_blink which should work on FreeBSD 5+ kernels with MAC support (KERNCONF=MAC).

At work we’ve bought new NetApp filers, and I’ve been tasked with the installation. One of the new things that I decided to inplement is iSCSI monitoring, anything critical should be monitored anyway.This is where I hit the first hurdle, nobody seems to do iSCSI monitoring with nagios. Google doesn’t show any check_iscsi examples. So I started investigating the available options. The main contender is iscsiadm from the open-iscsi stack. For monitoring purposes this package has a big problem; it uses daemons to save state, which probably speeds things up a fair bit, but totally ruins it for monitoring purposes.

Then I found http://code.google.com/p/freebsd-iscsi/ which after a careful look does exactly what’s needed, iSCSI sendtargets discovery and not much else. When adapting the software for nagios I quickly noticed some segfaults, which warranted further investigation. It turns out that a buffer of 1024 bytes is allocated, which is used for transmitting and receiving iSCSI messages. But the code which parses messages just looks at the datasegmentlength field in the header, which is used even when it exceeds the buffersize of 1024. When trying to find a simple solution for this problem I noticed the MaxRecvDataSegmentLength parameter which can be used during login to limit the length of messages. This looked like a quick fix, so I dediced to try it out. The NetApp filer I was talking to however didn’t appreciate the change.

When creating a sendtargets responce a NetApp will include every network interface which has iSCSI enabled. This will cause the response to be quite large if you have 10-15 active interfaces. So when trying to stuff all this information, about 1517 bytes normally in two 1500 bytes ethernet frames, in 512 bytes I hit the following panic:

Tue Feb 12 14:19:54 CET [mgr.stack.string:notice]: Panic string: ../driver/scsitarget/iswt/iswti_text.c:624: Assertion failure. in process iswti_iscsip_thread on release NetApp Release 7.2.4L1

The sad thing with NetApp is that most bugs like this will cause a reboot, which interrupts operation quite heavily. The main problem with this one is that if iSCSI is enabled it probably isn’t possible to stop people rebooting your filers.

One of my basic pet-peeves has been dhclient’s meddling with /etc/resolv.conf. I run my own resolvers and don’t like using ISP nameservers, which are often overloaded. Today I finally looked deeply enough into this to solve the issue on Debian/Ubuntu. Just add the following to /etc/dhcpd3/dhclient-enter-hooks.d/resolvconf.

# For safety, undefine the nasty default make_resolv_conf()
make_resolv_conf() {
    true
}

On FreeBSD/NetBSD the same can be accomplished by adding the above code to /etc/dhclient-enter-hooks. This overrides the default function used by dhclient with one that does nothing. Nice, clean, simple.

On FreeBSD most system configuration is centralized in /etc/rc.conf, which sets options for system startup and all well-behaved daemons. Most people however don’t know that rc.conf.local also is sourced by the same scripts. This makes it possible to use a generic rc.conf to specify site/company wide defaults, and configure host specific settings in rc.conf.local. One advantage is that rc.conf.local usually ends up being quite short which simplifies host management. Using this scheme works extremely well when it’s combined with cfengine and source control. Versioning and commit mailinglists can do truly amazing things for sysadmin productivity.

I carefully watched the Ubuntu respositories for linux 2.6.20 packages, and quickly noticed when the experimental packages were released. These however aren’t quite stable yet, the atheros driver (in restricted) currently oopses, which is somewhat problematic for me.

Someone has already reported the problem at launchpad, let’s see how quickly it’ll get fixed.

After some research I discovered that linux has IPv6 Stateful firewalling since 2.6.15. But the code won’t be usable until at least 2.6.20 because it doesn’t play nice with IPv4 NAT.

Initial Patches are available on the netfilter mailinglists, let’s hope they make 2.6.20.