Links

Ben Laurie blathering


Fun With Dot

As I’ve mentioned before, people don’t really talk much about the experience of writing and debugging code, so here’s another installation in my occasional series on doing just that.

Over the Easter weekend the weather has been pretty horrible, so, instead of having fun on my motorbike, I’ve been amusing myself in various ways: trying to finish up a paper I started last year, doing further work on OpenSSL and Deputy, cooking, playing with Tahoe (on which more later), updating my FreeBSD machines, and messing around with Graphviz.

Graphviz is one of my favourite toys. Basically, it lets you specify how a bunch of things are connected, and then it will draw them for you. A project I’d long had in my head was to take all the RFCs, work out which ones reference which other ones and have Graphviz draw it for me. Getting the data turned out to be pretty easy, but unfortunately the resulting dataset proves to be too much for poor old Graphviz, exposing all sorts of bugs in its drawing engine, which led to core dumps and/or complete garbage in the output files. Shame, early experiments promised some quite pretty output. Anyway, after banging my head against that for many hours, I gave up and instead did something I do every few months: got my various FreeBSD machines up-to-date. As part of that process, I had to look at the stuff that FreeBSD runs at startup, configured in /etc/rc.conf (and /etc/defaults/rc.conf), and actually done by scripts in /etc/rc.d and /usr/local/etc/rc.d.

This reminded me that these scripts expose their dependencies in comments, like this (from /etc/rc.d/pfsync)

# PROVIDE: pfsync
# REQUIRE: root FILESYSTEMS netif

So, I thought it would be fun to graph those dependencies – then at least I’d have one pretty thing to show for the weekend. Then, since it only took 15 minutes to do, I thought it might make an interesting subject for a post on how I go about coding such things.

So, first things first, I like some instant gratification, so step one is to eat the rc files and see if I can parse them. I wasn’t quite sure if /etc/rc.d had subdirectories, but since I had some code already to read all files in a directory and all its subdirectories (from my failed attempt at the RFCs) I just grabbed that and edited it slightly:

sub findRCs {
    my $dir = shift;

    local(*D);
    opendir(D, $dir) || croak "Can't open $dir: $!";
    while(my $f = readdir D) {
	next if $f eq '.' || $f eq '..';
	my $file="$dir/$f";
	if(-d $file) {
	    findRCs($file);
	    next;
	}
	readRC($file);
    }
}

This will call readRC for each file in the provided directory. My first version of readRC looked like this:

sub readRC {
    my $file = shift;

    my $rc = read_file($file);

    my($provide) = $rc =~ /^\# PROVIDE: (\S+)$/m;
    croak "Can't find PROVIDE in $file" if !$provide;

    print "$file: $provide\n";
}

Note that I assume that each file PROVIDEs only one thing, since I match \S+ (i.e. 1 or more non-whitespace characters), and force the matched string to span a whole line. This starts off well

/etc/rc.d/accounting: accounting
/etc/rc.d/amd: amd
/etc/rc.d/addswap: addswap
.
.
.

but ends

Can't find PROVIDE in /etc/rc.d/NETWORKING at ./graph-rc.pl line 13
main::readRC('/etc/rc.d/NETWORKING') called at ./graph-rc.pl line 30
main::findRCs('/etc/rc.d') called at ./graph-rc.pl line 35

oops. If we look at the offending file, we see

# PROVIDE: NETWORKING NETWORK
# REQUIRE: netif netoptions routing network_ipv6 isdnd ppp
# REQUIRE: routed mrouted route6d mroute6d resolv

OK, so it provides two things, it seems. Fair enough, I can fix that, I just have to elaborate the matching slightly

my($provide) = $rc =~ /^\# PROVIDE: (.+)$/m;
croak "Can't find PROVIDE in $file" if !$provide;

my @provide = split /\s/,$provide;

print "$file: ", join (', ',@provide), "\n";

In other words, match everything after PROVIDE: and then split it on whitespace. Notice that this file also has multiple REQUIRE lines – lucky I noticed that, it could easily have escaped my attention. Anyway, after this modification, I can read the whole of /etc/rc.d. Now I need to match the requirements, which I do like this

my(@lrequire) = $rc =~ /^# REQUIRE: (.+)$/mg;
my @require = split /\s/, join(' ', @lrequire);

Another test, just printing what I extracted (print ' ', join (', ',@require), "\n";) and this seems to work fine. So far I’ve only been testing with /etc/rc.d, but now I’m almost ready to start graphing, I also test /usr/local/etc/rc.d

Can't find PROVIDE in /usr/local/etc/rc.d/sshd_localhost.sh at ./graph-rc.pl line 13
main::readRC('/usr/local/etc/rc.d/sshd_localhost.sh') called at ./graph-rc.pl line 35
main::findRCs('/usr/local/etc/rc.d') called at ./graph-rc.pl line 40

OK, so this is a very old rc file of my own and it has no require/provides stuff. In fact, it totally departs from the spec. Whatever … I decide to just skip files that don’t include REQUIRE

    if($rc !~ /PROVIDE/) {
	print STDERR "Skipping $file\n";
	return;
    }

A quick test confirms that it only skips that one file, and now everything works. OK, so time to graph! All I need to do is generate a file in a format Graphviz can read, which is amazingly easy. First I have to output a header

print "digraph rcs {\n";
print " node [fontname=\"Courier\"];\n";

then a line for each dependency

    foreach my $p (@provide) {
	foreach my $r (@require) {
	    print "  $r -> $p; \n";
	}

and finally a trailer

print "}\n";

This produces a file that looks like this

digraph rfcs {
  node [fontname="Courier"];
  mountcritremote -> accounting; 
  rpcbind -> amd; 
  ypbind -> amd; 
  nfsclient -> amd; 
  cleanvar -> amd; 
.
.
.
}

which I can just feed to dot (one of the Graphviz programs), like so

dot -v -Tpng -o ~/tmp/rc.png /tmp/xx

and I get a lovely shiny graph. But while I’m admiring it, I notice that ramdisk has a link to itself, which seems a bit rum. On closer inspection, /etc/rc.d/ramdisk says

# PROVIDE: ramdisk
# REQUIRE: localswap

which doesn’t include a self-reference. Odd. Looking at the output from my script I notice

ramdisk -> ramdisk-own;

Guessing wildly that dot doesn’t like the “-“, I modify the output slightly

    foreach my $p (@provide) {
	foreach my $r (@require) {
	    print "  \"$r\" -> \"$p\"; \n";
	}
    }
}

and bingo, it works. Putting it all together, here’s the final script in full

#!/usr/bin/perl -w

use strict;
use File::Slurp;
use Carp;

sub readRC {
    my $file = shift;

    my $rc = read_file($file);

    if($rc !~ /PROVIDE/) {
	print STDERR "Skipping $file\n";
	return;
    }

    my($provide) = $rc =~ /^\# PROVIDE: (.+)$/m;
    croak "Can't find PROVIDE in $file" if !$provide;
    my @provide = split /\s/, $provide;

    my(@lrequire) = $rc =~ /^# REQUIRE: (.+)$/mg;
    my @require = split /\s/, join(' ', @lrequire);

    foreach my $p (@provide) {
	foreach my $r (@require) {
	    print "  \"$r\" -> \"$p\"; \n";
	}
    }
}

sub findRCs {
    my $dir = shift;

    local(*D);
    opendir(D, $dir) || croak "Can't open $dir: $!";
    while(my $f = readdir D) {
	next if $f eq '.' || $f eq '..';
	my $file="$dir/$f";
	if(-d $file) {
	    findRCs($file);
	    next;
	}
	readRC($file);
    }
}

print "digraph rfcs {\n";
print "  node [fontname=\"Courier\"];\n";

while(my $dir=shift) {
    findRCs($dir);
}

print "}\n";

and running it

./graph-rc.pl /etc/rc.d /usr/local/etc/rc.d > /tmp/xx
dot -v -Tpng -o ~/tmp/rc.png /tmp/xx

And finally, here’s the graph. Interesting that randomness is at the root!

RC dependencies

5 Comments

  1. […] an example of some work I am doing and then at some much latter point in time somebody else does something analogous; where does that fit in Clays little framework. To me this is parallel play (extreme […]

    Pingback by Ascription is an Anathema to any Enthusiasm » Blog Archive » Group kinds and metrics — 26 Mar 2008 @ 15:29

  2. This is very neat. It is possible to get a list of the run order by using rcorder

    rcorder /etc/rc.d/*

    DragonFly and MidnightBSD also have some debugging options to help out frustrated admins. See the -o and -p flags. In fact, -p gives you the provide keywords.

    Comment by Anonymous — 28 Mar 2008 @ 16:36

  3. […] Laurie recently observed “people don’t really talk much about the experience of writing and debugging code.”  […]

    Pingback by Ascription is an Anathema to any Enthusiasm » Blog Archive » clbuild cl-xmpp ejabberd — 31 Mar 2008 @ 13:33

  4. Excellent,Ben. Here’s an odd-but-true observation: at first glance it looked like a quick freehand pencil drawing of a tank (as in military). Second glance, a freehand drawing of a huge ship, an ocean tanker of sorts.

    Is there a simple script for those of a less tech bent to use this? Sure would be interesting. Tank you. I mean T*H*ank you.

    –Dean

    Comment by Dean Landsman — 9 Apr 2008 @ 17:01

  5. […] Laurie recently observed “people don’t really talk much about the experience of writing and debugging code.” I […]

    Pingback by Ascription is an Anathema to any Enthusiasm › clbuild cl-xmpp ejabberd — 12 Apr 2009 @ 14:56

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress