As I’ve mentioned before, people don’t really talk much about the experience of writing and debugging code, so here’s another installation in my occasional series on doing just that.
Over the Easter weekend the weather has been pretty horrible, so, instead of having fun on my motorbike, I’ve been amusing myself in various ways: trying to finish up a paper I started last year, doing further work on OpenSSL and Deputy, cooking, playing with Tahoe (on which more later), updating my FreeBSD machines, and messing around with Graphviz.
Graphviz is one of my favourite toys. Basically, it lets you specify how a bunch of things are connected, and then it will draw them for you. A project I’d long had in my head was to take all the RFCs, work out which ones reference which other ones and have Graphviz draw it for me. Getting the data turned out to be pretty easy, but unfortunately the resulting dataset proves to be too much for poor old Graphviz, exposing all sorts of bugs in its drawing engine, which led to core dumps and/or complete garbage in the output files. Shame, early experiments promised some quite pretty output. Anyway, after banging my head against that for many hours, I gave up and instead did something I do every few months: got my various FreeBSD machines up-to-date. As part of that process, I had to look at the stuff that FreeBSD runs at startup, configured in /etc/rc.conf (and /etc/defaults/rc.conf), and actually done by scripts in /etc/rc.d and /usr/local/etc/rc.d.
This reminded me that these scripts expose their dependencies in comments, like this (from /etc/rc.d/pfsync)
# PROVIDE: pfsync
# REQUIRE: root FILESYSTEMS netif
So, I thought it would be fun to graph those dependencies - then at least I’d have one pretty thing to show for the weekend. Then, since it only took 15 minutes to do, I thought it might make an interesting subject for a post on how I go about coding such things.
So, first things first, I like some instant gratification, so step one is to eat the rc files and see if I can parse them. I wasn’t quite sure if /etc/rc.d had subdirectories, but since I had some code already to read all files in a directory and all its subdirectories (from my failed attempt at the RFCs) I just grabbed that and edited it slightly:
sub findRCs {
my $dir = shift;
local(*D);
opendir(D, $dir) || croak "Can't open $dir: $!";
while(my $f = readdir D) {
next if $f eq '.' || $f eq '..';
my $file="$dir/$f";
if(-d $file) {
findRCs($file);
next;
}
readRC($file);
}
}
This will call readRC for each file in the provided directory. My first version of readRC looked like this:
sub readRC {
my $file = shift;
my $rc = read_file($file);
my($provide) = $rc =~ /^\# PROVIDE: (\S+)$/m;
croak "Can't find PROVIDE in $file" if !$provide;
print "$file: $provide\n";
}
Note that I assume that each file PROVIDEs only one thing, since I match \S+ (i.e. 1 or more non-whitespace characters), and force the matched string to span a whole line. This starts off well
/etc/rc.d/accounting: accounting
/etc/rc.d/amd: amd
/etc/rc.d/addswap: addswap
.
.
.
but ends
Can't find PROVIDE in /etc/rc.d/NETWORKING at ./graph-rc.pl line 13
main::readRC('/etc/rc.d/NETWORKING') called at ./graph-rc.pl line 30
main::findRCs('/etc/rc.d') called at ./graph-rc.pl line 35
oops. If we look at the offending file, we see
# PROVIDE: NETWORKING NETWORK
# REQUIRE: netif netoptions routing network_ipv6 isdnd ppp
# REQUIRE: routed mrouted route6d mroute6d resolv
OK, so it provides two things, it seems. Fair enough, I can fix that, I just have to elaborate the matching slightly
my($provide) = $rc =~ /^\# PROVIDE: (.+)$/m;
croak “Can’t find PROVIDE in $file” if !$provide;
my @provide = split /\s/,$provide;
print “$file: “, join (’, ‘,@provide), “\n”;
In other words, match everything after PROVIDE: and then split it on whitespace. Notice that this file also has multiple REQUIRE lines - lucky I noticed that, it could easily have escaped my attention. Anyway, after this modification, I can read the whole of /etc/rc.d. Now I need to match the requirements, which I do like this
my(@lrequire) = $rc =~ /^# REQUIRE: (.+)$/mg;
my @require = split /\s/, join(' ', @lrequire);
Another test, just printing what I extracted (print ' ', join (', ',@require), "\n";) and this seems to work fine. So far I’ve only been testing with /etc/rc.d, but now I’m almost ready to start graphing, I also test /usr/local/etc/rc.d…
Can't find PROVIDE in /usr/local/etc/rc.d/sshd_localhost.sh at ./graph-rc.pl line 13
main::readRC('/usr/local/etc/rc.d/sshd_localhost.sh') called at ./graph-rc.pl line 35
main::findRCs('/usr/local/etc/rc.d') called at ./graph-rc.pl line 40
OK, so this is a very old rc file of my own and it has no require/provides stuff. In fact, it totally departs from the spec. Whatever … I decide to just skip files that don’t include REQUIRE
if($rc !~ /PROVIDE/) {
print STDERR "Skipping $file\n";
return;
}
A quick test confirms that it only skips that one file, and now everything works. OK, so time to graph! All I need to do is generate a file in a format Graphviz can read, which is amazingly easy. First I have to output a header
print "digraph rcs {\n";
print " node [fontname=\"Courier\"];\n”;
then a line for each dependency
foreach my $p (@provide) {
foreach my $r (@require) {
print " $r -> $p; \n";
}
and finally a trailer
print "}\n";
This produces a file that looks like this
digraph rfcs {
node [fontname="Courier"];
mountcritremote -> accounting;
rpcbind -> amd;
ypbind -> amd;
nfsclient -> amd;
cleanvar -> amd;
.
.
.
}
which I can just feed to dot (one of the Graphviz programs), like so
dot -v -Tpng -o ~/tmp/rc.png /tmp/xx
and I get a lovely shiny graph. But while I’m admiring it, I notice that ramdisk has a link to itself, which seems a bit rum. On closer inspection, /etc/rc.d/ramdisk says
# PROVIDE: ramdisk
# REQUIRE: localswap
which doesn’t include a self-reference. Odd. Looking at the output from my script I notice
ramdisk -> ramdisk-own;
Guessing wildly that dot doesn’t like the “-”, I modify the output slightly
foreach my $p (@provide) {
foreach my $r (@require) {
print " \"$r\" -> \"$p\"; \n";
}
}
}
and bingo, it works. Putting it all together, here’s the final script in full
#!/usr/bin/perl -w
use strict;
use File::Slurp;
use Carp;
sub readRC {
my $file = shift;
my $rc = read_file($file);
if($rc !~ /PROVIDE/) {
print STDERR "Skipping $file\n";
return;
}
my($provide) = $rc =~ /^\# PROVIDE: (.+)$/m;
croak "Can't find PROVIDE in $file" if !$provide;
my @provide = split /\s/, $provide;
my(@lrequire) = $rc =~ /^# REQUIRE: (.+)$/mg;
my @require = split /\s/, join(' ', @lrequire);
foreach my $p (@provide) {
foreach my $r (@require) {
print " \"$r\" -> \"$p\"; \n";
}
}
}
sub findRCs {
my $dir = shift;
local(*D);
opendir(D, $dir) || croak "Can't open $dir: $!";
while(my $f = readdir D) {
next if $f eq '.' || $f eq '..';
my $file="$dir/$f";
if(-d $file) {
findRCs($file);
next;
}
readRC($file);
}
}
print "digraph rfcs {\n";
print " node [fontname=\"Courier\"];\n”;
while(my $dir=shift) {
findRCs($dir);
}
print “}\n”;
and running it
./graph-rc.pl /etc/rc.d /usr/local/etc/rc.d > /tmp/xx
dot -v -Tpng -o ~/tmp/rc.png /tmp/xx
And finally, here’s the graph. Interesting that randomness is at the root!
Share This
