Ben Laurie blathering

31 Mar 2008

More Bullshit from Phorm

Filed under: Anonymity/Privacy,Digital Rights,Security — Ben @ 14:54

Phorm continue to sob that us whining privacy advocates are misrepresenting their system

Phorm’s chairman and chief executive, Kent Ertugrul, said yesterday the firm was the victim of misinformation. “What is so strange about this is that if you were to put on a board what we do and what has been written about us and map the two, you would find there is very little correlation,” he said.

I’d be more than happy to compare what I’ve said to what their system actually does, only … when the Open Rights Group nominated me to be briefed by Phorm (in my capacity as both a director of ORG and a subject matter expert) they declined, on the basis that I work for a competitor, despite my assurance that I would not be acting for Google in any way, as is always the case when I do stuff for ORG. But, hey, trust is a one-way street, apparently, if you are Phorm – as one of the surveilled, I must trust them, but that’s no reason they should trust me, is it?

Strangely they were quite happy to brief two of my colleagues in detail, without any NDA – and my colleagues are planning to produce a full, public report of that briefing. With a bit of luck, they’ll have addressed all my concerns, but who knows? I wasn’t there to assist in that process.

Interestingly, they go on to say

“What we would like to do is issue a challenge to the privacy community to select some of their most technically savvy representatives and form an inspection committee. We would be delighted, on a recurring basis, to give those people the ability to spot inspect what it is we do.”

which rather emphasizes one of the core problems with their system: it requires everyone to trust that all this data they have gathered without consent is actually handled as they claim it is handled.

I do hope Phorm will be paying the going rate for this valuable service – but probably I won’t find out because I expect that, despite my obvious qualifications, I will be excluded from such a group. It wouldn’t do to have anyone too expert looking at their system, after all.

30 Mar 2008

Microsoft Implement The Evil Bit

Filed under: Anonymity/Privacy,Security — Ben @ 18:41

Thanks to the Shindig mailing list, I’ve just noticed this gem from Microsoft.

The essence here is that third party sites inside frames might invade your privacy by setting cookies, so IE6, by default, doesn’t let them set cookies. But, if they promise to be good, then it will allow them to be bad. Isn’t that marvellous?

What I think is particularly excellent about Microsoft’s support article is that they tell you how to suppress the behaviour by setting an appropriate P3P policy … but they don’t tell you what this policy really means, nor suggest that you should only set the policy if you actually conform to it.

Of course, you can tell it’s a Microsoft protocol because it takes 21 bytes to do what the original proposal could do in a single bit.

25 Mar 2008

Federated Messaging Meets Federated Identity

Filed under: Distributed stuff,Identity Management — Ben @ 23:51

XMPP, OAuth and OpenID. Social networking in real-time. Interesting. Peter Saint-Andre thinks we should talk about it.

Sign up here.

24 Mar 2008

Fun With Dot

Filed under: Programming — Ben @ 14:59

As I’ve mentioned before, people don’t really talk much about the experience of writing and debugging code, so here’s another installation in my occasional series on doing just that.

Over the Easter weekend the weather has been pretty horrible, so, instead of having fun on my motorbike, I’ve been amusing myself in various ways: trying to finish up a paper I started last year, doing further work on OpenSSL and Deputy, cooking, playing with Tahoe (on which more later), updating my FreeBSD machines, and messing around with Graphviz.

Graphviz is one of my favourite toys. Basically, it lets you specify how a bunch of things are connected, and then it will draw them for you. A project I’d long had in my head was to take all the RFCs, work out which ones reference which other ones and have Graphviz draw it for me. Getting the data turned out to be pretty easy, but unfortunately the resulting dataset proves to be too much for poor old Graphviz, exposing all sorts of bugs in its drawing engine, which led to core dumps and/or complete garbage in the output files. Shame, early experiments promised some quite pretty output. Anyway, after banging my head against that for many hours, I gave up and instead did something I do every few months: got my various FreeBSD machines up-to-date. As part of that process, I had to look at the stuff that FreeBSD runs at startup, configured in /etc/rc.conf (and /etc/defaults/rc.conf), and actually done by scripts in /etc/rc.d and /usr/local/etc/rc.d.

This reminded me that these scripts expose their dependencies in comments, like this (from /etc/rc.d/pfsync)

# PROVIDE: pfsync

So, I thought it would be fun to graph those dependencies – then at least I’d have one pretty thing to show for the weekend. Then, since it only took 15 minutes to do, I thought it might make an interesting subject for a post on how I go about coding such things.

So, first things first, I like some instant gratification, so step one is to eat the rc files and see if I can parse them. I wasn’t quite sure if /etc/rc.d had subdirectories, but since I had some code already to read all files in a directory and all its subdirectories (from my failed attempt at the RFCs) I just grabbed that and edited it slightly:

sub findRCs {
    my $dir = shift;

    opendir(D, $dir) || croak "Can't open $dir: $!";
    while(my $f = readdir D) {
	next if $f eq '.' || $f eq '..';
	my $file="$dir/$f";
	if(-d $file) {

This will call readRC for each file in the provided directory. My first version of readRC looked like this:

sub readRC {
    my $file = shift;

    my $rc = read_file($file);

    my($provide) = $rc =~ /^\# PROVIDE: (\S+)$/m;
    croak "Can't find PROVIDE in $file" if !$provide;

    print "$file: $provide\n";

Note that I assume that each file PROVIDEs only one thing, since I match \S+ (i.e. 1 or more non-whitespace characters), and force the matched string to span a whole line. This starts off well

/etc/rc.d/accounting: accounting
/etc/rc.d/amd: amd
/etc/rc.d/addswap: addswap

but ends

Can't find PROVIDE in /etc/rc.d/NETWORKING at ./ line 13
main::readRC('/etc/rc.d/NETWORKING') called at ./ line 30
main::findRCs('/etc/rc.d') called at ./ line 35

oops. If we look at the offending file, we see

# REQUIRE: netif netoptions routing network_ipv6 isdnd ppp
# REQUIRE: routed mrouted route6d mroute6d resolv

OK, so it provides two things, it seems. Fair enough, I can fix that, I just have to elaborate the matching slightly

my($provide) = $rc =~ /^\# PROVIDE: (.+)$/m;
croak "Can't find PROVIDE in $file" if !$provide;

my @provide = split /\s/,$provide;

print "$file: ", join (', ',@provide), "\n";

In other words, match everything after PROVIDE: and then split it on whitespace. Notice that this file also has multiple REQUIRE lines – lucky I noticed that, it could easily have escaped my attention. Anyway, after this modification, I can read the whole of /etc/rc.d. Now I need to match the requirements, which I do like this

my(@lrequire) = $rc =~ /^# REQUIRE: (.+)$/mg;
my @require = split /\s/, join(' ', @lrequire);

Another test, just printing what I extracted (print ' ', join (', ',@require), "\n";) and this seems to work fine. So far I’ve only been testing with /etc/rc.d, but now I’m almost ready to start graphing, I also test /usr/local/etc/rc.d

Can't find PROVIDE in /usr/local/etc/rc.d/ at ./ line 13
main::readRC('/usr/local/etc/rc.d/') called at ./ line 35
main::findRCs('/usr/local/etc/rc.d') called at ./ line 40

OK, so this is a very old rc file of my own and it has no require/provides stuff. In fact, it totally departs from the spec. Whatever … I decide to just skip files that don’t include REQUIRE

    if($rc !~ /PROVIDE/) {
	print STDERR "Skipping $file\n";

A quick test confirms that it only skips that one file, and now everything works. OK, so time to graph! All I need to do is generate a file in a format Graphviz can read, which is amazingly easy. First I have to output a header

print "digraph rcs {\n";
print " node [fontname=\"Courier\"];\n";

then a line for each dependency

    foreach my $p (@provide) {
	foreach my $r (@require) {
	    print "  $r -> $p; \n";

and finally a trailer

print "}\n";

This produces a file that looks like this

digraph rfcs {
  node [fontname="Courier"];
  mountcritremote -> accounting; 
  rpcbind -> amd; 
  ypbind -> amd; 
  nfsclient -> amd; 
  cleanvar -> amd; 

which I can just feed to dot (one of the Graphviz programs), like so

dot -v -Tpng -o ~/tmp/rc.png /tmp/xx

and I get a lovely shiny graph. But while I’m admiring it, I notice that ramdisk has a link to itself, which seems a bit rum. On closer inspection, /etc/rc.d/ramdisk says

# PROVIDE: ramdisk
# REQUIRE: localswap

which doesn’t include a self-reference. Odd. Looking at the output from my script I notice

ramdisk -> ramdisk-own;

Guessing wildly that dot doesn’t like the “-“, I modify the output slightly

    foreach my $p (@provide) {
	foreach my $r (@require) {
	    print "  \"$r\" -> \"$p\"; \n";

and bingo, it works. Putting it all together, here’s the final script in full

#!/usr/bin/perl -w

use strict;
use File::Slurp;
use Carp;

sub readRC {
    my $file = shift;

    my $rc = read_file($file);

    if($rc !~ /PROVIDE/) {
	print STDERR "Skipping $file\n";

    my($provide) = $rc =~ /^\# PROVIDE: (.+)$/m;
    croak "Can't find PROVIDE in $file" if !$provide;
    my @provide = split /\s/, $provide;

    my(@lrequire) = $rc =~ /^# REQUIRE: (.+)$/mg;
    my @require = split /\s/, join(' ', @lrequire);

    foreach my $p (@provide) {
	foreach my $r (@require) {
	    print "  \"$r\" -> \"$p\"; \n";

sub findRCs {
    my $dir = shift;

    opendir(D, $dir) || croak "Can't open $dir: $!";
    while(my $f = readdir D) {
	next if $f eq '.' || $f eq '..';
	my $file="$dir/$f";
	if(-d $file) {

print "digraph rfcs {\n";
print "  node [fontname=\"Courier\"];\n";

while(my $dir=shift) {

print "}\n";

and running it

./ /etc/rc.d /usr/local/etc/rc.d > /tmp/xx
dot -v -Tpng -o ~/tmp/rc.png /tmp/xx

And finally, here’s the graph. Interesting that randomness is at the root!

RC dependencies

20 Mar 2008

Am I Reassured?

Filed under: Open Source — Ben @ 10:09

Mike Jones (of Microsoft) tells me I was wrong to be worried about Microsoft’s Open Specification Promise. I wasn’t actually that worried, but now I am.

He says

The “analysis” tries to insinuate that since Microsoft doesn’t promise that future revisions of specifications covered by the Open Specification Promise will be automatically covered unless Microsoft is involved in developing them, that it’s not safe to rely on the OSP for current versions either. This is of course false, as the OSP is an irrevocable promise that Microsoft will never sue anyone for using any of the covered specifications (unless they sue Microsoft for using the same specification, which is a normal exception in all such non-assertion covenants).

Clearly the point is not that current specifications might stop being covered. The problem is that if future versions of the specs are not covered then it will become irrelevant that current ones are – there’s no point in implementing a standard that no-one uses anymore. And given Microsoft’s track record in the area of extending specifications until no-one can implement them, this seems like a very real risk.

He then points to a response in the context of OOXML that is supposed to further reassure. But it does the opposite

This section points out that the OSP only applies to listed versions of covered specifications. True, except that we have already committed to extending it to ISO/IEC DIS 29500 when it is approved in our filing with ISO/IEC. For ODF, IBM in their ISP takes the identical approach. Strange how things that seem appropriate for ODF are not appropriate for Open XML.

In other words, Microsoft can do exactly what I am concerned about, except that in the case of OOXML (which I really don’t care about) they’ve promised not to. Nice for word processors, perhaps. Not so nice for security people, who are covered by no such promise.

OSP covers specifications not code

Not true. The OSP is a promise to not assert patents that are necessarily infringed by implementations of covered specifications. Like all similar patent non-asserts (including the Sun and IBM versions for ODF) the promise covers that part of a product that implements that specification (and not other parts that have nothing to do with the specification). While the Sun covenant is silent about conformance to the specification, the OSP allows implementers the freedom to implement any (or all) parts of a covered specification and to the extent they do implement those portions (also known as conform to those parts) they are covered by the promise for those parts. Contrast that to the IBM pledge that requires total conformance and so programming errors or absence of something required by the spec (but not by an implementer’s product) means that the promise is totally void for that product.

I just don’t get this. It starts with “not true” but then goes on to confirm that it is true! That is, the code is only usable insofar as it is used to implement a covered specification. Programming errors would void their promise just like it does IBM’s. And similarly, re-use of the code for a different purpose would also not be covered. In other words, the specification is covered, not the code.

If Microsoft can’t coherently defend themselves against this analysis but still want us to believe it is incorrect, that seems cause for concern, don’t you think?


Despite Kim’s promise in his blog

That doesn’t mean it is trivial to figure out the best legal mecahnisms for making the intellectual property and even the code available to the ecosystem. Lawyers are needed, and it takes a while. But I can guarantee everyone that I have zero intention of hoarding Minimal Disclosure Tokens or turning U-Prove into a proprietary Microsoft technology silo.

Like, it’s 2008, right? Give me a break, guys!

I’ve now heard through several different channels that Microsoft want to “ensure interoperability”. Well. Interoperability with what, I ask? In order for things to be interoperable, they must adhere to a standard. And for Microsoft to ensure interoperability, they have to both licence the intellectual property such that it can only be used in conformance to that standard and they have to control the standard.

I don’t know about you, but that sure sounds like a “proprietary Microsoft technology silo” to me.

18 Mar 2008

Pork Chops in Cider

Filed under: Recipes — Ben @ 13:33

This recipe was originally cooked for me by my friend Honk. Well, something somewhat like it was.

pork chops
mixed herbs
olive oil
more garlic
cooking apples

Smear the pork chops with crushed garlic, mixed herbs, salt and pepper. I’m lazy, so I only do one side. Grill them so they’re nicely browned on each side but not cooked all the way through. Starting with them slightly frozen might help.

Meanwhile, roughly chop the onions and cook them in olive oil until sweet. Add butter, sliced mushrooms, more crushed garlic, salt and pepper. Be generous with the butter. Cook the mushrooms very gently until they’re soft and laden with butter. Add cider, quarter of a can at a time, reducing it to almost nothing each time. Somewhere in there, add sliced cooking apple. When the apple is nearly soft and you still have some cider left, add the grilled pork chops to this mess. Cook them in the final reduction of cider. Once that’s done, turn off the heat and add cream.

Eat. And feel fat.

For guidance for 4 pork chops I used four flat mushrooms, three 750 cl cans of cider and two cooking apples. Cream I leave to your conscience. Obviously the chops need some fat on them or this will be totally boring. In fact, I lust after the chops I once saw in The Ginger Pig which were over half fat. Yum.

Sauteed potatoes are good with this. And peas if you’re lazy. Spinach if less so.

13 Mar 2008

Microsoft’s Open Specification Promise

Filed under: Open Source,Programming — Ben @ 20:11

The Software Freedom Law Centre has published an analysis of the OSP. I don’t really care whether the OSP is compatible with the GPL, but their other points are a concern for everyone relying on the OSP, whether they write free software or not.

Bad Phorm?

Filed under: Anonymity/Privacy,Security — Ben @ 14:28

As anyone even half-awake knows, there has been a storm of protest over Phorm. I won’t reiterate the basic arguments, but I am intrigued by a couple of inconsistencies and/or misleading statements I’m spotting from Phorm’s techies.

In an interview in The Register, Phorm’s “top boffin” Marc Burgess says

What the profiler does is it first cleans the data. It’s looking at two sets of information: the information in the request that’s sent to the website and then information in the page that comes back.

From the request it pulls out the URL, and if that URL is a well known search engine such as Google or Yahoo! it’ll also look for the search terms that are in the request.

And then from the information returned by the website, the profiler looks at the content. The first thing it does is it ignores several classes of information that could potentially be sensitive. So there’s no form fields, no numbers, no email addresses (that is something containing an “@”) and anything containing a title like Mr or Mrs.

he says “there’s no form fields”. But this is in the response from the webserver. Form fields in the request sent to the webserver are fair game, it seems. In other words, Phorm are quite happy to spy on what you type, but will ignore form fields sent to you by the server – well, that’s big of them: those fields are usually empty. It’s interesting that many people have picked this up as a contradiction (that is, how can there be no form fields if you are looking at search terms, which are entered into a form field?) – but it has been carefully worded so that it is not contradictory, just easy to misinterpret.

Phorm can completely adhere to this public statement and yet still look at everything you typed. Note also that they only talk about filtering senstive data in the response and not in the request. So nothing, it seems, is really sacred.

Incidentally, they are silent about what they do with the body of the request (usually when you submit a form, the fields end up in the body rather than the URL). That fills me with curiosity.

Even ORG swallow this bit of propaganda (from ORG’s post)

Phorm assigns a user’s browser a unique identifying number, which, it is claimed, nobody can associate with your IP address, not even your ISP.

Of course, this is nonsense. The ISP can easily associate the identifying number with your IP address – all they have to do is watch the traffic and see which IP address sends the cookie with the Phorm ID in it. In fact, they could probably use the Phorm box for this, since it already sees all the data.

and Phorm’s CEO, Kent Ertegrul, again in the interview with The Register

It’s important to understand the distinction between actually recording stuff and concluding stuff. All of our systems sit inside BT’s network. Phorm has no way of going into the system and querying “what was cookie 1000062 doing?”. And even if we did we have no way of knowing who 1000062 was. And even if we did all we could pull out of it is product categories. There’s just no way of understanding where you’ve been, what you’ve done, what you’ve searched for.

They say this, but we have to take their word for it. Obviously the fact it sits inside BT’s network is no barrier to them connecting to it. Clearly they could just look at the traffic traversing the system and know exactly what cookie 1000062 is doing. And which IP address is doing it, which doesn’t tell you who is doing it, but certainly narrows it down. Analysis of the data will almost certainly allow identification of the individual concerned, of course.

Not, of course, that taking people’s word for their privacy practices is unacceptable – it is pretty much unavoidable. What I object to is Phorm’s attempts to convince us that it is impossible for them to misbehave. Of course, it is not.

Now let’s take a look at BT’s FAQ

Is my data still viewed when I am not participating?

No, when you don’t participate or switch the system off — it’s off. 100%. No browsing data whatsoever is looked at or processed by BT Webwise. . We should be clear: the Phorm servers are located in BT’s network and browsing data is not transmitted outside. Even if you are opted out, websites will still show you ads (as they do now) but these will not be ads from the Phorm service and they will not be more relevant to your browsing. In addition, you will also not get extra protection from fraudulent websites.

This is just obviously a lie. Since opt-out is controlled by a cookie, the system must look at your browsing data in order to determine whether you have the opt-out cookie or not. Naughty BT.

Furthermore, it is difficult to imagine how they could architect a system where your data did not traverse some box doing interception, though it may, of course, decide not to look at that data. But once more we’d have to take their word for it. How can we ever be sure they are not? Only by having our data not go to the box at all.

Talk Talk say they are going to architect their system in this way, somewhere in the comments on this post. I await details with interest – I can’t see how they can do it, except by either pushing the traffic through some other interception box, which doesn’t really change the situation at all, or by choosing whether to send to the Phorm box on the basis of IP address – which does not identify the user, so, for example, I could find myself opted-in by my children, without my knowledge!

All these worries apply to the system working as intended. What would happen if the Phorm box got pwned, I dread to think. I hope they’ve done their homework on hardening it! Of course, since they have “no access to the system”, it’ll be interesting to see how they plan to keep it up-to-date as attacks against it evolve.

11 Mar 2008

RFC 5155

Filed under: Anonymity/Privacy,Crypto,Distributed stuff — Ben @ 11:43

After nearly 4 years of mind-bending minutiae of DNS (who would’ve thought it could be so complicated?), political wrangling and the able assistance of many members of the DNSSEC Working Group, particularly my co-authors, Roy Arends, Geoff Sisson and David Blacka, the Internet Draft I started in April 2004, “DNSSEC NSEC2 Owner and RDATA Format (or; avoiding zone traversal using NSEC)” now known as “DNS Security (DNSSEC) Hashed Authenticated Denial of Existence” has become RFC 5155. Not my first RFC, but my first Standards Track RFC. So proud!

Matasano Chargen explain why this RFC is needed, complete with pretty pictures. They don’t say why its complicated, though. The central problem is that although we all think of DNS as a layered system neatly corresponding to the dots in the name, it isn’t.

So, you might like to think, and it is often explained this way, that when I look up I first ask the servers for . who the nameserver for com is. Then I ask the com nameservers where the nameservers for is, who I then ask for the nameservers for and finally ask them for the address of

But it isn’t as easy as that. In fact, the zone can contain an entry without delegating This makes proving the non-existence of a name by showing the surrounding pair rather more challenging. The non-cryptographic version (NSEC) solved it by cunningly ordering the names so that names that were “lower” in the tree came immediately after their parents. Like this:

So, proving that, say, doesn’t exist means showing the pair (, Note that this pair does not prove the nonexistence of as you might expect from a simple lexical ordering. Unfortunately, once you’ve hashed a name, you’ve lost information about how many components there were in the name and so forth, so this cunning trick doesn’t work for NSEC3.

It turns out that in general, to prove the nonexistence of a name using NSEC you have to show at most two records, one to prove the name itself doesn’t exist, and the other to show that you didn’t delegate some parent of it. Often the same record can do both.

In NSEC3, it turns out, you have to show at most three records. And if you can understand why, then you understand DNS better than almost anyone else on the planet.

6 Mar 2008

Microsoft Buys Credentica

Kim and Stefan blog about Microsoft’s acquisition of Stefan’s selective disclosure patents and technologies, which I’ve blogged about many times before.

This is potentially great news, especially if one interprets Kim’s

Our goal is that Minimal Disclosure Tokens will become base features of identity platforms and products, leading to the safest possible intenet. I don’t think the point here is ultimately to make a dollar. It’s about building a system of identity that can withstand the ravages that the Internet will unleash.

in the most positive way. Unfortunately, comments such as this from Stefan

Microsoft plans to integrate the technology into Windows Communication Foundation and Windows Cardspace.

and this from Microsoft’s Privacy folk

When this technology is broadly available in Microsoft products (such as Windows Communication Foundation and Windows Cardspace), enterprises, governments, and consumers all stand to benefit from the enhanced security and privacy that it will enable.

sound more like the Microsoft we know and love.

I await developments with interest.

2 Mar 2008

Users Are Stupid…

Filed under: Rants,Security — Ben @ 16:08

… and they won’t take sensible security decisions, so we have to dumb everything down for them. Or at least, that’s what we whine. So, I have to ask, WTF is this all about?

Stupid Dialog

How is anyone supposed to figure out what to do now? Surely we know which of those three errors it was? So why are we giving the user such appallingly crap feeback?

And I can totally imagine how the conversation with Majestic’s webmaster is going to go…

Me: Hey, I got this error. Apparently you’re either using a CA that’s not in a list I’m not going to give to you, or you screwed up your server config so your certificate is incomplete, or you are a phisher and I’ve just given you my password, or you aren’t a phisher but your cert and website don’t match. Please fix it.

Webmaster: O RLY? Well, I’ve got this email that says your wine order is either in the post, not in the post, cancelled or coming to me instead. Laugh it up, big boy.

Powered by WordPress