Ben Laurie blathering

19 Jun 2008

FF3: Better Late Than Never

Filed under: Open Source — Ben @ 14:38

Apparently there’s a launch party for Firefox 3 in London, open to all. Tonight.

15 May 2008

Debian and OpenSSL: The Last Word?

Filed under: Open Source,Programming,Security — Ben @ 15:59

I am reliably informed that, despite my previous claim, at least one member of the OpenSSL team does read openssl-dev religiously. For which he should be commended. I read it sometimes, too, but not religiously.

So, forget I said that you don’t reach the OpenSSL developers by posting on openssl-dev.

14 May 2008

Debian and OpenSSL: The Aftermath

Filed under: Open Source,Programming,Security — Ben @ 10:09

There have been an astonishing number of comments on my post about the Debian OpenSSL debacle, clearly this is a subject people have strong feelings about. But there are some points raised that need addressing, so here we go.

Firstly, many, many people seem to think that I am opposed to removing the use of uninitialised memory. I am not. As has been pointed out, this leads to undefined behaviour – and whilst that’s probably not a real issue given the current state of compiler technology, I can certainly believe in a future where compilers are clever enough to work out that on some calls the memory is not initialised and take action that might be unfortunate. I would also note in passing that my copy of K&R (second edition) does not discuss this issue, and ISO/IEC 9899, which some have quoted in support, rather post-dates the code in OpenSSL. To be clear, I am now in favour of addressing this issue correctly.

And this leads me to the second point. Many people seem to be confused about what change was actually made. There were, in fact, two changes. The first concerned a function called ssleay_rand_add(). As a developer using OpenSSL you would never call this function directly, but it is usually (unless a custom PRNG has been substituted, as happens in FIPS mode, for example) called indirectly via RAND_add(). This call is the only way entropy can be added to the PRNG’s pool. OpenSSL calls RAND_add() on buffers that may not have been initialised in a couple of places, and this is the cause of the valgrind warnings. However, rather than fix the calls to RAND_add(), the Debian maintainer instead removed the code that added the buffer handed to ssleay_rand_add() to the pool. This meant that the pool ended up with essentially no entropy. Clearly this was a very bad idea.

The second change was in ssleay_rand_bytes(), a function that extracts randomness from the pool into a buffer. Again, applications would access this via RAND_bytes() rather than directly. In this function, the contents of the buffer before it is filled are added to the pool. Once more, this could be uninitialised. The Debian developer also removed this call, and that is fine.

The third point: several people have come to the conclusion that OpenSSL relies on uninitialised memory for entropy. This is not so. OpenSSL gets its entropy from a variety of platform-dependent sources. Uninitialised memory is merely a bonus source of potential entropy, and is not counted as “real” entropy.

Fourthly, I said in my original post that if the Debian maintainer had asked the developers, then we would have advised against such a change. About 50% of the comments on my post point to this conversation on the openssl-dev mailing list. In this thread, the Debian maintainer states his intention to remove for debugging purposes a couple of lines that are “adding an unintialiased buffer to the pool”. In fact, the first line he quotes is the first one I described above, i.e. the only route to adding anything to the pool. Two OpenSSL developers responded, the first saying “use -DPURIFY” and the second saying “if it helps with debugging, I’m in favor of removing them”. Had they been inspired to check carefully what these lines of code actually were, rather than believing the description, then they would, indeed, have noticed the problem and said something, I am sure. But their response can hardly be taken as unconditional endorsement of the change.

Fifthly, I said that openssl-dev was not the way to ensure you had the attention of the OpenSSL team. Many have pointed out that the website says it is the place to discuss the development of OpenSSL, and this is true, it is what it says. But it is wrong. The reality is that the list is used to discuss application development questions and is not reliably read by the development team.

Sixthly, my objection to the fix Debian put in place has been misunderstood. The issue is not that they did not fully reverse their previous patch – as I say above, the second removal is actually fine. My issue is that it was committed to a public repository five days before an advisory was issued. Only a single attacker has to notice that and realise its import in order to start exploiting vulnerable systems – and I will be surprised if that has not happened.

I think that’s about enough clarification. The question is: what should we do to avoid this happening again? Firstly, if package maintainers think they are fixing a bug, then they should try to get it fixed upstream, not fix it locally. Had that been done in this case, there is no doubt none of this would have happened. Secondly, it seems clear that we (the OpenSSL team) need to find a way that people can reliably communicate with us in these kinds of cases.

The problem with the second is that there are a lot of people who think we should assist them, and OpenSSL is spectacularly underfunded compared to most other open source projects of its importance. No-one that I am aware of is paid by their employer to work full-time on it. Despite the widespread use of OpenSSL, almost no-one funds development on it. And, indeed, many commercial companies who absolutely depend on it refuse to even acknowledge publicly that they use it, despite the requirements of the licence, let alone contribute towards it in any way.

I welcome any suggestions to improve this situation.

Incidentally, some of the comments are not exactly what I would consider appropriate, and there’s a lot of repetition. I moderate comments on my blog, but only to remove spam (and the occasional cockup, such as people posting twice, not realising they are being moderated). I do not censor the comments, so don’t blame me for their content!

13 May 2008

Vendors Are Bad For Security

Filed under: Open Source,Programming,Security — Ben @ 14:09

I’ve ranted about this at length before, I’m sure – even in print, in O’Reily’s Open Sources 2. But now Debian have proved me right (again) beyond my wildest expectations. Two years ago, they “fixed” a “problem” in OpenSSL reported by valgrind[1] by removing any possibility of adding any entropy to OpenSSL’s pool of randomness[2].

The result of this is that for the last two years (from Debian’s “Etch” release until now), anyone doing pretty much any crypto on Debian (and hence Ubuntu) has been using easily guessable keys. This includes SSH keys, SSL keys and OpenVPN keys.

What can we learn from this? Firstly, vendors should not be fixing problems (or, really, anything) in open source packages by patching them locally – they should contribute their patches upstream to the package maintainers. Had Debian done this in this case, we (the OpenSSL Team) would have fallen about laughing, and once we had got our breath back, told them what a terrible idea this was. But no, it seems that every vendor wants to “add value” by getting in between the user of the software and its author.

Secondly, if you are going to fix bugs, then you should install this maxim of mine firmly in your head: never fix a bug you don’t understand. I’m not sure I’ve ever put that in writing before, but anyone who’s worked with me will have heard me say it multiple times.

Incidentally, while I am talking about vendors who are bad for security, it saddens me to have to report that FreeBSD, my favourite open source operating system, are also guilty. Not only do they have local patches in their ports system that should clearly be sent upstream, but they also install packages without running the self-tests. This has bitten me twice by installing broken crypto, most recently in the py-openssl package.

[1] Valgrind is a wonderful tool, I recommend it highly.

[2] Valgrind tracks the use of uninitialised memory. Usually it is bad to have any kind of dependency on uninitialised memory, but OpenSSL happens to include a rare case when its OK, or even a good idea: its randomness pool. Adding uninitialised memory to it can do no harm and might do some good, which is why we do it. It does cause irritating errors from some kinds of debugging tools, though, including valgrind and Purify. For that reason, we do have a flag (PURIFY) that removes the offending code. However, the Debian maintainers, instead of tracking down the source of the uninitialised memory instead chose to remove any possibility of adding memory to the pool at all. Clearly they had not understood the bug before fixing it.

P.S. I’d link to the offending patch in Debian’s source repository. If I could find a source repository. But I can’t.


Thanks to Cat Okita, I have now found the repo. Here’s the offending patch. But I have to admit to being astonished again by the fix, which was committed five days before the advisory! Do these guys have no clue whatsoever?

26 Apr 2008

Do We Need Credentica?

Filed under: Anonymity,Crypto,Open Source,Privacy,Security — Ben @ 20:22

I read that IBM have finally contributed Idemix to Higgins.

But … I am puzzled. Everyone knows that the reason Idemix has not been contributed sooner is because it infringes the Credentica patents. At least, so says Stefan – I wouldn’t know, I haven’t checked. But it seems plausible that at least IBM think that’s true.

So, what’s changed? Have IBM decided that Idemix does not infringe? Or did Microsoft let them publish? Or what?

If its the former, then do others agree? And if its the latter, then in what sense is this open source? If IBM have some kind of special permission with regard to the patents, that is of no assistance to the rest of us.

It seems to me that someone needs to do some explaining. But if the outcome is that Idemix really is open source, then what is the relevance of Credentica?

Incidentally, I wanted to take a look at what it is that IBM have actually released, but there doesn’t seem to be anything there.

Can Phorm Intercept SSL?

Filed under: Crypto,Open Source,Privacy — Ben @ 18:24

Someone asked me to comment on a thread over at BadPhorm on SSL interception.

In short, the question is: can someone in Phorm’s position decrypt SSL somehow? The fear is driven by the existence of appliances that do just this. But these appliances need to do one of two special things to work.

The first possibility is where the appliance is deployed in a corporate network to monitor traffic going from browsers inside the corporation to SSL servers outside. In this case, what you do is you have the SSL appliance act as a CA, and you install its CA certificate in each browser’s store of trusted CAs. Then when the appliance sees an SSL request go past it quickly creates (some would say “forges”) a certificate for the server the request is destined for and instead of routing the connection on to the real server, instead answers it itself, using the newly created certificate. Because the browser trusts the appliance’s CA this all looks perfectly fine and it will proceed without a warning. The appliance then creates an outgoing connection to the real server and acts as a proxy between the browser and server, thus getting access to the plaintext of the interaction.

I’d note in passing that in Netronome’s diagram they show a “trust relationship” between the webserver and the SSL appliance. This is not correct. There need be no relationship at all between the webserver and the appliance – indeed it would be fair to say that many a webserver operator would view what the appliance is doing as downright sneaky. Or dishonest, even.

But, in any case, inside the corporation this behaviour seems fair enough to me – they’re paying for the browser, the machine it runs on, the network connection and the employee’s time. I guess they have a right to see the data.

Could Phorm do this? Well, they could try to persuade anyone stupid enough to install a CA certificate of theirs in their browser, and then yes, indeed, this trick would work for them. More of the story: don’t install such certificates. Note that last time I looked if you wanted to register to do online returns for VAT you had to install one of these things. Oops!

Or, they could get certified as a CA and get automatically installed in everyone’s browser. I’m pretty sure, however, that such a use of a CA key would find them in breach of the conditions attached to their certification.

So, in short, Phorm can only do this to people who don’t understand what’s going on – i.e. 99% of Internet users. But not me.

The second scenario is to deploy the SSL interception appliance at the webserver end of the network (at least, this is how its usually done), and have it sniff incoming connections to the webserver. However, to break these connections it needs to have a copy of the webserver’s private key. I’m reasonably confident that the vast majority of webserver operators will not be handing over their private keys to Phorm, so even “99%” users are safe from this attack.

By the way, if you want to see this one in action, then you can: the excellent network sniffer, Wireshark, can do it. Full instructions can be found here. No need to buy an expensive appliance.

25 Apr 2008

Yet Another Version Control System (and an Apache Module)

Filed under: Distributed stuff,Open Source — Ben @ 22:32

I recently finished off mod_digest for Canonical. To you: the guys that make Ubuntu.

In the process I was forced to use yet another distributed version control system, Bazaar. Once I’d figured out that the FreeBSD port was devel/bazaar-ng and not devel/bazaar, I quite liked it. All these systems are turning out to be pretty much the same, so it’s the bells and whistles that matter. In the case of Bazaar the bell (or whistle) I liked was this

$ bzr push
Using saved location: s

Yes! In Monotone, I’m permanently confused about branches and repos and, well, stuff. Mercurial makes me edit a config file to set a default push location. Bazaar remembers what I did last time. How obvious is that?

16 Apr 2008

Nice Review of Caja

Filed under: Capabilities,Open Source,Programming,Security — Ben @ 1:41

Tim Oren posted about Caja.

…this adds up to a very good chance that something that’s right now fairly obscure could turn into a major force in Web 2.0 within months, not years. Because Caja modifies the de facto definition of JavaScript, it would have an immediate impact on any scripts and sites that are doing things regarded as unsafe in the new model. If you’ve got a Web 2.0 based site, get ready for a project to review for ‘Caja-safety’. If the Caja model spreads, then the edges of the sandbox are going to get blurry. Various users and sites will be able to make choices to allow more powerful operations, and figuring out which ones are significant and allow enhanced value could be a fairly torturous product management challenge, and perhaps allow market entry chances for more powerful forms of widgets and Facebook-style ‘apps’.

End of message.

20 Mar 2008

Am I Reassured?

Filed under: Open Source — Ben @ 10:09

Mike Jones (of Microsoft) tells me I was wrong to be worried about Microsoft’s Open Specification Promise. I wasn’t actually that worried, but now I am.

He says

The “analysis” tries to insinuate that since Microsoft doesn’t promise that future revisions of specifications covered by the Open Specification Promise will be automatically covered unless Microsoft is involved in developing them, that it’s not safe to rely on the OSP for current versions either. This is of course false, as the OSP is an irrevocable promise that Microsoft will never sue anyone for using any of the covered specifications (unless they sue Microsoft for using the same specification, which is a normal exception in all such non-assertion covenants).

Clearly the point is not that current specifications might stop being covered. The problem is that if future versions of the specs are not covered then it will become irrelevant that current ones are – there’s no point in implementing a standard that no-one uses anymore. And given Microsoft’s track record in the area of extending specifications until no-one can implement them, this seems like a very real risk.

He then points to a response in the context of OOXML that is supposed to further reassure. But it does the opposite

This section points out that the OSP only applies to listed versions of covered specifications. True, except that we have already committed to extending it to ISO/IEC DIS 29500 when it is approved in our filing with ISO/IEC. For ODF, IBM in their ISP takes the identical approach. Strange how things that seem appropriate for ODF are not appropriate for Open XML.

In other words, Microsoft can do exactly what I am concerned about, except that in the case of OOXML (which I really don’t care about) they’ve promised not to. Nice for word processors, perhaps. Not so nice for security people, who are covered by no such promise.

OSP covers specifications not code

Not true. The OSP is a promise to not assert patents that are necessarily infringed by implementations of covered specifications. Like all similar patent non-asserts (including the Sun and IBM versions for ODF) the promise covers that part of a product that implements that specification (and not other parts that have nothing to do with the specification). While the Sun covenant is silent about conformance to the specification, the OSP allows implementers the freedom to implement any (or all) parts of a covered specification and to the extent they do implement those portions (also known as conform to those parts) they are covered by the promise for those parts. Contrast that to the IBM pledge that requires total conformance and so programming errors or absence of something required by the spec (but not by an implementer’s product) means that the promise is totally void for that product.

I just don’t get this. It starts with “not true” but then goes on to confirm that it is true! That is, the code is only usable insofar as it is used to implement a covered specification. Programming errors would void their promise just like it does IBM’s. And similarly, re-use of the code for a different purpose would also not be covered. In other words, the specification is covered, not the code.

If Microsoft can’t coherently defend themselves against this analysis but still want us to believe it is incorrect, that seems cause for concern, don’t you think?


Despite Kim’s promise in his blog

That doesn’t mean it is trivial to figure out the best legal mecahnisms for making the intellectual property and even the code available to the ecosystem. Lawyers are needed, and it takes a while. But I can guarantee everyone that I have zero intention of hoarding Minimal Disclosure Tokens or turning U-Prove into a proprietary Microsoft technology silo.

Like, it’s 2008, right? Give me a break, guys!

I’ve now heard through several different channels that Microsoft want to “ensure interoperability”. Well. Interoperability with what, I ask? In order for things to be interoperable, they must adhere to a standard. And for Microsoft to ensure interoperability, they have to both licence the intellectual property such that it can only be used in conformance to that standard and they have to control the standard.

I don’t know about you, but that sure sounds like a “proprietary Microsoft technology silo” to me.

13 Mar 2008

Microsoft’s Open Specification Promise

Filed under: Open Source,Programming — Ben @ 20:11

The Software Freedom Law Centre has published an analysis of the OSP. I don’t really care whether the OSP is compatible with the GPL, but their other points are a concern for everyone relying on the OSP, whether they write free software or not.

5 Feb 2008

Caja in the News

Filed under: Capabilities,Open Source,Programming,Security — Ben @ 20:07

It seems MySpace’s developer launch today is causing Caja to get splattered all over the place.

27 Jan 2008

Open Source Is Just Economics

Filed under: General,Open Source,Programming — Ben @ 5:39

A number of conversations I have had recently indicate to me that a lot of the world still doesn’t get what’s behind open source. It’s easy: economics.

The first thing you can trivially explain is why people work on open source at all. This has been a source of a vast amount of speculation, particularly irritatingly by sociologists. Ben Hyde has a fantastic list to which I will only add the explanation I love to hate: geek pride. We do it just to show off to each other.

Nope, it’s all bollocks – the motivation is simple: by solving your common problem together, you reduce your costs. There is absolutely no point in financing five different companies to produce five different products that don’t quite do what you want – far better to tweak the open source thing to do exactly what you need (often expressed as “scratching your itch” around the ASF).

Some people whine that, because this is an option open only to geeks, open source is not really available to completely open participation. Well, kinda. If you aren’t a geek yourself, you can always hire one. What do you mean, you don’t want to spend your money on free stuff? Why not? We all spend our time on it. Time that we could convert into money, if we so chose.

So why don’t we? Because participating in the open source projects we participate in is worth more to us, in purely monetary terms, in the long run. This is why I no longer have much to do with Apache: it does what I need. I have no itch to scratch.

This leads me into the second easily explainable fact. People complain that open source projects don’t care about users. It’s true. They don’t – they care about people who are participating in the costs of producing the software. If you aren’t contributing, why would your voice matter?

Of course, you have to be careful when applying these obvious truths to what you see around you. For example, the presence of companies like Red Hat in the market complicates analysis. They have their own set of economic drivers, including the needs of their customers, which they then apply to the calculation around their participation in various projects. As the reach of open source extends, so do end users actually start to get an indirect say in what happens. But it costs them. Money.

Back in the good old days, it was so much simpler. All it cost me then was time.

25 Jan 2008

Caja, Shindig and OpenSocial

Filed under: Open Source,Programming,Security — Ben @ 23:07

Its been a while since I wrote about Caja but we’ve been working hard on it and it has come along in leaps and bounds, thanks to my excellent team at Google.

Today I’m very pleased to be able to point you at a test gadget container which supports Cajoling of gadgets. This is based on the open source OpenSocial container, Shindig.

Here’s the announcement, and there’s also some documentation on how to get things working with Caja. We’ve even included a couple of malicious gadgets which are defeated by Caja.

Feedback, as always, welcome.

3 Dec 2007

Caja and OpenSocial

Filed under: Capabilities,Open Source,Programming,Security — Ben @ 2:03

An obvious place to use Caja is, of course, in OpenSocial. So, a bunch of us at Google have been experimenting with this use case and the first outcome is an update to the container sample which allows you to try running your gadget Caja-ised (gotta think of a better name for that). We even have instructions on how to Caja-ise your gadget.

We haven’t tried many gadgets yet, but the good news is the example gadgets worked with (almost[1]) no change. It seems clear that more complex gadgets are not likely to survive without at least some change but we don’t yet know how hard that’s going to be. Feedback, as always, welcome! And don’t forget to join the mailing list to discuss it.

[1] Right now, because Caja-ised code gets pushed into its own sandbox, you have to export any functions that need to be visible to the rest of the page (for example functions that get called when you click a button) – right now, you have to explicitly perform that export but we expect to be able to remove that requirement.

16 Nov 2007

Quilt and SVN: A Slightly Unhappy Marriage

Filed under: Open Source,Programming — Ben @ 5:48

Now that Caja is out in the wild, and I can’t use Google’s internal development tools, I find quilt is coming in handy (why not mercurial queues? I’d prefer it, but the version I can easily install is too old, currently). But, surprisingly for a tool that was designed to assist in open source development, it turns out quilt is a bit weird about co-existing with version control systems.

The issue comes when you finally get approval for your patch and you commit it to the tree. At this point, you want to delete it from the patch series – but quilt won’t let you, because it is applied. If you pop it, then you’ll undo what you’ve just committed. So, what to do? Here’s my ad-hoc recipe

quilt pop -a
patch -p1 < patches/the-bottom-patch svn ci quilt delete the-bottom-patch

and there you are, done. You can even do this retroactively if you forgot to do it as you go along – just miss out the svn ci step. Once you’re back up-to-date you should find that you are still in sync with the head of the tree (assuming no-one committed in the meantime).

14 Nov 2007

Caja Code is Available

Filed under: Capabilities,Open Source,Programming,Security — Ben @ 15:51

Yesterday we put the initial (incomplete) version of the Caja code up at

From now on, all development will be done out in the open. External developers are welcome to come and play, too. Join the mailing list. Write code! Find bugs! Laugh at my mistakes! Have fun!

9 Nov 2007

Groklaw Interviews Becky Hogge on the BBC

Filed under: Digital Rights,Open Source — Ben @ 16:02

I’ve only recently started reading Groklaw, but it is fast becoming one of my favourite blogs. Today they have an interview with Becky Hogge, Executive Director of the Open Rights Group, on the BBC’s iPlayer and rights strategies.

She rightly distances herself from the folderol over BBC’s relationship with Microsoft and focusses on the bigger issues

Q: OK. Now, it was widely reported that the BBC signed a letter of intent with Microsoft which covered the iPlayer, DRM, and other cooperation. Have you seen the document? Is the document available? Do you know what it says?

Becky Hogge: I don’t know what it says, I haven’t seen it, and I don’t know if it’s available. Like I say, the Open Rights Group, we’re trying to move away from this Microsoft issue and look further into the future for the BBC. The BBC has got itself into a really sticky situation with iPlayer and with DRM, and I think it must be feeling bad at this point. What the Open Rights Group are trying to say here is that yes, these problems are real, a lot of our supporter base are using Linux operating systems and even though they’re paying their license fee, they’re unable to access iPlayer services. But we’d like to find solutions for the BBC, rather than more problems. And our big solution is that it needs to start reexamining the rights models. For the sake of public broadcast.

8 Nov 2007

Is The GPL Open Source’s DRM?

Filed under: Open Source — Ben @ 6:08

I was trying to explain the difference between BSD and GPL to a non-open source person the other day and halfway through I suddenly realised that what I was saying sounded just like DRM.

What does DRM do? It seeks to control the ways in which the recipient of some content can use that content.

What does the GPL do? It seeks to control the ways in which the recipient of some code can use that code.

Just sayin’.

5 Nov 2007

Ancient History

Filed under: Open Source — Ben @ 16:17

A new, and rather nice, mail archive searching thingy has been launched. Out of curiosity, I used it to try to find the lost-in-the-mists-of-time birthdate of Apache-SSL. I did – it’s the 16th of October, 1995. Roughly.

Along the way, somewhat narcissisticly, I found a few other things. My first post to the httpd-dev list, admittedly forwarded, on the 5th of September 1995. I had no idea I’d started on SSL so soon after joining the Apache project.

Finally, me arguing against using the GPL (Brian Behlendorf has sometimes claimed that my opposition to the GPL was one of the reasons Apache started with a BSD-style licence).

« Previous PageNext Page »

Powered by WordPress