Ben Laurie blathering

Intent Is The Problem

Of late, I keep banging into the problem that people want systems to be “secure by default”: they don’t want to pester the user about security. They want the system to just do the right thing. The problem is, this just isn’t possible. One example I like to give is “rm -rf *“. Clearly this command is sometimes a very bad idea, and sometimes exactly what you want to do. If some piece of code I mistakenly trusted runs that command on my behalf, I might be very sad about it. Therefore, any system that wants to be “secure” has to somehow know that when I move to some directory and type rm -rf * I mean it, and when I run a piece of code I’m expecting to (say) edit some text, I don’t mean it, and it should not be allowed to do it.

How can the system discover this? Clearly it must be through some user action. The user must behave differently in some way in the two cases, so that the system can discover his intent. Therefore it is impossible to be “secure” without, in some way, consulting the user about his intent.

Obviously we can try to minimise the intrusiveness of the consultation – for example, this is the impetus behind the “designation is authorisation” paradigm that is so natural in capability systems. But we cannot make it go away.

ChromeOS provides us with some interesting examples. If we are going to have an operating system that only lets you use a browser, then clearly we’re going to have to let that browser do some things we would not normally expect a browser to do, like access the webcam or interact with your USB devices. There is simply no way to have those operations be secure by default – some web pages should have access to the camera and some should not, and there’s no way to tell which is which without involving the user.

Of course, we’ve traditionally allowed any program we install on a conventional operating system to access these things if it wants to, but the stupidity of that practice becomes very clear when we instead worry about what a web page can do. Why do we continue to grant these broad permissions to executables? Once more, it is largely because we don’t want to bother the user with these microdecisions (we saw what a great idea that was with Vista), but hopefully the increasing power of the web will force us to figure out good ways to discern intent without getting in the user’s way. It seems to me that one opportunity we have with web interfaces is that we can place the APIs at a higher level. This allows us to ask the user more meaningful questions than when the security boundary is at the system call level – and obviously by “ask questions” I include ways to discern the intent of the user without explicitly asking him, as is done, for example, in a file open dialog: clearly what is indicated is a single file which the user wants to open – modern browsers enforce that decision transparently, whereas modern operating systems just provide the file name as a hint to the executable – which can open any file it pleases.

Will the web teach us a better way? I don’t know, but one thing is clear: we can’t ignore these problems in the browser. “Stupid user shouldn’t have installed that evil executable” does not translate well into “stupid user shouldn’t have visited that evil web page”. We’re going to have to find some way to consult the user; we won’t be able to brush the problem under the table as we have done in operating systems.

One approach I am very interested in is to somehow use collective behaviour to make smarter default decisions. But more on that another time.

A final thought on the subject: what lunacy caused us to design systems where “cat foo” gets any more privilege than a read handle to foo plus write handles to stdout and stderr?


  1. What’s up with you idiot? There’s one huge thing you left out of your whole entire article. USER RIGHTS MANAGEMENT in a MULTI-USER operating system. You can’t run the rm -rf command on anything outside of your user accounts home folder. The only way you can is if you’re root. That’s a very big layer of security on there, temporarily accessing root while in a standard user account (now you know what the multi-user thing is great for).

    Here’s the thing about root. You’re supposed to it sparingly on a computer, which doesn’t mean all the time. Windows users had it wrong for many years, then again, windows users are babies they cry when the start menu icon has changed. But, back to my point, root is supposed to be sparingly used for approval functions. You don’t need to run as root all the time until you to say install a program. If it’s a program you trust and want to install, then enter the root password, and you as the administrator just approved a program to be on your computer from within the non-volatile user account.

    Here’s the other thing about linux, how do you know you’re going to install safe software? Repositories are virus and malware free. Now whether a coder put together a crappy program that deletes your stuff on purpose without you knowing that’s in the repositories, nothing like that has happened yet, and if it did, that program would gain a negative reputation really fast and be removed from the repositories.

    What really is good security is a properly used multi-user aspect of an operating system and smarter computer users.

    Comment by shamil — 7 Dec 2009 @ 2:01

  2. Randy Farmer wrote up a nice example of not just ‘collective behaviour’ but potentially social behaviour:

    Comment by Kevin Marks — 7 Dec 2009 @ 3:51

  3. @shamil

    First: calm down. Then learn to read, then learn to comprehend what you’re reading, then learn about capability-based security, then re-read TFA, and then finally re-think your rant and apologize the author for calling him an idiot. Security is not a solved problem and Unix’s model of security is much more far from perfection than you think.

    Comment by bsah — 7 Dec 2009 @ 12:19

  4. In capability-based systems like EROS, my understanding is that programs get assigned capabilities by the install process. The install produces a kind of shortcut with embedded capabilities.
    With ChromeOS or other browser, this paradigm could not work, as there is no install. You can use a website just by visiting it.
    In that case, I think that a lazy approach works better. The website gets nothing by default, only a manager which can be used to obtain more permissions by asking the user. The manager can be persistent and also shield the user from spam (“remember this”).

    The issue remains that the browser itself has access to the webcam, so if an exploit appears that compromises the browser, the webcam is compromised…
    To solve this, you have to look “one turtle” below. The OS and design for processes needs to be based on capabilities too.

    Comment by Julien Couvreur — 8 Dec 2009 @ 1:14

  5. Whilst trying to think of a pertinent analogy it suddenly occurred to me that what you’re talking about is government, and what type of government we want our OSs to be. Frankly I’m feeling a bit disenchanted with democracy today, and even then it’s a toss-up between all the different flavours. One thing JS Mill did bang on about a bit was the dodginess of “good dictators” which he describes as

    …absolute power, in the hands of an eminent individual, would ensure a virtuous and intelligent performance of all the duties of government. Good laws would be established and enforced, bad laws would be reformed; the best men would be placed in all situations of trust; justice would be as well administered, the public burthens would be as light and as judiciously imposed, every branch of administration would be as purely and as intelligently conducted, as the circumstances of the country and its degree of intellectual and moral cultivation would admit.

    Which gives us:

    One man of superhuman mental activity managing the entire affairs of a mentally passive people

    Leading to another problem:

    Wherever the sphere of action of human beings is artificially circumscribed, their sentiments are narrowed and dwarfed in the same proportion.

    So we end up with

    A good despotism … a government in which … all the collective interests of the people are managed for them, all the thinking that has relation to collective interests done for them, and in which their minds are formed by, and consenting to, this abdication of their own energies.

    Which I suppose leaves me wondering a bit about users’ capabilities. Hum.

    Comment by Barney — 8 Dec 2009 @ 16:43

  6. I was going to make the same comment that Kevin linked to, this is all about reputation systems. Yet another example of how much of a joke the current situation is is that we have almost no reputation systems in use today. The closest we have are closed systems like Anti-Virus software or Apple’s iPhone vetting, where they determine who’s good enough to go on the iPhone but is so constrained that you have to squint real hard to call that a reputation system. Reputation systems will be big very soon and will provide a process to crowd-source a lot of these security issues.

    Comment by sprewell — 9 Dec 2009 @ 13:56

  7. The web site visiting webcam problem is already here with Flash, although you’re right that Chrome OS needs to solve it too. I just tried

    This prompted me in the Flash applet to explicitly authorize the use of the webcam. On my MacBookPro, a little green light also came on at the top of the screen to let me know the camera was active. Visiting again yields another prompt.

    Comment by David Molnar — 13 Dec 2009 @ 22:33

  8. Seconding comment 3 by bsah: and see

    Comment by Antonomasia — 14 Dec 2009 @ 23:43

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress