the cups blog


SOUPS Keynote: Ross Anderson

Towards a Science of Security and Human Behaviour

Summary: Economics, Sociology, and Psychology can give important insights on security and how to make it more effective.  The current incentive structure makes it so that users are left to their own devices, mistakes, and misconceptions.

Security to Economics, How did I get there?
– People used to think security was all about crypto, authentication, firewalls, etc
– But, people realized this wasn’t enough, things weren’t getting better.

Economics and Security
– Since 2000, he has been applying economic analysis to IT security and dependability
– It explains the failure better!

Security fails b/c the incentives are wrong.

New view of InfoSec.
– Systems are insecure b/c the incentives are wrong.
Bank customers suffer when poor design makes fraud and fishing easier
– Insecurity is often and “externality” or a side-effect, like environmental pollution

New uses of Infosec
– Support business models
Xerox tied ink to printers, to increase the price of ink.
Car makers can charge more for parts for certain vehicles

IT Economics
– 1st feature, Network effects
Metcalfe’s law – value of a network is the square of the # of users
Real networks – phones, fax, email
Virtual networks – PC architecture vs. Mac
Network effects tend to lead to dominant firm markets where the winner takes all.

2nd common feature of IT product and service markets is high fixed costs and low marginal costs.
– Competition can drive down prices to marginal cost of production, but this can make it hard to recover capital investment, unless stopped by patent, brand, compatibility.
– These effects can also lead to dominant-firm market structures.

3rd feature is that switching from one product or service to another is expensive.
Shapiro-Varian theorem: the net present value of a software company is the total switching costs

So major effort goes into managing switching costs, i.e. IPod and the music for your iPod, you are locked in to your product.

IT Economics and Security:
High fixed/low marginal costs, network effects and switching costs tend to lead to dominant-firm markets with the big first-mover advantage
– Time to market is critical
-Ship fast and fix it later is a rational action plan

When building a network monopoly, you must appeal to vendors of complementary products
– Lack of security in earlier versions of Windows made it easier to develop applications.
– So did the choice of security technologies that dump usability costs on teh users
– Once you are a monopoly, lock it all down!

Economics and Usability
– Make your products usable by newbies…  but more usable with practice!
– To what extent can you make skill a source of asymmetric lockin? (i.e. you learn all the keyboard shortcuts)
– Hypothesis: this underlies the failure of user programmability to get traction!
– How many features should a product have?
– Marginal benefit of new feature concentrated in some target market
– Marginal costs is spread over all users – so we get chronic featurities!
– At equilibrium, any programmable thing will just be on the edge of unacceptability of a significant number of users
– The same thing happens with laws, services, etc

Why are so many security products ineffective?
– Akerlof’s Nobel Prize winning paper: The Market for Lemons, which introduced asymmetric information
– People don’t know what things are the lemons, so they will pay full price…
– Security products are a ‘lemons market’

Products worse than useless:
– Why do Volvo drivers have more accidents? Adverse selection and moral hazard- people think they are safer so they act more dangerously.
– Application to trust: Ben Edelman, ‘Adverse selection on online trust certifications (WEIS 06)
– Websites wtih a TRUSTe certification are more than twice as likely to be malicious
– The top Google ad is about twice as likely as the top free search results to be malicious

– People say they value privacy but act otherwise
– Why is there this privacy gap
– Odlyzko – Technology makes price discrimination easier and more attractive
– Acquisti et al – People care about privacy when buying clothes, but not cameras
– Loewenstein et al – It’s not clear that there are stable and coherent privacy preferences
– Students disclose more for How bad RU and less with detailed privacy notice

Conflict Theory:
– Does the defense of a country or a system depend on the least effort, or on the best effort, or on the sum of efforts
– The last is optimal, the first is aweful
– Software is a mix, it depends on the worst effort of the least careful programmer, the best effort of the security architect, and the sum of efforts of the testers
– Moral: hire fewer better programmers, more testers, top architects

Skewed Incentives:
– Why do large companies spend so much and little companies so little on security?
– If you are the Director of the NSA and you have a hack for XP and Vista, do you tell Bill Gates?
– If you do, you protect 300 million Americans
– If you don’t, you can hack 400 million Europeans, 1000 million Chinese, etc
– If the Chinese hack US systems, they keep quiet.  If you hack their systems, you can brag about it to the President (increase your budget)
– Offense favored over defense

Security and Policy

Security and Sociology
– Can we use evolutionary game theory ideas to figure out how networks evolve?
– Idea: run many simulations between different attack/defense strategies

Psychology and Security
– Phishing
Banks react to phishing with a “blame and train” efforts towards customers
But we know, this doesn’t work
– We train people to keep on clicking OK and ‘learned helplessness’ goes much wider
– People didn’t notice the missing SSL padlock icon

Social psychology is relevant
– People deny the evidence of their eyes to conform to a group
– People will do immoral things if ordered to
– Roles and group dynamics are enough
– Disturbing case of “Officer Scott” – a guy who called McDonald’s and ordered managers to strip search female employees.  McDonald’s couldn’t be bothered to train their employees to resist immoral orders from police.

How do systems resist abuse of authority?

– Why does Terrorism work?
– It’s evolved to exploit a large number of our heuristics and biases
– Availability heuristic, mortality salience, anchoring, loss aversion in uncertainty, wariness of hostile intent, violation of moral sentiments, credence given to images, reaction against out-group, sensitivity to change
– The good news: biases affect novel events more, and so can be overcome by experience.

– Central to evolution of homo sapiens
– Self deception
– What are the effects on policy?

– People don’t care enough about computer security, and they care much more about terrorism.

For more information:
Workshop on the Economics of Information Security
Workshop on Security and Human Behavior
_ Security Engineering by Ross Anderson