Cloud Security

If you surfed up to the Amazon Web Services homepage today, you’d be forgiven thinking all has been well in the Amazon cloud.

Recent news stories highlight new features and capabilities, including a SQL-like SELECT API for Amazon SimpleDB and the (significant news) that Cloud Compute is now available in the EU.

More worryingly, however and missing from the Amazon AWS homepage, is that Amazon just rolled out an important security fix to correct a basic, but significant, cryptographic weakness in their request signing code.  The flaw affects their database API (SimpleDB) and compute API (EC2) services.

Colin Percival discovered the weakness and reported it to Amazon back in May (!).  Thats 7.5 months ago.  Here’s the “executive summary” from Colins’ write-up:

AWS signature version 1 is insecure

The important bit first: If you are making Query (aka REST) requests to Amazon SimpleDB, to Amazon Elastic Compute Cloud (EC2), or to Amazon Simple Queue Service (SQS) over HTTP, and there is any way for an attacker to provide you with data which you use to construct your request, switch to HTTPS or start using AWS signature version 2 now. For example, if you allow users to add arbitrary “tags” to documents, and you use SimpleDB to store those tags, this means you. (Amazon Flexible Payments Service (FPS) and Amazon Devpay also use the same insecure signature method, but they already require the use of HTTPS. Amazon S3 and other services use different signature methods.)

So in a nutshell, if you only ever accessed SimpleDB or EC2 services over HTTPS, you’re not impacted.

But if you did use the non-encrypted transport, you were vulnerable to a man-in-the-middle attack (MITM) that could expose your data hosted in SimpleDB and your EC2 compute instances.

Overall, Colin is generous in his praise of the response from the Amazon team - although did express concern at the time it took for the vulnerably to be closed:

I reported this issue to Amazon via an email to Jeff Barr, the “Lead Web Services Evangelist” at Amazon on May 1st, and while it took a long time — 7.5 months — for it to be fixed, I’m happy to say that Amazon took this issue seriously at all times, and the lengthy timeline was simply because of the large amount of work involved. Jeff forwarded my email to someone working on SimpleDB (I’ve been asked not to mention names), who confirmed that they agreed that this was a problem. As part of their review of my findings, Amazon’s security people realized that this also affected EC2 and SQS — in my initial investigation I had only looked at SimpleDB — and at the beginning of July they agreed to send me their planned signature version 2 so that I could review it.

Based on the timeline provided by Colin, it took about 2 months for Amazon to assess the extent of the exposure across their other service APIs and come up with a proposed fix.  That’s a considerable length of time given the simplicity of the weakness.  They then took over 5 months to actually implement the fix.  I don’t care how many client libraries you offer Mr Cloud Provider, thats still a very, very long time to leave your customers exposed while you get a fix into production.

Back to Colin as he describes the vulnerable signing process:

AWS signature version 1 signs an HTTP query string as follows:

1. Split the query string based on ‘&’ and ‘=’ characters into a series of key-value pairs.
2. Sort the pairs based on the keys.
3. Append the keys and values together, in order, to construct one big string (key1 + value1 + key2 + value2 + … ).
4. Sign that string using HMAC-SHA1 and your secret access key.

Did you spot the problem?  Colin did and explains as follows:

When Amazon invented this signature scheme, they forgot about one of the foremost design principles relating to cryptographic signatures: Collisions are BAD! In a well-designed signature system, it should be computationally infeasible to construct two different messages which have the same signature; this prevents substitution attacks where an attacker convinces the key holder to sign a “harmless” message, and then attaches that signature to a different message. Looking at how AWS signature version 1 is computed, it’s easy to see how to construct collisions: Because there are no delimiters between the keys and values, the signature for “foo=bar” is identical to the signature for “foob=ar”; moreover, the signature for “foo=bar&fooble=baz” is the same as the signature for “foo=barfooblebaz”.

Oops.  He then goes on to explain how a man in the middle attack could exploit the weakness.

Taking a step back, what are the concerns with this weakness and the way Amazon handles security vulnerability reports in general?

Vulnerability Reporting

• Amazon doesn’t provide a visible mechanism for researchers/customers to report security vulnerabilities.  You can report ‘abuse’ (i.e. your EC2 image is under attack) via a contact form on the AWS website but unless you know someone at Amazon, you have to guess how to report a security vulnerability.
• Amazon has no published policy on how they handle vulnerability reports.  Nothing to explain their commitment to following up, promptly responding to reports, overall process etc.  Now, it seems they do follow up and overall Colin says good things but that timeline sends shivers up my spine.
• Amazon doesn’t have a dedicated security page, mailing list, RSS feed or pigeon carrier for alerting customers of security issues.  This makes it a PITA for enlightended customers to even know they are at  risk (unless they setup  Google News Alerts) and comes across to me at least as a trifle arrogant.

Vulnerability Notification

• Amazon hasn’t notified their customers yet that their SimpleDB and EC2 instances were vulnerable for the past 7.5 months (I’m a customer and I’ve yet to receive an email).  Obviously the vulnerability itself goes back further than 7.5 months.
• Amazon have not published how customers can determine if the issue was exploited (back to the visibility issue again).   Clearly, Amazon doesn’t have full visibliity of the network between the customer and their data centers, however what analysis has Amazon performed on the logs *only they have access to* that could highlight an issue? (frankly, due to the nature of this issue and the lack of event logs/audit trails I suspect the simple answer is there isn’t much they can do here).

Secure Software Development

• Amazons’ internal code review process failed to detect a basic cryptography flaw.  Note, although the error is pretty basic, errors do happen.  The real problem here is the failure of peer review processes to identify the problem prior to going into production.  Cryptography routines along with other sensitive security routines should be subject to greater security analysis as clearly the stakes are higher.  If they did in fact have this code reviewed for security weaknesses, this is a huge oversight.  So this is now fixed, what else is vulnerable?  It would be great to learn that the lengthy delay in delivering the fix was due to Amazon have all their source code reviewed for security issues…
• Amazon have made no public commitment regarding how security is factored into their software development lifecycle.  Until now they could argue they didn’t need to - they hadn’t messed up (or at least, we don’t think they did).

The geek in me loves Amazon AWS and the promise that it holds, but the Security Professional rooted in the “here and now” is seeing too many red flags (the issues are wider than the handling of this one bug).

If anyone from Amazon reads this and wants to talk security roadmap and go “beyond the whitepaper”, you know where I am.

If you are curious about Cloud Computing and security, don’t miss out on future posts: subscribe by RSS or subscribe by email.

Categories: Cloud Computing Security, Infrastucture as a Service
Tags:

The expression “a chain is only as secure as its weakest link” seems pretty appropriate with the release of an updated test report on Web Browser password security. In a nutshell, your browser may not be protecting your stored/cached passwords the way you might hope. Browser security is already important for people that shop or bank online.  However, when your IT Infrastructure is Cloud based, browser security becomes a critical organisational security factor.  If an attacker can gain your Cloud credentials, how much of your Cloud Compute, Storage and SaaS can they control, influence or destroy. In their first report, Chaplin Information Services highlighted two key required features of browser password managers:

Internet password managers need to know two things to be secure: Which website is requesting a password? And, to which website is a password being delivered?

Check out the scorecard below:

Test definitions are available here.

Clearly, there is still some way to go.

A few observations:

• The timing of the announcement is interesting.  According to CIS, they notified Google about the issues prior to the release of Chrome.  On December 12th, Google removed the beta label and Chrome is considered production code.  It will be interesting to see if Google comment on this as on the surface this does leave some awkward questions.  Rather than pre-judge now, lets see if/how they respond.
• This issue is compounded with Cloud Computing as the Cloud Management interfaces exposed by the Cloud providers cannot - in most cases - be restricted to specific IP ranges (not a total solution but a significant mitigant).  For example, if an attacker steals your Amazon AWS password, she can directly access the AWS management pages, claim your AccessID and Secret Key and perform a Cloud takeover from any source IP address of her choosing.
• The lack of Cloud visibility means you have no administrative audit trail to (a) pro-actively monitor for misuse of your credentials and (b) perform rapid response.  On that last point, you would need to reach out to your provider and ask them to tell you which source IP accessed your Cloud account (in a future post, I’ll talk about the complete lack of *formal* Incident Response provisions in Cloud SLAs).
• Browser bugs are hardly rare - what makes this batch insidious is the ease with which the weaknesses can be exploited - no special exploit code required.  Expect to see this abused.
• This is a bad week week for browser bugs.  An emergency hotfix from Microsoft to fix an actively exploited heap overflow is released today, along with fixes from Opera rated as highy critical.
• The market for browser security products will continue to grow (duh!).

The most practical recommendation I can give is to recommend all Cloud related administration is carried out in a separate Virtual Machine.  This is good advice for internal IT administrators today - regardless of whether they use Cloud services or not.  The main thrust is to ensure that your email client and “normal” web browser are isolated from your administration infrastructure.  The need for separation of administration environments becomes increasingly vital as organisations adopt Cloud Services.

If you are curious, you can test your browser.

Categories: Cloud Computing Security
Tags:

Last month I was interviewed for a podcast with SearchSoftwareQuality.com.

We talked about some of the advantages Cloud Computing could bring to software development and testing.  Notice I say ‘could’ - I continue to see great potential benefits but some of these require us to rethink how we do things as ‘end-users’ and depend on the Cloud Computing ecosystem maturing enough to deliver them (e.g. security monitoring of Cloud API calls).

This was recorded prior to the Microsoft Azure announcement hence the “software + services” model wasn’t covered.

Anyway, the podcast is broken into 3 x 8 minute segments (I think I broke the spoken word count ;-):

• General benefits of cloud computing for software development
• Cloud computing’s impact on agile development practices, software testing, and e-commerce
• Security elements surrounding cloud computing, such as software monitoring, implementing security patches, and the reduction of data leakage.

You can access the podcast segments here.

My thanks to Michelle and Keith over at TechTarget for the opportunity.

What About You?

Apart from general feedback on whether the podcast was helpful or not, I’m interested to hear if you’ve started any Cloud based development projects - please share in the comments.

Categories: Cloud Computing Security
Tags:

Hardly a shock but validates the need for a meaningful security conversation between Cloud providers and potential Cloud customers…

As part of our ongoing research into Cloud Computing, IDC recently conducted a survey of 244 IT executives/CIOs and their line-of-business (LOB) colleagues about their companies’ use of, and views about, IT Cloud Services. Successful suppliers will need to address both the biggest challenges of cloud services, and the biggest traditional IT user issues.In part 1, we looked at current and future adoption of IT cloud services. In part 2, we looked at users’ views about the key benefits and challenges of IT cloud services.

What is your Cloud Provider doing to address your security concerns?

Categories: Cloud Computing Security
Tags:

Privacy is a lot like universal healthcare.  Many agree its a good idea in concept, but few people want to pay for it.

Richard Stallman - the man that gave us GNU - doesn’t trust Cloud providers with his data and says you shouldn’t either.  Richard believes we should store our private data on our own computers using ‘free’ (as in freedom) software.  The ironic part for Richard is that a significant portion of the Cloud is powered by open source software which he indirectly created (think gcc).

Richard sees it as a question of control.  Control is important but it isn’t the only variable.  Rather, I see it as a question of control, competence and economics.

The quick rebuttal to Richards’ view is this: the average computer user is not as smart as you.  Control is not the same as competence.  Control is about exercising choice, not about requiring everyone in the world to develop sufficient skills to protect complex hardware and software systems (aka their computer) against ever increasing threats.

My view is that privacy is not ‘free’.  It comes at a cost.  Whether you run your own systems or rely on someone else to do it, there is a cost.  There is cost in designing and implementing mechanisms to support privacy.  Beyond upfront costs there are ongoing expenditures to ensure privacy is maintained e.g. maintaining access control lists, testing and applying security patches, data leakage prevention etc.  None of these things are ‘free’.

If we agree that privacy costs money then how much is your privacy worth?

Stop for a second - think of a number…  

Now did we all think of the same number?

The problem with a one size fits all approach to privacy is that we each place a different value on it.

Checking in on the EPIC site, I saw this:  

A new report from Pew Internet and American Life Project indicates that “cloud computing” applications, such as web-based email and other web apps, are raising new privacy concerns. The report Use of Cloud Computing: Applications and Services found that 69% of online Americans use webmail services, store data online, or use software programs such as word processing applications whose functionality is located on the web. At the same time, “users report high levels of concern when presented with scenarios in which companies may put their data to uses of which they may not be aware.” For example, 90% of respondents said that they “would be very concerned if the company at which their data were stored sold it to another party,” 80% say “they would be very concerned if companies used their photos or other data in marketing campaigns,” and 68% of “users of at least one of the six cloud applications say they would be very concerned if companies who provided these services analyzed their information and then displayed ads to them based on their actions.”

What does that tell us?

The average (American) Internet user finds Cloud services convenient but has concerns about how their privacy might be affected by Cloud providers actions (duh!).  The survey identifies a lack of awareness in how private data is used in some consumer based Cloud services (consistent with web advertising awareness surveys).  

Unfortunately, the results of this survey are not very actionable.  The survey doesn’t mention whether these are all ‘free’ Cloud services (we can only assume they are) or ask the respondents what their expectations of privacy are and how much they would be willing to pay for different privacy assurance levels. 

On a sidenote, respondents were not asked if they had actually read the privacy agreement for the services they signed up to.  But the providers know if they did or not…  Or at least, they have the data to figure it out.  At sign up time they can measure the time between displaying the privacy agreement and the user clicking ‘I accept’.  If its just a few seconds then its pretty obvious there was more scrolling than reading going on.  But I think we can probably guess the answer without the data ;-).

I believe we need to be able to link expectation of privacy with cost.

• How much are you willing to pay for privacy?  What level of privacy assurance do you need?
• How much is your Cloud Provider paying to protect your privacy today?  What privacy services could they reasonably offer if they had customers willing to pay?  How might this compare with how you manage your private data on your home computer today?

The cynical view is that we expect privacy but don’t want to pay for it.  Its a bit like uptime - there is a parallel universe out there, where internal IT departments allegedly meet their 99.999% uptime SLAs, but when Gmail goes down, the Sergey Brin witchcraft dolls come out.

From a provider perspective, the “cost” of privacy invariably gets bundled under that line item called ‘Information Security’.  And don’t be fooled, the cost of privacy in reality is more than the salary of the person employed to be the privacy advocate (if there is one).  If we can’t see how much our providers are spending on our privacy then how can we judge if they are spending enough?  And what is enough?  And what can I get if I’m willing to pay a little extra?

Personally, I would rather we get some transparency around privacy costs and assessment of offerings.  However, without a sufficiently sized market of customers willing to pay for privacy assurance and Cloud Providers willing to be more open, I won’t hold my breath.

What about you?  Would you be prepared to pay for privacy?  Should providers be more transparent about what they do and don’t do and how they do it?
 
 

Categories: Cloud Computing Security
Tags:

I’ve been sitting on this for a while and I’m glad I can now finally say it…

I’m delighted to announce that I have been invited to present at the World Summit of Cloud Computing, to be held in Israel on 1-2 December 2008.

The event is organised by Avner Algom from the IGT (Israeli Association of Grid Technologies). Putting my invitation to one side, I have to say its a stunning lineup of speakers. Its a who’s who of Cloud players. Avner has clearly done his homework!

Obviously I’ll be talking about the security aspects of Cloud Computing, delving into some of the areas I’ve written about here and some new material that I’m currently working on.

If you work for a company that is consdering future plans and Cloud Computing, you might want to take a look over the agenda. Compared to some other conferences, the ticket prices seem very reasonable to me.

Registration is now open.

If you have any questions, feel free to leave a comment below. I’ll do my best to get them answered. Also, if you know anyone that might benefit from 2 days in a beautiful part of Israel getting up to speed on Cloud Computing, feel free to send them this link.

Categories: Cloud Computing Security
Tags: speaking