Cloud Security

Much has been said about public IaaS providers that expressly forbid customers from running network scans against their cloud hosted infrastructure.  Failure to comply with the Terms of Service can result in account suspension or termination (ouch!).  This post is my attempt to suggest a way forward.  I welcome your feedback…

As has been noted before, a blanket ban on legitimate scanning activity by customers of their own infrastructure (whether outsourced or not) undermines security assurance processes and can make regulatory compliance impossible; e.g. PCI DSS mandates network vulnerability scanning as a control.

Vulnerability scanning is a stalwart practice of the Information Security community. Enterprises invest considerable time and money developing vulnerability management programs to help assess IT security risk across applications and infrastructure. Specifically, vulnerability scanners help identify potential security weaknesses at scale; e.g. missing patches, default passwords, coding or configuration weaknesses.

Vulnerability scanning is front of mind for Internet exposed or partner connected infrastructure. However, when said infrastructure is owned and/or operated by a service provider, some of the existing challenges associated with vulnerability scanning are magnified:

• Scans can cause outages.  This can happen if the scanning policy includes Denial of Service checks or the scanning engine is configured with “aggressive” settings; e.g. connection entries in firewall state tables get exhausted.  Its also possible for scans to tickle obscure bugs in the target - or devices enroute to the target.  Even without a full-on outage, poorly configured scans can still negatively impact performance or availability for other customers of shared infrastructure.
• Identifying unauthorised scans. Without a trusted, robust process for “blessing” or approving source IP addresses of customer scan engines, service providers cannot distinguish legitimate scans from scans with the evil bit set.  Sure, they can use whois to determine source network ownership but even if the scan originates from a customer owned network, this does not necessarily mean it is authorised!  Given this, many providers take the stance that all scans are treated as hostile unless pre-agreed.
• Scanning may trigger automated or manual actions by the provider. A common automated response from a provider is to apply traffic shaping to slow down the scan, or simply block the client IP address via an ACL update.  This can lead to false negatives; i.e. vulnerabilities present are not discovered as the scanner IP was automagically identified as a noisy vulnerability scanner and auto-throttled/blocked.  Even half smart attackers can quickly deduces the presence of auto-response mechanisms (”huh, no response now”) so either switches to slow probes from multiple sources or goes for gold with a one-shot exploit.

Enterprise customers on dedicated infrastructure at Tier 1 web hosting providers will either contract the hosting company (or their security partner) to perform vulnerability scans or do it themselves.  Either way, for scanning to happen, agreement will need to be reached on scan scope, types of scans to be run (scanning tools & policies), time windows and source IP addresses used.  Beyond that are the process issues of how results will be communicated, integration with ticketing systems etc.

The provider will limit the scan scope to the dedicated infrastructure allocated to the customer - the scanning of shared infrastructure by the customer is generally a ‘no no’.  This, along with management networks will be scanned by the provider to meet customer compliance mandates or security policies.

With Cloud “Infrastructure as a Service” providers, things get a little more complicated.

• A cloud is multi-tenant; i.e. the cloud platform is shared to multiple customers through software abstraction.  The provider will naturally be concerned with the impact of any scanning activity, particularly if it causes any SLA violations.
• Further, cloud customers can spin up infrastructure on demand.  New virtual servers can be  brought to life automagically to handle increased load.  This increased infrastructure footprint is still subject to the same compliance mandates though; i.e. it must be scanned within some time period of its appearance.  Even if spinning up copies of “known good/secure” virtual machine (VM), you still need to scan them.   New vulnerablities are published all the time, along with corresponding vulnerability checks - hence the need for both regular scans and representative scans.  Further, vulnerbility scanning isn’t just testing the VM, its also helping you verify the security controls outside the VM that are designed to protect it; e.g. a providers’ software firewall.  Picking and choosing which pieces of your hosted infrastructure to scan is a slippery slope to selective exposure if not handled with care.
• Finally, we shouldn’t discount the “Clouding around” factor.  Credit card payments for “instant on” infrastructure changes the dynamic between cloud consumer and cloud provider.  Similar to low end, consumer oriented shared hosting before it, you may never speak with, let alone meet, an employee of your provider before you use their services.  There simply isn’t a conversation about scanning (the “conversation” today is a monologue found in the Terms of Service).  Plus, if the provider fails to meet your needs, you can drop them at a moments notice and switch to another (Cloud baggage permitting…).  In other words, its either not possible, or not convenient to call up your provider to agree the principle and logistics of scanning the services they host on your behalf.  Enterprise customers - or at least their security teams - will be wanting that conversation and can likely strike a deal with a modified ToS to allow scanning of some sort but this seems unncessarily exclusionist to me.

We can address these issues through a mix of provider open-mindedness, policy, process, technology and contract.

For cloud providers to attract certain customers, they may need to soften their policy on vulnerability scanning.  Taking a hardline “no” stance precludes some workloads from ever entering the cloudosphere (with bigger consequences for enterprises seeking a strategic cloud partner).  A preferred scenario has the cloud provider showing some understanding of enterprise prospects assurance needs and defining scanning parameters acceptable to their own operations risk tolerance.

Scanning is not an “unknown” risk, rather its a very well understood activity with quantifiable elements (packet rate, state table usage etc).  Normal rate limiting could be temporarily or permanently loosened for customer approved IP addresses to enable scans against a customers cloud IP addresses (not API endpoints or cloud providers websites!) to complete in a reasonable time window.  Besides, Internet systems are scanned, probed and attacked constantly by script kiddies, Internet surveyors and an assortment of bots and other lifeforms.  So the bad guys get to scan because they don’t care and yet the customer, who wants to do the “right thing”, is not allowed to.  Is that rational?

Assuming a cloud provider with a more measured approach towards vulnerability scanning of customer cloud infrastructure, we now need a simple, mutually trusted mechanism to agree scan sources, rate limits etc.  Something like an “ScanAuth” (Scan Authorize) API call offered by cloud providers that a customer can call with parameters for conveying source IP address(es) that will perform the scanning, and optionally a subset of their Cloud hosted IP addresses, scan start time and/or duration. This request would be signed by the customers API secret/private key as per other privileged API calls. The provider receiving the request can rely on the digital signature as proof that a scan is authorised with the associated parameters. After the provider has processed the scan authorisation request, the provider could return a status code approving or denying the request (with a possible reason code to allow resubmission with more acceptable parameters).  This response can optionally include rate limits which the customer can use to tune the intensity of their scanner.

The provider can now whitelist the customer provided scanner IP(s) for the duration of the requested scanning window such that active countermeasures like anti-DoS controls are not triggered, resulting in a ‘cleaner’ scan (and hence a more accurate report).

Should the scanning activity exceed any specified limits, or communicate with IP addresses not associated with customer virtual machines, the provider could instantly blacklist the scanning IP or apply traffic shaping.

The bottom line: when everyone is clear on the need, approval process, scan parameters and abuse policy, this can be done with very little fuss.

A “ScanAuth” API call empowers the customer (or their nominated 3rd party) to scan their hosted Cloud infrastructure confident in the knowledge they won’t fall foul of the providers Terms of Service. This avoids a situation where either a customers Cloud services are interrupted by an angry provider (availability fail!) or in the worst case, getting kicked off the Cloud entirely.  Clearly, a lose/lose scenario.

What do you think?

If you are curious about Cloud Computing and security, don’t miss out on future posts: subscribe by RSS or subscribe by email.

Categories: Cloud Computing Security
Tags:

Have you ever performed a security review or audit of a 3rd party hosting provider before your employer signs on the dotted line?  Did you ever “have that moment?”.  It’s that time when exhausted from review fatigue you find yourself banging your head on the desk screaming ‘there must be a faster way’.  Well, you’re not the only one…

The scene goes something like this:

The provider rolls their eyes as yet another customer security team sends in their 500 deeply probing security questions, transmitted in some homegrown template in Word, Excel or $diety forbid, Powerpoint.  The customer security team, naturally suspicious of the provider and irked by managements apparent keenness to outsource the farm, has created the security questionnaire from hell:

• it’s the result of 100 hours of internal team meetings
• it’s gone through 14 drafts, 20 reviewers inboxes, 76 yellow highlighter comment fields and was printed at least 6 times
• it only asks IT security questions (no input from other relevant functions such legal/compliance/audit - HA!)
• it’s laced with a few tricky landmine questions based on potential security issues raised (but not satisfactorily answered) in online forums and provider support forums
• it contains 25 attachments detailing all the company security policies that *must* be followed (huh, Bluetooth policy requirements for a cloud storage provider…interesting)

In the context of cloud providers, they are slammed - a raft of audits in progress right now - with more expected soon.  The provider is experiencing an ADoS (Audit Denial of Service).  Instead of innovating new service offerings (including security!), the talented security professional at the provider is stuck cut and pasting answers from internal cheatsheets to customers questionnaires in the knowledge that the customer likely has no idea how much money it would cost to fulfill some of these security requests.  The sheer number of questions is confusing given that the customer IT team had stated they were only looking to host non-critical, non-sensitive data…

Audits are time consuming, repetitive across customers, costly and generally a motivational drain for everyone involved.  Moreover in the context of Cloud, time consuming audits seriously delays a key benefit of cloud - agility.  Its the “on demand” part of “Infrastructure on demand” that is a primary benefit of cloud.  If the security review process takes 3 months to complete, how much business opportunity has your employer lost?  Don’t like that question?  OK, another one: how much time could you have spent doing something more interesting?

Which leads me to some questions:

• what does the cost/benefit ratio look like of the “questionnaire security review method”? (to be clear, I’m not arguing against the need for security reviews)
• why do we all use different format questionnaires? (note: format)
• why are we asking these questions? (are the bulk of our questions simply an expression of our policy asked in a question format?)
• how many of these questions/policies are predictable and duplicated?  As in, you and I ask some of the same questions…we may differ in the details (e.g. password complexity..eek!) but we both probably ask the same base question even if our thresholds around answers are slightly different.
• what if we were to agree a set of common questions/policy statements?  We don’t all have to subscribe to them, we can pick the ones that reflect our policy…  There could be thousands, you search, pick and mix just like an iTunes playlist (Ed: Genius!)
• for those standard policy questions, could we “digitize” them and express them electronically?  Could the provider host a policy oracle that we could post these questions to?
• for those “uncommon” questions that the providers oracle cannot automagically answer, could we agree a standard way to “ask/transmit” those with some simple agreements about response formats? (um, freetext fields ;-).
• ultimately, could we “digitize” a significant portion of our questions to get near instant answers? (and could we make that multi-lingual…)
• would the provider recognise this as a benefit too?
• would the provider also see the legitimate opportunity this presents to charge for higher assurance services around cloud compute/storage/network based on our policy requirements?  “You want triple cycle, double buffering?  You got it - for an extra 5c per MB”).  Yes, the cost of  your security policies in a pay per drink model are revealed!
• would the provider recognise the opportunity to offer incentives to customers for choosing this low friction path of policy compliance instead of tying up their skilled employees filling out ad-hoc questionnaires?

Is there an existing system/application/protocol whereby I can transmit my policy requirements to a provider, they can respond in real-time with compliance level and any additional costs, with less structured/known requirements responded to by a human (but transmitted the same way)?  In other words, I’m looking for human driven, machine to machine policy exchange/agreement.

I propose that the benefit of quickly ascertaining policy compatibility along with any additional costs involved would reduce the on-ramp to cloud, reduce switching costs, drive a form of policy interoperability and take us closer to where we need to be in the long run: the ability to express security policy for a single unit of compute/storage/network in a cloud.  Ultimately, I want to be able to tie my security policy to the information asset I need to protect and push that to a cloud broker who performs policy reconciliation to determine which of my approved provider(s) can meet my needs without any human intervention (yeah, I can hope ;-).

And before everyone jumps on me and says ‘but the point of an on-site audit/security review is to get assurance that the provider is doing what they claim they are doing” I’d like to point out that policy and assurance are two different things.  Before you and the provider invest time in the optional on-site audit, why not get the bulk of the policy questions out the way in a fast and low cost manner? (i.e. “death to the questoinnaire?”).

If you’re following along thus far, you’ll also see the possibility for trusted 3rd party auditors to digitally ’sign’ individual policy statements made by cloud providers they have audited. That signature could itself reflect the assurance level you need.  This in turn could help drive the nascent cyberinsurance market for cloud…assuming the auditor is open to counterclaims by the insurer ;-).

If you do need to go on-site (and assuming the cloud provider tells you where “on-site” is ;-), you’ll have a list of items the provider categorically stated they do, meaning you can cherry pick the areas where you want to deep dive for assurance.  If upon inspection you find reality does not match stated policy, you can scream bloody murder.  Providers that mislead customers will soon get known.

Thoughts?

Categories: Cloud Computing Security
Tags: audit, compliance, questionnaire, security review

In a move I found a tad ‘uncloudlike’, ZDNet reports that SUN UK CTO Wayne Horkan is trying to pull together a UK specific Cloud Security group.

On the one hand I totally understand the need for a nation to protect its own interests - particularly where national critical infrastructure is concerned, but on the other, it “feels” a bit strange that an initiative like this is coming from a vendor with a vested interest in Cloud.

Here’s the quote:

Sun’s UK chief technology officer is working with major British public and private organisations to set up a cross-sector forum to resolve cloud-computing security issues.

Cloud-computing systems could become as important as the UK critical national infrastructure, and they need to be secured in an appropriate manner, Wayne Horkan told ZDNet UK on Thursday. The Sun executive said he is working on setting up the forum alongside organisations such as the CBI, Microsoft and Accenture; government departments such as Berr, Dius and the Treasury; and the government’s chief scientific advisor, Professor John Beddington.

“I’m concerned about the security of the supply,” Horkan said at the Cloud Expo Europe conference in London. “If cloud computing becomes a utility, it’s important to me that the UK as a nation state has good security of supply. It’s important that the UK has the appropriate capability in cloud computing.”

He then goes on to cite privacy concerns.

It’s plain to see that the majority of Cloud offerings are from US based companies.  Nearly every briefing I’m invited to is EST or PST.  In fact, I can’t remember even speaking with a UK Cloud provider.   Of the many media requests for comments, all but one were from the US.

I can’t help smelling fear in this effort. As a Brit, I would love to see a UK group coming together to innovate, support and promote the fledgling UK Cloud industry.  Perhaps that will be one of the goals of the group - if so, I don’t think that is ’security’ specific (unless we are talking security innovation).

Development of UK specific Data Privacy guidance in relation to Cloud should be led and enforced by the Information Commissioners Office.

I also feel this will do little to advance security of the Cloud overall. With the positive news yesterday that the UK based Jericho forum and the Cloud Security Alliance (CSA) have formally agreed to “work together”, isn’t this inward looking approach just fragmenting our efforts?  Why not direct the security talent that would comprise this group towards the CSA or ENISA.

Security is a *global* issue.  I’m struggling to see how country specific cloud security interest groups “fit” when we talk about globally distributed systems.  What next - Cloud UN? ;-).

I don’t disagree with the need to protect supply, but I would much prefer to see the UK government driving an initiative like this as part of their critical infrastructure protection strategy.  A strategy around UK Cloud innovation would be nice too ;-).

Perhaps I am being overly pessimistic or missing something. What do you think of a country specific Cloud security group set up by a technology company? A US based technology company no less… ;-).

Categories: Cloud Computing Security
Tags: CSA, ENISA, SUN

The Cloud Security Alliance is seeking your input to develop and improve upon version 1.0 of the guidance document they announced at RSA.

Launched last month, the founders are security professionals from Cloud customers and Security in the Cloud providers (with sponsorship coming from the latter).  The Technical Adviser is friend and fellow security professional Chris Hoff.

From the Introduction on page 5 of the guidance document:

The Cloud Security Alliance is a grassroots effort to facilitate the mission to create and apply best practices to secure cloud computing. Incorporated as a not-for-profit organization, our efforts will seek to provide a voice for security practitioners. However, recognizing that a secure cloud is a shared responsibility, we will be inclusive of all organizations and points of view to fulfill this mission.
What follows is our initial report, outlining areas of concern and guidance for organizations adopting cloud computing. The intention is to provide security practitioners with a comprehensive roadmap for being proactive in developing positive and secure relationships with cloud providers. Much of this guidance is also quite relevant to the cloud provider to improve the quality and security of their service offerings. As with any initial foray, there will certainly be guidance that we could improve upon. We will quite likely modify the number of domains and change the focus of some areas of concern. We seek your help to improve this guidance to make version 2.0 of this document an even better asset to the security practitioner and cloud provider.

How To Get Involved

This is a real opportunity to shape the future security of Cloud. With sufficient participants, a mature guidance document and strong awareness, I believe a group like this can make a real impact on the future of Cloud Security. Its my view that this advances the Cloud Security conversation which is a major reason why I started this blog and will be contributing as I can.

If you’ve been sitting on the sidelines up to now, I encourage you to get involved and contribute as little or as much as you can.

Getting started is easy:

1. Join the CSA linkedin.com group to become an official member of the group (I’m already a member).

2. Review and give feedback to the CSA guidance document via the CSA Google Group.

Finally, the CSA have a number of  events planned to spread the word, including Gluecon (Denver), ISSA CISO Forum (Chicago) and the Cloud Computing Expo Europe in Prague, Czech Republic.  More info here.

Categories: Cloud Computing Security
Tags: Cloud Security Alliance, CSA, Guidance

Today, nScaled announces the Legal Cloud in Beta. This is a vertical specific cloud targeting the “infrastructure on demand” (IaaS) requirements for international law firms:

Legal Cloud today announced that several top, international law firms had signed up as early testers of its virtual data center services for the legal market. The Legal Cloud is operating a ‘closed beta’ with select law firms interested in reducing the costs of their existing collocation facilities, finding a way to implement a business continuity program without duplicating private infrastructure or simply planning for their future primary and secondary infrastructure facilities.

What Makes This Different?

From their blurb:

The founders of the Legal Cloud have been working in the legal technology industry for over a decade. We understand that the needs of international law firms are different to other industries. Our data centers are optimized to meet the needs of law firms. Our choice of technologies, performance, data storage, latency, service level agreements, security and features have all been specifically devised to support the needs of the legal industry (source).

Why This Is Important From a Cloud Security Perspective?

• This cloud is designed around the needs of a specific industry:  with a well defined set of clients in mind it can cater to the groups specific operational and security needs
• These are not just “any customers”: international law firms that will have legal, compliance and security requirements over and above your “average” cloud customer today.  This needs to be a cloud with ‘higher assurance’ features to gain the trust and buy-in of legal CIOs
• The security conversation suddenly becomes a lot more focused: we are not talking about a general “one size fits all” cloud anymore and facing the disharmony of varying customers security needs and provider capabilities.  This may sound trivial but security conversations can get painful fast when customer and provider come from different worlds.
• In a view I’ve held for a longtime, its a taste of things to come: banking clouds, healthcare clouds, federal clouds (happening now).  Yes, there are other industry specific clouds (e.g. Salesforce Service Cloud) and they have their own security requirements, but arguably less assurance will be demanded by customers.
• The customers become an important lobby group for future security feature requests: instead of X voices asking for completely different things, the community of Legal Cloud users will state requirements “loud and clear” and if nScaled doesn’t listen, provide an opportunity for “Another Legal Cloud” to steal customers.
• The success of this cloud will be judged by many: if nScaled delivers on their promise, they will benefit from first mover advantage and become the “standard” for legal cloud.  From my UK experience, the legal community is cautious about new technologies and is a pretty tight-nit group, so if sufficient “established” legal firms move its not hard to imagine many more following (well, I’m sure that’s what nScaled hopes ;-).

What Is On Offer?

Legal Cloud is offering the following on a services basis:

• Fully virtualized data centers
• Business Continuity Service
• Active Cloud Servers
• Unlimited Storage
• Snapshot recovery points

And here’s how it looks from a 50,000ft:

What Do They Say About Security?

After a brave headline of “Security Guaranteed” (sure to rile anyone in information security), they go on to state:

The security of your data is of paramount importance. Here is how we guarantee it’s security.

Secure Data Centers

Our data centers are highly secure and redundant precision environments backed by the Fanatical Support of Rackspace. (SAS-70 Compliant)

Secure Virtual Private Networks

We extend your network into the Legal Cloud using VPN (Virtual Private Network) and VLAN (Virtual LAN) technologies. Your data is encrypted during transit with IPsec. Within the Legal Cloud, your data is segregated in logically separate areas from other clients data and attached only to your private networking. This gives each client their own private network and storage in the Cloud.

Data Encryption

Client Data is encrypted from client source servers to target devices using strong encryption protocols.

Not on the public Internet

The legal Cloud is not exposed to the public Internet. It is actually an extention of each clients internal network, each seperated by strong security protocols.

Service Level Agreements

We are working on appropriate SLA’s for our legal customers during the beta period.

Psychologically, I suspect the most significant reassurance for many CISOs will be this one single sentence: “Not on the public Internet”.  Beyond that, use of IPsec will make this feel very much like a standard 3rd party ‘partner’ connection.  I don’t see any mention of storage encryption options as yet, nor any further detail on the logical separation - once I’ve had a briefing and can speak more to the security aspects, I’ll post more.

P.S nScaled have annouced a couple of webinars aimed at their target audience here.

Categories: Infrastucture as a Service
Tags: Legal Cloud, nScaled

The slides from my talk at Black Hat Europe 2009 are now available [PDF].

From comments I received afterwards, I got positive feedback despite running out of time (my fault entirely).  I’ve been pleasantly surprised by the number of people asking for copies of the slides, but do bear in mind the slides are somewhat ‘terse’ as they are primarily talking points for me to bounce off of (as it were).

Should anything not be clear, feel free to leave a comment below and I’ll do my best to clarify.

I’d also like to take this chance to thank Jeff Moss, Ping and the rest of the Black Hat crew for doing such a professional job running the conference - it was confidence inspiring to be in such capable hands.

Categories: Cloud Computing Security
Tags: Black Hat, presentations