<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head><body text="#000000" bgcolor="#FFFFFF"><br>
Getting off topic here but the Capital One data breach was not the
result of a cloud provider failure or cloud provider security hole. The
only audits that would have identified the problem would have been
client-side audits. <br>
<br>
It was a failure of the "shared security model" where AWS is very clear
about the boundaries of what they are supposed to do vs what the
client/user is responsible for and in this situation the failure was
clearly on the Capital One side although there is wiggle room to perhaps
blame an as-yet-unamed WAF vendor (see below). <br>
<br>
The short summary is :<br>
<br>
- A commercial WAF (web application firewall) appliance (and not the
AWS managed WAF service) was either misconfigured or actually had a
security vulnerability<br>
- Capital One was using this WAF appliance to protect an internet
facing service<br>
- The attacker was able to either gain a shell on the WAF or access a
vulnerability that let her access the EC2 instance metadata on the WAF
appliance <br>
- Inside EC2 instance metadata were the constantly rotating IAM EC2
instance-role credentials that the WAF itself used to talk to AWS APIs<br>
<br>
So the attacker was able to steal the rotating credential set used by
the WAF out of instance metadata and use those transient API keys to go
hunting in S3 buckets for private data<br>
<br>
This is where Capital One will be getting some serious side eye ..<br>
<br>
1) It is legit that a WAF may need S3 service access if (for instance)
it dumps logs there but why did it's permission set include read access
to buckets hosting sensitive data? In a least-priv model the permissions
given to the WAF appliance should have been "you can only access the
log bucket and nothing else" <br>
<br>
2) S3 buckets hosting sensitive data probably should have had source-IP
access rules applied to them. The attacker apparently did not steal the
data via the WAF -- she used the WAF to pull the API keys out and then
accessed S3 via other methods using the ripped keys. This meant that she
was accessing from IPs that pretty clearly were not internal or
CapitalOne managed. <br>
<br>
2) This is in the hindsight-20/20 category but if they were running
sophisticated security tooling they presumably should have been able to
detect that an API Keypair was suddenly acting "out of character" and
accessing things that it had never touched in the past. This is the "we
have the capability to alert on anomalous behavior" feature that a lot
of people want or think they can buy ready to go off the shelf. I sort
of give them a pass on this because right now this part of the security
industry is full of bullshit vendors selling "AI" solutions that will
magically solve all your problems. This is still a very hard problem in
2019 -- finding the anomalous needle in a haystack without crushing
your analysts under a wave of false positives <br>
<br>
<br>
Chris<br>
<br>
<br>
<span>
</span><br>
<blockquote type="cite"
cite="mid:AM0PR08MB424109B787A7C29ED2C6EFC4A0DE0@AM0PR08MB4241.eurprd08.prod.outlook.com"
style="border: 0px none ! important;">
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvHr"
style="margin:30px 25px 10px 25px;"><div
style="width:100%;border-top:2px solid #EDF1F4;padding-top:10px;"> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:49%;">
<a style="color:#485664
!important;padding-right:6px;font-weight:500;text-decoration:none
!important;" href="mailto:jaquilina@eagleeyet.net"
moz-do-not-send="true">Jonathan Aquilina</a></div> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:48%;text-align:
right;"> <font color="#909AA4"><span style="padding-left:6px">August
1, 2019 at 1:05 AM</span></font></div> </div></div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvBody"
__pbrmquotes="true"
style="color:#909AA4;margin-left:24px;margin-right:24px;"><div>Hi
Gerald,<br><br>I think the question is how do these cloud providers let
such misconfigurations get through to production systems. Arent audits
carried out to ensure that this doesn’t happen?<br><br>Regards,<br>Jonathan<br><br>-----Original
Message-----<br>From: Beowulf <a class="moz-txt-link-rfc2396E" href="mailto:beowulf-bounces@beowulf.org"><beowulf-bounces@beowulf.org></a> On
Behalf Of Gerald Henriksen<br>Sent: Thursday, 1 August 2019 02:46<br>To:
<a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a><br>Subject: Re: [Beowulf] Lustre on google cloud<br><br></div><div><br>Not
sure what the Capital One data breach has to do with the cloud, it was
(yet again?) misconfigured software that allowed the theft.<br><br>_______________________________________________<br>Beowulf
mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing To
change your subscription (digest mode or unsubscribe) visit
<a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><br>_______________________________________________<br>Beowulf
mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>To
change your subscription (digest mode or unsubscribe) visit
<a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><br></div>
</div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvHr"
style="margin:30px 25px 10px 25px;"><div
style="width:100%;border-top:2px solid #EDF1F4;padding-top:10px;"> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:49%;">
<a style="color:#485664
!important;padding-right:6px;font-weight:500;text-decoration:none
!important;" href="mailto:ghenriks@gmail.com" moz-do-not-send="true">Gerald
Henriksen</a></div> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:48%;text-align:
right;"> <font color="#909AA4"><span style="padding-left:6px">July
31, 2019 at 8:45 PM</span></font></div> </div></div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvBody"
__pbrmquotes="true"
style="color:#909AA4;margin-left:24px;margin-right:24px;"><div><br>Not
sure what the Capital One data breach has to do with the cloud, it<br>was
(yet again?) misconfigured software that allowed the theft.<br><br>_______________________________________________<br>Beowulf
mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>To
change your subscription (digest mode or unsubscribe) visit
<a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><br></div>
</div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvHr"
style="margin:30px 25px 10px 25px;"><div
style="width:100%;border-top:2px solid #EDF1F4;padding-top:10px;"> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:49%;">
<a style="color:#485664
!important;padding-right:6px;font-weight:500;text-decoration:none
!important;" href="mailto:jaquilina@eagleeyet.net"
moz-do-not-send="true">Jonathan Aquilina</a></div> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:48%;text-align:
right;"> <font color="#909AA4"><span style="padding-left:6px">July
31, 2019 at 12:10 AM</span></font></div> </div></div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvBody"
__pbrmquotes="true"
style="color:#909AA4;margin-left:24px;margin-right:24px;">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"
lang="EN-GB">Hi Jon,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"
lang="EN-GB">They now have Lustre through FSx or what ever AWS have
called it. I am not sure you guys have heard about the capital one data
breach but at times im still rather weary of the cloud.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"
lang="EN-GB">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"
lang="EN-GB">Jonathan<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"
lang="en-MT"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> Jonathan Engwall <a class="moz-txt-link-rfc2396E" href="mailto:engwalljonathanthereal@gmail.com"><engwalljonathanthereal@gmail.com></a>
<br>
<b>Sent:</b> Wednesday, 31 July 2019 01:03<br>
<b>To:</b> Douglas Eadline <a class="moz-txt-link-rfc2396E" href="mailto:deadline@eadline.org"><deadline@eadline.org></a><br>
<b>Cc:</b> Jonathan Aquilina <a class="moz-txt-link-rfc2396E" href="mailto:jaquilina@eagleeyet.net"><jaquilina@eagleeyet.net></a>; Beowulf
Mailing List <a class="moz-txt-link-rfc2396E" href="mailto:Beowulf@beowulf.org"><Beowulf@beowulf.org></a>; Chris Samuel
<a class="moz-txt-link-rfc2396E" href="mailto:chris@csamuel.org"><chris@csamuel.org></a><br>
<b>Subject:</b> Re: [Beowulf] Lustre on google cloud<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">AWS has a host of free tier sercives you should
blend together. Elastic Beanstalk and Lambda (AWS proprietary lambda)
can move lots of data below a cost level.<o:p></o:p></p>
<div>
<p class="MsoNormal">Your volume will automatically cause billing
obviously. I have a friend at AWS. Maybe something new is going on, I
can check up with him.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Jul 29, 2019, 11:24 AM Douglas Eadline <<a
href="mailto:deadline@eadline.org" moz-do-not-send="true">deadline@eadline.org</a>>
wrote:<o:p></o:p></p>
</div>
</div>
</div>
<br><fieldset class="mimeAttachmentHeader"></fieldset><br><div>_______________________________________________<br>Beowulf
mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>To
change your subscription (digest mode or unsubscribe) visit
<a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><br></div>
</div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvHr"
style="margin:30px 25px 10px 25px;"><div
style="width:100%;border-top:2px solid #EDF1F4;padding-top:10px;"> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:49%;">
<a style="color:#485664
!important;padding-right:6px;font-weight:500;text-decoration:none
!important;" href="mailto:engwalljonathanthereal@gmail.com"
moz-do-not-send="true">Jonathan Engwall</a></div> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:48%;text-align:
right;"> <font color="#909AA4"><span style="padding-left:6px">July
30, 2019 at 7:03 PM</span></font></div> </div></div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvBody"
__pbrmquotes="true"
style="color:#909AA4;margin-left:24px;margin-right:24px;">
<meta http-equiv="content-type" content="text/html; charset=utf-8"><div
dir="auto">AWS has a host of free tier sercives you should blend
together. Elastic Beanstalk and Lambda (AWS proprietary lambda) can move
lots of data below a cost level.<div dir="auto">Your volume will
automatically cause billing obviously. I have a friend at AWS. Maybe
something new is going on, I can check up with him.</div></div><br>
<br><fieldset class="mimeAttachmentHeader"></fieldset><br><div>_______________________________________________<br>Beowulf
mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>To
change your subscription (digest mode or unsubscribe) visit
<a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><br></div>
</div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvHr"
style="margin:30px 25px 10px 25px;"><div
style="width:100%;border-top:2px solid #EDF1F4;padding-top:10px;"> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:49%;">
<a style="color:#485664
!important;padding-right:6px;font-weight:500;text-decoration:none
!important;" href="mailto:deadline@eadline.org" moz-do-not-send="true">Douglas
Eadline</a></div> <div
style="display:inline-block;white-space:nowrap;vertical-align:middle;width:48%;text-align:
right;"> <font color="#909AA4"><span style="padding-left:6px">July
29, 2019 at 2:04 PM</span></font></div> </div></div>
<div xmlns="http://www.w3.org/1999/xhtml" class="__pbConvBody"
__pbrmquotes="true"
style="color:#909AA4;margin-left:24px;margin-right:24px;"><blockquote
type="cite"><pre wrap="">What would be the reason for getting such large data sets back on premise?
Why not leave them in the cloud for example in an S3 bucket on amazon or
google data store.
</pre></blockquote><pre wrap="">I think this touches on the ownership issue I have seen some
people mention (I think Addison Snell or i360). That is, you own
the data but not the infrastructure.
To use the "data lake" analogy, you start
out creating a swimming pool in the cloud. You own
the water, but it is in someone else's pool. Manageable.
At some point your little pool becomes a big lake. Moving the lake,
for any number of reasons, become a really big issue and possibly
unmanageable.
"For any number of reasons" can be cost, performance, access,
etc. and the issues you never imagined (a black swan as it were)
Just like everything else, it all depends ... (and how risk adverse
you are).
--
Doug
</pre><blockquote type="cite"><pre wrap="">Regards,
Jonathan
-----Original Message-----
From: Beowulf <a class="moz-txt-link-rfc2396E" href="mailto:beowulf-bounces@beowulf.org"><beowulf-bounces@beowulf.org></a> On Behalf Of Chris Samuel
Sent: Sunday, 28 July 2019 03:36
To: <a class="moz-txt-link-abbreviated" href="mailto:beowulf@beowulf.org">beowulf@beowulf.org</a>
Subject: Re: [Beowulf] Lustre on google cloud
On Friday, 26 July 2019 4:46:56 AM PDT John Hearns via Beowulf wrote:
</pre><blockquote type="cite"><pre wrap="">Terabyte scale data movement into or out of the cloud is not scary in
2019.
You can move data into and out of the cloud at basically the line rate
of your internet connection as long as you take a little care in
selecting and tuning your firewalls and inline security devices.
Pushing 1TB/day etc.
into the cloud these days is no big deal and that level of volume is
now normal for a ton of different markets and industries.
</pre></blockquote><pre wrap="">Whilst this is true as Chris points out this does not mean that there
won't be data transport costs imposed by the cloud provider (usually for
egress).
All the best,
Chris
--
Chris Samuel : <a class="moz-txt-link-freetext" href="http://www.csamuel.org/">http://www.csamuel.org/</a> : Berkeley, CA, USA
_______________________________________________
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
<a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a>
_______________________________________________
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
<a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a>
</pre></blockquote><pre wrap="">
</pre>
</div>
</blockquote>
<br>
</body></html>