[Beowulf] Lustre on google cloud

[Jonathan Aquilina]

Hi Jon,

They now have Lustre through FSx or what ever AWS have called it. I am not sure you guys have heard about the capital one data breach but at times im still rather weary of the cloud.

Regards,
Jonathan

From: Jonathan Engwall <engwalljonathanthereal at gmail.com>
Sent: Wednesday, 31 July 2019 01:03
To: Douglas Eadline <deadline at eadline.org>
Cc: Jonathan Aquilina <jaquilina at eagleeyet.net>; Beowulf Mailing List <Beowulf at beowulf.org>; Chris Samuel <chris at csamuel.org>
Subject: Re: [Beowulf] Lustre on google cloud

AWS has a host of free tier sercives you should blend together. Elastic Beanstalk and Lambda (AWS proprietary lambda) can move lots of data below a cost level.
Your volume will automatically cause billing obviously. I have a friend at AWS. Maybe something new is going on, I can check up with him.

On Mon, Jul 29, 2019, 11:24 AM Douglas Eadline <deadline at eadline.org<mailto:deadline at eadline.org>> wrote:

> What would be the reason for getting such large data sets back on premise?
> Why not leave them in the cloud for example in an S3 bucket on amazon or
> google data store.

I think this touches on the ownership issue I have seen some
people mention (I think Addison Snell or i360). That is, you own
the data but not the infrastructure.

To use the "data lake" analogy, you start
out creating a swimming pool in the cloud. You own
the water, but it is in someone else's pool. Manageable.
At some point your little pool becomes a big lake. Moving the lake,
for any number of reasons, become a really big issue and possibly
unmanageable.

"For any number of reasons" can be cost, performance, access,
etc. and the issues you never imagined (a black swan as it were)

Just like everything else, it all depends ... (and how risk adverse
you are).

--
Doug



>
> Regards,
> Jonathan
>
> -----Original Message-----
> From: Beowulf <beowulf-bounces at beowulf.org<mailto:beowulf-bounces at beowulf.org>> On Behalf Of Chris Samuel
> Sent: Sunday, 28 July 2019 03:36
> To: beowulf at beowulf.org<mailto:beowulf at beowulf.org>
> Subject: Re: [Beowulf] Lustre on google cloud
>
> On Friday, 26 July 2019 4:46:56 AM PDT John Hearns via Beowulf wrote:
>
>> Terabyte scale data movement into or out of the cloud is not scary in
>> 2019.
>> You can move data into and out of the cloud at basically the line rate
>> of your internet connection as long as you take a little care in
>> selecting and tuning your firewalls and inline security devices.
>> Pushing  1TB/day etc.
>> into the cloud these days is no big deal and that level of volume is
>> now normal for a ton of different markets and industries.
>
> Whilst this is true as Chris points out this does not mean that there
> won't be data transport costs imposed by the cloud provider (usually for
> egress).
>
> All the best,
> Chris
> --
>   Chris Samuel  :  http://www.csamuel.org/  :  Berkeley, CA, USA
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
>


--
Doug

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20190731/a856af6d/attachment-0001.html>