[Beowulf] Re: OT: LTO Ultrium (3) throughput?
gerry.creager at tamu.edu
Sat Jul 5 08:03:26 PDT 2008
In my data management exploits, I'm inclined to have first-tier (iSCSI)
disk, second-tier (AoE) disk, and third-tier (remote site) storage. If
I can manage the remote site as another storage server farm, with
rotating media, great. If I can manage it with robotic tape, great. I
still duplicate Tier 2 data to Tier 3 for disaster recovery.
A lot of this depends on how serious you are about being able to get
your data back. Even though I can tap the ultimate archival site for
the meteorological data I retain, translating it from netcdf to database
is time-consuming and requires a human to babysit at times.
Being able to respond nearly immediately to user requests for data from
the Tier 1 data makes our services more valuable (and makes my work with
data assimilation for weather models easier/faster). I retain some 90
days of data on Tier 1. Requests for data floated off to Tier 2 take
longer to fill but the data holdings are, for all intents and purposes,
permanent. Takes longer to get the data off but users know and
understand that, and a simple e-mail tells 'em it's ready.
Permanent, less-volatile Tier 3 storage is disaster-recovery stuff.
Similarly, for hurricanes making US landfall, we also store data away on
DVD to make its retrieval a (little) bit easier to locate. We use a
database to maintain an inventory of where things are on disk, with
significant file metadata, but sometimes it's easier to go to the DVD
storage case to retrieve that stuff.
If you're not as worried about how you'll recover your data after the
inevitable storage failure (ask me about burning a RAID shelf down, some
day), then not worrying about diversity in data storage/management isn't
as big an issue.
Jon Aquilina wrote:
> what i dont understand is why someone would want to invest in something
> that is already quite expensive instead of using a method which is not
> expensive and in a way provides double redundency.
> On Sat, Jul 5, 2008 at 2:27 PM, Chris Samuel <csamuel at vpac.org
> <mailto:csamuel at vpac.org>> wrote:
> ----- "John Hearns" <john.hearns at streamline-computing.com
> <mailto:john.hearns at streamline-computing.com>> wrote:
> > - the smart thing being that any files which are unchanged since the
> > last backup are links to the first copy of the file. So your vault
> > size does not grow and grow endlessly. You can roll back to any given
> > date.
> FWIW BackupPC claims to do the same, extending that to
> duplicate copies across multiple machines. Of course
> then you want to be sure that the single copy you have
> on disk doesn't go bad..
> Christopher Samuel - (03) 9925 4751 - Systems Manager
> The Victorian Partnership for Advanced Computing
> P.O. Box 201, Carlton South, VIC 3053, Australia
> VPAC is a not-for-profit Registered Research Agency
> Beowulf mailing list, Beowulf at beowulf.org <mailto:Beowulf at beowulf.org>
> To change your subscription (digest mode or unsubscribe) visit
> Jonathan Aquilina
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
More information about the Beowulf