[Beowulf] High Performance for Large Database

[Michael Will]

On Wednesday 27 October 2004 03:39 am, Keith Murphy wrote:
> Check out this url http://www.linuxlabs.com/clusgres.html they look like
> they have a solution for scaleable Postgres
> 
> Kindest Regards
> 
> Keith Murphy
> Dolphin Interconnect

Hey Keith, that is a really cool link. What interconnect does
that lock them into again, though?

On a more serious side:

They advertise a beowulf-with-shared-memory solution, which demands
low latency high bandwidth interconnects, and AFAIK they only support
Dolphin Interconnect (SCI). Has anybody tried their product yet and can
comment on its efficiency and scalability ? It does sound promising for
any SMP type software that does not run well on a cluster because of
its lack of shared memory.

Also check out mysql.com's in-ram database product - they created a
database that is not relying on any shared memory, but instead redundandly 
distributes the data out onto a cluster, using RAM only and claiming to
be really fast.

http://www.mysql.com/products/cluster/

And then there is oracle that advertises together with Infinicon, HP and AMD they
would have set a new TPC-H One-Terabyte record:

http://www.oracle.com/corporate/press/home/index.html

Michael

> 818-292-5100
> kmurphy at dolphinics.com
> www.dolphinics.com
> ----- Original Message ----- 
> From: "Laurence Liew" <laurenceliew at yahoo.com.sg>
> To: "Joshua Marsh" <icub3d at gmail.com>
> Cc: <beowulf at beowulf.org>
> Sent: Wednesday, October 27, 2004 3:29 AM
> Subject: Re: [Beowulf] High Performance for Large Database
> 
> 
> > Hi,
> >
> > You may wish to search thru the beowulf list or google for "beowulf and
> > databases and postgresql"... there were a couple of threads on exeactly
> > this issue.
> >
> > Very briefly
> >
> > 1. Beowulf clusters CANNOT help make Postgresql or any databases run
> > faster. You need the database code to be modified to do that (think
> > Oracle 10g). I met a company at Supercomputer 03 last year that had
> > Mysql running on a cluster... you may wish to query for them.
> >
> > 2. You could try to sponsor the development of a parallel postrgresql -
> > talk to the postgresql development team... when I broached the idea in
> > 1998.. there was some interest.. unfortunately.. I could not afford the
> > development/sponsorship costs then.
> >
> > 3. Try running Postgresql on a cluster filesystem like PVFS - it is not
> > gauranteed as it probably fails the ACID test for a SQL compliant
> > database. The basic idea is that if we cannot parallelise the database -
> > we make the underlying IO parallel and hence boost the IO performance of
> > the system.. and any applications that run on them.. and this includes
> > Postgresql.
> >
> > Hope this helps.
> >
> > Cheers!
> > Laurence
> > Scalable Systems
> > Singapore
> >
> >
> >
> > Joshua Marsh wrote:
> > > Hi all,
> > >
> > > I'm currently working on a project that will require fast access to
> > > data stored in a postgreSQL database server.  I've been told that a
> > > Beowulf cluster may help increase performance.  Since I'm not very
> > > familar with Beowulf clusters, I was hoping that you might have some
> > > advice or information on whether a cluster would increase performance
> > > for a PostgreSQL database.  The major tables accessed are around
> > > 150-200 million records.  On a stand alone server, it can take several
> > > minutes to perform a simple select query.
> > >
> > > It seems like once we start pricing for servers with 16+ processors
> > > and 64+ GB of RAM, the prices sky rocket.  If I can acheive high
> > > performance with a cluster, using 15-20 dual processor machines, that
> > > would be great.
> > >
> > > Thanks for any help you may have!
> > >
> > > -Josh
> > > _______________________________________________
> > > Beowulf mailing list, Beowulf at beowulf.org
> > > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> > >
> >
> 
> 
> ----------------------------------------------------------------------------
> ----
> 
> 
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org
> > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> >
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 

-- 
Michael Will, Linux Sales Engineer
NEWS: We have moved to a larger iceberg :-)
NEWS: 300 California St., San Francisco, CA.
Tel:  415-954-2822  Toll Free:  888-PENGUIN
Fax:  415-954-2899 
www.penguincomputing.com