Windows HPC

[Art Edwards]

Please see insertions below.
On Thu, Aug 08, 2002 at 03:09:17PM -0400, Paul Redfern wrote:
> Dear Robert,
> Thank you for your point of view. To clarify, the grant is not for two
> machines and software, but it is actually for multiple machines over a
> four year period, development projects, staff, research, training, etc.
> Microsoft software donations are not included in the grant. Our goal is
> to help clients around the world who are interested in HPC on Windows to
> port, parallelize, and optimize their applications and systems. Those
> clients may choose to install very large clusters; our goal, however, is
> to enable their success whether it's a workstation cluster, midrange
> system, or a top 500 machine.
> 
> As far as Windows HPC goes, here's a few of the reasons universities and
> businesses are telling us they're interested in Windows clusters.
> 
> ** Reliability. We ran a 256-processor Dell cluster with Windows 2000
> and collected all errors (OS, I/O, hardware) on a secure web site for 6
> months. MIT analyzed and independently verified the up-time--99.9986%. 
> ** Existing infrastructure. Many companies (especially small and medium
> size businesses), universities, and government agencies already run
> Windows servers. The capability to scale an application from a Windows
> desktop to Windows servers interests them because they can do it on
> their existing infrastructure and on an operating system they are
> familiar with (everyone doesn't want to learn LINUX).
> "LINUX-only-HPC-or-no-HPC" may be a non-starter for many researchers and
> businesses. It ignores the Windows installed base as well as server
> growth projections (analysts predict Windows will be 1/3 of the overall
> server market by 2005; the other 1/3 will be Linux/UNIX and 1/3
Who are the analysts? 
> proprietary/legacy OS--many choices, perhaps a good thing?). Each year
> attendance by business at SCXX gets smaller and smaller. The same
> universities/labs tend to participate year after year. Is it possible
> that HPC on Windows might help expand the HPC community? That new
> classes of users might develop new applications? That businesses might
> begin to think of HPC as a service that can be accessed seamlessly from
> their Windows desktop on existing infrastructure rather than built out
> as a separate island of performance with its own hardware, OS, staff and
> inherent costs. Anyone can run Windows and some organizations want to
> leverage the environment they have.
The implicit myth is that anyone can't run Linux. My wife, and actress,
is running linux very happily using mozilla and star office. Linux is
also much easier to install when one considers the compilers, editors,
graphics, etc. that come with Linux. Also, upgrading is both free and
effortless over the web. 

> ** TCO. Studies show that the most costly element of any total cost of
> ownership study is people not software. Our clients find they can
> dramatically reduce the number of systems staff needed to run their
> Windows cluster because of the systems manageability tools provided. To
> date, the HPC community has not considered TCO as a metric. It would be
> interesting if people costs were truly measured in universities and
> government labs like they are in businesses. "Slave labor" is often used
> as a descriptor for grad students/post docs for a reason. Many
> businesses, perhaps most, want to reduce people costs. They tell us they
> want their people to focus on their "core" business, such as doing
> structural engineering design (rather than each becoming experts in
> source coding or dealing with systems administration issues). 
> ** Standardization. Many organizations are beginning to look at
> standardization as a way to reduce TCO. Windows standardization may be a
> motivation to pursue Windows HPC. A major oil company recently completed
> a TCO study that found they could reduce their IT expenditures by $210
> million per year if they standardized on Windows. Air Products and
> Chemicals, one of our clients, is standardizing their corporation
> worldwide on Windows starting with engineering and parallel HPC
> applications and then moving to the glass house. 
> ** Manageability. We were able to dramatically reduce the size of our
> systems staff with Windows clusters. More people are now available to
> focus on their science. Because they are easy to manage, we add Windows
> clusters without adding staff. We installed a 256-processor cluster and
> had it up and running, pinging processors in less than 10 hours. We've
> found that adding to existing clusters and adding new clusters as well
> as storage is fast and easy with Windows. 
Equally so with Linux. Perhaps you should talk with scientists that add
their own machines and administer them with no measurable impact on
their productivity. Might I also mention that we do not pay a per node
price for software?
> ** Desktop savings. Rather than having 2 desktops per person (one
> UNIX/Linux and one Windows) like we used to, we can now simplify things
> and go with 1 laptop and dock. Employees love it. They get their laptops
> refreshed more often. And the facility is wireless. Everyone can take
> their work home with them too (ok, not really a plus).
We have done the same thing with Linux. Now that there is star office
and evolution, I have no need for an MS box. It is a real savings not
only in hardware but in software!
> ** Performance. Results reported by Windows users are as good as or
> better than other OS's according to the users we've talked to and
> benches being run.
> ** Data access. We are using SQL server and our clients are using other
> commercial databases as data delivery engines for HPC rather than flat
> files. A lot of programming errors can be eliminated that way. There are
> a wide variety of databases to choose from that operate on the Windows
> OS.
> ** Visualization. CAVEs are now available on Windows. We are operating
> ours with 3 Dell workstations with Wildcat cards. Users love the ability
> to drag their data to H: drive, walk over the CAVE, and there it is, up
> and running. Great usability. Less visualization staff specialists (self
> service instead). And, lot's of new applications because it's easier to
> use, such as architecture design classes and engineering students
> designing new structural materials for the space shuttle replacement.
> ** Feature functionality. Some clients want Web services with HPC or
> .NET cluster on the back end. They want the parallel processing accessed
> seamlessly through a Windows program front end such as Excel.   
> ** Application availability. As Dan aptly pointed out, there are
> Windows-only applications that companies want to run in parallel and do
> not want to port. For example, El Paso energy recently contacted with
> CTC to parallelize Estimas' Time Regression Analysis software to run
> against the company's database. At Cornell we have new classes of users
> who would have never considered HPC in the past. Business school
> researchers are running large-scale manufacturing optimization problems
> for electronics companies and finance simulations (that were previously
> run on desktops for days), social scientists are doing secure data
> analysis with parallel SAS on government data, and lab biologists, who
> only know Windows, are now interested in HPC for the first time. A DOE
> lab is planning on running VASP on a Windows cluster. One hard core UNIX
> enthusiast, who first said no way he'd run on a Windows system, is now a
> regular user. When asked about Windows, he now says, "I'm OS agnostic; I
> just want to get my work done." Note: parallel SAS is a commercially
> available application for Windows, like SQL Server and VASP. CTC is
> working with a number of commercial vendors on parallel versions of
> their standard applications.
> 
> I hope this provides a little insight into some people's motiations. If
> anyone is interested in how Windows can help you with your research,
> engineering, or business intelligence, feel free to drop me an email.
> We'd be glad to work with you. CTC and our OEM/ISV team plan many
> exciting things to come. 
> Paul
> 

Just out of curiosity, now that CTC has lost NSF funding, who, besides
intel, dell and microsoft funds the center?

> Paul Redfern
> CTC High-Performance Solutions
> http://www.ctc-hpc.com
> tel 607-254-8693
> 
> 
> 
> -----Original Message-----
> From: Robert G. Brown [mailto:rgb at phy.duke.edu] 
> Sent: Tuesday, August 06, 2002 1:01 PM
> To: Paul Redfern
> Cc: beowulf at beowulf.org
> Subject: Re: Windows HPC
> 
> On Tue, 6 Aug 2002, Paul Redfern wrote:
> 
> > FYI .
> > 
> > Cornell Joins Forces with Dell, Intel, and Microsoft to Expand Usage
> of
> > High Performance Cluster Computing in the Corporate Data Center
> > 
> > Aug. 5, 2002 - Cornell Theory Center (CTC) today announced an
> agreement
> > with Dell, Intel, and Microsoft to develop and deliver CTC
> > High-Performance Solutions, a suite of industry standards-based
> > high-performance computing (HPC) solutions and services for business,
> > government and academic clients. The agreement provides $60 million
> > worth of resources over the next four years to aid in solutions
> > development.
> 
> Microsoft discovers the beowulf concept?  Again?
> 
> $60 million worth of "resources" -- would that be $30 million worth of
> hardware and $30 million dollars worth of full retail Microsoft software
> (which costs them nothing but media to provide)?  Let's see, 425 new
> Dell servers at an absolutely insane highball price of $5K each is all
> of $2.5M, adding $5K in infrastructure costs including gold-plated
> networking -- hard for me to see more than $5 million in hardware, $10
> million if they completely replace the other 425 "servers" (with enough
> slop in my estimates to likely build a building just to house all this).
> 
> So how about $10M in hardware and infrastructure, $10M in salaries
> (that's enough salary money for at least 10 full professors and their
> students for four years even at Cornell:-) and $40M in MS software?
> Enquiring minds like to know...
> 
> Hmmm, Win2K server licenses for (say) 1000 seats -- ooo, that's a lot of
> money.  Compilers and programming tools are expensive.  No compute node
> is complete without Office;-).  And so forth.
> 
> Now the interesting question is -- who in their right mind would ever
> buy into this?  A linux cluster can be installed completely over the
> network from PXE boots in an unbelievably short period of time, and its
> software maintenance is completely automatable.  A single person can run
> a cluster with hundreds of nodes, and we can build a cluster with
> hundreds of nodes for around $2500/node INCLUDING the cost of building
> server-level infrastructure (AC and power).  Software costs scale
> perfectly (ranging from marginal costs of perhaps 3-5 minutes of human
> time per node per month after configuring the cluster servers and
> installing the first node, all opportunity cost and $0-100 per node for
> software per se, depending on what you choose to install and how many
> nodes you've got).
> 
> All the source is open, so one can FIX the numerous problems encountered
> trying to write and implement HPC parallel code running over a network.
> Or at least understand them and work around them.
> 
> Then there is security.  WinXX security is legendary.  Really.
> 
> Finally, there are all the lovely aspects of Microsoft's rumored future
> pricing models.  Not only will one have to buy the original software,
> but (whether you like it or not, or need to or not) one will have to
> repurchase it all over again, year after year, and the joy of being
> locked in by the outrageous DMCA for anything "interesting" that you
> ever develop that uses proprietary libraries or tools to implement.
> 
> Not to mention the personal satisfaction that comes from helping
> perpetuate the largest and most ruthless monopoly that the world has
> ever seen, which quite literally reduces your personal freedom as it
> manipulates the political process to protect its "right" to squash
> competition and have the use of Microsoft products mandated by law.
> 
> I'm not impressed.  I can hardly blame Cornell for taking the money --
> even an inflated $60M undoubtedly makes them a lot of money, and
> starving professors do have to eat (I'm being serious here, not
> facetious -- I'm a starving professor myself:-).
> 
> Still, give me TWENTY million dollars and I will install a cluster with
> many times the capacity implied here and run it four years and do all
> sorts of cool research and development, especially if I get actual cash
> and not the preconfigured servers that Intel/Dell tends to give away in
> these projects, which have heavily inflated price tags and heavily
> deflated price/performance.  Oh, and this includes spending $1M or two
> building or remodelintg a facility for the cluster.
> 
>    rgb
> 
> > 
> > CTC High-Performance Solutions will be based on Dell PowerEdge
> servers;
> > IntelR XeonT and ItaniumR family processors and tools; and running
> > Microsoft Server software. This combination is designed to provide
> > customers with the performance and availability once only achieved by
> > proprietary supercomputers at a fraction of the price. CTC will double
> > the size of its existing 425-server Dell, Intel and Windows-based HPC
> > clusters as a result of this agreement. With the standards-based
> > technologies in CTC's clusters, it can provide users with documented
> > high performance, reliability and security while functioning at
> > significantly reduced total cost of ownership when compared to the
> > proprietary supercomputer CTC previously used. 
> > 
> > HPC clustering has been successfully used in university and research
> > environments for years to solve complex problems, but also has many
> > practical applications for business such as scalable online
> transaction
> > processing with Web clients, decision support systems, engineering
> > design and analysis, bioinformatics and more. CTC High-Performance
> > Solutions will apply its Windows HPC expertise to accelerate the
> > deployment and scale out of Windows-based IT infrastructure in the
> > private sector. 
> > 
> > CTC High-Performance Solutions will develop robust Windows HPC
> solution
> > stacks for broad industry deployment, and will include HPC services
> such
> > as UNIX to Windows code porting, optimization, and porting to parallel
> > environments; systems planning and integration; systems and
> applications
> > training and testing; benchmarking. CTC will also offer
> high-performance
> > Web services based on Microsoft's .NET software and SQL Server. CTC's
> > TechExchange Consortium will provide members with more immediate
> access
> > to IT technologies and will help drive the evolution of Windows HPC. 
> > 
> > In addition, CTC will establish a technology showcase for
> > proof-of-concept applications for HPCC in the financial district of
> New
> > York City. This facility will be linked to related activities in CTC's
> > Ithaca, N.Y., laboratories and will serve as the setting for customers
> > to pilot projects. 
> > 
> > "Establishment of CTC High-Performance Solutions comes at a time when
> > all sectors of the economy face increasing competition, pressure on
> > margins, and the need to demonstrate strong and quick returns on
> > investment," said Thomas F. Coleman, CTC director and Cornell computer
> > scientist. "With our expanded relationships and combined strengths, we
> > can show companies, government agencies, and academic institutions how
> > to expand their technical computing environment, while reducing their
> > overall IT budget. They can take their existing expensive, proprietary
> > systems, which are often islands of performance requiring extra
> systems
> > staff, and replace them with a more flexible, scale-out clustered
> > environment that is expandable and that fits in the overall
> > Windows-based office environment." 
> > 
> > "Cornell Theory Center is playing an important leadership role in
> > Windows Server-based high-performance computing," said Brian
> Valentine,
> > Microsoft Senior Vice President, Windows Division. "They were first to
> > move completely to Windows for HPC. They have shown that it works in
> the
> > most demanding settings. And they will be instrumental in moving HPC
> out
> > of the research environment and into the mainstream industry. As we
> work
> > together with CTC, Dell, and Intel, the efforts coming out of this
> > agreement will very clearly show Windows brings the highest value to
> > high-performance computing applications and companies' business
> systems
> > on an industry standards-based IT platform."
> > 
> > "The flexibility, performance and cost-effectiveness of Dell PowerEdge
> > servers with Intel technology is becoming more and more attractive to
> > customers in research-intensive industries outside of the university,
> > due in part to initiatives like CTC's Windows program," said Russ
> Holt,
> > vice president of Dell's Enterprise Systems Group. "Through Dell's own
> > HPCC program, we continue to see customers replacing legacy,
> proprietary
> > systems with Intel-based HPC clusters." 
> > 
> > "Intel continues to invest in HPC to propel the industry forward and
> > drive innovation using Intel's volume economics model - delivering
> > absolute performance, price/performance, flexibility and choice to
> > enable supercomputing for the masses," said Mike Fister, senior vice
> > president and general manager, Intel Enterprise Platforms Group.
> "Using
> > the industry-leading floating point performance of the Intel Itanium 2
> > processor and the world-class price/performance of the Intel Xeon
> > processor, CTC High-Performance Solutions will help accelerate the
> > migration of leading-edge computational research into the corporate
> data
> > center of the future."
> > 
> > "This tremendous investment by Dell, Intel and Microsoft in the
> Cornell
> > Theory Center is a true vote of confidence in the intellectual power
> of
> > one of our State's finest academic institutions," said Governor
> Pataki.
> > "Industry, university and government collaboration is critical to
> > economic success in our State and throughout the nation, especially in
> > the fast-paced world of information technology. This project is a
> prime
> > example of how expertise at New York State's top-flight universities
> can
> > help industry solve complex problems that will benefit all sectors,
> > public and private." 
> > 
> > For more information about CTC High-Performance Solutions, visit
> > http://www.ctc-hpc.com.
> > 
> > About the Cornell Theory Center
> > CTC is a high-performance computing and interdisciplinary research
> > center located on the Ithaca campus of Cornell University. CTC
> currently
> > operates a Dell/Intel/Windows cluster complex consisting of more than
> > 900 processors. Scientific and engineering projects supported by CTC
> > represent a vast variety of disciplines, including bioinformatics,
> > behavioral and social sciences, computer science, engineering,
> > geosciences, mathematics, physical sciences, and business.
> > 
> > About CTC Systems
> > CTC's Systems are configured into general purpose, strategic
> > application, and dedicated clusters. Among the dedicated research
> > clusters housed at CTC are a 64-node system devoted to computational
> > materials, 64 nodes for computational biology solutions, 32 nodes to
> > support the USDA-ARS Center for Agricultural Bioinformatics, and 32
> > nodes dedicated to social and economic research. CTC also provides a
> > novel Windows/Dell/Intel 3D, stereo immersive CAVE environment for
> > scientific visualization. 
> > 
> > Note: Intel, Itanium and Xeon are trademarks or registered trademarks
> of
> > Intel Corporation or its subsidiaries in the United States and other
> > countries. Dell and PowerEdge are trademarks or registered trademarks
> of
> > Dell Corporation in the United States and/or other countries.
> Microsoft,
> > Windows, SQL Server, and .NET are trademarks or registered trademarks
> of
> > Microsoft Corp. in the United States and/or other countries. Other
> names
> > and brands may be claimed as the property of others.
> > 
> > Paul Redfern            
> > 
> > red at tc.cornell.edu
> > 
> >  
> > 
> > 
> 
> Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
> Duke University Dept. of Physics, Box 90305
> Durham, N.C. 27708-0305
> Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu
> 
> 
> 
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf