Fwd: [Mauiusers] March 2003 Newsletter
Ron Chen
ron_chen_123 at yahoo.com
Wed Mar 12 20:49:37 PST 2003
FYI,
-Ron
--- help at supercluster.org wrote:
> Dear user,
> In today's newsletter we are announcing the
> availability of MAUI SCHEDULER
> 3.2.6, continued rapid growth of MAUI usage, added
> resource manager
> support and more.
>
>
> News/Announcements:
>
> 1. Usage of MAUI SCHEDULER is growing significantly
> (Downloads exceed
> 250% of previous year)
> 2. MAUI SCHEDULER 3.2.6 (Alpha) is now available
> 3. Added resource manager support (SGE 5.3, LSF
> 5.1, Loadleveler 3.1,
> PBSPro 5.2)
> 4. Support for the Department of Energy's SSS
> standard (Scalable System
> Software)
> 5. New XML support on Client and Peer Services
> 6. CLUSTER RESOURCES, INC. - Free consultation
> offer
>
> Tips and Tricks:
> · How to use MAUI's Advanced Preemption Support
>
>
> Please support our MAUI awareness initiative and add
> a link to -
> supercluster.org or clusterresources.com or add a
> web banner that shows
> you're a proud user
> (www.supercluster.org/mauilink.html)
>
>
>
*****************************************************************************
> Usage of MAUI SCHEDULER is growing significantly
>
> There has been continued strong growth of MAUI
> SCHEDULER adoption, with
> 2002 downloads reaching 250% of the previous year,
> and the growth
> continues. Web site hits last year were over 80,000
> hits per month, and
> have already grown an additional 50% in just the
> first quarter of this
> year. Contributing to the growth has been
> significant feature
> advancements, improved usability, and support for a
> broader range of
> resource managers. Also, your efforts to grow the
> community of
> MAUI-knowledgeable experts have been significant and
> much appreciated.
> Please feel free to suggest feature enhancements at
> any time by sending
> and email to support at clusterresources.com, and thank
> you again for helping
> to get the word out with your peers.
>
>
>
*****************************************************************************
> MAUI SCHEDULER 3.2.6 (Alpha) is now available
>
> MAUI SCHEDULER 3.2.6 (Alpha) is now available for
> use. It has no known
> stability or compatibility issues and is already
> being used on over 100
> systems. To gain access to 3.2.6 please contact us
> at
> help at supercluster.org. We will email you a username
> and password and
> then you can proceed to download it at
>
> http://supercluster.org/downloads/maui/alpha.
>
> This release adds support for the SSS 1.0
> Protocol, provides additional
> resource manager interfaces, and utilizes XML for
> client and peer
> interfaces (see sections below for additional
> details). Further, 3.2.6
> adds 64 bit support as well as a feature known as
> 'Standalone Mode'.
> Standalone Mode allows you to perform simple load
> balancing with MAUI
> SCHEDULER without requiring a resource manager.
> Using the COMPUTEHOST
> parameter, sites can specify the list of nodes they
> want Maui to consider
> and track. The new mbal command allows a user to
> load balance either
> logins or remote command execution. If a command
> argument is specified,
> MAUI will launch this command on the most lightly
> loaded node. If no
> command is specified, a login shell to that node
> will be launched.
>
> While partially available in 3.2.5, more advanced
> preemption features
> have been added to MAUI SCHEDULER since our last
> newsletter (See Tips and
> Tricks: How to use MAUI's Advanced Preemption
> Support).
>
>
>
*****************************************************************************
> Added resource manager support (SGE 5.3, LSF 5.1,
> Loadleveler 3.1, PBSPro 5.2)
>
> In an effort to maintain compatibility with the
> latest of both open and
> proprietary resource managers, we have added support
> for SGE 5.3, LSF
> 5.3, Loadleveler 3.1 and PBSPro 5.2. MAUI SCHEDULER
> now supports these
> latest four additions plus OpenPBS 2.3, BProc, WIKI
> and SSS 1.0. For
> integration guides for SGE, PBSPro and others visit
> http://supercluster.org/documentation.
>
>
>
*****************************************************************************
> Support for the Department of Energy's SSS Protocol
> (Scalable System
> Software)
>
> MAUI SCHEDULER 3.2.6 adds support for SSS 1.0. The
> SSS protocol is a
> resource management protocol that provides
> standardized interfaces for
> resource managers, queue managers, allocation
> managers, cluster
> schedulers, grid schedulers, event management
> systems, and information
> services. Reference implementations of these tools
> have been and are
> being developed which target the massive compute
> systems of the future.
> Design criteria are being used to ensure efficient,
> secure, and flexible
> control of clusters possessing in excess of 50,000
> processors. SSS
> Protocols and tools are being developed by the U.S.
> Department of Energy
> via a collaboration of 8 research laboratories.
>
> MAUI SCHEDULER 3.2.6 and higher can be configured to
> utilize resource
> management tools conforming to the SSS protocol. As
> part of this
> initiative, there have also been significant
> enhancements to the
> scheduler client and peer interfaces, the scheduling
> feature set,
> security options, and scalability. Maui/SSS
> installations are currently
> being tested at a number of your favorite leading
> edge sites. Further
> details can be found on the Scalable Software
> Systems Homepage -
> (http://www.scidac.org/ScalableSystems).
>
>
>
*****************************************************************************
> New XML support on Client and Peer Services
>
> MAUI SCHEDULER 3.2.6 and higher have added XML
> support for Client and
> Peer Services. For a subset of commands, the
> '--format=XML' command line
> flag can be used to provide an XML response string.
> This string may
> present data in a format that is more amenable to
> use by peer services,
> databases, and GUIs. Clients at various stages of
> XML support now
> include mshow (showq), showstate, showres, checknode
> , mjobctl (runjob,
> setjobhold, setspri, canceljob), and mresctl
> (setres, releaseres).
>
>
>
*****************************************************************************
> CLUSTER RESOURCES INC. - Free consultation offer
>
> Call CLUSTER RESOURCES, INC. today (801) 873-3400
> and get a free
> consultation on what MAUI SCHEDULER and SILVER can
> do for you and how
> best to deploy it within your environment. The
> Catch: It's Free, but we
> will better understand your needs and you will
> better understand what we
> can do for you and how our services can make you
> more successful. Maybe
> you'll realize how much money we can save you.
>
> The developers of Maui and founders of
> Supercluster.org formed CLUSTER
> RESOURCES INC., in an effort to provide customers
> access to professional
> services supporting their use of MAUI SCHEDULER, &
> SILVER. See
> www.clusterresources.com for details.
>
>
>
*****************************************************************************
> Tips & Tricks:
>
> How to use MAUI's Advanced Preemption Support
>
> MAUI SCHEDULER has had limited preemption support
> since 3.0.7, but Maui
> 3.2.6 has added important new preemption features.
> Note: Preemption
> features are currently available for PBS,
> Loadleveler and LSF
> environments.
>
> Basic preemption of 3.0.7 and higher:
> It is almost always the case that sites possess
> workloads of varying
> importance. While it may be critical that some jobs
> obtain resources
> immediately, other jobs are less 'response time'
> sensitive but have an
> insatiable hunger for compute cycles, consuming
> every available cycle for
> weeks, months or even years on end. The preemption
> feature enables
> systems to run low priority, preemptible jobs
> whenever something more
> pressing is not running. MAUI SCHEDULER makes this
> feature available
> through its QoS-based preemption system. Certain
> QoS's may be marked
> with the flag PREEMPTOR, others with the flag
> PREEMPTEE. With this
> configuration, low priority, preemptee jobs can be
> started whenever idle
> resources are available. These jobs will be allowed
> to run until a
> higher priority preemptor job arrives, at which
> point the preemptee job
> will be checkpointed if possible and vacated. This
> allows near immediate
> resource access for the higher priority preemptor
> job.
>
> Using this approach, a cluster can maintain near
> 100% system utilization
> while still delivering excellent turnaround time to
> the jobs of greatest
> value. Use of the preemption system need not be
> limited to controlling
> low priority jobs. Other uses include optimistic
> scheduling and
> development job support.
>
> Example:
> QOSCFG[high] PRIORITY=10000 QFLAGS=PREEMPTOR
> QOSCFG[med]
> QOSCFG[low] QFLAGS=PREEMPTEE
>
> Advanced preemption of 3.2.6 and higher:
> MAUI SCHEDULER 3.2.6 and higher has preemption-based
> backfill and policy
> violation based preemption. The first feature,
> preemption-based
> backfill, allows an organization to combine the
> familiar backfill
> optimization policy with preemption. The value in
> this is that it now
> allows 'optimistic' scheduling. Normally backfill
> allows high priority
> jobs to receive their 'earliest job start'
> guarantee, and allows lower
> priority jobs to be backfilled into available
> resources as long as they
> do not interfere with the 'earliest job start' of
> the high priority jobs.
> Adding in the preemption element allows even more of
> the low priority
> jobs to be run, since now they can be
> 'optimistically' started even if
> they cannot be finished in a particular open slot of
> resources. If the
> low priority backfill job runs too long, it would
> simply be preempted,
> and if possible, checkpointed at the partially
> completed point. When
> there was another available resource opening, the
> low priority job could
> continue. Preemption based backfill is not a
> panacea and may not improve
> performance in all cases. Each site should consider
> their workload and
> environment closely before enabling this policy, so
> feel free to contact
> us for a site-specific recommendation.
>
> The second feature, 'policy violation based
> preemption', is actually
> quite a useful feature. This feature allows an
> organization to specify
> per job policy based limitations, and then, if said
> limits are violated,
> allow another job that is compliant to preempt the
> violating job. This
> allows an organization that has competing internal
> interests to manage
> within a set of established policies in such a
> manner that violations do
> not simply cancel a job or cause it to fail, but
> rather automatically
> allow it to be preempted by compliant job requests,
> and then allow it to
> continue once resources are available again. This
> maintains fairness,
> maximizes utilization, and adds the flexibility for
> violating jobs to run
> on available resources when they don't compete with
> jobs that are in
> compliance.
>
> Example:
>
> # suspend jobs which violate memory usage limits and
> share resources
> # with other jobs
> RESOURCELIMITPOLICY MEM:BLOCKEDWORKLOADONLY:SUSPEND
>
> # cancel jobs which exceed CPU targets for more than
> 10 minutes
> RESOURCELIMITPOLICY PROC:ALWAYS:CANCEL:00:10:00
>
> Documentation on this policy and other 3.2.6
> features is available to all
> registered visitors; to gain access to 3.2.6 as well
> as the supporting
> documentation please contact us at
> help at supercluster.org
>
>
>
*****************************************************************************
> Highlight/Did you Know:
>
> Did you know supercluster.org/clusterresources.com
> reached the 1,000,000
> hits/year mark in 2002, and already this year we are
> at a 50% higher run
> rate, just part way into the first quarter.
>
> ***** View online at
> http://clusterresources.com/newsletter/march2003.htm
> *****
>
______________________________________________________
> Support Maui - link to supercluster.org or
> clusterresources.com using a
> URL or the web banners found at
> www.supercluster.org/mauilink.html
>
> Contact Us:
> Tech. Support, Consulting, Custom Development
> sales at clusterresources.com
> Product Feedback help at supercluster.org
>
> --
>
--------------------------------------------------------
> Supercluster Development Group
> Scheduling and Resource Management of Clusters and
> Grids
> Maui Home Page - http://supercluster.org/maui
> Silver Home Page - http://supercluster.org/silver
> Documentation -
> http://supercluster.org/documentation
>
>
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://supercluster.org/mailman/listinfo/mauiusers
__________________________________________________
Do you Yahoo!?
Yahoo! Web Hosting - establish your business online
http://webhosting.yahoo.com
More information about the Beowulf
mailing list