From eugen at leitl.org  Fri Aug  3 06:19:54 2012
From: eugen at leitl.org (Eugen Leitl)
Date: Fri, 3 Aug 2012 15:19:54 +0200
Subject: [Beowulf] MPI-3.0 public draft and call for comments
Message-ID: <20120803131953.GT12615@leitl.org>

----- Forwarded message from Rolf Rabenseifner <rabenseifner at hlrs.de> -----

From: Rolf Rabenseifner <rabenseifner at hlrs.de>
Date: Fri,  3 Aug 2012 14:32:20 +0200 (CEST)
To: eugen at leitl.org
Subject: MPI-3.0 public draft and call for comments

[Our apologies if you receive multiple copies or if you are not related to MPI]

Dear MPI user,

The MPI-Forum is about to ratify the MPI-3.0, a new version of the MPI standard.
As part of this process, we are soliciting feedback on the current draft standard.
The draft document can be found here:

 http://meetings.mpi-forum.org/mpi3.0_draft_2.pdf 

We are seeking the following feedback in particular:

- Small requests/corrections that can be resolved before finishing 3.0.
- Requests for clarification of unclear text or undefined behavior.
- Undetected severe inconsistencies or bugs that may delay the
  standard publication.
- Wishes for additional enhancements or new functionality
  to be taken into account after 3.0 is published.

Please comment before Sep. 6, 2012 as follows:

1. Subscribe to: http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments
2. Then send your comments to: mpi-comments at mpi-forum.org , 
   together with the URL of the version of the MPI standard and 
   the page and line numbers on which you are commenting. 

Messages sent from an unsubscribed e-mail address will not be considered.

Thank you in advance for your help.

Best regards,

The MPI Forum 

 
( www.mpi-forum.org ) 

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE


From deadline at eadline.org  Fri Aug  3 17:50:40 2012
From: deadline at eadline.org (Douglas Eadline)
Date: Fri, 3 Aug 2012 20:50:40 -0400
Subject: [Beowulf] List status and test
Message-ID: <f699f7402e63081f01ccd45ede3a3958.squirrel@mail.eadline.org>


The list has been down for about two weeks. Penguin is
working on the situation. I am also talking with
Penguin about some community members assisting with
moderation.

-- 
Doug

-- 
Mailscanner: Clean



From diep at xs4all.nl  Wed Aug  8 15:40:23 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Thu, 9 Aug 2012 00:40:23 +0200
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
Message-ID: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>

hi,

How do you guys monitor the CPU core temperatures?

if i run lm_sensors, it's 30C higher at every node than a few nodes i  
tried compare with windows.
Also under full load it reports temperatures like end 60s and up to  
78C i've seen reported.
Am guessing it should be 30-40+ at most.

It blows cool air from and outside the cpu's. Nothing is even 'warm'.

Nodes here: supermicro X7DWE inside Xeons L5420. They are not  
overclocked.

I also downloaded some similar motherboards definitions - seems they  
uploaded it for motherboards with dual core Xeons
and such, not for the quadcores. None of those defines 'corrects' the  
temperature of the quadcore Xeons, they basically kick out
readings that are not getting used.

Now i bet several clusters/supercomputers had these cpu's. How did  
you solve this problem with the intel L5420's?

Maybe someone still has the lm_sensors script lying around somewhere  
fixing it for the intel Xeons?

Thanks in advance,
Vincent






From hearnsj at googlemail.com  Thu Aug  9 00:21:38 2012
From: hearnsj at googlemail.com (John Hearns)
Date: Thu, 9 Aug 2012 08:21:38 +0100
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
Message-ID: <CAPqNE2Ucj-WkvcHtE97HXemEZ_u1eY+38RnokAM7tM3wb4c9Zg@mail.gmail.com>

Well, I don't use lm_sensors for a start!
Use the ipmitool utility to probe the readings from BMC cards (ILO,
DRAC, they're the same thing).
I don;t trust the absolute calibration of the sensors - generally
you're looking at setting a limit on which to alarm or shutdown so
just take a reading under no load on the CPU and call that the
'normal' reading.
I may be wrong. YMMV.

On 08/08/2012, Vincent Diepeveen <diep at xs4all.nl> wrote:
> hi,
>
> How do you guys monitor the CPU core temperatures?
>
> if i run lm_sensors, it's 30C higher at every node than a few nodes i
> tried compare with windows.
> Also under full load it reports temperatures like end 60s and up to
> 78C i've seen reported.
> Am guessing it should be 30-40+ at most.
>
> It blows cool air from and outside the cpu's. Nothing is even 'warm'.
>
> Nodes here: supermicro X7DWE inside Xeons L5420. They are not
> overclocked.
>
> I also downloaded some similar motherboards definitions - seems they
> uploaded it for motherboards with dual core Xeons
> and such, not for the quadcores. None of those defines 'corrects' the
> temperature of the quadcore Xeons, they basically kick out
> readings that are not getting used.
>
> Now i bet several clusters/supercomputers had these cpu's. How did
> you solve this problem with the intel L5420's?
>
> Maybe someone still has the lm_sensors script lying around somewhere
> fixing it for the intel Xeons?
>
> Thanks in advance,
> Vincent
>
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>


From samuel at unimelb.edu.au  Sun Aug 12 20:53:56 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Mon, 13 Aug 2012 13:53:56 +1000
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <CAPqNE2Ucj-WkvcHtE97HXemEZ_u1eY+38RnokAM7tM3wb4c9Zg@mail.gmail.com>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
	<CAPqNE2Ucj-WkvcHtE97HXemEZ_u1eY+38RnokAM7tM3wb4c9Zg@mail.gmail.com>
Message-ID: <50287A54.2090700@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/08/12 17:21, John Hearns wrote:

> Use the ipmitool utility to probe the readings from BMC cards
> (ILO, DRAC, they're the same thing).

We do the same..

cheers!
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAoelQACgkQO2KABBYQAh+PPQCeLRo7Czlo6iVJ460TVlYVBqYD
I/MAoJP9DpuAeDaKAOjbGbA7iHzyoB3h
=KEkX
-----END PGP SIGNATURE-----


From diep at xs4all.nl  Mon Aug 13 00:03:58 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Mon, 13 Aug 2012 09:03:58 +0200
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <50287A54.2090700@unimelb.edu.au>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
	<CAPqNE2Ucj-WkvcHtE97HXemEZ_u1eY+38RnokAM7tM3wb4c9Zg@mail.gmail.com>
	<50287A54.2090700@unimelb.edu.au>
Message-ID: <81038DC3-155D-4A5D-AE62-970548C5F3A3@xs4all.nl>

I'll try that - thanks.

Maybe the intels measure in fahrenheit and convert wrong to Celcius  
for lm_sensors.

If they first add 32 (instead of reduce) then multiply by 5 and  
divide by 9,
that gives roughly 34C difference, which more or less seems to be the  
close to the error it has.



On Aug 13, 2012, at 5:53 AM, Christopher Samuel wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 09/08/12 17:21, John Hearns wrote:
>
>> Use the ipmitool utility to probe the readings from BMC cards
>> (ILO, DRAC, they're the same thing).
>
> We do the same..
>
> cheers!
> Chris
> - --
>  Christopher Samuel        Senior Systems Administrator
>  VLSCI - Victorian Life Sciences Computation Initiative
>  Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>  http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlAoelQACgkQO2KABBYQAh+PPQCeLRo7Czlo6iVJ460TVlYVBqYD
> I/MAoJP9DpuAeDaKAOjbGbA7iHzyoB3h
> =KEkX
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From bernard at vanhpc.org  Mon Aug 13 10:42:57 2012
From: bernard at vanhpc.org (Bernard Li)
Date: Mon, 13 Aug 2012 10:42:57 -0700
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <81038DC3-155D-4A5D-AE62-970548C5F3A3@xs4all.nl>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
	<CAPqNE2Ucj-WkvcHtE97HXemEZ_u1eY+38RnokAM7tM3wb4c9Zg@mail.gmail.com>
	<50287A54.2090700@unimelb.edu.au>
	<81038DC3-155D-4A5D-AE62-970548C5F3A3@xs4all.nl>
Message-ID: <CA+3XN_KTp21_vdcZv96-qYTvuYLySvinh3KJ_ywa9TqjpOO-9w@mail.gmail.com>

Hi Vincent:

For what it's worth, Supermicro have their own monitoring tool called
SuperDoctor, you may get slightly different readings from it than from
lm_sensors:

http://www.supermicro.com/products/accessories/software/SuperDoctorIII.cfm

Cheers,

Bernard

On Mon, Aug 13, 2012 at 12:03 AM, Vincent Diepeveen <diep at xs4all.nl> wrote:
> I'll try that - thanks.
>
> Maybe the intels measure in fahrenheit and convert wrong to Celcius
> for lm_sensors.
>
> If they first add 32 (instead of reduce) then multiply by 5 and
> divide by 9,
> that gives roughly 34C difference, which more or less seems to be the
> close to the error it has.
>
>
>
> On Aug 13, 2012, at 5:53 AM, Christopher Samuel wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 09/08/12 17:21, John Hearns wrote:
>>
>>> Use the ipmitool utility to probe the readings from BMC cards
>>> (ILO, DRAC, they're the same thing).
>>
>> We do the same..
>>
>> cheers!
>> Chris
>> - --
>>  Christopher Samuel        Senior Systems Administrator
>>  VLSCI - Victorian Life Sciences Computation Initiative
>>  Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>>  http://www.vlsci.org.au/      http://twitter.com/vlsci
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAlAoelQACgkQO2KABBYQAh+PPQCeLRo7Czlo6iVJ460TVlYVBqYD
>> I/MAoJP9DpuAeDaKAOjbGbA7iHzyoB3h
>> =KEkX
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From dnlombar at ichips.intel.com  Mon Aug 13 13:36:02 2012
From: dnlombar at ichips.intel.com (David N. Lombard)
Date: Mon, 13 Aug 2012 13:36:02 -0700
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
Message-ID: <20120813203602.GA24478@nlxcldnl2.cl.intel.com>

On Thu, Aug 09, 2012 at 12:40:23AM +0200, Vincent Diepeveen wrote:
> hi,
> 
> How do you guys monitor the CPU core temperatures?

Nowadays, I *first* look in /sys. For example, this file is on my WS:

	/sys/devices/platform/coretemp.0/temp2_input

This the temp (in thousandth's of a deg C) of /one/ of the cores.
If you look in the coretemp.0 directory (where 0 is one of the sockets,
you'll see other pseudo files, like the alarm et al.

The names may well vary depending on platform and kernel version.

> Maybe someone still has the lm_sensors script lying around somewhere  
> fixing it for the intel Xeons?

Today, I'll only use lm_sensors for things the kernel doesn't explicitly
export in /sys or /proc.

-- 
David N. Lombard, Intel, Irvine, CA
I do not speak for Intel Corporation; all comments are strictly my own.


From samuel at unimelb.edu.au  Mon Aug 13 21:11:03 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Tue, 14 Aug 2012 14:11:03 +1000
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <20120813203602.GA24478@nlxcldnl2.cl.intel.com>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
	<20120813203602.GA24478@nlxcldnl2.cl.intel.com>
Message-ID: <5029CFD7.8050407@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 14/08/12 06:36, David N. Lombard wrote:

> Nowadays, I *first* look in /sys. For example, this file is on my
> WS:
> 
> /sys/devices/platform/coretemp.0/temp2_input
> 
> This the temp (in thousandth's of a deg C) of /one/ of the cores. 
> If you look in the coretemp.0 directory (where 0 is one of the
> sockets, you'll see other pseudo files, like the alarm et al.
> 
> The names may well vary depending on platform and kernel version.

Nice find, I wasn't seeing that on any of my systems until I realised
that the coretemp kernel module isn't loaded by default.. :-)

modprobe coretemp  # is your friend :-)

cheers,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlApz9cACgkQO2KABBYQAh8uuACdFidJXDE/ZcWg7JT3n6C9PMz2
KakAnAm/YBSXp6BsvuSONOV3BPb/MKn1
=SmQs
-----END PGP SIGNATURE-----


From samuel at unimelb.edu.au  Mon Aug 13 21:14:15 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Tue, 14 Aug 2012 14:14:15 +1000
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <5029CFD7.8050407@unimelb.edu.au>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
	<20120813203602.GA24478@nlxcldnl2.cl.intel.com>
	<5029CFD7.8050407@unimelb.edu.au>
Message-ID: <5029D097.7060507@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 14/08/12 14:11, Christopher Samuel wrote:

> modprobe coretemp  # is your friend :-)

Of course that's for Intel Core based CPUs (including Atom), for AMD
ones I guess you'll be wanting k10temp or k8temp!

Have a look in /lib/modules/`uname -r`/kernel/drivers/hwmon to see
where they live (and what else is there).

- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAp0JcACgkQO2KABBYQAh+LkwCeNFUVAyjqqcdKO70agwEN1Uwo
wOcAnRs3u4hg9B82TURCxMOAX0BTCm7U
=Ssic
-----END PGP SIGNATURE-----


From adrian.mcdonald at canterbury.ac.nz  Tue Aug  7 23:10:33 2012
From: adrian.mcdonald at canterbury.ac.nz (adrian.mcdonald at canterbury.ac.nz)
Date: Wed, 8 Aug 2012 14:10:33 +0800
Subject: [Beowulf] Delivery reports about your e-mail
Message-ID: <20120808061050.12E13A040798@beoscyld.localdomain>

The original message was received at Wed, 8 Aug 2012 14:10:33 +0800 from canterbury.ac.nz [100.159.183.150]

----- The following addresses had permanent fatal errors -----
<beowulf at beowulf.org>

----- Transcript of session follows -----
... while talking to host beowulf.org.:
554 5.0.0 Service unavailable; [152.137.145.87] blocked using relays.osirusoft.com, reason: Blocked
Session aborted, reason: lost connection

-------------- next part --------------
A non-text attachment was scrubbed...
Name: letter.zip
Type: application/octet-stream
Size: 28982 bytes
Desc: not available
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120808/eecd8b9c/attachment.obj>

From andrew.holway at gmail.com  Thu Aug  9 00:34:14 2012
From: andrew.holway at gmail.com (Andrew Holway)
Date: Thu, 9 Aug 2012 09:34:14 +0200
Subject: [Beowulf] lm_sensors and clusters and wrong intel cpu readings
In-Reply-To: <CAPqNE2Ucj-WkvcHtE97HXemEZ_u1eY+38RnokAM7tM3wb4c9Zg@mail.gmail.com>
References: <2214BA2D-ADCE-4189-9627-D1B980CA85CF@xs4all.nl>
	<CAPqNE2Ucj-WkvcHtE97HXemEZ_u1eY+38RnokAM7tM3wb4c9Zg@mail.gmail.com>
Message-ID: <CAEiui-uHg+hoLZungAhc58ieFtR8FdSHZaorko35x5qBDc_ouA@mail.gmail.com>

On AMD sensors at least the reading is a 'relative value' with 70C
indicating an overheat.

The processor is fine unless it is actually clocking its self down to
a lower ACPI power state. It is seemingly impossible to overheat
modern CPUS. When its clocking down you should see messages in
/var/log/messages and also the ipmi SEL log.

Dont trust IPMI. Ever. lm_sensors actually reads the raw value from
the CPU and requires a specifically written kernel module to do so.
Who knows what kind of junk math the ipmi does.

Supermicro IPMI implementations are particularly bad at reporting temp
properly(like really awful).

Most of my experience here is with AMD. ymmv :)



2012/8/9 John Hearns <hearnsj at googlemail.com>:
> Well, I don't use lm_sensors for a start!
> Use the ipmitool utility to probe the readings from BMC cards (ILO,
> DRAC, they're the same thing).
> I don;t trust the absolute calibration of the sensors - generally
> you're looking at setting a limit on which to alarm or shutdown so
> just take a reading under no load on the CPU and call that the
> 'normal' reading.
> I may be wrong. YMMV.
>
> On 08/08/2012, Vincent Diepeveen <diep at xs4all.nl> wrote:
>> hi,
>>
>> How do you guys monitor the CPU core temperatures?
>>
>> if i run lm_sensors, it's 30C higher at every node than a few nodes i
>> tried compare with windows.
>> Also under full load it reports temperatures like end 60s and up to
>> 78C i've seen reported.
>> Am guessing it should be 30-40+ at most.
>>
>> It blows cool air from and outside the cpu's. Nothing is even 'warm'.
>>
>> Nodes here: supermicro X7DWE inside Xeons L5420. They are not
>> overclocked.
>>
>> I also downloaded some similar motherboards definitions - seems they
>> uploaded it for motherboards with dual core Xeons
>> and such, not for the quadcores. None of those defines 'corrects' the
>> temperature of the quadcore Xeons, they basically kick out
>> readings that are not getting used.
>>
>> Now i bet several clusters/supercomputers had these cpu's. How did
>> you solve this problem with the intel L5420's?
>>
>> Maybe someone still has the lm_sensors script lying around somewhere
>> fixing it for the intel Xeons?
>>
>> Thanks in advance,
>> Vincent
>>
>>
>>
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From admin at heels4u.com  Wed Aug 15 18:00:18 2012
From: admin at heels4u.com (WordPress Plugin Turns Any Site Into An App That Installs Directly Onto Any iPhone)
Date: Wed, 15 Aug 2012 20:00:18 -0500
Subject: [Beowulf] Your friend WordPress Plugin Turns Any Site Into An App
	That Installs Directly Onto Any iPhone has recommended this
	great product from heels4u and more
Message-ID: <578b036729cbf7428e3dd4297e0a16cd@heels4u.com>

Hi Buddy!

Your friend, WordPress Plugin Turns Any Site Into An App That Installs
Directly Onto Any iPhone, thought that you would be interested in
MADRID-03-BKPU from heels4u and more.

WordPress Plugin Turns Any Site Into An App That Installs Directly Onto Any
iPhone sent a note saying:

Hi,

 Nowadays, our technology continues to rise to a whole new era. New
different gadgets which make them convenient enough to
 people. Now what if i tell you that you can now turn any WordPress site
into an Apple web app in under 20 minutes?

 Yes!!!it is now possible with the Wordpress Plugin!!

 Newly launched product Wordpress Plugin One Click Mobile Web App converts
any site into a 'web app'
 then forces visitors to install your app directly onto their
iPhone,iPad,iPod or anny other Apple device.

 For additional information kindly watch the video at for guidance and
overview of the product:

 http://www.youtube.com/watch?v=IMLlJDVGZ8g&feature=player_embedded

 sO what are you waiting for,order now this awesome gadgets and start
enjoying its different features.

 Order now at:

 http://oneclick-mobileapp.net.tc/

 Thanks

 Dwight
























































































































































































































































.

----------------------------------------------------------------------------------------

To view the product, click on the link below or copy and paste the link into
your web browser:

http://heels4u.com/shop/index.php?main_page=product_info&products_id=26183

Regards,

heels4u and more
http://heels4u.com/shop/


-----
IMPORTANT: For your protection and to prevent malicious use, all emails sent
via this web site are logged and the contents recorded and available to the
store owner. If you feel that you have received this email in error, please
send an email to admin at heels4u.com




This email address was given to us by you or by one of our customers. If you
feel that you have received this email in error, please send an email to
admin at heels4u.com 
This email is sent in accordance with the US CAN-SPAM Law in effect
01/01/2004. Removal requests can be sent to this address and will be honored
and respected.



From eagles051387 at gmail.com  Fri Aug 17 00:13:00 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 17 Aug 2012 09:13:00 +0200
Subject: [Beowulf] Moderation of the Beowulf Mailing List
In-Reply-To: <CACqOxA4fbRO39gJpbGcHHqif_X8gaE1H6x3Ge_8Kyk8aQ_Z9sA@mail.gmail.com>
References: <CACqOxA4fbRO39gJpbGcHHqif_X8gaE1H6x3Ge_8Kyk8aQ_Z9sA@mail.gmail.com>
Message-ID: <502DEEFC.4010205@gmail.com>

I am looking at the moderation tasks and it seems like there are already 
a number of spam messages waiting there. Question I have is do i reject 
the messages and discard them as well as add the sender to either the 
reject or discard list?

How are we going to proceed with these emails?

On 17/08/2012 00:55, Arend Dittmer wrote:
> Hi Everybody,
>
> I am with Penguin Computing, who is hosting the beowulf mailing 
> list. Thank you for volunteering to share the burden of moderating the 
> list. Doug Eadline suggested that everybody agrees to this mission 
> statement as a common denominator:
>
> "/As moderators we strive to preserve a focused and open discussion//
> of HPC technical issues. We will support civil discourse and
> disagreement, but reserve the right to moderate personal attacks
> and inappropriate content of a non-technical nature. Mild commercial
> promotion of projects and products is permissible if the topic has been
> /
> /suggested by prior non-commercial list conversations or queries./"
>
> I would like to share the list administrator password with you. The 
> easiest and safest way to share is probably via SMS. If you could 
> please send me your cell phone numbers so that I can send you the 
> admin password. I promise ... I will delete every number as soon as I 
> have sent the password. No follow-up calls from Penguin sales ... :)
>
> One issue all of you may want to look at its the topic of spam 
> filtering. Right now e-mail from unknown sources is held, and the 
> queue can quickly get pretty lengthy. Joe Landman offered to help with 
> installing a spam filter. Joe, please let me know if you need 
> any privileges beyond list admin for this.
>
> Thanks again for volunteering
>
> Arend
>
> -- 
> Arend Dittmer
> Penguin Computing
> (415) 954-2863
> ---
> /Follow us on Twitter: @PenguinHPC/
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120817/4ef7ad6a/attachment.html>

From diep at xs4all.nl  Fri Aug 17 05:03:39 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Fri, 17 Aug 2012 14:03:39 +0200
Subject: [Beowulf] Doing i/o at a small cluster
Message-ID: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>

hi,

Which free or very cheap distributed file system choices do i have  
for a 8 node cluster that has QDR infiniband (mellanox)?
Each node could have a few harddrives. Up to 8 or so SATA2. Could  
also use some raid cards.

And i'm investigating what i need.

I'm investigating to generate the 7 men EGTBs at the cluster. This is  
a big challenge.
To generate it is high i/o load. I'm looking at around a 4 GB/s i/o  
from which a tad more than
1GB/s is write and a tad less than 3GB/s is readspeed from harddrives.

This for 3+ months nonstop. Provided the CPU's can keep up with that.  
Otherwise a few months more.

This 4GB/s i/o is aggregated speed.

What raid system you'd recommend here?

A problem is the write speed + read speed i need. From what i  
understand at the edges of drives the speed is
roughly 133MB/s SATA2 moving down to a 33MB/s at the innersides.

Is that roughly correct?

Of course there will be many solutions. I could use some raid cards  
or i could equip each node with some drives.
Raid card is probably sata-3 nowadays. Didn't check speeds there.

Total storage is some dozen to a few dozens of terabytes.

Does the filesystem automatically optimize for writing at the edges  
instead of starting at the innerside?
which 'raid' level would you recommend for this if any is appropriate  
at all :)

How many harddrives would i need? What failure rate can i expect with  
modern SATA drives there?
I had several fail at a raid0+1 system before when generating some  
EGTBs some years ago.

Thanks in advance for tips/hints and suggestions!

Note there is more questions. Like which buffer size i must read/ 
write. Most files get streamed.
 From 2 files that i do reading from, i read big blocks from a random  
spot in that file. Each file is
a couple of hundreds of gigabyte.

I used to grab chunks of 64KB from each file, but don't see how to  
get to gigabytes a second i/o with
todays hardware that manner.

Am considering now to read blocks of 10MB. Which size will get me  
there to the maximum bandwidth the i/o
can deliver?

Note that each node is busy with a bunch of files in this manner.

So if a big distributed file system doesn't work out well i can try  
JBOD at each node.

Kind Regards,
Vincent




From james.p.lux at jpl.nasa.gov  Fri Aug 17 06:56:04 2012
From: james.p.lux at jpl.nasa.gov (Lux, Jim (337C))
Date: Fri, 17 Aug 2012 13:56:04 +0000
Subject: [Beowulf] Moderation of the Beowulf Mailing List
In-Reply-To: <502DEEFC.4010205@gmail.com>
Message-ID: <CC539A96.240E1%james.p.lux@jpl.nasa.gov>

I can tell you want we do on another list where I'm a moderator..

If it's an "autogenerated spam"  just dump it silently automatically and unsubscribe (these are easy.. They have email addresses like asdfgh at bogusdomain.com)

I can find out what we use for our filters.

If it's spam that's "out of character" with the previous posts from the apparent source, assume it's spoofed origin, dump it silently.  If it's someone we (moderators) know, we might send them an email saying we got the spoofed source, but since there's not a heck of a lot anyone can do, it's of dubious usefulness.




From: Jonathan Aquilina <eagles051387 at gmail.com<mailto:eagles051387 at gmail.com>>
Date: Fri, 17 Aug 2012 09:13:00 +0200
To: Arend Dittmer <adittmer at penguincomputing.com<mailto:adittmer at penguincomputing.com>>
Cc: "beowulf at beowulf.org<mailto:beowulf at beowulf.org>" <beowulf at beowulf.org<mailto:beowulf at beowulf.org>>
Subject: Re: [Beowulf] Moderation of the Beowulf Mailing List

I am looking at the moderation tasks and it seems like there are already a number of spam messages waiting there. Question I have is do i reject the messages and discard them as well as add the sender to either the reject or discard list?

How are we going to proceed with these emails?

On 17/08/2012 00:55, Arend Dittmer wrote:
Hi Everybody,

I am with Penguin Computing, who is hosting the beowulf mailing list. Thank you for volunteering to share the burden of moderating the list. Doug Eadline suggested that everybody agrees to this mission statement as a common denominator:

"As moderators we strive to preserve a focused and open discussion
of HPC technical issues. We will support civil discourse and
disagreement, but reserve the right to moderate personal attacks
and inappropriate content of a non-technical nature. Mild commercial
promotion of projects and products is permissible if the topic has been
suggested by prior non-commercial list conversations or queries."

I would like to share the list administrator password with you. The easiest and safest way to share is probably via SMS. If you could please send me your cell phone numbers so that I can send you the admin password. I promise ... I will delete every number as soon as I have sent the password. No follow-up calls from Penguin sales ... :)

One issue all of you may want to look at its the topic of spam filtering. Right now e-mail from unknown sources is held, and the queue can quickly get pretty lengthy. Joe Landman offered to help with installing a spam filter. Joe, please let me know if you need any privileges beyond list admin for this.

Thanks again for volunteering

Arend

--
Arend Dittmer
Penguin Computing
(415) 954-2863
---
Follow us on Twitter: @PenguinHPC


_______________________________________________ Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120817/173ad96f/attachment.html>

From eagles051387 at gmail.com  Fri Aug 17 07:16:20 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 17 Aug 2012 16:16:20 +0200
Subject: [Beowulf] Moderation of the Beowulf Mailing List
In-Reply-To: <CC539A96.240E1%james.p.lux@jpl.nasa.gov>
References: <CC539A96.240E1%james.p.lux@jpl.nasa.gov>
Message-ID: <502E5234.3020700@gmail.com>

Ya I have basically dropped all the ones that are caught. They are all 
spam just dropped another big bunch again just now.

Arend I am quite suprised Spamassassin hasnt been setup and integrated 
with mailman. I am not sure if that is possible, but I can find out and 
let you know later on today, that is assuming they have an IRC channel.

On 17/08/2012 15:56, Lux, Jim (337C) wrote:
> I can tell you want we do on another list where I'm a moderator..
>
> If it's an "autogenerated spam"  just dump it silently automatically 
> and unsubscribe (these are easy.. They have email addresses like 
> asdfgh at bogusdomain.com)
>
> I can find out what we use for our filters.
>
> If it's spam that's "out of character" with the previous posts from 
> the apparent source, assume it's spoofed origin, dump it silently.  If 
> it's someone we (moderators) know, we might send them an email saying 
> we got the spoofed source, but since there's not a heck of a lot 
> anyone can do, it's of dubious usefulness.
>
>
>
>
> From: Jonathan Aquilina <eagles051387 at gmail.com 
> <mailto:eagles051387 at gmail.com>>
> Date: Fri, 17 Aug 2012 09:13:00 +0200
> To: Arend Dittmer <adittmer at penguincomputing.com 
> <mailto:adittmer at penguincomputing.com>>
> Cc: "beowulf at beowulf.org <mailto:beowulf at beowulf.org>" 
> <beowulf at beowulf.org <mailto:beowulf at beowulf.org>>
> Subject: Re: [Beowulf] Moderation of the Beowulf Mailing List
>
> I am looking at the moderation tasks and it seems like there are 
> already a number of spam messages waiting there. Question I have is do 
> i reject the messages and discard them as well as add the sender to 
> either the reject or discard list?
>
> How are we going to proceed with these emails?
>
> On 17/08/2012 00:55, Arend Dittmer wrote:
>> Hi Everybody,
>>
>> I am with Penguin Computing, who is hosting the beowulf mailing 
>> list. Thank you for volunteering to share the burden of moderating 
>> the list. Doug Eadline suggested that everybody agrees to this 
>> mission statement as a common denominator:
>>
>> "/As moderators we strive to preserve a focused and open discussion//
>> of HPC technical issues. We will support civil discourse and
>> disagreement, but reserve the right to moderate personal attacks
>> and inappropriate content of a non-technical nature. Mild commercial
>> promotion of projects and products is permissible if the topic has been
>> /
>> /suggested by prior non-commercial list conversations or queries./"
>>
>> I would like to share the list administrator password with you. The 
>> easiest and safest way to share is probably via SMS. If you could 
>> please send me your cell phone numbers so that I can send you the 
>> admin password. I promise ... I will delete every number as soon as I 
>> have sent the password. No follow-up calls from Penguin sales ... :)
>>
>> One issue all of you may want to look at its the topic of spam 
>> filtering. Right now e-mail from unknown sources is held, and the 
>> queue can quickly get pretty lengthy. Joe Landman offered to help 
>> with installing a spam filter. Joe, please let me know if you need 
>> any privileges beyond list admin for this.
>>
>> Thanks again for volunteering
>>
>> Arend
>>
>> -- 
>> Arend Dittmer
>> Penguin Computing
>> (415) 954-2863
>> ---
>> /Follow us on Twitter: @PenguinHPC/
>>
>
> _______________________________________________ Beowulf mailing list, 
> Beowulf at beowulf.org <mailto:Beowulf at beowulf.org> sponsored by Penguin 
> Computing To change your subscription (digest mode or unsubscribe) 
> visit http://www.beowulf.org/mailman/listinfo/beowulf 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120817/0110c7fd/attachment.html>

From ellis at cse.psu.edu  Fri Aug 17 07:42:23 2012
From: ellis at cse.psu.edu (Ellis H. Wilson III)
Date: Fri, 17 Aug 2012 10:42:23 -0400
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
Message-ID: <502E584F.9080908@cse.psu.edu>

On 08/17/12 08:03, Vincent Diepeveen wrote:
> hi,
>
> Which free or very cheap distributed file system choices do i have
> for a 8 node cluster that has QDR infiniband (mellanox)?
> Each node could have a few harddrives. Up to 8 or so SATA2. Could
> also use some raid cards.

Lots of choices, but are you talking about putting a bunch of disks in 
all those PCs or having one I/O server?  The latter is the classic 
solution but there are ways to do the former.

Short answer is there are complicated ways to fling your hdds into 
distributed machines using PVFS and get good performance provided you 
are okay with those non-posix semantics and guarantees.  There are also 
ways to get decent performance from the Hadoop Distributed File System, 
which can handle a distributed set of nodes and internal HDDs well, but 
for a /constrained set of applications./  Based on your previous posts 
about GPUs and whatnot, I'm going to assume you will have little to zero 
interest in Hadoop.  Last, there's a new NFS version out (pNFS, or NFS 
v4.1) that you can probably use to great impact with proper tuning.  No 
comments on tuning it however, as I haven't yet tried myself.  That may 
be your best out of the box solution.

Also, I assume you're talking about QDR 1X here, so just 8Gb/s per node. 
Correct me if that's wrong.

> And i'm investigating what i need.
>
> I'm investigating to generate the 7 men EGTBs at the cluster. This is
> a big challenge.

For anyone who doesn't know (probably many who aren't into chess, I had 
to look this up myself), EGTB is end game table bases, and more info is 
available at:
http://en.wikipedia.org/wiki/Endgame_tablebase

Basically it's just a giant dump of exhaustive moves for N men left on 
the board.

> To generate it is high i/o load. I'm looking at around a 4 GB/s i/o
> from which a tad more than
> 1GB/s is write and a tad less than 3GB/s is readspeed from harddrives.
>
> This for 3+ months nonstop. Provided the CPU's can keep up with that.
> Otherwise a few months more.
>
> This 4GB/s i/o is aggregated speed.

I would LOVE to hear what Joe has to say on this, but going out on a 
limb here, it will be almost impossible to get that much out of your 
HDDs with 8 nodes without serious planning and an extremely narrow 
use-case.  I assume you are talking about putting drives in each node at 
this point, because with just QDR you cannot feed aggregate 4GB/s 
without bonding from one node.

We need to know more about generating this tablebase -- I can only 
assume you are planning to do analyses on it after you generate all 
possible combinations, right?  We need to know more about how that 
follow-up analysis can be divided before commenting on possible storage 
solutions.  If everything is totally embarrassingly parallel you're in a 
good spot to not bother with a parallel filesystem.  In that case you 
just might be able to squeeze 4GB/s out of your drives.

But with all the nodes accessing all the disks at once, hitting 4GB/s 
with just strung together FOSS software is really tough for anything but 
the most basic and most embarrassingly parallel stuff.  It requires 
serious tuning over months or buying a product that has already done 
this (e.g. a solution like Joe's company Scalable Informatics makes or 
Panasas, the company I work for, makes).  People always love to say, 
"Oh, that's 100MB/s per drive!  So with 64 drives I should be able to 
get 6.4GB/s!  Yea!"  Sadly, that's really only the case when these 
drives are accessed completely sequentially and completely separately 
(i.e. not put together into a distributed filesystem).

> What raid system you'd recommend here?

Uh, you looking for software or hardware or object RAID?

> A problem is the write speed + read speed i need. From what i
> understand at the edges of drives the speed is
> roughly 133MB/s SATA2 moving down to a 33MB/s at the innersides.
>
> Is that roughly correct?

I hate this as much as anybody, but........ It Depends (TM).
You talking plain-jane "dd".  Sure, that might be reasonable for certain 
vendors.

> Of course there will be many solutions. I could use some raid cards
> or i could equip each node with some drives.
> Raid card is probably sata-3 nowadays. Didn't check speeds there.
>
> Total storage is some dozen to a few dozens of terabytes.
>
> Does the filesystem automatically optimize for writing at the edges
> instead of starting at the innerside?
> which 'raid' level would you recommend for this if any is appropriate
> at all :)

Again, depends on RAID card and whatnot.  Some do, some don't.

> How many harddrives would i need? What failure rate can i expect with
> modern SATA drives there?
> I had several fail at a raid0+1 system before when generating some
> EGTBs some years ago.

Yup, things will break especially during the shakeout (first few days or 
weeks).  I assume you're buying commodity drives here, not enterprise, 
so you should prepare for upwards of, /after the shakeout/, maybe 4-8 of 
your drives to fail or start throwing SMART errors in the first year 
(ball-parking it here based solely on experience).  Rebuilds will suck 
for you with lots of data unless you have really thought that out 
(typically limited to speed of a single disk -- therefore 2TB drive 
rebuilding itself at 50MB/s (that's best case scenario) is like 11 
hours.  I hope you haven't bought all your drives from the same batch 
from the same manufacturer as well -- that often results in very similar 
failure times (i.e. concurrent failures in a day).  Very non-uniform.

> Note there is more questions. Like which buffer size i must read/
> write. Most files get streamed.
>   From 2 files that i do reading from, i read big blocks from a random
> spot in that file. Each file is
> a couple of hundreds of gigabyte.
>
> I used to grab chunks of 64KB from each file, but don't see how to
> get to gigabytes a second i/o with
> todays hardware that manner.
>
> Am considering now to read blocks of 10MB. Which size will get me
> there to the maximum bandwidth the i/o
> can deliver?

I actually do wonder if Hadoop won't work for you.  This sounds like a 
very Hadoop-like workload, assuming you are OK with write-once read-many 
semantics.  But I need to know way more about what you want to do with 
the data afterwards.  Moving data off of HDFS sucks.

Best,

ellis


From prentice.bisbal at rutgers.edu  Fri Aug 17 07:38:33 2012
From: prentice.bisbal at rutgers.edu (Prentice Bisbal)
Date: Fri, 17 Aug 2012 10:38:33 -0400
Subject: [Beowulf] Prentice has a new job.
Message-ID: <502E5769.7020202@rutgers.edu>

Beowulfers,

Now that the Beowulf mailing list appears to be functional again, I 
wanted to announce a recent job change.

At the end of June I left the Institute for Advanced Study to become the 
Manager of IT for the Rutgers Discovery Informatics Institute (aka 
RDI2). This is a new HPC institute at Rutgers University that is just 
getting started. Right now, our only HPC resource is a two-rack IBM Blue 
Gene /P with 2048 nodes (8196 cores) cores, but we hope to add a Blue 
Gene /Q in the not-too-distant future.

If there's anyone else on this list who's supported Blue Genes or 
LoadLeveler, please drop me a line, I'd like to hear from you.

For the curious, you can learn more here: http://rdi2.rutgers.edu

-- 
Prentice Bisbal
Manager of Information Technology
Rutgers Discovery Informatics Institute (RDI2)
Rutgers University
http://rdi2.rutgers.edu



From diep at xs4all.nl  Fri Aug 17 09:04:33 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Fri, 17 Aug 2012 18:04:33 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <502E584F.9080908@cse.psu.edu>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
Message-ID: <0C42D0AC-897E-4276-A9BD-027DB5A7F5FA@xs4all.nl>


On Aug 17, 2012, at 4:42 PM, Ellis H. Wilson III wrote:

> On 08/17/12 08:03, Vincent Diepeveen wrote:
>> hi,
>>
>> Which free or very cheap distributed file system choices do i have
>> for a 8 node cluster that has QDR infiniband (mellanox)?
>> Each node could have a few harddrives. Up to 8 or so SATA2. Could
>> also use some raid cards.
>

Thanks for your quick response!

> Lots of choices, but are you talking about putting a bunch of disks in
> all those PCs or having one I/O server?  The latter is the classic
> solution but there are ways to do the former.
>
> Short answer is there are complicated ways to fling your hdds into
> distributed machines using PVFS and get good performance provided you
> are okay with those non-posix semantics and guarantees.  There are  
> also

I need a solution just for this workload, doesn't need to work for  
anything else.
So if i have to run around the church to get something done with it,
yet it is delivering me the speed i need, then that's cceptable here.

> ways to get decent performance from the Hadoop Distributed File  
> System,
> which can handle a distributed set of nodes and internal HDDs well,  
> but
> for a /constrained set of applications./  Based on your previous posts
> about GPUs and whatnot, I'm going to assume you will have little to  
> zero
> interest in Hadoop.  Last, there's a new NFS version out (pNFS, or NFS
> v4.1) that you can probably use to great impact with proper  
> tuning.  No
> comments on tuning it however, as I haven't yet tried myself.  That  
> may
> be your best out of the box solution.
>
> Also, I assume you're talking about QDR 1X here, so just 8Gb/s per  
> node.
> Correct me if that's wrong.

This is correct. the motherboards are pci-e 2.0.

Each node is a $200 machine with 2 x L5420, seaburg chipset.

You will realize i don't have money for professional solutions.

>
>> And i'm investigating what i need.
>>
>> I'm investigating to generate the 7 men EGTBs at the cluster. This is
>> a big challenge.
>
> For anyone who doesn't know (probably many who aren't into chess, I  
> had
> to look this up myself), EGTB is end game table bases, and more  
> info is
> available at:
> http://en.wikipedia.org/wiki/Endgame_tablebase
>
> Basically it's just a giant dump of exhaustive moves for N men left on
> the board.

Correct. Basically you need to do a lot of calculations to produce 1  
byte of useful
positions. In the end 1 byte stores 5 positions.

The endresult will be a 82TB of data when it's uncompressed, i'll  
store it compressed.
To generate it is at least 3 months of nonstop i/o at 4GB/s, or more  
months if that i/o is less.

Simple compressed it will easily fit at a 3TB drive. Yet in  
compressed state it's not usable
to generate.

Historic attempts such as the famous attempts from Ken Thompson  
stored in less efficient manners which also
compresses not so well.

>
>> To generate it is high i/o load. I'm looking at around a 4 GB/s i/o
>> from which a tad more than
>> 1GB/s is write and a tad less than 3GB/s is readspeed from  
>> harddrives.
>>
>> This for 3+ months nonstop. Provided the CPU's can keep up with that.
>> Otherwise a few months more.
>>
>> This 4GB/s i/o is aggregated speed.
>
> I would LOVE to hear what Joe has to say on this, but going out on a
> limb here, it will be almost impossible to get that much out of your
> HDDs with 8 nodes without serious planning and an extremely narrow
> use-case.  I assume you are talking about putting drives in each  
> node at
> this point, because with just QDR you cannot feed aggregate 4GB/s
> without bonding from one node.

Yes i realize that. In principle you're looking at a 1050 files or so  
that get effectively generated
and to generate each file half a dozen of huge files get created.

Now in theory generating them is embarrassingly parallel except that  
to generate 1 large Set of EGTBs
requires around a 3TB of working set size.

So with a core or 64 that would mean one needs the immense size of  
some TB's times 64 cores.

Therefore generating it all embarrassingly parallel is pretty  
expensive sort of i/o i would need.

I intend to have each node of 8 cores work on 1 or 2 sets at the same  
time. Probably at 1.
So 8 cores require then a maximum of 3TB.

So far it seemed maybe that i could get away equip each node with 4  
drives of 1.5TB in raid10,
or 2 drives in raid0 (and then just restart it when a drive fails).

yet there is a catch.

Now comes the achillesheel. The start of the generation it needs to  
access real quickly some earlier
generated sets; the initial generation of the conversion bitmap needs  
to access other EGTBs,
as pieces can promote especially. Lucky this is a single pass, but  
it's a slow and intensive pass.

In such case accessing over the network is important.

So there is a huge locality except for 1 pass. The real fast  
generation that hammers onto the drives and reads quick and writes
fast, that can be done entirely local. The first pass generating a  
single 'exchange bitmap', needs to lookup to
EGTBs earlier generate. For example if we have the EGTB  KQRP KRP  
then it has to lookup to the much larger
EGTB that holds KQRB KRP and a few others.

So the cohesion to the other nodes drives is limited to say a few  
percent of the total i/o getting done.

As we speak about complex file management here, it's difficult to do  
this by hand.

In other words, efficient usage of the available harddrive space is  
important.

>
> We need to know more about generating this tablebase -- I can only
> assume you are planning to do analyses on it after you generate all
> possible combinations, right?  We need to know more about how that
> follow-up analysis can be divided before commenting on possible  
> storage
> solutions.  If everything is totally embarrassingly parallel you're  
> in a
> good spot to not bother with a parallel filesystem.  In that case you
> just might be able to squeeze 4GB/s out of your drives.
>
> But with all the nodes accessing all the disks at once, hitting 4GB/s
> with just strung together FOSS software is really tough for  
> anything but
> the most basic and most embarrassingly parallel stuff.  It requires
> serious tuning over months or buying a product that has already done
> this (e.g. a solution like Joe's company Scalable Informatics makes or
> Panasas, the company I work for, makes).  People always love to say,
> "Oh, that's 100MB/s per drive!  So with 64 drives I should be able to
> get 6.4GB/s!  Yea!"  Sadly, that's really only the case when these
> drives are accessed completely sequentially and completely separately
> (i.e. not put together into a distributed filesystem).
>
>> What raid system you'd recommend here?
>
> Uh, you looking for software or hardware or object RAID?

I'm open to something that can work for me.

Didn't buy the harddrives yet and doubt i will. Just investigating
what i need to get the job done.

After that i will know the price of the drives.

>
>> A problem is the write speed + read speed i need. From what i
>> understand at the edges of drives the speed is
>> roughly 133MB/s SATA2 moving down to a 33MB/s at the innersides.
>>
>> Is that roughly correct?
>
> I hate this as much as anybody, but........ It Depends (TM).
> You talking plain-jane "dd".  Sure, that might be reasonable for  
> certain
> vendors.
>
>> Of course there will be many solutions. I could use some raid cards
>> or i could equip each node with some drives.
>> Raid card is probably sata-3 nowadays. Didn't check speeds there.
>>
>> Total storage is some dozen to a few dozens of terabytes.
>>
>> Does the filesystem automatically optimize for writing at the edges
>> instead of starting at the innerside?
>> which 'raid' level would you recommend for this if any is appropriate
>> at all :)
>
> Again, depends on RAID card and whatnot.  Some do, some don't.
>
>> How many harddrives would i need? What failure rate can i expect with
>> modern SATA drives there?
>> I had several fail at a raid0+1 system before when generating some
>> EGTBs some years ago.
>
> Yup, things will break especially during the shakeout (first few  
> days or
> weeks).  I assume you're buying commodity drives here, not enterprise,
> so you should prepare for upwards of, /after the shakeout/, maybe  
> 4-8 of
> your drives to fail or start throwing SMART errors in the first year
> (ball-parking it here based solely on experience).  Rebuilds will suck
> for you with lots of data unless you have really thought that out
> (typically limited to speed of a single disk -- therefore 2TB drive
> rebuilding itself at 50MB/s (that's best case scenario) is like 11
> hours.  I hope you haven't bought all your drives from the same batch
> from the same manufacturer as well -- that often results in very  
> similar
> failure times (i.e. concurrent failures in a day).  Very non-uniform.

Yes i always have had many failures of harddrives if you do EGTB work.

Usually 100% breaks within a few years. Only 1 old SCSI drive  
survived actually.
all the parallel ATA's and SATA's they one day all die.

Biggest failure rate is obvously first few weeks indeed, after that  
most die
after 1 or 2 year of intensive usage.

>
>> Note there is more questions. Like which buffer size i must read/
>> write. Most files get streamed.
>>   From 2 files that i do reading from, i read big blocks from a  
>> random
>> spot in that file. Each file is
>> a couple of hundreds of gigabyte.
>>
>> I used to grab chunks of 64KB from each file, but don't see how to
>> get to gigabytes a second i/o with
>> todays hardware that manner.
>>
>> Am considering now to read blocks of 10MB. Which size will get me
>> there to the maximum bandwidth the i/o
>> can deliver?
>
> I actually do wonder if Hadoop won't work for you.  This sounds like a
> very Hadoop-like workload, assuming you are OK with write-once read- 
> many
> semantics.  But I need to know way more about what you want to do with
> the data afterwards.  Moving data off of HDFS sucks.

Compress it with 7-zip and move it away indeed. It'll compress to 3TB  
that 82 TB uncompressed i guess.
Probably during the generation of that 82TB data i already must start  
compressing to save
space.

Majority of those chessfiles is useless crap actually. Just a few are  
really interesting to have.
The interesting ones are actually a few smaller ones with many pawns,  
but to generate them you need to first do all that work :)

Then after that supercompress them using chesstechnical knowledge,  
that'll get that 82TB to a 100GB or so, but it will take
many years to do so as that's very compute intensive and not high  
priority.

>
> Best,
>
> ellis
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From diep at xs4all.nl  Fri Aug 17 12:32:20 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Fri, 17 Aug 2012 21:32:20 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <502E584F.9080908@cse.psu.edu>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
Message-ID: <1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>

The idea someone brought me on by means of a private email is
to use a distributed file system and split each drive in 2 partitions.

the outside which is fastest for local storage and the inside for a  
global distributed partition for long term
storage of endresults and automatically compressing with a scripts  
results and decompressing when it
seems soon a specific EGTB is needed.

Then using 3 disks a node i can get a 133MB-150MB /s on the outside  
of the drives in a raid-0.
That'll be around a 3TB the minimum needed for generation.

And the inside then gets a partition that uses redundancy, maybe  
raid-6 ?
any thoughts there.

So say a node or 4 i can dedicate to this. that's 12 drives.

Then i'll take 6 months instead of 3 months to generate but i have 4  
other nodes free for other jobs.

Also i need to pay less to harddrives then. Question now is whether  
i'll go for the 3TB then or the 2TB.

As for the filesystem that's most interesting to do this. Is gluster  
a good idea for this?

Can it handle this split between partitions in local and global? Does  
it have raid-6 or maybe some other
sort of redundancy you'd advice?

As for hadoop that's a java thing you know. If i want to get my  
cluster hacked from India i know an easier way to get that done :)

Thanks in Advance,
Vincent


On Aug 17, 2012, at 4:42 PM, Ellis H. Wilson III wrote:

> On 08/17/12 08:03, Vincent Diepeveen wrote:
>> hi,
>>
>> Which free or very cheap distributed file system choices do i have
>> for a 8 node cluster that has QDR infiniband (mellanox)?
>> Each node could have a few harddrives. Up to 8 or so SATA2. Could
>> also use some raid cards.
>
> Lots of choices, but are you talking about putting a bunch of disks in
> all those PCs or having one I/O server?  The latter is the classic
> solution but there are ways to do the former.
>
> Short answer is there are complicated ways to fling your hdds into
> distributed machines using PVFS and get good performance provided you
> are okay with those non-posix semantics and guarantees.  There are  
> also
> ways to get decent performance from the Hadoop Distributed File  
> System,
> which can handle a distributed set of nodes and internal HDDs well,  
> but
> for a /constrained set of applications./  Based on your previous posts
> about GPUs and whatnot, I'm going to assume you will have little to  
> zero
> interest in Hadoop.  Last, there's a new NFS version out (pNFS, or NFS
> v4.1) that you can probably use to great impact with proper  
> tuning.  No
> comments on tuning it however, as I haven't yet tried myself.  That  
> may
> be your best out of the box solution.
>
> Also, I assume you're talking about QDR 1X here, so just 8Gb/s per  
> node.
> Correct me if that's wrong.
>
>> And i'm investigating what i need.
>>
>> I'm investigating to generate the 7 men EGTBs at the cluster. This is
>> a big challenge.
>
> For anyone who doesn't know (probably many who aren't into chess, I  
> had
> to look this up myself), EGTB is end game table bases, and more  
> info is
> available at:
> http://en.wikipedia.org/wiki/Endgame_tablebase
>
> Basically it's just a giant dump of exhaustive moves for N men left on
> the board.
>
>> To generate it is high i/o load. I'm looking at around a 4 GB/s i/o
>> from which a tad more than
>> 1GB/s is write and a tad less than 3GB/s is readspeed from  
>> harddrives.
>>
>> This for 3+ months nonstop. Provided the CPU's can keep up with that.
>> Otherwise a few months more.
>>
>> This 4GB/s i/o is aggregated speed.
>
> I would LOVE to hear what Joe has to say on this, but going out on a
> limb here, it will be almost impossible to get that much out of your
> HDDs with 8 nodes without serious planning and an extremely narrow
> use-case.  I assume you are talking about putting drives in each  
> node at
> this point, because with just QDR you cannot feed aggregate 4GB/s
> without bonding from one node.
>
> We need to know more about generating this tablebase -- I can only
> assume you are planning to do analyses on it after you generate all
> possible combinations, right?  We need to know more about how that
> follow-up analysis can be divided before commenting on possible  
> storage
> solutions.  If everything is totally embarrassingly parallel you're  
> in a
> good spot to not bother with a parallel filesystem.  In that case you
> just might be able to squeeze 4GB/s out of your drives.
>
> But with all the nodes accessing all the disks at once, hitting 4GB/s
> with just strung together FOSS software is really tough for  
> anything but
> the most basic and most embarrassingly parallel stuff.  It requires
> serious tuning over months or buying a product that has already done
> this (e.g. a solution like Joe's company Scalable Informatics makes or
> Panasas, the company I work for, makes).  People always love to say,
> "Oh, that's 100MB/s per drive!  So with 64 drives I should be able to
> get 6.4GB/s!  Yea!"  Sadly, that's really only the case when these
> drives are accessed completely sequentially and completely separately
> (i.e. not put together into a distributed filesystem).
>
>> What raid system you'd recommend here?
>
> Uh, you looking for software or hardware or object RAID?
>
>> A problem is the write speed + read speed i need. From what i
>> understand at the edges of drives the speed is
>> roughly 133MB/s SATA2 moving down to a 33MB/s at the innersides.
>>
>> Is that roughly correct?
>
> I hate this as much as anybody, but........ It Depends (TM).
> You talking plain-jane "dd".  Sure, that might be reasonable for  
> certain
> vendors.
>
>> Of course there will be many solutions. I could use some raid cards
>> or i could equip each node with some drives.
>> Raid card is probably sata-3 nowadays. Didn't check speeds there.
>>
>> Total storage is some dozen to a few dozens of terabytes.
>>
>> Does the filesystem automatically optimize for writing at the edges
>> instead of starting at the innerside?
>> which 'raid' level would you recommend for this if any is appropriate
>> at all :)
>
> Again, depends on RAID card and whatnot.  Some do, some don't.
>
>> How many harddrives would i need? What failure rate can i expect with
>> modern SATA drives there?
>> I had several fail at a raid0+1 system before when generating some
>> EGTBs some years ago.
>
> Yup, things will break especially during the shakeout (first few  
> days or
> weeks).  I assume you're buying commodity drives here, not enterprise,
> so you should prepare for upwards of, /after the shakeout/, maybe  
> 4-8 of
> your drives to fail or start throwing SMART errors in the first year
> (ball-parking it here based solely on experience).  Rebuilds will suck
> for you with lots of data unless you have really thought that out
> (typically limited to speed of a single disk -- therefore 2TB drive
> rebuilding itself at 50MB/s (that's best case scenario) is like 11
> hours.  I hope you haven't bought all your drives from the same batch
> from the same manufacturer as well -- that often results in very  
> similar
> failure times (i.e. concurrent failures in a day).  Very non-uniform.
>
>> Note there is more questions. Like which buffer size i must read/
>> write. Most files get streamed.
>>   From 2 files that i do reading from, i read big blocks from a  
>> random
>> spot in that file. Each file is
>> a couple of hundreds of gigabyte.
>>
>> I used to grab chunks of 64KB from each file, but don't see how to
>> get to gigabytes a second i/o with
>> todays hardware that manner.
>>
>> Am considering now to read blocks of 10MB. Which size will get me
>> there to the maximum bandwidth the i/o
>> can deliver?
>
> I actually do wonder if Hadoop won't work for you.  This sounds like a
> very Hadoop-like workload, assuming you are OK with write-once read- 
> many
> semantics.  But I need to know way more about what you want to do with
> the data afterwards.  Moving data off of HDFS sucks.
>
> Best,
>
> ellis
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From diep at xs4all.nl  Fri Aug 17 13:03:27 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Fri, 17 Aug 2012 22:03:27 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
	<CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
Message-ID: <E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>

The homepage looks very commercial and they have a free trial on it.
You refer to the free trial?

I'll leave it at that.

Putting everything in 1 basket means extra machine that burns juice
of course. That's the first disadvantage. Not the most serious one of  
course
as i could equip one of the nodes with it.

Means buy raid controller. That's extra cost. That depends upon what  
it costs.

The mellanox infiniband can on paper handle it, it's 8 GB/s.

So if i would use it for 4 nodes 2GB/s from which majority reads,
should be possible maybe. Of course reads or writes no big difference  
to the network.

Don't think the mellanox has any problems there. It'll do it handsdown.

Asking for trouble though for the motherboards to load a network that  
much i would
  suppose.

But it does mean that every node and every diskread and write you do,
that they all hammer at the same time at that single basket.

I don't see how you can get out of 1 cheap box good performance like  
that.

What's write latency for a diskwrite?

I'm no expert there. 7 milliseconds or so?

If i read 10MB blocks from disk from a semi-random location at a time  
and i have 32-64 cores doing  that,
it's gonna get at against 2GB/s around a 200 packets of 10MB a second  
both read and
write. Majority will be reads, say 70-30 or so.

Which raid controller can handle that?

Probably not a $200 one i suppose? Start adding zero's?



On Aug 17, 2012, at 9:45 PM, Andrew Holway wrote:

> How about something like putting all your disks in one basket and
> getting a ZFS / NFSoRDMA solution such as nexenta.
>
> They have a nice open source distribution.
>
> 2012/8/17 Vincent Diepeveen <diep at xs4all.nl>:
>> The idea someone brought me on by means of a private email is
>> to use a distributed file system and split each drive in 2  
>> partitions.
>>
>> the outside which is fastest for local storage and the inside for a
>> global distributed partition for long term
>> storage of endresults and automatically compressing with a scripts
>> results and decompressing when it
>> seems soon a specific EGTB is needed.
>>
>> Then using 3 disks a node i can get a 133MB-150MB /s on the outside
>> of the drives in a raid-0.
>> That'll be around a 3TB the minimum needed for generation.
>>
>> And the inside then gets a partition that uses redundancy, maybe
>> raid-6 ?
>> any thoughts there.
>>
>> So say a node or 4 i can dedicate to this. that's 12 drives.
>>
>> Then i'll take 6 months instead of 3 months to generate but i have 4
>> other nodes free for other jobs.
>>
>> Also i need to pay less to harddrives then. Question now is whether
>> i'll go for the 3TB then or the 2TB.
>>
>> As for the filesystem that's most interesting to do this. Is gluster
>> a good idea for this?
>>
>> Can it handle this split between partitions in local and global? Does
>> it have raid-6 or maybe some other
>> sort of redundancy you'd advice?
>>
>> As for hadoop that's a java thing you know. If i want to get my
>> cluster hacked from India i know an easier way to get that done :)
>>
>> Thanks in Advance,
>> Vincent
>>
>>
>> On Aug 17, 2012, at 4:42 PM, Ellis H. Wilson III wrote:
>>
>>> On 08/17/12 08:03, Vincent Diepeveen wrote:
>>>> hi,
>>>>
>>>> Which free or very cheap distributed file system choices do i have
>>>> for a 8 node cluster that has QDR infiniband (mellanox)?
>>>> Each node could have a few harddrives. Up to 8 or so SATA2. Could
>>>> also use some raid cards.
>>>
>>> Lots of choices, but are you talking about putting a bunch of  
>>> disks in
>>> all those PCs or having one I/O server?  The latter is the classic
>>> solution but there are ways to do the former.
>>>
>>> Short answer is there are complicated ways to fling your hdds into
>>> distributed machines using PVFS and get good performance provided  
>>> you
>>> are okay with those non-posix semantics and guarantees.  There are
>>> also
>>> ways to get decent performance from the Hadoop Distributed File
>>> System,
>>> which can handle a distributed set of nodes and internal HDDs well,
>>> but
>>> for a /constrained set of applications./  Based on your previous  
>>> posts
>>> about GPUs and whatnot, I'm going to assume you will have little to
>>> zero
>>> interest in Hadoop.  Last, there's a new NFS version out (pNFS,  
>>> or NFS
>>> v4.1) that you can probably use to great impact with proper
>>> tuning.  No
>>> comments on tuning it however, as I haven't yet tried myself.  That
>>> may
>>> be your best out of the box solution.
>>>
>>> Also, I assume you're talking about QDR 1X here, so just 8Gb/s per
>>> node.
>>> Correct me if that's wrong.
>>>
>>>> And i'm investigating what i need.
>>>>
>>>> I'm investigating to generate the 7 men EGTBs at the cluster.  
>>>> This is
>>>> a big challenge.
>>>
>>> For anyone who doesn't know (probably many who aren't into chess, I
>>> had
>>> to look this up myself), EGTB is end game table bases, and more
>>> info is
>>> available at:
>>> http://en.wikipedia.org/wiki/Endgame_tablebase
>>>
>>> Basically it's just a giant dump of exhaustive moves for N men  
>>> left on
>>> the board.
>>>
>>>> To generate it is high i/o load. I'm looking at around a 4 GB/s i/o
>>>> from which a tad more than
>>>> 1GB/s is write and a tad less than 3GB/s is readspeed from
>>>> harddrives.
>>>>
>>>> This for 3+ months nonstop. Provided the CPU's can keep up with  
>>>> that.
>>>> Otherwise a few months more.
>>>>
>>>> This 4GB/s i/o is aggregated speed.
>>>
>>> I would LOVE to hear what Joe has to say on this, but going out on a
>>> limb here, it will be almost impossible to get that much out of your
>>> HDDs with 8 nodes without serious planning and an extremely narrow
>>> use-case.  I assume you are talking about putting drives in each
>>> node at
>>> this point, because with just QDR you cannot feed aggregate 4GB/s
>>> without bonding from one node.
>>>
>>> We need to know more about generating this tablebase -- I can only
>>> assume you are planning to do analyses on it after you generate all
>>> possible combinations, right?  We need to know more about how that
>>> follow-up analysis can be divided before commenting on possible
>>> storage
>>> solutions.  If everything is totally embarrassingly parallel you're
>>> in a
>>> good spot to not bother with a parallel filesystem.  In that case  
>>> you
>>> just might be able to squeeze 4GB/s out of your drives.
>>>
>>> But with all the nodes accessing all the disks at once, hitting  
>>> 4GB/s
>>> with just strung together FOSS software is really tough for
>>> anything but
>>> the most basic and most embarrassingly parallel stuff.  It requires
>>> serious tuning over months or buying a product that has already done
>>> this (e.g. a solution like Joe's company Scalable Informatics  
>>> makes or
>>> Panasas, the company I work for, makes).  People always love to say,
>>> "Oh, that's 100MB/s per drive!  So with 64 drives I should be  
>>> able to
>>> get 6.4GB/s!  Yea!"  Sadly, that's really only the case when these
>>> drives are accessed completely sequentially and completely  
>>> separately
>>> (i.e. not put together into a distributed filesystem).
>>>
>>>> What raid system you'd recommend here?
>>>
>>> Uh, you looking for software or hardware or object RAID?
>>>
>>>> A problem is the write speed + read speed i need. From what i
>>>> understand at the edges of drives the speed is
>>>> roughly 133MB/s SATA2 moving down to a 33MB/s at the innersides.
>>>>
>>>> Is that roughly correct?
>>>
>>> I hate this as much as anybody, but........ It Depends (TM).
>>> You talking plain-jane "dd".  Sure, that might be reasonable for
>>> certain
>>> vendors.
>>>
>>>> Of course there will be many solutions. I could use some raid cards
>>>> or i could equip each node with some drives.
>>>> Raid card is probably sata-3 nowadays. Didn't check speeds there.
>>>>
>>>> Total storage is some dozen to a few dozens of terabytes.
>>>>
>>>> Does the filesystem automatically optimize for writing at the edges
>>>> instead of starting at the innerside?
>>>> which 'raid' level would you recommend for this if any is  
>>>> appropriate
>>>> at all :)
>>>
>>> Again, depends on RAID card and whatnot.  Some do, some don't.
>>>
>>>> How many harddrives would i need? What failure rate can i expect  
>>>> with
>>>> modern SATA drives there?
>>>> I had several fail at a raid0+1 system before when generating some
>>>> EGTBs some years ago.
>>>
>>> Yup, things will break especially during the shakeout (first few
>>> days or
>>> weeks).  I assume you're buying commodity drives here, not  
>>> enterprise,
>>> so you should prepare for upwards of, /after the shakeout/, maybe
>>> 4-8 of
>>> your drives to fail or start throwing SMART errors in the first year
>>> (ball-parking it here based solely on experience).  Rebuilds will  
>>> suck
>>> for you with lots of data unless you have really thought that out
>>> (typically limited to speed of a single disk -- therefore 2TB drive
>>> rebuilding itself at 50MB/s (that's best case scenario) is like 11
>>> hours.  I hope you haven't bought all your drives from the same  
>>> batch
>>> from the same manufacturer as well -- that often results in very
>>> similar
>>> failure times (i.e. concurrent failures in a day).  Very non- 
>>> uniform.
>>>
>>>> Note there is more questions. Like which buffer size i must read/
>>>> write. Most files get streamed.
>>>>   From 2 files that i do reading from, i read big blocks from a
>>>> random
>>>> spot in that file. Each file is
>>>> a couple of hundreds of gigabyte.
>>>>
>>>> I used to grab chunks of 64KB from each file, but don't see how to
>>>> get to gigabytes a second i/o with
>>>> todays hardware that manner.
>>>>
>>>> Am considering now to read blocks of 10MB. Which size will get me
>>>> there to the maximum bandwidth the i/o
>>>> can deliver?
>>>
>>> I actually do wonder if Hadoop won't work for you.  This sounds  
>>> like a
>>> very Hadoop-like workload, assuming you are OK with write-once read-
>>> many
>>> semantics.  But I need to know way more about what you want to do  
>>> with
>>> the data afterwards.  Moving data off of HDFS sucks.
>>>
>>> Best,
>>>
>>> ellis
>>> _______________________________________________
>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>>> Computing
>>> To change your subscription (digest mode or unsubscribe) visit
>>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit  
>> http://www.beowulf.org/mailman/listinfo/beowulf



From andrew.holway at gmail.com  Fri Aug 17 12:45:49 2012
From: andrew.holway at gmail.com (Andrew Holway)
Date: Fri, 17 Aug 2012 21:45:49 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
Message-ID: <CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>

How about something like putting all your disks in one basket and
getting a ZFS / NFSoRDMA solution such as nexenta.

They have a nice open source distribution.

2012/8/17 Vincent Diepeveen <diep at xs4all.nl>:
> The idea someone brought me on by means of a private email is
> to use a distributed file system and split each drive in 2 partitions.
>
> the outside which is fastest for local storage and the inside for a
> global distributed partition for long term
> storage of endresults and automatically compressing with a scripts
> results and decompressing when it
> seems soon a specific EGTB is needed.
>
> Then using 3 disks a node i can get a 133MB-150MB /s on the outside
> of the drives in a raid-0.
> That'll be around a 3TB the minimum needed for generation.
>
> And the inside then gets a partition that uses redundancy, maybe
> raid-6 ?
> any thoughts there.
>
> So say a node or 4 i can dedicate to this. that's 12 drives.
>
> Then i'll take 6 months instead of 3 months to generate but i have 4
> other nodes free for other jobs.
>
> Also i need to pay less to harddrives then. Question now is whether
> i'll go for the 3TB then or the 2TB.
>
> As for the filesystem that's most interesting to do this. Is gluster
> a good idea for this?
>
> Can it handle this split between partitions in local and global? Does
> it have raid-6 or maybe some other
> sort of redundancy you'd advice?
>
> As for hadoop that's a java thing you know. If i want to get my
> cluster hacked from India i know an easier way to get that done :)
>
> Thanks in Advance,
> Vincent
>
>
> On Aug 17, 2012, at 4:42 PM, Ellis H. Wilson III wrote:
>
>> On 08/17/12 08:03, Vincent Diepeveen wrote:
>>> hi,
>>>
>>> Which free or very cheap distributed file system choices do i have
>>> for a 8 node cluster that has QDR infiniband (mellanox)?
>>> Each node could have a few harddrives. Up to 8 or so SATA2. Could
>>> also use some raid cards.
>>
>> Lots of choices, but are you talking about putting a bunch of disks in
>> all those PCs or having one I/O server?  The latter is the classic
>> solution but there are ways to do the former.
>>
>> Short answer is there are complicated ways to fling your hdds into
>> distributed machines using PVFS and get good performance provided you
>> are okay with those non-posix semantics and guarantees.  There are
>> also
>> ways to get decent performance from the Hadoop Distributed File
>> System,
>> which can handle a distributed set of nodes and internal HDDs well,
>> but
>> for a /constrained set of applications./  Based on your previous posts
>> about GPUs and whatnot, I'm going to assume you will have little to
>> zero
>> interest in Hadoop.  Last, there's a new NFS version out (pNFS, or NFS
>> v4.1) that you can probably use to great impact with proper
>> tuning.  No
>> comments on tuning it however, as I haven't yet tried myself.  That
>> may
>> be your best out of the box solution.
>>
>> Also, I assume you're talking about QDR 1X here, so just 8Gb/s per
>> node.
>> Correct me if that's wrong.
>>
>>> And i'm investigating what i need.
>>>
>>> I'm investigating to generate the 7 men EGTBs at the cluster. This is
>>> a big challenge.
>>
>> For anyone who doesn't know (probably many who aren't into chess, I
>> had
>> to look this up myself), EGTB is end game table bases, and more
>> info is
>> available at:
>> http://en.wikipedia.org/wiki/Endgame_tablebase
>>
>> Basically it's just a giant dump of exhaustive moves for N men left on
>> the board.
>>
>>> To generate it is high i/o load. I'm looking at around a 4 GB/s i/o
>>> from which a tad more than
>>> 1GB/s is write and a tad less than 3GB/s is readspeed from
>>> harddrives.
>>>
>>> This for 3+ months nonstop. Provided the CPU's can keep up with that.
>>> Otherwise a few months more.
>>>
>>> This 4GB/s i/o is aggregated speed.
>>
>> I would LOVE to hear what Joe has to say on this, but going out on a
>> limb here, it will be almost impossible to get that much out of your
>> HDDs with 8 nodes without serious planning and an extremely narrow
>> use-case.  I assume you are talking about putting drives in each
>> node at
>> this point, because with just QDR you cannot feed aggregate 4GB/s
>> without bonding from one node.
>>
>> We need to know more about generating this tablebase -- I can only
>> assume you are planning to do analyses on it after you generate all
>> possible combinations, right?  We need to know more about how that
>> follow-up analysis can be divided before commenting on possible
>> storage
>> solutions.  If everything is totally embarrassingly parallel you're
>> in a
>> good spot to not bother with a parallel filesystem.  In that case you
>> just might be able to squeeze 4GB/s out of your drives.
>>
>> But with all the nodes accessing all the disks at once, hitting 4GB/s
>> with just strung together FOSS software is really tough for
>> anything but
>> the most basic and most embarrassingly parallel stuff.  It requires
>> serious tuning over months or buying a product that has already done
>> this (e.g. a solution like Joe's company Scalable Informatics makes or
>> Panasas, the company I work for, makes).  People always love to say,
>> "Oh, that's 100MB/s per drive!  So with 64 drives I should be able to
>> get 6.4GB/s!  Yea!"  Sadly, that's really only the case when these
>> drives are accessed completely sequentially and completely separately
>> (i.e. not put together into a distributed filesystem).
>>
>>> What raid system you'd recommend here?
>>
>> Uh, you looking for software or hardware or object RAID?
>>
>>> A problem is the write speed + read speed i need. From what i
>>> understand at the edges of drives the speed is
>>> roughly 133MB/s SATA2 moving down to a 33MB/s at the innersides.
>>>
>>> Is that roughly correct?
>>
>> I hate this as much as anybody, but........ It Depends (TM).
>> You talking plain-jane "dd".  Sure, that might be reasonable for
>> certain
>> vendors.
>>
>>> Of course there will be many solutions. I could use some raid cards
>>> or i could equip each node with some drives.
>>> Raid card is probably sata-3 nowadays. Didn't check speeds there.
>>>
>>> Total storage is some dozen to a few dozens of terabytes.
>>>
>>> Does the filesystem automatically optimize for writing at the edges
>>> instead of starting at the innerside?
>>> which 'raid' level would you recommend for this if any is appropriate
>>> at all :)
>>
>> Again, depends on RAID card and whatnot.  Some do, some don't.
>>
>>> How many harddrives would i need? What failure rate can i expect with
>>> modern SATA drives there?
>>> I had several fail at a raid0+1 system before when generating some
>>> EGTBs some years ago.
>>
>> Yup, things will break especially during the shakeout (first few
>> days or
>> weeks).  I assume you're buying commodity drives here, not enterprise,
>> so you should prepare for upwards of, /after the shakeout/, maybe
>> 4-8 of
>> your drives to fail or start throwing SMART errors in the first year
>> (ball-parking it here based solely on experience).  Rebuilds will suck
>> for you with lots of data unless you have really thought that out
>> (typically limited to speed of a single disk -- therefore 2TB drive
>> rebuilding itself at 50MB/s (that's best case scenario) is like 11
>> hours.  I hope you haven't bought all your drives from the same batch
>> from the same manufacturer as well -- that often results in very
>> similar
>> failure times (i.e. concurrent failures in a day).  Very non-uniform.
>>
>>> Note there is more questions. Like which buffer size i must read/
>>> write. Most files get streamed.
>>>   From 2 files that i do reading from, i read big blocks from a
>>> random
>>> spot in that file. Each file is
>>> a couple of hundreds of gigabyte.
>>>
>>> I used to grab chunks of 64KB from each file, but don't see how to
>>> get to gigabytes a second i/o with
>>> todays hardware that manner.
>>>
>>> Am considering now to read blocks of 10MB. Which size will get me
>>> there to the maximum bandwidth the i/o
>>> can deliver?
>>
>> I actually do wonder if Hadoop won't work for you.  This sounds like a
>> very Hadoop-like workload, assuming you are OK with write-once read-
>> many
>> semantics.  But I need to know way more about what you want to do with
>> the data afterwards.  Moving data off of HDFS sucks.
>>
>> Best,
>>
>> ellis
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From n030552 at yahoo.com.tw  Fri Aug 17 14:51:23 2012
From: n030552 at yahoo.com.tw (sandy)
Date: Sat, 18 Aug 2012 05:51:23 +0800
Subject: [Beowulf] =?utf-8?b?YmVvd3VsZuaIkOeri+mmmea4r+WFrOWPuA==?=
Message-ID: <B3D5CD26C0324BF27B9726E9E61733A5@hvgxwijjq>

??????????

????????????020-87690812???????
 (???????????QQ?MSN?????????)

 ???????
 
 - ??????
 - ??????
 - ?????1?
 - ????20?
 - 1????2 ???????????????????
 - ?????????NC1??????????
 - ???????????
 - ?????????????
 - ??????????


????  ??? QQ: 1875287993 ?QQ??????2) MSN:hksbc7 at hotmail.com

3kj


Dear Sir,
Success bridge consultant Co., ltd is a leading business consultancy and company 
reagister agency in Hong kong .
Our service guideline is to provide one-stop service with hige quality for our 
clients.
Our service scope
1. Hong kong company formation.
2. Hong kong business bank account opening.
3. Taxation and accounting.
4. Trademar hk registration.
We provide consultation on investment projects and design plans for clients who 
to invest in Hong kong .
Best Regards
 


From ellis at cse.psu.edu  Sat Aug 18 06:19:20 2012
From: ellis at cse.psu.edu (Ellis H. Wilson III)
Date: Sat, 18 Aug 2012 09:19:20 -0400
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <0C42D0AC-897E-4276-A9BD-027DB5A7F5FA@xs4all.nl>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<0C42D0AC-897E-4276-A9BD-027DB5A7F5FA@xs4all.nl>
Message-ID: <502F9658.6090801@cse.psu.edu>

On 08/17/2012 12:04 PM, Vincent Diepeveen wrote:
> Yes i realize that. In principle you're looking at a 1050 files or so
> that get effectively generated
> and to generate each file half a dozen of huge files get created.
>
> Now in theory generating them is embarrassingly parallel except that to
> generate 1 large Set of EGTBs
> requires around a 3TB of working set size.
>
> Now comes the achillesheel. The start of the generation it needs to
> access real quickly some earlier
> generated sets; the initial generation of the conversion bitmap needs to
> access other EGTBs,
> as pieces can promote especially. Lucky this is a single pass, but it's
> a slow and intensive pass.
>
> In such case accessing over the network is important.
>
> So there is a huge locality except for 1 pass. The real fast generation
> that hammers onto the drives and reads quick and writes
> fast, that can be done entirely local. The first pass generating a
> single 'exchange bitmap', needs to lookup to
> EGTBs earlier generate. For example if we have the EGTB KQRP KRP then it
> has to lookup to the much larger
> EGTB that holds KQRB KRP and a few others.
>
> So the cohesion to the other nodes drives is limited to say a few
> percent of the total i/o getting done.
>
> As we speak about complex file management here, it's difficult to do
> this by hand.
>
> In other words, efficient usage of the available harddrive space is
> important.

Without knowing more about your workload (can you avoid 
read-modify-writes?  how small or large are your individual I/Os and can 
you adjust them?) this really seems to be ideal for Hadoop...

I've dug into the code and have a reasonably firm understanding of the 
nitty-gritty for PVFS 1 and 2, PanFS, NFS and HDFS, and I can promise 
you this, on the surface at least, appears to be well suited for the 
MapReduce paradigm.

I understand your hesitation about Java (not the security ones since 
this cluster should not be exposed directly to the internet anyhow...but 
that's beside the point), but let me put it to you this way:  I've got a 
cluster with 50 individual drives all in separate 50, just dual core 
machines, which are just connected via plain old 1Gb ethernet, and I can 
push close to 4.5GB/s when doing embarrassingly parallel, mainly 
sequential workloads with them.  Obviously if I start to do anything 
more random or less embarrassingly parallel this number is halved if not 
worse.  If your work allows you to operate in the environment that 
MapReduce and HDFS allows and encourages, I would strongly suggest you 
pursue that route.  That's the only distributed environment I can think 
of off of the top of my head that can properly handle (out of the box) 
the division you strike between local and remote accesses.

> Compress it with 7-zip and move it away indeed. It'll compress to 3TB

As a side-note -- Hadoop provides support for compression on transfers 
that might help you immensely.  You can pick from a few, but LZO tends 
to be the best one for speed/compression for my workloads.  This could 
really help you when you need to do that 1 pass where all nodes are 
exchanging with each other.

Best of luck!

ellis


From diep at xs4all.nl  Sat Aug 18 07:55:12 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Sat, 18 Aug 2012 16:55:12 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <CAEiui-uBb_rpYiWJS55FF+M2-YYVAuG361k4SH=GqLtCPVR_9Q@mail.gmail.com>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
	<CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
	<E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>
	<CAEiui-uBb_rpYiWJS55FF+M2-YYVAuG361k4SH=GqLtCPVR_9Q@mail.gmail.com>
Message-ID: <476FB5BC-84A7-4298-A311-F327FEBB1495@xs4all.nl>


On Aug 18, 2012, at 1:04 PM, Andrew Holway wrote:

> 2012/8/17 Vincent Diepeveen <diep at xs4all.nl>:
>> The homepage looks very commercial and they have a free trial on it.
>> You refer to the free trial?
>
> http://nexentastor.org/ - Sorry wrong link. Its a commercially backed
> open source project.
>
>> Means buy raid controller. That's extra cost. That depends upon what
>> it costs.
>
> You just need to attach the disks to some SATA port. ZFS does all the
> raid stuff internally in software.
>
>> But it does mean that every node and every diskread and write you do,
>> that they all hammer at the same time at that single basket.
>
> ZFS seems to handle this kind of stuff elegantly. As with hardware
> raid each disk can be accessed individually. NFSoRDMA would ensure
> speedy access times.
>
>>
>> I don't see how you can get out of 1 cheap box good performance like
>> that.
>
> Try it and see. It would certainly be much less of a headache than
> some kind of distributed filesystem which, in my opinion is complete
> overkill for a 4 node cluster. All of the admins that I know that look
> after these systems have the haunted look of a village elder that must
> choose which of the villages daughters must be fed to the lustre
> monster every 6 months.
>
> Dont forget to put in as much memory as you can afford and ideally an
> SSD For read cache (assuming that you access the same blocks over and
> over in some fashion)

I designed something myself in datastructure that's close to ZFS  
according to someone at the time working for Sun in Bangalore;
this was before ZFS was popular, or even introduced (am not sure - it  
was 2001-2002 or so),
but am no aware how the filesystem has been expanded since then to  
satisfy professional needs :)

Note i wasn't aware it works sincethen in Linux as opensource. Does it?

My thing is streaming a dataset of around a 1.3TB over and over again  
and each time something in
the dataset gets modified. So the output is a bitstream that you  
store and this bitstream is, all cores together
storing 1.3TB or so.

Note if i write TB it's terabyte. All those raidcards write Gb =  
gigaBIT.

1.3TB of SSD a node would speed it up considerable, but that's too  
expensive.

I do agree about maintenance, but my cluster ain't larger than 8  
nodes here and i do want that
performance of 0.5TB/s a node, so in case of 8 nodes it should be 4GB/ 
s agreggated bandwidth to the i/o
and not the say nearly 800MB/s that most raidcards, that are cheap on  
ebay, seem to deliver.

So some sort of distributed file system seems the best option, and a  
lot cheaper and a lot faster than a dedicated fileserver
that will not be able to keep up.






From diep at xs4all.nl  Sat Aug 18 08:05:35 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Sat, 18 Aug 2012 17:05:35 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <502F9658.6090801@cse.psu.edu>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<0C42D0AC-897E-4276-A9BD-027DB5A7F5FA@xs4all.nl>
	<502F9658.6090801@cse.psu.edu>
Message-ID: <5E4E5819-985A-4F79-9FDA-DCE64A79CF59@xs4all.nl>


On Aug 18, 2012, at 3:19 PM, Ellis H. Wilson III wrote:

> On 08/17/2012 12:04 PM, Vincent Diepeveen wrote:
>> Yes i realize that. In principle you're looking at a 1050 files or so
>> that get effectively generated
>> and to generate each file half a dozen of huge files get created.
>>
>> Now in theory generating them is embarrassingly parallel except  
>> that to
>> generate 1 large Set of EGTBs
>> requires around a 3TB of working set size.
>>
>> Now comes the achillesheel. The start of the generation it needs to
>> access real quickly some earlier
>> generated sets; the initial generation of the conversion bitmap  
>> needs to
>> access other EGTBs,
>> as pieces can promote especially. Lucky this is a single pass, but  
>> it's
>> a slow and intensive pass.
>>
>> In such case accessing over the network is important.
>>
>> So there is a huge locality except for 1 pass. The real fast  
>> generation
>> that hammers onto the drives and reads quick and writes
>> fast, that can be done entirely local. The first pass generating a
>> single 'exchange bitmap', needs to lookup to
>> EGTBs earlier generate. For example if we have the EGTB KQRP KRP  
>> then it
>> has to lookup to the much larger
>> EGTB that holds KQRB KRP and a few others.
>>
>> So the cohesion to the other nodes drives is limited to say a few
>> percent of the total i/o getting done.
>>
>> As we speak about complex file management here, it's difficult to do
>> this by hand.
>>
>> In other words, efficient usage of the available harddrive space is
>> important.
>
> Without knowing more about your workload (can you avoid
> read-modify-writes?  how small or large are your individual I/Os  
> and can
> you adjust them?) this really seems to be ideal for Hadoop...
>
> I've dug into the code and have a reasonably firm understanding of the
> nitty-gritty for PVFS 1 and 2, PanFS, NFS and HDFS, and I can promise
> you this, on the surface at least, appears to be well suited for the
> MapReduce paradigm.
>
> I understand your hesitation about Java (not the security ones since
> this cluster should not be exposed directly to the internet  
> anyhow...but
> that's beside the point), but let me put it to you this way:  I've  
> got a
> cluster with 50 individual drives all in separate 50, just dual core
> machines, which are just connected via plain old 1Gb ethernet, and  
> I can
> push close to 4.5GB/s when doing embarrassingly parallel, mainly
> sequential workloads with them.  Obviously if I start to do anything

The important question is: how much system time do you have left if  
you run java?

I need 100% of the CPU power. So a filesystem that doesn't require much
of a system time is most interesting.

I really need each core of every node badly. Having the filesystem  
compress
things for me is something i just can't afford. Finished EGTBs i can  
wr.............ite
a small script for to get compressed after a while...

Now the EGTBs themselves aren't secret anyhow, as they will get spreaded
amongst Diep users, but i feel you seriously underestimate the  
security risk of
java. Once something evil is on the inside of your building, it WILL  
get out
somehow. So to speak because of a car driving closeby and someone who
swings open a door for half a minute.

A chain of software that controls high frequency and low frequency
you are total powerless against.

the hard reality of todays society is that it's cheap to have people  
do this and
it gets massively done. HPC being the most attractive target, and  
they don't care
if they only find tic-tac-toe type software, as long as someone pays  
their bill it'll
keep happening.

Enough about that.

So i just skip as much javasoftware as i can avoid that's the cheap  
way out.

> more random or less embarrassingly parallel this number is halved  
> if not
> worse.  If your work allows you to operate in the environment that
> MapReduce and HDFS allows and encourages, I would strongly suggest you
> pursue that route.  That's the only distributed environment I can  
> think
> of off of the top of my head that can properly handle (out of the box)
> the division you strike between local and remote accesses.
>
>> Compress it with 7-zip and move it away indeed. It'll compress to 3TB
>
> As a side-note -- Hadoop provides support for compression on transfers
> that might help you immensely.  You can pick from a few, but LZO tends
> to be the best one for speed/compression for my workloads.  This could
> really help you when you need to do that 1 pass where all nodes are
> exchanging with each other.
>
> Best of luck!
>
> ellis
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From andrew.holway at gmail.com  Sat Aug 18 04:04:42 2012
From: andrew.holway at gmail.com (Andrew Holway)
Date: Sat, 18 Aug 2012 13:04:42 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
	<CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
	<E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>
Message-ID: <CAEiui-uBb_rpYiWJS55FF+M2-YYVAuG361k4SH=GqLtCPVR_9Q@mail.gmail.com>

2012/8/17 Vincent Diepeveen <diep at xs4all.nl>:
> The homepage looks very commercial and they have a free trial on it.
> You refer to the free trial?

http://nexentastor.org/ - Sorry wrong link. Its a commercially backed
open source project.

> Means buy raid controller. That's extra cost. That depends upon what
> it costs.

You just need to attach the disks to some SATA port. ZFS does all the
raid stuff internally in software.

> But it does mean that every node and every diskread and write you do,
> that they all hammer at the same time at that single basket.

ZFS seems to handle this kind of stuff elegantly. As with hardware
raid each disk can be accessed individually. NFSoRDMA would ensure
speedy access times.

>
> I don't see how you can get out of 1 cheap box good performance like
> that.

Try it and see. It would certainly be much less of a headache than
some kind of distributed filesystem which, in my opinion is complete
overkill for a 4 node cluster. All of the admins that I know that look
after these systems have the haunted look of a village elder that must
choose which of the villages daughters must be fed to the lustre
monster every 6 months.

Dont forget to put in as much memory as you can afford and ideally an
SSD For read cache (assuming that you access the same blocks over and
over in some fashion)


From andrew.holway at gmail.com  Sat Aug 18 10:17:17 2012
From: andrew.holway at gmail.com (Andrew Holway)
Date: Sat, 18 Aug 2012 19:17:17 +0200
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <476FB5BC-84A7-4298-A311-F327FEBB1495@xs4all.nl>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
	<CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
	<E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>
	<CAEiui-uBb_rpYiWJS55FF+M2-YYVAuG361k4SH=GqLtCPVR_9Q@mail.gmail.com>
	<476FB5BC-84A7-4298-A311-F327FEBB1495@xs4all.nl>
Message-ID: <CAEiui-vR8txhsGKmb+3No30mh7DLRZoon0XTctUPzunF0ZECAA@mail.gmail.com>

> So some sort of distributed file system seems the best option, and a
> lot cheaper and a lot faster than a dedicated fileserver
> that will not be able to keep up.

a) ZFS doesn't use hardware raid. at all. ever.

b) 500MB/s (actually 1GB/s) of I/O will chew up a quite large amount
of resource. Just think of all that nasty nasty TCP that your going to
have to do. Your essentially doubling the I/O requirements of each
node. Also, most distributed filesystems have a quite heavy CPU
overhead. Recalculating hashes and the such like is dirty work.

c) ZFS would use one or two SSD's as a read cache. Having an SSD only
volume is a quite rustic way of doing things nowadays.

d) I think the only tool that does what your describing, gluster,
really sucks because it is still FUSE.

e) ZFS is not yet on linux. Nexenta is based on what used to be OpenSolaris.

f) It does sound like a good fit for hadoop actually.

ta for now.

>
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From diep at xs4all.nl  Sun Aug 19 06:09:56 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Sun, 19 Aug 2012 15:09:56 +0200
Subject: [Beowulf] Compressor LZO versus others
In-Reply-To: <502F9658.6090801@cse.psu.edu>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<0C42D0AC-897E-4276-A9BD-027DB5A7F5FA@xs4all.nl>
	<502F9658.6090801@cse.psu.edu>
Message-ID: <41D7C702-5E93-4D5C-BD5C-AE073085B1FA@xs4all.nl>


On Aug 18, 2012, at 3:19 PM, Ellis H. Wilson III wrote:

[snip]

>
> As a side-note -- Hadoop provides support for compression on transfers
> that might help you immensely.  You can pick from a few, but LZO tends
> to be the best one for speed/compression for my workloads.  This could
> really help you when you need to do that 1 pass where all nodes are
> exchanging with each other.
>
> Best of luck!
>
> ellis

Hi,

I compared compressors for my dataset. Let me first of all mention to  
you that the official compressorbenchmarks are a joke.
They're tiny benchmarks with lots of different sorts of stuff to  
compress.

Now we all realize how important all that is.

to compres LARGE data, you suddenly need total other sorts of  
compressors. Speed sure matters, you can't use too slow of a compressor.

For whatever i do, 7-zip is total superior and relative fast to  
compress.
LZO beats it speedwise bigtime. No surprises there.

Now even doing something tiny differnt already gets called a  
different algorithm in compressing world so if i would quote  
algorithmic names,
i will have it all wrong of course.

Huffman and similar stuff that's basically able to compress buckets,  
that's pretty simple of course.
The 'new generation' compressors that's already here for a year or 15 
+ by now, they are doing multidimensional compression,
something far superior for huge datasets of course, as there is  
always multidimensional relatinships.

Most of such compressors are rather slow. 7-zip is relative fast.

I'm a bit amazed that there is no good 7-zip port for linux. In fact  
i had to compile something myself for Scientific Linux
and it is some total crap type of 'decompress only' sort of 7-zip.  
Some stuff i found by google it all didn't work. Wrong ELF binaries
this or wrong glibc that etc.

So then i compared LZO with another real fast compressor programmed  
by Andrew Kadatch. It's also from 90s and the huge advantage for
computerchess is that it decompresses into buckets. Not sure whether  
the compressor is 'free'. As a decompressor you can find the code in
several EGTB programs under which from Eugene Nalimov.

It's also Huffman and as a difference to LZO, Andrew Kadatch is a tad  
better in analyzing the data. Realize it doesn't use a dictionary,
which explains much of todays better compression standards, it's  
really compressing things in buckets, which is an advantage if you just
want those 64 kilobyte of data out of the database from some random  
spot in the file...

Here is the results:

The original file used for every compressor. A small EGTB of 1.8GB:

-rw-rw-r--. 1 diep diep 1814155128 Aug 19 10:37 knnknp_w.dtb

LZO (default compression):

-rw-rw-r--. 1 diep diep  474233006 Aug 19 10:37 knnknp_w.dtb.lzo

7-zip (default compression):

-rw-rw-r--. 1 diep diep 160603822 Aug 18 19:33 ../7z/33p/knnknp_w.dtb.7z

Andrew Kadatch:

-rw-rw-r--. 1 diep diep  334258087 Aug 19 14:37 knnknp_w.dtb.emd

We see kadatch is a 140MB smaller in size than LZO, that's a lot at  
474MB total size for the lzo
and it's 10% of total size of the original data.

So LZO in fact is so bad it doesn't even beat another Huffman  
compressor. A fast bucket compressor not using a dictionary at all is  
hammering it.

As for decompression speeds (not compressoin but the DECOMPRESSION  
speed is what matters to me), EMD is roughly same speed like LZO.
Maybe EMD tad faster, but you won't notice it. Of course being real  
old program, EMD has a 32 bits limit, it won't work over 2GB and the  
type of files
i proces will be quite larger. More like a 162.5GB each :)

So i'll search some further what proggie to use.

Note that as for real good compression there is much better  
compressing programs of course than 7-zip.
Much much better. There is some guys busy with this. Most of them  
really take a long time though to decompress (besides compression time),
which makes them less interesting.











> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From ellis at cse.psu.edu  Sun Aug 19 08:45:29 2012
From: ellis at cse.psu.edu (Ellis H. Wilson III)
Date: Sun, 19 Aug 2012 11:45:29 -0400
Subject: [Beowulf] Compressor LZO versus others
In-Reply-To: <41D7C702-5E93-4D5C-BD5C-AE073085B1FA@xs4all.nl>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<0C42D0AC-897E-4276-A9BD-027DB5A7F5FA@xs4all.nl>
	<502F9658.6090801@cse.psu.edu>
	<41D7C702-5E93-4D5C-BD5C-AE073085B1FA@xs4all.nl>
Message-ID: <50310A19.1020700@cse.psu.edu>

On 08/19/2012 09:09 AM, Vincent Diepeveen wrote:
> Here is the results:
>
> The original file used for every compressor. A small EGTB of 1.8GB:
>
> -rw-rw-r--. 1 diep diep 1814155128 Aug 19 10:37 knnknp_w.dtb
>
> LZO (default compression):
>
> -rw-rw-r--. 1 diep diep  474233006 Aug 19 10:37 knnknp_w.dtb.lzo
>
> 7-zip (default compression):
>
> -rw-rw-r--. 1 diep diep 160603822 Aug 18 19:33 ../7z/33p/knnknp_w.dtb.7z
>
> Andrew Kadatch:
>
> -rw-rw-r--. 1 diep diep  334258087 Aug 19 14:37 knnknp_w.dtb.emd
>
> We see kadatch is a 140MB smaller in size than LZO, that's a lot at
> 474MB total size for the lzo
> and it's 10% of total size of the original data.
>
> So LZO in fact is so bad it doesn't even beat another Huffman
> compressor. A fast bucket compressor not using a dictionary at all is
> hammering it.

Thanks for these insightful findings Vincent.  Unless I missed 
something, I didn't see timings for these algorithms.  I would be very 
interested to see these compressions wrapped in a 'time' command and 
please make sure to flush your buffer cache in between.  In Hadoop LZO 
seems to be the defacto standard for its widespread use, speed both of 
compression and decompression, and relatively high compression ratio 
compared to very bare-bones compressors.

So seeing these results, alongside the 1) time to compress when data is 
solely on HDD and 2) time to decompress when data is solely on HDD would 
be really, really helpful.

For Hadoop, since compression is mainly used to "package" data up prior 
to network transfer (and obviously it gets "unpackaged" on the other 
side if it needs to be used), the balance between speed and compression 
is a fine balance, dependent on your network and CPU capabilities.

Please let me know if you get around to running these experiments and if 
you find another compressor out there that is excellent and I'll have to 
consider it for my use in Hadoop!

Best,

ellis


From diep at xs4all.nl  Sun Aug 19 10:46:57 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Sun, 19 Aug 2012 19:46:57 +0200
Subject: [Beowulf] Compressor LZO versus others
In-Reply-To: <50310A19.1020700@cse.psu.edu>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<0C42D0AC-897E-4276-A9BD-027DB5A7F5FA@xs4all.nl>
	<502F9658.6090801@cse.psu.edu>
	<41D7C702-5E93-4D5C-BD5C-AE073085B1FA@xs4all.nl>
	<50310A19.1020700@cse.psu.edu>
Message-ID: <0A714E1D-1FF1-4041-9909-753031F848B2@xs4all.nl>


On Aug 19, 2012, at 5:45 PM, Ellis H. Wilson III wrote:

> On 08/19/2012 09:09 AM, Vincent Diepeveen wrote:
>> Here is the results:
>>
>> The original file used for every compressor. A small EGTB of 1.8GB:
>>
>> -rw-rw-r--. 1 diep diep 1814155128 Aug 19 10:37 knnknp_w.dtb
>>
>> LZO (default compression):
>>
>> -rw-rw-r--. 1 diep diep  474233006 Aug 19 10:37 knnknp_w.dtb.lzo
>>
>> 7-zip (default compression):
>>
>> -rw-rw-r--. 1 diep diep 160603822 Aug 18 19:33 ../7z/33p/ 
>> knnknp_w.dtb.7z
>>
>> Andrew Kadatch:
>>
>> -rw-rw-r--. 1 diep diep  334258087 Aug 19 14:37 knnknp_w.dtb.emd
>>
>> We see kadatch is a 140MB smaller in size than LZO, that's a lot at
>> 474MB total size for the lzo
>> and it's 10% of total size of the original data.
>>
>> So LZO in fact is so bad it doesn't even beat another Huffman
>> compressor. A fast bucket compressor not using a dictionary at all is
>> hammering it.
>
> Thanks for these insightful findings Vincent.  Unless I missed
> something, I didn't see timings for these algorithms.  I would be very
> interested to see these compressions wrapped in a 'time' command and
> please make sure to flush your buffer cache in between.  In Hadoop LZO
> seems to be the defacto standard for its widespread use, speed both of
> compression and decompression, and relatively high compression ratio
> compared to very bare-bones compressors.

I understand your question about 'time'.

Now i'll do something that seems terrible arrogant, but really i'm no  
expert
on compression. there is a few guys, especially 1 Dutch guy, who has  
a website
DEDICATED measuring everything painfully accurate.

It's better for me to email to him a few EGTBs, like i did do a year  
or 10 ago with him
and have him toy and then you get really any number you want and not  
just 1 compressor.

He'll try a whole range maybe for you.

As for me. I want a BIG space reduction and compressing i do just 1  
time. Decompressing must
be fast however.

The compression rate of LZO is just too laughable to even take  
serious in this case.

It's simply using years 70 algorithms and applies them very bad. It's  
fast sure, but we have plenty of
cores everywhere to compress and especially decompress; didn't even  
speak of GPU's yet.

3 times worse than 7-zip which is a relative fast compressor to  
decompress. At a single core it can write
more than the full write bandwidth of 1 drive here, it's achieving  
the maximum of 1 drive and that's just
simple C code. For windows it seems to have also assembler. Not sure  
whether the linux compile uses that;
I doubt it.

Now i don't know about you, but i have more cores than drives, as  
drives are painfully expensive.

So i'm not interested in : "how much faster does it decompress". I  
simply want a decompress speed where 1 core can get close
to the bandwidth of 1 drive which is simply, if i look to garantuees  
of manufacturers a 60-80 MB/s sustained they
garantuee kind of with peaks up to 133MB/s. I decompressed a terabyte  
last few hours here with 7-zip and it's
achieving a 50MB/s+ a core or so there.

Yet i do have far more cores available than bandwidth to the drives.  
It's easy nowadays to scale cores in fact.

LZO in fact  isn't achieving a much higher speed there obviously,  
simply i tested at 1 drive; so there isn't a faster i/o
speed than 100MB/s simply. So LZO doesn't offer an advantage there.  
It's simply 3x worse as when storing that massive
data that i store, you really want a good space reduction.

For me the space reduction really is *critical* important.

The 7 men are 82TB uncompressed. If i look to an 'attempt' at the 8  
men that's several petabytes. So if i want to
store the final result at an array that's just some dozens of  
terabyte, then i need a good compressor.

LZO loses me factor 3 more in diskspace - that's just not acceptable  
here.

Data that has been finished calculating, i never again basically need  
to modify it. I guess that's true for most.
It does need relative fast decompression though.

In fact if there would be a compressor that can get me a much better  
compression than this yet takes 10 times
the time, and decompresses "only" 2 times slower than 7-zip, i would  
use it.

What i do know from experiments 10 years ago is that the best  
compressor back then managed around a
factor 2.5 smaller result than 7-zip. That was much smaller EGTBs  
that was experimented upon, but of course
it took a single cpu back then (if i remember well a K7 at 1.x Ghz)  
around a 24 hours to compress a testset of a 200MB or so,
and more importantly also the same time to decompress it, and a 400MB  
ram or so.

That was a guy from new zealand with some crappy DOS type interface  
if i remember well...

Note all of this compresses much better for me than Paq what was  
topping the lists back then, but of course a typical
'testset' product. Using algorithm X for data d1 and algorithm Y for  
data d2; just careful parameter tuned - useless compressor
to me when i try to get it to work.

In all this you really want to leave the 80s in compressing  
technology now.

Why isn't there a good open source port that works great for linux  
for 7-zip by the way?

Especially one where you don't need more RAM than the filesize it's  
decompressing...

It's a magnificent compressor and i don't know anything that rivals  
it in terms of speed and the great result it achieves given
the slow speed you've got to the i/o on a core by core basis averaged.

So my hunt for a good compressor under linux isn't over yet...

Kind Regards,
Vincent

>
> So seeing these results, alongside the 1) time to compress when  
> data is
> solely on HDD and 2) time to decompress when data is solely on HDD  
> would
> be really, really helpful.

Yeah well decompressing a terabyte with just 4GB of RAM here i bet it  
all wasn't
on the HDD yet.

>
> For Hadoop, since compression is mainly used to "package" data up  
> prior
> to network transfer (and obviously it gets "unpackaged" on the other
> side if it needs to be used), the balance between speed and  
> compression
> is a fine balance, dependent on your network and CPU capabilities.
>
> Please let me know if you get around to running these experiments  
> and if
> you find another compressor out there that is excellent and I'll  
> have to
> consider it for my use in Hadoop!
>
> Best,
>
> ellis
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From samuel at unimelb.edu.au  Sun Aug 19 16:45:15 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Mon, 20 Aug 2012 09:45:15 +1000
Subject: [Beowulf] Moderation of the Beowulf Mailing List
In-Reply-To: <502DEEFC.4010205@gmail.com>
References: <CACqOxA4fbRO39gJpbGcHHqif_X8gaE1H6x3Ge_8Kyk8aQ_Z9sA@mail.gmail.com>
	<502DEEFC.4010205@gmail.com>
Message-ID: <50317A8B.9010900@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 17/08/12 17:13, Jonathan Aquilina wrote:

> I am looking at the moderation tasks and it seems like there are
> already a number of spam messages waiting there.

I'd had no notification of any moderation needed yet so I was a little
confused - then I checked the University spam filter and found them
all caught there.  I *think* that's a good thing.. :-/

cheers,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAxeosACgkQO2KABBYQAh9EaQCeOtjeF6WC6EWyBFkguR3JdgPn
lKsAnAkFPp7oswL+L0faVifLX7DhPS/+
=KtIE
-----END PGP SIGNATURE-----


From duke.lists at gmx.com  Sun Aug 19 21:55:22 2012
From: duke.lists at gmx.com (Duke Nguyen)
Date: Mon, 20 Aug 2012 11:55:22 +0700
Subject: [Beowulf] advice for newbie
Message-ID: <5031C33A.3060304@gmx.com>

Hi folks,

First let me start that I am total novice with cluster and/or beowulf. I 
am familiar with unix/linux and have a few years working in a cluster 
(HPC) environment, but I never have a chance to design and admin a cluster.

Now my new institute decides to build a (small) cluster for our next 
research focus area: genome research. The requirements are simple: 
expandable and capable of doing genome research. The budget is low, 
about $15,000, and we have decided:

  * cluster is a box cluster, not rack (well, mainly because our funding 
is low)
  * cluster OS is scientific linux with OpenMPI
  * cluster is about 16-node with a master node, expandable is a must

Now next step for us is to decide hardwares and other aspects:

  * any recommendation for a reliable 24-port gigabit switch for the 
cluster? I heard of HP ProCurve 2824 but it is a little bit hard to find 
it in my country
  * should our boxes be diskless or should they have a hard disk inside? 
I am still not very clear the advantages if the clients has about 80GB 
hard disk internally except that their OS are independent and does not 
depend on the master node, and maybe faster data processing (temporay), 
but 80GB each for genome research is too small
  * hard drives/data storage: we want to have storage of about 10TB but 
I am not sure how to design this. Should all the hard disk be in the 
master node, or they can be on each of the node, or should it be a NAS?
  * any recommendation for a mainboard (gigabit network, at least 4 RAM 
slots) about $200-$300 good for cluster?

I would love to hear any advice/suggestion from you, especially if you 
had built a similar cluster with similar purpose.

Thank you in advance,

Duke.



From eagles051387 at gmail.com  Sun Aug 19 22:27:47 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Mon, 20 Aug 2012 07:27:47 +0200
Subject: [Beowulf] Fwd: Your message to Beowulf awaits moderator approval
In-Reply-To: <5031CA28.5020602@unimelb.edu.au>
References: <mailman.39162.1344878858.2391.beowulf@beowulf.org>
	<5031B7EB.9060507@gmx.com> <5031CA28.5020602@unimelb.edu.au>
Message-ID: <5031CAD3.2060205@gmail.com>

On 20/08/2012 07:24, Christopher Samuel wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 20/08/12 14:07, Duke Nguyen wrote:
>
>> Hi Jonathan and Christopher,
> Hiya,
>
>> I am a new comer to the list. I sent two messages to the list a
>> week ago but I still have not heard anything from list moderators.
>> Is there any chance that my messages got into the spam list?
> Hmm, I've just had a look and there are only 4 messages
> waiting for moderation and they're all spam I'm afraid.
>
> I checked the uni's spam system and the messages I'd had
> weren't about moderation but spams that had got through to
> the list. :-(
>
> So it appears I'm not getting any alerts about messages
> that are pending moderation. :-(
>
> cheers,
> Chris
> - -- 
>   Christopher Samuel        Senior Systems Administrator
>   VLSCI - Victorian Life Sciences Computation Initiative
>   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>   http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlAxyigACgkQO2KABBYQAh8xGQCfUGgYDOD5iDeQ239RHTDJdmJQ
> MaMAnR/bkVnobamKZCekxi8WtPR8k5pS
> =A+Ae
> -----END PGP SIGNATURE-----
I am not either christopher. I just occasionally refresh the page and 
moderate as needed.

Arend is it possible to get an email with the link to the mailman back 
end so we know when there are emails needing moderation?


From diep at xs4all.nl  Mon Aug 20 00:12:38 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Mon, 20 Aug 2012 09:12:38 +0200
Subject: [Beowulf] advice for newbie
In-Reply-To: <5031C33A.3060304@gmx.com>
References: <5031C33A.3060304@gmx.com>
Message-ID: <CD2708F3-AF3D-4563-AA19-27BE6D70075D@xs4all.nl>

Look at ebay, there is cheap machines there $150 or less for 8 core  
Xeons L5420,
i'm using those as well. For that $150 you get 8GB ram as well.

Boot them over the network.

That's 16 * $150 = $3k.

Now there might be a difference between my and your wishes.
For me the network is important so i bought pci-e 2.0 motherboards;

In short i bought each component separate which is more expensive  
than $150.

And those rackmounts make huge noise; i'm doing it here with 14 CM  
fans that's not so noisy,
but i assume your institute has a spare room that may make people who  
walk their deaf before
they're 30.

As for you, buy 8 drives SATA from 2 TB and a $30 raid card from ebay  
Those raid cards 2nd hand
are dirt cheap especially the pci-x ones and with 8 drives you won't  
get a bigger bandwidth
anyway and most those online nodes have that.

They're 100 euro a piece here those drives. So that's 800 euro = $1k  
or so. And $30 from
ebay for a total superior raid card. Put them in a ready rackmount  
that allows such drives,
it's a couple of hundreds of dollars on ebay, with the same L5420's  
and motherboards and 8GB
ram. So say a $500 for that rackmount with drives you can plug in.

Put in the drives, build a raid6 partition and your fileserver , the  
17th machine, it's ready to serve you
at around a 700MB/s readspeed.

Now i don't know the latest about genome research; the last Phd  
student i helped out
there, his university used 90s software to do his research.

That really required big crunching for months for each calculation,  
at hundreds of cpu's,
yet new commercial software finished within 15 minutes each  
calculation at a single core.

That 90s software uses MPI if i recall but that'll depend upon what  
sort of software your
guys want to use.

You might want to decide next to buy the cheapest gigabit switch you  
can get, in order to
boot all the nodes over the network using pci-e.

It's possible those motherboards won't boot over infiniband, some might.

Then i'd really advice you buy a cheap 2nd hand switch infiniband,  
maybe DDR, of $300 or so.
Cables $20 a piece times 17 = $140, and a bunch of 2nd hand DDR  
infiniband cards and put
each one in each machine.

So after boot over the gigabit switch, assuming the motherboards  
don't boot over infiniband,
they might boot actually over infiniband in which case you don't need  
the gigabit switch,
then in that case infiniband will take over and can serve you either  
as 10 gigabit network
cards or for the MPI that much software in that area needs.

So all what's left is buy a 17 infiniband cards DDR 2nd hand off  
ebay. Not sure about prices,
maybe a $80 maybe $100. a piece. Let's say it's under $1600

Now you're done with a 16 node cluster with 17 machines from which 1  
is fileserver,
for a part of the budget you had in mind.

Just it's noisy and loud.

Also it's pretty low power compared to alternatives. It'll eat a 180  
watt a node or so under full load. It's 170 watt a node
here under full load (but that's with a much better psu).

As for software to install in case you decide for infiniband, your  
choices are limited as OpenFED doesn't give
much alternatives.

Fedora Core or Scientific Linux are for free and probably your only 2  
options if you want to use free software
that are easy to get things done as you want to.

Then install OpenFED that has the openmpi and other infiniband stuff  
for free.

Probably Debian works as well, provided you use the exact kernel that  
OpenFED recommends.
Any other kernel won't work. So you have to download some older  
debian then, get the exact kernel recommended
and then i guess OpenFED will install as well.

Good Luck,
Vincent

On Aug 20, 2012, at 6:55 AM, Duke Nguyen wrote:

> Hi folks,
>
> First let me start that I am total novice with cluster and/or  
> beowulf. I
> am familiar with unix/linux and have a few years working in a cluster
> (HPC) environment, but I never have a chance to design and admin a  
> cluster.
>
> Now my new institute decides to build a (small) cluster for our next
> research focus area: genome research. The requirements are simple:
> expandable and capable of doing genome research. The budget is low,
> about $15,000, and we have decided:
>
>   * cluster is a box cluster, not rack (well, mainly because our  
> funding
> is low)
>   * cluster OS is scientific linux with OpenMPI
>   * cluster is about 16-node with a master node, expandable is a must
>
> Now next step for us is to decide hardwares and other aspects:
>
>   * any recommendation for a reliable 24-port gigabit switch for the
> cluster? I heard of HP ProCurve 2824 but it is a little bit hard to  
> find
> it in my country
>   * should our boxes be diskless or should they have a hard disk  
> inside?
> I am still not very clear the advantages if the clients has about 80GB
> hard disk internally except that their OS are independent and does not
> depend on the master node, and maybe faster data processing  
> (temporay),
> but 80GB each for genome research is too small
>   * hard drives/data storage: we want to have storage of about 10TB  
> but
> I am not sure how to design this. Should all the hard disk be in the
> master node, or they can be on each of the node, or should it be a  
> NAS?
>   * any recommendation for a mainboard (gigabit network, at least 4  
> RAM
> slots) about $200-$300 good for cluster?
>
> I would love to hear any advice/suggestion from you, especially if you
> had built a similar cluster with similar purpose.
>
> Thank you in advance,
>
> Duke.
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From diep at xs4all.nl  Mon Aug 20 00:17:39 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Mon, 20 Aug 2012 09:17:39 +0200
Subject: [Beowulf] advice for newbie
In-Reply-To: <CD2708F3-AF3D-4563-AA19-27BE6D70075D@xs4all.nl>
References: <5031C33A.3060304@gmx.com>
	<CD2708F3-AF3D-4563-AA19-27BE6D70075D@xs4all.nl>
Message-ID: <79D1A9BA-269B-48F3-B8E5-37ECA4B0B7FE@xs4all.nl>


On Aug 20, 2012, at 9:12 AM, Vincent Diepeveen wrote:

> Look at ebay, there is cheap machines there $150 or less for 8 core  
> Xeons L5420,
> i'm using those as well. For that $150 you get 8GB ram as well.
>
> Boot them over the network.
>
> That's 16 * $150 = $3k.
>
> Now there might be a difference between my and your wishes.
> For me the network is important so i bought pci-e 2.0 motherboards;
>
> In short i bought each component separate which is more expensive  
> than $150.
>
> And those rackmounts make huge noise; i'm doing it here with 14 CM  
> fans that's not so noisy,
> but i assume your institute has a spare room that may make people  
> who walk their deaf before
> they're 30.
>
> As for you, buy 8 drives SATA from 2 TB and a $30 raid card from  
> ebay Those raid cards 2nd hand
> are dirt cheap especially the pci-x ones and with 8 drives you  
> won't get a bigger bandwidth
> anyway and most those online nodes have that.

online nodes => those $150k nodes have an empty pci-x slot

Note pci-x is a lot slower than pci-e, yet for file server with  
limited amount of drives you won't
get more than 700 MB/s out of it anyway and pci-x easily delivers that.

>
> They're 100 euro a piece here those drives. So that's 800 euro =  
> $1k or so. And $30 from
> ebay for a total superior raid card. Put them in a ready rackmount  
> that allows such drives,
> it's a couple of hundreds of dollars on ebay, with the same L5420's  
> and motherboards and 8GB
> ram. So say a $500 for that rackmount with drives you can plug in.
>
> Put in the drives, build a raid6 partition and your fileserver ,  
> the 17th machine, it's ready to serve you
> at around a 700MB/s readspeed.
>
> Now i don't know the latest about genome research; the last Phd  
> student i helped out
> there, his university used 90s software to do his research.
>
> That really required big crunching for months for each calculation,  
> at hundreds of cpu's,
> yet new commercial software finished within 15 minutes each  
> calculation at a single core.
>
> That 90s software uses MPI if i recall but that'll depend upon what  
> sort of software your
> guys want to use.
>
> You might want to decide next to buy the cheapest gigabit switch  
> you can get, in order to
> boot all the nodes over the network using pci-e.

correction : pxe

>
> It's possible those motherboards won't boot over infiniband, some  
> might.
>
> Then i'd really advice you buy a cheap 2nd hand switch infiniband,  
> maybe DDR, of $300 or so.
> Cables $20 a piece times 17 = $140, and a bunch of 2nd hand DDR  
> infiniband cards and put
> each one in each machine.
>
> So after boot over the gigabit switch, assuming the motherboards  
> don't boot over infiniband,
> they might boot actually over infiniband in which case you don't  
> need the gigabit switch,
> then in that case infiniband will take over and can serve you  
> either as 10 gigabit network
> cards or for the MPI that much software in that area needs.
>
> So all what's left is buy a 17 infiniband cards DDR 2nd hand off  
> ebay. Not sure about prices,
> maybe a $80 maybe $100. a piece. Let's say it's under $1600
>
> Now you're done with a 16 node cluster with 17 machines from which  
> 1 is fileserver,
> for a part of the budget you had in mind.
>
> Just it's noisy and loud.
>
> Also it's pretty low power compared to alternatives. It'll eat a  
> 180 watt a node or so under full load. It's 170 watt a node
> here under full load (but that's with a much better psu).
>
> As for software to install in case you decide for infiniband, your  
> choices are limited as OpenFED doesn't give
> much alternatives.
>
> Fedora Core or Scientific Linux are for free and probably your only  
> 2 options if you want to use free software
> that are easy to get things done as you want to.
>
> Then install OpenFED that has the openmpi and other infiniband  
> stuff for free.
>
> Probably Debian works as well, provided you use the exact kernel  
> that OpenFED recommends.
> Any other kernel won't work. So you have to download some older  
> debian then, get the exact kernel recommended
> and then i guess OpenFED will install as well.
>
> Good Luck,
> Vincent
>
> On Aug 20, 2012, at 6:55 AM, Duke Nguyen wrote:
>
>> Hi folks,
>>
>> First let me start that I am total novice with cluster and/or  
>> beowulf. I
>> am familiar with unix/linux and have a few years working in a cluster
>> (HPC) environment, but I never have a chance to design and admin a  
>> cluster.
>>
>> Now my new institute decides to build a (small) cluster for our next
>> research focus area: genome research. The requirements are simple:
>> expandable and capable of doing genome research. The budget is low,
>> about $15,000, and we have decided:
>>
>>   * cluster is a box cluster, not rack (well, mainly because our  
>> funding
>> is low)
>>   * cluster OS is scientific linux with OpenMPI
>>   * cluster is about 16-node with a master node, expandable is a must
>>
>> Now next step for us is to decide hardwares and other aspects:
>>
>>   * any recommendation for a reliable 24-port gigabit switch for the
>> cluster? I heard of HP ProCurve 2824 but it is a little bit hard  
>> to find
>> it in my country
>>   * should our boxes be diskless or should they have a hard disk  
>> inside?
>> I am still not very clear the advantages if the clients has about  
>> 80GB
>> hard disk internally except that their OS are independent and does  
>> not
>> depend on the master node, and maybe faster data processing  
>> (temporay),
>> but 80GB each for genome research is too small
>>   * hard drives/data storage: we want to have storage of about  
>> 10TB but
>> I am not sure how to design this. Should all the hard disk be in the
>> master node, or they can be on each of the node, or should it be a  
>> NAS?
>>   * any recommendation for a mainboard (gigabit network, at least  
>> 4 RAM
>> slots) about $200-$300 good for cluster?
>>
>> I would love to hear any advice/suggestion from you, especially if  
>> you
>> had built a similar cluster with similar purpose.
>>
>> Thank you in advance,
>>
>> Duke.
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit  
>> http://www.beowulf.org/mailman/listinfo/beowulf
>



From samuel at unimelb.edu.au  Mon Aug 20 20:41:47 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Tue, 21 Aug 2012 13:41:47 +1000
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <CAEiui-vR8txhsGKmb+3No30mh7DLRZoon0XTctUPzunF0ZECAA@mail.gmail.com>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
	<CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
	<E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>
	<CAEiui-uBb_rpYiWJS55FF+M2-YYVAuG361k4SH=GqLtCPVR_9Q@mail.gmail.com>
	<476FB5BC-84A7-4298-A311-F327FEBB1495@xs4all.nl>
	<CAEiui-vR8txhsGKmb+3No30mh7DLRZoon0XTctUPzunF0ZECAA@mail.gmail.com>
Message-ID: <5033037B.70804@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 19/08/12 03:17, Andrew Holway wrote:

> d) I think the only tool that does what your describing, gluster, 
> really sucks because it is still FUSE.

FUSE doesn't necessarily mean slow, I benchmarked NTFS-3G (FUSE)
against XFS years ago and was surprised to find it matched XFS for
read & write performance.

> e) ZFS is not yet on linux.

http://zfsonlinux.org/

It's what LLNL are going to use (using?) for their Lustre OSD's on
Sequoia.  Packaged for Ubuntu via a PPA too.

cheers,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAzA3sACgkQO2KABBYQAh8nbwCffDQPUeEqoXcLOCKf+yNFNpHG
hOIAnAia9gr0z8r530vmsZwrFMSloUaH
=Tv15
-----END PGP SIGNATURE-----


From landman at scalableinformatics.com  Mon Aug 20 21:45:25 2012
From: landman at scalableinformatics.com (Joe Landman)
Date: Tue, 21 Aug 2012 00:45:25 -0400
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <5033037B.70804@unimelb.edu.au>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
	<CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
	<E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>
	<CAEiui-uBb_rpYiWJS55FF+M2-YYVAuG361k4SH=GqLtCPVR_9Q@mail.gmail.com>
	<476FB5BC-84A7-4298-A311-F327FEBB1495@xs4all.nl>
	<CAEiui-vR8txhsGKmb+3No30mh7DLRZoon0XTctUPzunF0ZECAA@mail.gmail.com>
	<5033037B.70804@unimelb.edu.au>
Message-ID: <50331265.5090205@scalableinformatics.com>

On 08/20/2012 11:41 PM, Christopher Samuel wrote:

> Sequoia.  Packaged for Ubuntu via a PPA too.

Whoa ...  really?

We've stayed far away from it due to the incompatible licensing ...  you 
can't legally ship binaries.  Is Ubuntu shipping a build environment 
rather than binaries?

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615



From samuel at unimelb.edu.au  Mon Aug 20 22:05:50 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Tue, 21 Aug 2012 15:05:50 +1000
Subject: [Beowulf] Doing i/o at a small cluster
In-Reply-To: <50331265.5090205@scalableinformatics.com>
References: <655109DC-2BC0-40CE-BF6F-2C0743E81810@xs4all.nl>
	<502E584F.9080908@cse.psu.edu>
	<1D99AFA3-BD03-4010-A227-073CACD5D850@xs4all.nl>
	<CAEiui-vJQLDFY4wkbjGG8Xzf_Bsyp88ub5aDmMbS8iv6qtdBig@mail.gmail.com>
	<E42AB19C-E4BD-4D50-9549-999C08290C7D@xs4all.nl>
	<CAEiui-uBb_rpYiWJS55FF+M2-YYVAuG361k4SH=GqLtCPVR_9Q@mail.gmail.com>
	<476FB5BC-84A7-4298-A311-F327FEBB1495@xs4all.nl>
	<CAEiui-vR8txhsGKmb+3No30mh7DLRZoon0XTctUPzunF0ZECAA@mail.gmail.com>
	<5033037B.70804@unimelb.edu.au>
	<50331265.5090205@scalableinformatics.com>
Message-ID: <5033172E.2030908@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 21/08/12 14:45, Joe Landman wrote:

> On 08/20/2012 11:41 PM, Christopher Samuel wrote:
> 
>>> Sequoia.  Packaged for Ubuntu via a PPA too.
> 
> Whoa ...  really?

https://launchpad.net/~zfs-native  :-)

> We've stayed far away from it due to the incompatible licensing ...
> you can't legally ship binaries.  Is Ubuntu shipping a build
> environment rather than binaries?

The ZFS on Linux team (not Canonical/Ubuntu) uses DKMS to build the
kernel modules, and separately package up the utilities.

I've used it for a bit and it was fine (though memory hungry of course).

cheers,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAzFy4ACgkQO2KABBYQAh+BvgCbBJmlt6IqYMJl6Ijt6u9SW6JV
qygAn38j/GG3xsTDMJUeQgCuN5Y/sEbM
=REdA
-----END PGP SIGNATURE-----


From james.p.lux at jpl.nasa.gov  Tue Aug 21 11:27:38 2012
From: james.p.lux at jpl.nasa.gov (Lux, Jim (337C))
Date: Tue, 21 Aug 2012 18:27:38 +0000
Subject: [Beowulf] packaged build environments
Message-ID: <CC592128.242B7%james.p.lux@jpl.nasa.gov>

A recent post brought up a question?

Are there quasi standarized ways to distribute the build system.  I've got a bunch of applications that use the gcc toolchain (targeting SPARC, as it happens), and we'd like a clean way to distribute the toolchain, in parallel with the sources. This instead of sending them a list of the half a dozen places to get all the pieces, and hoping that they get all the pieces with the correct versions. Then, they have to go through a raft of make-ing to build the host tools, then the target tools.  Yes, we have a script for it all, but if someone else has solved it in a more generalized fashion, I'd love to adopt it.

Jim Lux


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120821/bf87635a/attachment.html>

From diep at xs4all.nl  Wed Aug 22 11:27:11 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Wed, 22 Aug 2012 20:27:11 +0200
Subject: [Beowulf] advice for newbie
In-Reply-To: <5035076D.90008@gmx.com>
References: <5031C33A.3060304@gmx.com>
	<CD2708F3-AF3D-4563-AA19-27BE6D70075D@xs4all.nl>
	<79D1A9BA-269B-48F3-B8E5-37ECA4B0B7FE@xs4all.nl>
	<5035076D.90008@gmx.com>
Message-ID: <CD22313F-889A-46F7-9776-ABB5F08E8A7E@xs4all.nl>


On Aug 22, 2012, at 6:23 PM, Duke Nguyen wrote:

> Hi Vincent,
>
> Thanks so much for your detail messages with lots of suggestions. I  
> will try to understand/catch up with what you said.
>
> On 8/20/12 2:17 PM, Vincent Diepeveen wrote:
>> On Aug 20, 2012, at 9:12 AM, Vincent Diepeveen wrote:
>>
>>> Look at ebay, there is cheap machines there $150 or less for 8 core
>>> Xeons L5420,
>>> i'm using those as well. For that $150 you get 8GB ram as well.
>>>
>>> Boot them over the network.
>>>
>>> That's 16 * $150 = $3k.
>>>
>>> Now there might be a difference between my and your wishes.
>>> For me the network is important so i bought pci-e 2.0 motherboards;
>>>
>>> In short i bought each component separate which is more expensive
>>> than $150.
>
> Not sure if our wishes are different, but our plan was to buy  
> components separately as well (and ensembl them as boxes - which we  
> thought should be cheaper with the same-configuration boxes which  
> are selling out there). Based on my own experience on genome  
> research, network is very important too, so we want to have the  
> best network capable within our budget.




>
>>>
>>> And those rackmounts make huge noise; i'm doing it here with 14 CM
>>> fans that's not so noisy,
>>> but i assume your institute has a spare room that may make people
>>> who walk their deaf before
>>> they're 30.
>
> We do have a dedicated room for the cluster.
>

That's a huge difference with me. HUGE.

The only 2 disadvantages of the ebay offered rackmounts is that they  
are LOUD and not pci-e 2.0.

Ready assembled rackmounts really are cheaper in this case except if  
you intend to build a cluster of hundreds of nodes.
These guys selling on ebay they sell these rackmounts by the  
thousands. You can't build it cheaper than them just buying
in components. You'll lose it bigtime on transport costs and  
component prices.

>>>
>>> As for you, buy 8 drives SATA from 2 TB and a $30 raid card from
>>> ebay Those raid cards 2nd hand
>>> are dirt cheap especially the pci-x ones and with 8 drives you
>>> won't get a bigger bandwidth
>>> anyway and most those online nodes have that.
>> online nodes => those $150k nodes have an empty pci-x slot
>>
>> Note pci-x is a lot slower than pci-e, yet for file server with
>> limited amount of drives you won't
>> get more than 700 MB/s out of it anyway and pci-x easily delivers  
>> that.
>>
>>> They're 100 euro a piece here those drives. So that's 800 euro =
>>> $1k or so. And $30 from
>>> ebay for a total superior raid card. Put them in a ready rackmount
>>> that allows such drives,
>>> it's a couple of hundreds of dollars on ebay, with the same L5420's
>>> and motherboards and 8GB
>>> ram. So say a $500 for that rackmount with drives you can plug in.
>>>
>>> Put in the drives, build a raid6 partition and your fileserver ,
>>> the 17th machine, it's ready to serve you
>>> at around a 700MB/s readspeed.
>
> My understand this point is that we buy a file server with 8 or  
> more slots for hard drives with the same motherboard and 8 GB RAM.  
> What I dont understand is that how we can get a speed of 700MB/s  
> for this file server?

the raidcard delivers that speed. You also need a $30 second hand  
raidcard that you put in this machine.

It's true that if you would order a pci-e card doing the same, a new  
one, that this one is way faster. Yet that would require
more drives and therefore also a raidcard that can handle more than 8  
drives.

Now i don't know much yet about doing i/o over infiniband, but the  
idea is that you will use the infiniband network for i/o.
It has a bandwidth that's much larger than the gigabit ethernet  
that's built in.

So the agreggated speed you  can expect is 700MB/s.

If you have, like me a bigger hunger for such i/o speed, then feel  
free to say so, but it'll be involving giving each node its own  
scratch that is real fast.

So as the switch you can buy on ebay easily will have 24 ports, you  
can build a 24 node cluster easily as you have a separated room.

You will need enough ventilation of course - but i'm sure you know  
more there than i do :)

>
>>>
>>> Now i don't know the latest about genome research; the last Phd
>>> student i helped out
>>> there, his university used 90s software to do his research.
>>>
>>> That really required big crunching for months for each calculation,
>>> at hundreds of cpu's,
>>> yet new commercial software finished within 15 minutes each
>>> calculation at a single core.
>>>
>>> That 90s software uses MPI if i recall but that'll depend upon what
>>> sort of software your
>>> guys want to use.
>>>
>>> You might want to decide next to buy the cheapest gigabit switch
>>> you can get, in order to
>>> boot all the nodes over the network using pci-e.
>> correction : pxe
>>
>>> It's possible those motherboards won't boot over infiniband, some
>>> might.
>>>
>>> Then i'd really advice you buy a cheap 2nd hand switch infiniband,
>>> maybe DDR, of $300 or so.
>>> Cables $20 a piece times 17 = $140, and a bunch of 2nd hand DDR
>>> infiniband cards and put
>>> each one in each machine.
>>>
>>> So after boot over the gigabit switch, assuming the motherboards
>>> don't boot over infiniband,
>>> they might boot actually over infiniband in which case you don't
>>> need the gigabit switch,
>>> then in that case infiniband will take over and can serve you
>>> either as 10 gigabit network
>>> cards or for the MPI that much software in that area needs.
>>>
>>> So all what's left is buy a 17 infiniband cards DDR 2nd hand off
>>> ebay. Not sure about prices,
>>> maybe a $80 maybe $100. a piece. Let's say it's under $1600
>>>
>>> Now you're done with a 16 node cluster with 17 machines from which
>>> 1 is fileserver,
>>> for a part of the budget you had in mind.
>>>
>>> Just it's noisy and loud.
>
> I might have lost this point, so I will try to wrap it up:
>
>  * 16 nodes: pci-e motherboard, 2x 4 core xeons L5420, infiniband  
> DDR card, no hard drives, about 4-8GB RAM
>  * file server: rackmount with 8 hard SATA drives, raid card, 8GB  
> RAM, pci-e motherboard, 2x 4 core xeons L5420
>  * infiniband switch
>
> We will try to see if we can afford new hard wares first (prices  
> from ebay):
>
> pci-e motherboard: ~ 17x 100 = 1700
> 2x 4core L5420 ~ 17x 200 = 3400
> 8G DDR3 ~ 17x 100 = 1700
> infiniband card ~ 17x 100 = 1700
> infiniband cables ~ 17x 20 = 140
> 8 SATA ~ 1000
> RAID card ~ 50
> file server ~ 500?
> infiniband switch ~ 500?
> a server rack (or PC shelf) ~ ?
> 5-6kW PSU ~ ?
>
> So that will be around $11k (not including the shelf/rack and the  
> PSU). It looks like we can afford this system. Am I missing  
> anything else? Are there those above components for box PC? I did  
> some quick search on ebay and they are all seem to be for rack- 
> mount servers.
>
>>>
>>> Also it's pretty low power compared to alternatives. It'll eat a
>>> 180 watt a node or so under full load. It's 170 watt a node
>>> here under full load (but that's with a much better psu).
>>>
>>> As for software to install in case you decide for infiniband, your
>>> choices are limited as OpenFED doesn't give
>>> much alternatives.
>>>
>>> Fedora Core or Scientific Linux are for free and probably your only
>>> 2 options if you want to use free software
>>> that are easy to get things done as you want to.
>>>
>>> Then install OpenFED that has the openmpi and other infiniband
>>> stuff for free.
>>>
>>> Probably Debian works as well, provided you use the exact kernel
>>> that OpenFED recommends.
>>> Any other kernel won't work. So you have to download some older
>>> debian then, get the exact kernel recommended
>>> and then i guess OpenFED will install as well.
>
> Thanks for this suggestion about softwares. I think we will go with  
> SL6. I actually tried a 3-node cluster (2 clients and 1 master)  
> with SL6 and OpenMPI, and it works fine. For the infiniband cards,  
> I have zero experience but I assume it is not too hard to install/ 
> configure?
>
> Thanks,
>
> D.
>
>>>
>>> Good Luck,
>>> Vincent
>>>
>>> On Aug 20, 2012, at 6:55 AM, Duke Nguyen wrote:
>>>
>>>> Hi folks,
>>>>
>>>> First let me start that I am total novice with cluster and/or
>>>> beowulf. I
>>>> am familiar with unix/linux and have a few years working in a  
>>>> cluster
>>>> (HPC) environment, but I never have a chance to design and admin a
>>>> cluster.
>>>>
>>>> Now my new institute decides to build a (small) cluster for our  
>>>> next
>>>> research focus area: genome research. The requirements are simple:
>>>> expandable and capable of doing genome research. The budget is low,
>>>> about $15,000, and we have decided:
>>>>
>>>>    * cluster is a box cluster, not rack (well, mainly because our
>>>> funding
>>>> is low)
>>>>    * cluster OS is scientific linux with OpenMPI
>>>>    * cluster is about 16-node with a master node, expandable is  
>>>> a must
>>>>
>>>> Now next step for us is to decide hardwares and other aspects:
>>>>
>>>>    * any recommendation for a reliable 24-port gigabit switch  
>>>> for the
>>>> cluster? I heard of HP ProCurve 2824 but it is a little bit hard
>>>> to find
>>>> it in my country
>>>>    * should our boxes be diskless or should they have a hard disk
>>>> inside?
>>>> I am still not very clear the advantages if the clients has about
>>>> 80GB
>>>> hard disk internally except that their OS are independent and does
>>>> not
>>>> depend on the master node, and maybe faster data processing
>>>> (temporay),
>>>> but 80GB each for genome research is too small
>>>>    * hard drives/data storage: we want to have storage of about
>>>> 10TB but
>>>> I am not sure how to design this. Should all the hard disk be in  
>>>> the
>>>> master node, or they can be on each of the node, or should it be a
>>>> NAS?
>>>>    * any recommendation for a mainboard (gigabit network, at least
>>>> 4 RAM
>>>> slots) about $200-$300 good for cluster?
>>>>
>>>> I would love to hear any advice/suggestion from you, especially if
>>>> you
>>>> had built a similar cluster with similar purpose.
>>>>
>>>> Thank you in advance,
>>>>
>>>> Duke.
>>>>
>>>> _______________________________________________
>>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>>>> Computing
>>>> To change your subscription (digest mode or unsubscribe) visit
>>>> http://www.beowulf.org/mailman/listinfo/beowulf
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit  
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>



From diep at xs4all.nl  Wed Aug 22 12:42:40 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Wed, 22 Aug 2012 21:42:40 +0200
Subject: [Beowulf] advice for newbie - component list
In-Reply-To: <5035076D.90008@gmx.com>
References: <5031C33A.3060304@gmx.com>
	<CD2708F3-AF3D-4563-AA19-27BE6D70075D@xs4all.nl>
	<79D1A9BA-269B-48F3-B8E5-37ECA4B0B7FE@xs4all.nl>
	<5035076D.90008@gmx.com>
Message-ID: <DB54721E-D5ED-408A-A754-8BF087BD2115@xs4all.nl>

>
> I might have lost this point, so I will try to wrap it up:
>
>  * 16 nodes: pci-e motherboard, 2x 4 core xeons L5420, infiniband  
> DDR card, no hard drives, about 4-8GB RAM

i'd go with 8GB a node if i were you. 1GB/core is pretty much  
standard for 10+ years in HPC now.

>  * file server: rackmount with 8 hard SATA drives, raid card, 8GB  
> RAM, pci-e motherboard, 2x 4 core xeons L5420
>  * infiniband switch
>

All your nodes need pci-e for the DDR infiniband. Default most those  
motherboards have pci-x slot as well.
Note that pci-x is not pci-e, pci-x is older and therefore dirt cheap.

New pci-e raid card is a lot more expensive. hundreds of dollars.
This where it won't deliver you more bandwidth at 8 drives, only when  
using more than 8 drives and i doubt any node
could handle such bandwidth.

Reality of bandwidth is that you never want to stress in HPC things  
to the limit. Everything usually ughs out then.

My advice always is to really limit total bandwidth to 10-20% of the  
maximum the network in theory is capable of handling.

> We will try to see if we can afford new hard wares first (prices  
> from ebay):
>
> pci-e motherboard: ~ 17x 100 = 1700

you get a certificate in gold?

$100 for motherboard is way way too expensive. They're for $50-$60  
there on ebay.
Yet better is buy a barebone or ready rackmount.

> 2x 4core L5420 ~ 17x 200 = 3400

http://www.ebay.com/itm/Intel-Xeon-L5420-2-5GHz-12M-1333MHz-LGA-771- 
SLARP-OEM-/140817053377?pt=CPUs&hash=item20c959b6c1

They're $19.90 a piece on ebay a piece.

> 8G DDR3 ~ 17x 100 = 1700

they don't eat DDR3 ram. they eat FBDIMMS About $6 a gigabyte if you  
put in larger types ram. Say $50 for 8 GB in total.
Not everyday such offers there.

> infiniband card ~ 17x 100 = 1700

Yeah around a $60-$70 depends what card you get. The cards that can  
do pci-e 2.0 are useless anyway except if you get motherboards
with a seaburg chipset.

> infiniband cables ~ 17x 20 = 140
> 8 SATA ~ 1000



> RAID card ~ 50
> file server ~ 500?

At least 500.


> infiniband switch ~ 500?

saw them $300 on ebay last time i checked but that's a while ago. It  
always depends upon how many of such
specialized hardware is on ebay. If it's several suddenly it's dirt  
cheap and $100 otherwise it's $1k or $2k or more...

> a server rack (or PC shelf) ~ ?

A case for each node and 2 passive heatsinks for each machine might  
get in handy.

The ready rackmounts come with a PSU.

> 5-6kW PSU ~ ?

each machine has its own PSU.

A 16 node network with everything except the airconditioner to cool  
the room,
that's gonna eat you a 3 kilowatt roughly.

You really want to order ready rackmounts.

Here is example of a barebone if you want to be active yourself:

http://www.ebay.com/itm/1U-Server-Intel-S5000PSL-Barebone-2x-LGA771- 
CPU-DDR2-RAM-2-5-SATA-Hard-Dirves-/140831053892? 
pt=COMP_EN_Servers&hash=item20ca2f5844

And it has heatsinks and a psu. Note if you order there you probably  
want ready rackmounts anyway as you pay the transport costs already
gonna be $500 at least transport.

If you want to save shelf space maybe get this:

http://www.ebay.com/itm/Supermicro-SC809T-980B-1U-2-Node-Twin-Server- 
X7DWT-INF-Barebone-LGA771-Socket-/140826089474? 
pt=COMP_EN_Servers&hash=item20c9e39802]

that's 2 motherboards in 2 machine. It needs 4 cpu's and 2 x 8 GB ram  
(so 8 dimms i'd guess) and then you have 2 machines in 1 rackmount.

I didn't check what comes with it. You want 2 riser cards of course  
for the network.

Ask the guy for cpu's inside it and RAM and a small harddrive and 2  
riser cards as local scratch and you're probably done for the  
calculation nodes.
Just need then network gear.

Note that E53** ready rackmounts are a LOT cheaper on ebay. Under  
full load doesn't at that much power btw,
intel writes big novels always on that. If you have a special room  
burning a tad more energy is usually not a major problem...

Note there is much cheaper offers if you want things dirt cheap. I  
just picked the first link i saw on ebay, that's true for everything.
So no links so far were gotten by me on purpose.

This X7DWT from supermicro is possibly a seaburg chipset. that means  
that it can do pci-e 2.0
pci-e 2.0 is 8 GB/s (i'm sure someone corrects it if i write it  
wrong) versus pci-e 1.0 is at x8 doing half that speed.

Now i hope you don't need that bandwidth full speed of course...

>
> So that will be around $11k (not including the shelf/rack and the  
> PSU). It looks like we can afford this system. Am I missing  
> anything else? Are there those above components for box PC? I did  
> some quick search on ebay and they are all seem to be for rack- 
> mount servers.
>
>>>
>>> Also it's pretty low power compared to alternatives. It'll eat a
>>> 180 watt a node or so under full load. It's 170 watt a node
>>> here under full load (but that's with a much better psu).
>>>
>>> As for software to install in case you decide for infiniband, your
>>> choices are limited as OpenFED doesn't give
>>> much alternatives.
>>>
>>> Fedora Core or Scientific Linux are for free and probably your only
>>> 2 options if you want to use free software
>>> that are easy to get things done as you want to.
>>>
>>> Then install OpenFED that has the openmpi and other infiniband
>>> stuff for free.
>>>
>>> Probably Debian works as well, provided you use the exact kernel
>>> that OpenFED recommends.
>>> Any other kernel won't work. So you have to download some older
>>> debian then, get the exact kernel recommended
>>> and then i guess OpenFED will install as well.
>
> Thanks for this suggestion about softwares. I think we will go with  
> SL6. I actually tried a 3-node cluster (2 clients and 1 master)  
> with SL6 and OpenMPI, and it works fine. For the infiniband cards,  
> I have zero experience but I assume it is not too hard to install/ 
> configure?

They didn't make it real easy i feel as the manufacturers only have  
stuff at their website that works brilliant for
the expensive SLES and RHEL, yet with OpenFED you can get there and  
do things manual by typing a few commands.

There is manuals online.

OpenFED installed fine at default SL6.1 and SL6.2, what didn't work  
was latest GCC (i need at least 4.7.0)
I had to modify the OpenFED installation script for that.

For my programs gcc 4.7.0 at intel cpu's is a lot faster than 4.5  
series and before 4.5.3 (4.6.3 didn't check might be fast also).
for 64 bits programs 4.7.0 really is a lot faster than 4.5 and before.

You sure you have software that can use MPI by the way?
I suppose so as you refered to openmpi...

One should never order hardware without knowing you have software  
that can work for that specific hardware.

Realize how compatible all this infiniband gear is, it all works  
crisscross. Mellanox with others are cross compatible.
No wonder infiniband dominates right now in HPC.

Interesting i didn't see a cheap infiniband switch at ebay right now.  
That's probably because i was just looking for a Mellanox switch.
They are probably in high demand.

As for the network gear:

http://www.ebay.com/itm/Mellanox-Voltaire-Grid-Director-2036-36- 
DDR-20Gbps-ports-1U-Switch-/170889969337? 
pt=LH_DefaultDomain_0&hash=item27c9d5feb9

Auch $1150, switches sometimes are expensive...

My guess is it'll eat a 200 watt or so, max is 300 watt i guess  
(blindfolded guess based upon the switch that stands here).
When it works default here and idles, it's eating just over a 100  
watt by the way.

I'd go for that switch though and one by one collect the network  
cards off ebay or sit and wait until after summer as usual
some clusters get dismantled and equipment thrown to ebay.

No real cheap connect-X ddr cards this time on ebay, but you might  
get lucky, sometimes they're cheap there sometimes they're not.
There is also cheap topspin switches but i'm not sure about them.

Note this mellanox has 36 ports in case you want to order more nodes  
or directly do that.

Clusters tend to get useful if you really build bigger ones, as most  
software is not so every efficient. It loses big overhead.
Just a few nodes is same speed like a box with 48 cores. My cluster  
is on the brink of losing there in that sense (currently has
8 nodes but planned to increase to 16 when works well).

Ok seems the cheap network cards on ebay aren't connectX and that  
many of the cheap offers are gone.
Maybe you want to collect it network card by network card from ebay.

http://www.ebay.com/itm/HP-INFINIBAND-4X-DDR-PCIe-DUAL-PORT- 
HCA-409778-001-409376-B21-/130744094587? 
pt=COMP_EN_Servers&hash=item1e70f48f7b

This doesn't say it's connectX.
It is pci-e though.

Many of the other infiniband cards offered seem to be pci-x, don't  
get those except when bandwidth is not a big issue to you.

For example to me latency is important, more important than bandwidth  
(except for the EGTB crunching), so Connect-X then gets
recommended here on this list.

If bandwidth is not a big issue things change. Cheaper switch,  
cheaper network and cheaper cards, or maybe just a gigabit switch.

tons of cheap offers that were there a year ago on ebay, they seem  
all gone!

Maybe it's just bad luck and sit and wait a little until some of the  
big supercomputers get dismantled unleashing a flow of QDR network
cards to ebay - then the DDR ones will become dirt cheap i bet and  
everyone wants to get rid of them.

Usualy big new supercomputers take some years to prepare and then the  
big install happens especially in July when all scientists have a  
holiday
and no one runs on those machines. So some weeks from now the tide  
might change again...

So the question is to you: what sort of network do you need and what  
do you want?

>
> Thanks,
>
> D.
>
>>>
>>> Good Luck,
>>> Vincent
>>>
>>> On Aug 20, 2012, at 6:55 AM, Duke Nguyen wrote:
>>>
>>>> Hi folks,
>>>>
>>>> First let me start that I am total novice with cluster and/or
>>>> beowulf. I
>>>> am familiar with unix/linux and have a few years working in a  
>>>> cluster
>>>> (HPC) environment, but I never have a chance to design and admin a
>>>> cluster.
>>>>
>>>> Now my new institute decides to build a (small) cluster for our  
>>>> next
>>>> research focus area: genome research. The requirements are simple:
>>>> expandable and capable of doing genome research. The budget is low,
>>>> about $15,000, and we have decided:
>>>>
>>>>    * cluster is a box cluster, not rack (well, mainly because our
>>>> funding
>>>> is low)
>>>>    * cluster OS is scientific linux with OpenMPI
>>>>    * cluster is about 16-node with a master node, expandable is  
>>>> a must
>>>>
>>>> Now next step for us is to decide hardwares and other aspects:
>>>>
>>>>    * any recommendation for a reliable 24-port gigabit switch  
>>>> for the
>>>> cluster? I heard of HP ProCurve 2824 but it is a little bit hard
>>>> to find
>>>> it in my country
>>>>    * should our boxes be diskless or should they have a hard disk
>>>> inside?
>>>> I am still not very clear the advantages if the clients has about
>>>> 80GB
>>>> hard disk internally except that their OS are independent and does
>>>> not
>>>> depend on the master node, and maybe faster data processing
>>>> (temporay),
>>>> but 80GB each for genome research is too small
>>>>    * hard drives/data storage: we want to have storage of about
>>>> 10TB but
>>>> I am not sure how to design this. Should all the hard disk be in  
>>>> the
>>>> master node, or they can be on each of the node, or should it be a
>>>> NAS?
>>>>    * any recommendation for a mainboard (gigabit network, at least
>>>> 4 RAM
>>>> slots) about $200-$300 good for cluster?
>>>>
>>>> I would love to hear any advice/suggestion from you, especially if
>>>> you
>>>> had built a similar cluster with similar purpose.
>>>>
>>>> Thank you in advance,
>>>>
>>>> Duke.
>>>>
>>>> _______________________________________________
>>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>>>> Computing
>>>> To change your subscription (digest mode or unsubscribe) visit
>>>> http://www.beowulf.org/mailman/listinfo/beowulf
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit  
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>



From eagles051387 at gmail.com  Wed Aug 22 22:29:22 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Thu, 23 Aug 2012 07:29:22 +0200
Subject: [Beowulf] setting up mailman to work with spam assassin
In-Reply-To: <524FB151A20E8B4E997E1946E79494AEAFD168@MTMRH1420.atrema.deloitte.com>
References: <524FB151A20E8B4E997E1946E79494AEAFD168@MTMRH1420.atrema.deloitte.com>
Message-ID: <CAHdpx61oip91uRaaN0dHmYNLifhJPP0LDaozMp-Ntucz5_dz7w@mail.gmail.com>

This is what I found when googling which landed me at the mailman wiki.

Arend would you be able to set this up or has that task been handed over to
someone else?

---------- Forwarded message ----------
From: Aquilina, Jonathan (MT - Malta) <joaquilina at deloitte.com.mt>
Date: Tue, Aug 21, 2012 at 1:46 PM
Subject: setting up mailman to work with spam assassin
To: "eagles051387 at gmail.com" <eagles051387 at gmail.com>


  http://wiki.list.org/pages/viewpage.action?pageId=4030589****

*Disclaimer:*
The information contained in this message and in any attachment is
confidential and may be legally privileged. It is intended for the
recipient or recipients to whom it is addressed. If you are not an intended
recipient and receive this message in error please delete it and any
attachment and all copies of it and of any such attachment from your system
immediately, destroy any paper copies and notify the sender. If you are not
an intended recipient, you are not authorised to use, distribute, print or
copy any part of this message or of any attachment, whether directly or
indirectly. This message has been scanned for computer viruses, however we
do not accept any liability whatsoever in respect of any potential damage
caused by any virus which is not detected



-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120823/9d7ebf83/attachment.html>

From samuel at unimelb.edu.au  Thu Aug 23 18:06:06 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Fri, 24 Aug 2012 11:06:06 +1000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
Message-ID: <5036D37E.5030800@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

I'd like to solicit opinions from list members on changing the policy
of the list on messages from non-subscribers from the current default
of holding them for moderation to the position of rejecting them (with
an explanatory message).

The reason is that currently us new list admins get many dozens of
spams a day being held for moderation and it would be very hard to
spot a legitimate message (from either a new subscriber or from a new
member who is having messages held for moderation) amongst them.

It seems fairer to me for legitimate non-members to get an email
saying that their post has been rejected because they're not a member,
but pointing them to the web page to subscribe, rather than just a
message saying it's been held for moderation and then nothing (if it
gets missed).

It's also fairer to new members who are under moderation (as spammers
do sign up to lists too) as their messages won't get missed in the
general flood of non-member spam.

This is how all the other Mailman lists I run are configured and it's
not caused issues or complaints there.

Thoughts please!

cheers,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
=RDNl
-----END PGP SIGNATURE-----


From derekr42 at gmail.com  Thu Aug 23 18:20:00 2012
From: derekr42 at gmail.com (derekr42 at gmail.com)
Date: Fri, 24 Aug 2012 01:20:00 +0000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <1900879382-1345771245-cardhu_decombobulator_blackberry.rim.net-346624041-@b27.c5.bise6.blackberry>

Agreed. Your logic is sound.
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: Christopher Samuel <samuel at unimelb.edu.au>
Sender: beowulf-bounces at beowulf.org
Date: Fri, 24 Aug 2012 11:06:06 
To: Beowulf Mailing List<beowulf at beowulf.org>
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

I'd like to solicit opinions from list members on changing the policy
of the list on messages from non-subscribers from the current default
of holding them for moderation to the position of rejecting them (with
an explanatory message).

The reason is that currently us new list admins get many dozens of
spams a day being held for moderation and it would be very hard to
spot a legitimate message (from either a new subscriber or from a new
member who is having messages held for moderation) amongst them.

It seems fairer to me for legitimate non-members to get an email
saying that their post has been rejected because they're not a member,
but pointing them to the web page to subscribe, rather than just a
message saying it's been held for moderation and then nothing (if it
gets missed).

It's also fairer to new members who are under moderation (as spammers
do sign up to lists too) as their messages won't get missed in the
general flood of non-member spam.

This is how all the other Mailman lists I run are configured and it's
not caused issues or complaints there.

Thoughts please!

cheers,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
=RDNl
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From eagles051387 at gmail.com  Thu Aug 23 21:52:41 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 24 Aug 2012 06:52:41 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <CAHdpx62758aEVbr4Hw0mD2svM9uFZsYQ5tonMiGjQOcNFV_bMg@mail.gmail.com>

I dont know if you have noticed though it seems quite easy to do while we
hold them for moderation. There is the ability currently to where we add
them to the list and they will be automatically discarded or rejected
depending on the list they are added to. I dont know if that would be a
good idea to start using or not. It would be interesting to get some feed
back on that if its a good idea to start using or not.

On Fri, Aug 24, 2012 at 3:06 AM, Christopher Samuel
<samuel at unimelb.edu.au>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> I'd like to solicit opinions from list members on changing the policy
> of the list on messages from non-subscribers from the current default
> of holding them for moderation to the position of rejecting them (with
> an explanatory message).
>
> The reason is that currently us new list admins get many dozens of
> spams a day being held for moderation and it would be very hard to
> spot a legitimate message (from either a new subscriber or from a new
> member who is having messages held for moderation) amongst them.
>
> It seems fairer to me for legitimate non-members to get an email
> saying that their post has been rejected because they're not a member,
> but pointing them to the web page to subscribe, rather than just a
> message saying it's been held for moderation and then nothing (if it
> gets missed).
>
> It's also fairer to new members who are under moderation (as spammers
> do sign up to lists too) as their messages won't get missed in the
> general flood of non-member spam.
>
> This is how all the other Mailman lists I run are configured and it's
> not caused issues or complaints there.
>
> Thoughts please!
>
> cheers,
> Chris
> - --
>  Christopher Samuel        Senior Systems Administrator
>  VLSCI - Victorian Life Sciences Computation Initiative
>  Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>  http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> =RDNl
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120824/0fc88ebe/attachment.html>

From samuel at unimelb.edu.au  Thu Aug 23 21:59:48 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Fri, 24 Aug 2012 14:59:48 +1000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <CAHdpx62758aEVbr4Hw0mD2svM9uFZsYQ5tonMiGjQOcNFV_bMg@mail.gmail.com>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAHdpx62758aEVbr4Hw0mD2svM9uFZsYQ5tonMiGjQOcNFV_bMg@mail.gmail.com>
Message-ID: <50370A44.8000009@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24/08/12 14:52, Jonathan Aquilina wrote:

> I dont know if you have noticed though it seems quite easy to do
> while we hold them for moderation. There is the ability currently
> to where we add them to the list and they will be automatically
> discarded or rejected depending on the list they are added to.

The problem is that they're pretty much all from unique addresses, so
you'll never catch that much that way.

cheers!
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlA3CkQACgkQO2KABBYQAh9DwACfRuZXfe6xBeRQUKAge08NsIpj
bfwAnAz54hh1UALToPoCYEGJ956UAgY0
=0NE5
-----END PGP SIGNATURE-----


From eagles051387 at gmail.com  Thu Aug 23 22:05:57 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 24 Aug 2012 07:05:57 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <50370A44.8000009@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAHdpx62758aEVbr4Hw0mD2svM9uFZsYQ5tonMiGjQOcNFV_bMg@mail.gmail.com>
	<50370A44.8000009@unimelb.edu.au>
Message-ID: <CAHdpx60-Gs-6DiqT1KNxphfXnsN5szXsKUK8QNadiNKWbC0O1A@mail.gmail.com>

I did forward a how to for integration of spamassassin with mailman just
not sure who to send it to so I sent it to the entire list.

On Fri, Aug 24, 2012 at 6:59 AM, Christopher Samuel
<samuel at unimelb.edu.au>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 24/08/12 14:52, Jonathan Aquilina wrote:
>
> > I dont know if you have noticed though it seems quite easy to do
> > while we hold them for moderation. There is the ability currently
> > to where we add them to the list and they will be automatically
> > discarded or rejected depending on the list they are added to.
>
> The problem is that they're pretty much all from unique addresses, so
> you'll never catch that much that way.
>
> cheers!
> Chris
> - --
>  Christopher Samuel        Senior Systems Administrator
>  VLSCI - Victorian Life Sciences Computation Initiative
>  Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>  http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlA3CkQACgkQO2KABBYQAh9DwACfRuZXfe6xBeRQUKAge08NsIpj
> bfwAnAz54hh1UALToPoCYEGJ956UAgY0
> =0NE5
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120824/c3191c48/attachment.html>

From greg.matthews at diamond.ac.uk  Fri Aug 24 03:26:11 2012
From: greg.matthews at diamond.ac.uk (Gregory Matthews)
Date: Fri, 24 Aug 2012 11:26:11 +0100
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <503756C3.9000907@diamond.ac.uk>

this is not a good idea - your list can become a spam mirror and 
replying to spammers is bad in lots of ways. Get a good spam filter, 
MailScanner + SpamAssassin + selected blacklists + selected 
sendmail/postfix plugins.

GREG

On 24/08/12 02:06, Christopher Samuel wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> I'd like to solicit opinions from list members on changing the policy
> of the list on messages from non-subscribers from the current default
> of holding them for moderation to the position of rejecting them (with
> an explanatory message).
>
> The reason is that currently us new list admins get many dozens of
> spams a day being held for moderation and it would be very hard to
> spot a legitimate message (from either a new subscriber or from a new
> member who is having messages held for moderation) amongst them.
>
> It seems fairer to me for legitimate non-members to get an email
> saying that their post has been rejected because they're not a member,
> but pointing them to the web page to subscribe, rather than just a
> message saying it's been held for moderation and then nothing (if it
> gets missed).
>
> It's also fairer to new members who are under moderation (as spammers
> do sign up to lists too) as their messages won't get missed in the
> general flood of non-member spam.
>
> This is how all the other Mailman lists I run are configured and it's
> not caused issues or complaints there.
>
> Thoughts please!
>
> cheers,
> Chris
> - --
>   Christopher Samuel        Senior Systems Administrator
>   VLSCI - Victorian Life Sciences Computation Initiative
>   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>   http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> =RDNl
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>


-- 
Greg Matthews        01235 778658
Scientific Computing Group Leader
Diamond Light Source Ltd. OXON UK

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 





From eagles051387 at gmail.com  Fri Aug 24 03:39:59 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 24 Aug 2012 12:39:59 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <503756C3.9000907@diamond.ac.uk>
References: <5036D37E.5030800@unimelb.edu.au> <503756C3.9000907@diamond.ac.uk>
Message-ID: <CAHdpx63wmZyqY0rR_zTMHYyKpFTCqFTsYca+NP1CQUqatwFy_Q@mail.gmail.com>

Correct me if im wrong Arend doesnt mailman already have a black list
mechanism in a way?

On Fri, Aug 24, 2012 at 12:26 PM, Gregory Matthews <
greg.matthews at diamond.ac.uk> wrote:

> this is not a good idea - your list can become a spam mirror and
> replying to spammers is bad in lots of ways. Get a good spam filter,
> MailScanner + SpamAssassin + selected blacklists + selected
> sendmail/postfix plugins.
>
> GREG
>
> On 24/08/12 02:06, Christopher Samuel wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Hi all,
> >
> > I'd like to solicit opinions from list members on changing the policy
> > of the list on messages from non-subscribers from the current default
> > of holding them for moderation to the position of rejecting them (with
> > an explanatory message).
> >
> > The reason is that currently us new list admins get many dozens of
> > spams a day being held for moderation and it would be very hard to
> > spot a legitimate message (from either a new subscriber or from a new
> > member who is having messages held for moderation) amongst them.
> >
> > It seems fairer to me for legitimate non-members to get an email
> > saying that their post has been rejected because they're not a member,
> > but pointing them to the web page to subscribe, rather than just a
> > message saying it's been held for moderation and then nothing (if it
> > gets missed).
> >
> > It's also fairer to new members who are under moderation (as spammers
> > do sign up to lists too) as their messages won't get missed in the
> > general flood of non-member spam.
> >
> > This is how all the other Mailman lists I run are configured and it's
> > not caused issues or complaints there.
> >
> > Thoughts please!
> >
> > cheers,
> > Chris
> > - --
> >   Christopher Samuel        Senior Systems Administrator
> >   VLSCI - Victorian Life Sciences Computation Initiative
> >   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
> >   http://www.vlsci.org.au/      http://twitter.com/vlsci
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.11 (GNU/Linux)
> > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> >
> > iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> > kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> > =RDNl
> > -----END PGP SIGNATURE-----
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> >
>
>
> --
> Greg Matthews        01235 778658
> Scientific Computing Group Leader
> Diamond Light Source Ltd. OXON UK
>
> --
> This e-mail and any attachments may contain confidential, copyright and or
> privileged material, and are for the use of the intended addressee only. If
> you are not the intended addressee or an authorised recipient of the
> addressee please notify us of receipt by returning the e-mail and do not
> use, copy, retain, distribute or disclose the information in or attached to
> the e-mail.
> Any opinions expressed within this e-mail are those of the individual and
> not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
> attachments are free from viruses and we cannot accept liability for any
> damage which you may sustain as a result of software viruses which may be
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England
> and Wales with its registered office at Diamond House, Harwell Science and
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120824/73dc484e/attachment.html>

From bcostescu at gmail.com  Fri Aug 24 05:03:40 2012
From: bcostescu at gmail.com (Bogdan Costescu)
Date: Fri, 24 Aug 2012 14:03:40 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>

On Fri, Aug 24, 2012 at 3:06 AM, Christopher Samuel
<samuel at unimelb.edu.au> wrote:
> I'd like to solicit opinions from list members on changing the policy
> of the list on messages from non-subscribers from the current default
> of holding them for moderation to the position of rejecting them (with
> an explanatory message).

Does someone have a figure of how many non-subscribers posts we are
talking about ? I have the impression that lately I've seen only
messages coming from "regulars". If non-subscribers post very little,
then such a change would not be very annoying IMHO.

OTOH, I hope that archives (wherever they are) remain publicly available.

> The reason is that currently us new list admins get many dozens of
> spams a day being held for moderation

We should be very thankful to Don Becker for doing a wonderful job
with the list for many years :)

Cheers,
Bogdan


From j.sassmannshausen at ucl.ac.uk  Fri Aug 24 05:11:39 2012
From: j.sassmannshausen at ucl.ac.uk (=?iso-8859-1?q?J=F6rg_Sa=DFmannshausen?=)
Date: Fri, 24 Aug 2012 13:11:39 +0100
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <503756C3.9000907@diamond.ac.uk>
References: <5036D37E.5030800@unimelb.edu.au> <503756C3.9000907@diamond.ac.uk>
Message-ID: <201208241311.39808.j.sassmannshausen@ucl.ac.uk>

Hi all,

I am one of the many who cannot post to the list despite being a member of it 
for some years. :-/

I think the policy was that only members can post (good thing in my opinion) 
and new members are moderated. 
I am moderating an email list here as well and unfortunately the list's email 
address got to the outside world. I do get some spam but it is something I can 
cope with. 
Once a new member has established themselves as not being a spammer (and here 
I really mean the type of: buy vi online! I mean, I got vi installed on my 
machines so why shall I buy it? :D ) you can remove the moderation flag. If 
that member then becomes a spammer or troll you can them put back into 
moderation only.

Does that sound like a good idea? We do not have to rule out the usual anti-
spam mechanisms here.

My two cents from somebody who got put on the observer chair :D

Regards

J?rg

On Friday 24 August 2012 11:26:11 Gregory Matthews wrote:
> this is not a good idea - your list can become a spam mirror and
> replying to spammers is bad in lots of ways. Get a good spam filter,
> MailScanner + SpamAssassin + selected blacklists + selected
> sendmail/postfix plugins.
> 
> GREG
> 
> On 24/08/12 02:06, Christopher Samuel wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > Hi all,
> > 
> > I'd like to solicit opinions from list members on changing the policy
> > of the list on messages from non-subscribers from the current default
> > of holding them for moderation to the position of rejecting them (with
> > an explanatory message).
> > 
> > The reason is that currently us new list admins get many dozens of
> > spams a day being held for moderation and it would be very hard to
> > spot a legitimate message (from either a new subscriber or from a new
> > member who is having messages held for moderation) amongst them.
> > 
> > It seems fairer to me for legitimate non-members to get an email
> > saying that their post has been rejected because they're not a member,
> > but pointing them to the web page to subscribe, rather than just a
> > message saying it's been held for moderation and then nothing (if it
> > gets missed).
> > 
> > It's also fairer to new members who are under moderation (as spammers
> > do sign up to lists too) as their messages won't get missed in the
> > general flood of non-member spam.
> > 
> > This is how all the other Mailman lists I run are configured and it's
> > not caused issues or complaints there.
> > 
> > Thoughts please!
> > 
> > cheers,
> > Chris
> > - --
> > 
> >   Christopher Samuel        Senior Systems Administrator
> >   VLSCI - Victorian Life Sciences Computation Initiative
> >   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
> >   http://www.vlsci.org.au/      http://twitter.com/vlsci
> > 
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.11 (GNU/Linux)
> > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> > 
> > iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> > kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> > =RDNl
> > -----END PGP SIGNATURE-----
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit
> > http://www.beowulf.org/mailman/listinfo/beowulf

-- 
*************************************************************
J?rg Sa?mannshausen
University College London
Department of Chemistry
Gordon Street
London
WC1H 0AJ 

email: j.sassmannshausen at ucl.ac.uk
web: http://sassy.formativ.net

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html



From andrew.holway at gmail.com  Fri Aug 24 03:47:35 2012
From: andrew.holway at gmail.com (Andrew Holway)
Date: Fri, 24 Aug 2012 12:47:35 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <CAEiui-sXPVEUCoDjgRmBO-a72BBgTbSi6tp-35hvuCyRRKKALA@mail.gmail.com>

Maybe we should have a

'Please send us a brief email explaining why you would like to join
the list so that we know that your not a spammer"


2012/8/24 Christopher Samuel <samuel at unimelb.edu.au>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> I'd like to solicit opinions from list members on changing the policy
> of the list on messages from non-subscribers from the current default
> of holding them for moderation to the position of rejecting them (with
> an explanatory message).
>
> The reason is that currently us new list admins get many dozens of
> spams a day being held for moderation and it would be very hard to
> spot a legitimate message (from either a new subscriber or from a new
> member who is having messages held for moderation) amongst them.
>
> It seems fairer to me for legitimate non-members to get an email
> saying that their post has been rejected because they're not a member,
> but pointing them to the web page to subscribe, rather than just a
> message saying it's been held for moderation and then nothing (if it
> gets missed).
>
> It's also fairer to new members who are under moderation (as spammers
> do sign up to lists too) as their messages won't get missed in the
> general flood of non-member spam.
>
> This is how all the other Mailman lists I run are configured and it's
> not caused issues or complaints there.
>
> Thoughts please!
>
> cheers,
> Chris
> - --
>  Christopher Samuel        Senior Systems Administrator
>  VLSCI - Victorian Life Sciences Computation Initiative
>  Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>  http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> =RDNl
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From samuel at unimelb.edu.au  Fri Aug 24 05:50:35 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Fri, 24 Aug 2012 22:50:35 +1000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <503756C3.9000907@diamond.ac.uk>
References: <5036D37E.5030800@unimelb.edu.au> <503756C3.9000907@diamond.ac.uk>
Message-ID: <5037789B.6080904@unimelb.edu.au>

On 08/24/2012 08:26 PM, Gregory Matthews wrote:

> this is not a good idea - your list can become a spam mirror and
> replying to spammers is bad in lots of ways.

The Mailman software is *already* doing this for the Beowulf list as it 
replies to every message held for moderation with a link so that the 
sender can cancel it if they so wish.

> Get a good spam filter, MailScanner + SpamAssassin + selected
> blacklists + selected sendmail/postfix plugins.

That is needed as well (a point I made in private email amongst the list 
admins), but that is not in our control, that is up to Penguin Computing.

All the best,
Chris


From samuel at unimelb.edu.au  Fri Aug 24 05:55:18 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Fri, 24 Aug 2012 22:55:18 +1000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <201208241311.39808.j.sassmannshausen@ucl.ac.uk>
References: <5036D37E.5030800@unimelb.edu.au> <503756C3.9000907@diamond.ac.uk>
	<201208241311.39808.j.sassmannshausen@ucl.ac.uk>
Message-ID: <503779B6.3070006@unimelb.edu.au>

On 08/24/2012 10:11 PM, J?rg Sa?mannshausen wrote:

> Hi all,

Hiya!

> I am one of the many who cannot post to the list despite being a member of it
> for some years. :-/

Well I spotted your message in the queue and let it through, so welcome!

> I think the policy was that only members can post (good thing in my opinion)
> and new members are moderated.
> I am moderating an email list here as well and unfortunately the list's email
> address got to the outside world. I do get some spam but it is something I can
> cope with.
> Once a new member has established themselves as not being a spammer (and here
> I really mean the type of: buy vi online! I mean, I got vi installed on my
> machines so why shall I buy it? :D ) you can remove the moderation flag. If
> that member then becomes a spammer or troll you can them put back into
> moderation only.

That's precisely how the list works at the moment, in that new 
subscribers have posts moderated at first and that flag gets cleared 
after a while.  The only problem is that the list has been running on 
autopilot for quite a while (I don't know how long) and so people 
haven't been getting unmoderated or having posts approved.

> Does that sound like a good idea? We do not have to rule out the usual anti-
> spam mechanisms here.

Agreed, we need defence in depth on this. :-)

> My two cents from somebody who got put on the observer chair :D

You are now unmoderated, so you can get off that chair now. ;-)

cheers!
Chris


From eagles051387 at gmail.com  Fri Aug 24 05:27:52 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 24 Aug 2012 14:27:52 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>
Message-ID: <CAHdpx62rCmj96+pcbchaZoTP4DRA3=7haFcAF7s8XTWikt4H+Q@mail.gmail.com>

A few others and myself have been moderating the list to ensure only legit
subscribers get through. With that being said there are quite a fair bit of
emails coming in from non subscribers which is proving rather hard to keep
up with.

On Fri, Aug 24, 2012 at 2:03 PM, Bogdan Costescu <bcostescu at gmail.com>wrote:

> On Fri, Aug 24, 2012 at 3:06 AM, Christopher Samuel
> <samuel at unimelb.edu.au> wrote:
> > I'd like to solicit opinions from list members on changing the policy
> > of the list on messages from non-subscribers from the current default
> > of holding them for moderation to the position of rejecting them (with
> > an explanatory message).
>
> Does someone have a figure of how many non-subscribers posts we are
> talking about ? I have the impression that lately I've seen only
> messages coming from "regulars". If non-subscribers post very little,
> then such a change would not be very annoying IMHO.
>
> OTOH, I hope that archives (wherever they are) remain publicly available.
>
> > The reason is that currently us new list admins get many dozens of
> > spams a day being held for moderation
>
> We should be very thankful to Don Becker for doing a wonderful job
> with the list for many years :)
>
> Cheers,
> Bogdan
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120824/04fe9461/attachment.html>

From samuel at unimelb.edu.au  Fri Aug 24 06:01:05 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Fri, 24 Aug 2012 23:01:05 +1000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>
Message-ID: <50377B11.3080608@unimelb.edu.au>

On 08/24/2012 10:03 PM, Bogdan Costescu wrote:

> Does someone have a figure of how many non-subscribers posts we are
> talking about ? I have the impression that lately I've seen only
> messages coming from "regulars". If non-subscribers post very little,
> then such a change would not be very annoying IMHO.

Well in the 6 hours since I last checked at work there have were over 30 
new messages waiting to be processed, of which 1 was from a subscriber 
still moderated and 1 was a legitimate post from a non-member 
(presumably subscribed at a different address as they replied to my list 
post).

The other 30 plus were just spam.

> OTOH, I hope that archives (wherever they are) remain publicly available.

I don't think anyone is planning to change those at all.

http://www.beowulf.org/pipermail/beowulf/

> We should be very thankful to Don Becker for doing a wonderful job
> with the list for many years :)

Oh indeed!  Very much so...

All the best,
Chris


From landman at scalableinformatics.com  Fri Aug 24 06:13:55 2012
From: landman at scalableinformatics.com (Joe Landman)
Date: Fri, 24 Aug 2012 09:13:55 -0400
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <50377E13.7060000@scalableinformatics.com>

Apologies on the delay:


On 08/23/2012 09:06 PM, Christopher Samuel wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> I'd like to solicit opinions from list members on changing the policy
> of the list on messages from non-subscribers from the current default
> of holding them for moderation to the position of rejecting them (with
> an explanatory message).
>
> The reason is that currently us new list admins get many dozens of
> spams a day being held for moderation and it would be very hard to
> spot a legitimate message (from either a new subscriber or from a new
> member who is having messages held for moderation) amongst them.
>
> It seems fairer to me for legitimate non-members to get an email
> saying that their post has been rejected because they're not a member,
> but pointing them to the web page to subscribe, rather than just a
> message saying it's been held for moderation and then nothing (if it
> gets missed).
>
> It's also fairer to new members who are under moderation (as spammers
> do sign up to lists too) as their messages won't get missed in the
> general flood of non-member spam.
>
> This is how all the other Mailman lists I run are configured and it's
> not caused issues or complaints there.
>
> Thoughts please!
>

Subscriber only is good.

> cheers,
> Chris
> - --
>   Christopher Samuel        Senior Systems Administrator
>   VLSCI - Victorian Life Sciences Computation Initiative
>   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>   http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> =RDNl
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615


From james.p.lux at jpl.nasa.gov  Fri Aug 24 06:32:04 2012
From: james.p.lux at jpl.nasa.gov (Lux, Jim (337C))
Date: Fri, 24 Aug 2012 13:32:04 +0000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <50377E13.7060000@scalableinformatics.com>
Message-ID: <CC5CCF3B.2466D%james.p.lux@jpl.nasa.gov>


I'm one of the moderators on the (moderated) Tesla Coil Mailing List,
using Mailman, and Chip Atkinson (list owner) has set up something that
silently discards a lot of the spam.  I'd say we get maybe <10% posts from
non-subscribers coming through along with the rest of the traffic.  Of
that 10% maybe half or a third is something not worthy of response.  The
rest of the non-subscriber posts, we just bounce with a nice message
saying "please subscribe, and your post will go through".

The list has been around for 10-20 years, so it must receive plenty of
spam at the front end.  If you're interested, I can put you in touch with
Chip and he'd probably be happy to explain what he does.

Jim

 
On 8/24/12 6:13 AM, "Joe Landman" <landman at scalableinformatics.com> wrote:

>Apologies on the delay:
>
>
>On 08/23/2012 09:06 PM, Christopher Samuel wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi all,
>>
>> I'd like to solicit opinions from list members on changing the policy
>> of the list on messages from non-subscribers from the current default
>> of holding them for moderation to the position of rejecting them (with
>> an explanatory message).
>>
>> The reason is that currently us new list admins get many dozens of
>> spams a day being held for moderation and it would be very hard to
>> spot a legitimate message (from either a new subscriber or from a new
>> member who is having messages held for moderation) amongst them.
>>
>> It seems fairer to me for legitimate non-members to get an email
>> saying that their post has been rejected because they're not a member,
>> but pointing them to the web page to subscribe, rather than just a
>> message saying it's been held for moderation and then nothing (if it
>> gets missed).
>>
>> It's also fairer to new members who are under moderation (as spammers
>> do sign up to lists too) as their messages won't get missed in the
>> general flood of non-member spam.
>>
>> This is how all the other Mailman lists I run are configured and it's
>> not caused issues or complaints there.
>>
>> Thoughts please!
>>
>
>Subscriber only is good.
>
>> cheers,
>> Chris
>> - --
>>   Christopher Samuel        Senior Systems Administrator
>>   VLSCI - Victorian Life Sciences Computation Initiative
>>   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>>   http://www.vlsci.org.au/      http://twitter.com/vlsci
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
>> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
>> =RDNl
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit
>>http://www.beowulf.org/mailman/listinfo/beowulf
>>
>
>
>-- 
>Joseph Landman, Ph.D
>Founder and CEO
>Scalable Informatics Inc.
>email: landman at scalableinformatics.com
>web  : http://scalableinformatics.com
>        http://scalableinformatics.com/sicluster
>phone: +1 734 786 8423 x121
>fax  : +1 866 888 3112
>cell : +1 734 612 4615
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>To change your subscription (digest mode or unsubscribe) visit
>http://www.beowulf.org/mailman/listinfo/beowulf



From landman at scalableinformatics.com  Fri Aug 24 06:39:48 2012
From: landman at scalableinformatics.com (Joe Landman)
Date: Fri, 24 Aug 2012 09:39:48 -0400
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <CC5CCF3B.2466D%james.p.lux@jpl.nasa.gov>
References: <CC5CCF3B.2466D%james.p.lux@jpl.nasa.gov>
Message-ID: <50378424.6060000@scalableinformatics.com>

On 08/24/2012 09:32 AM, Lux, Jim (337C) wrote:
>
> I'm one of the moderators on the (moderated) Tesla Coil Mailing List,
> using Mailman, and Chip Atkinson (list owner) has set up something that
> silently discards a lot of the spam.  I'd say we get maybe <10% posts from
> non-subscribers coming through along with the rest of the traffic.  Of
> that 10% maybe half or a third is something not worthy of response.  The
> rest of the non-subscriber posts, we just bounce with a nice message
> saying "please subscribe, and your post will go through".

FWIW, I did offer to set up spam filtering for this.  Need to know how 
they process their mail, and we can set up an annotation pipeline 
(complete with Bayesian, heuristic, and other filters).  At the end of 
the process, we have a pretty good score and cutoff for spam vs 
not-spam.  We use that for our filtering.  Since we get a couple 
thousand emails running through the filters a day, we get maybe 3-4 
false positives, and a few false negatives that we correct by hand 
(training the Bayesian network).



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615


From landman at scalableinformatics.com  Fri Aug 24 09:10:54 2012
From: landman at scalableinformatics.com (Joe Landman)
Date: Fri, 24 Aug 2012 12:10:54 -0400
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <50377B11.3080608@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>
	<50377B11.3080608@unimelb.edu.au>
Message-ID: <5037A78E.5060400@scalableinformatics.com>

I am thinking we need a Monty Python Captcha

"What is your name"

"What is your quest"

"What is the airspeed of an unladen swallow"

...


On 08/24/2012 09:01 AM, Christopher Samuel wrote:
> On 08/24/2012 10:03 PM, Bogdan Costescu wrote:
>
>> Does someone have a figure of how many non-subscribers posts we are
>> talking about ? I have the impression that lately I've seen only
>> messages coming from "regulars". If non-subscribers post very little,
>> then such a change would not be very annoying IMHO.
>
> Well in the 6 hours since I last checked at work there have were over 30
> new messages waiting to be processed, of which 1 was from a subscriber
> still moderated and 1 was a legitimate post from a non-member
> (presumably subscribed at a different address as they replied to my list
> post).
>
> The other 30 plus were just spam.
>
>> OTOH, I hope that archives (wherever they are) remain publicly available.
>
> I don't think anyone is planning to change those at all.
>
> http://www.beowulf.org/pipermail/beowulf/
>
>> We should be very thankful to Don Becker for doing a wonderful job
>> with the list for many years :)
>
> Oh indeed!  Very much so...
>
> All the best,
> Chris
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615


From prentice.bisbal at rutgers.edu  Fri Aug 24 06:43:31 2012
From: prentice.bisbal at rutgers.edu (Prentice Bisbal)
Date: Fri, 24 Aug 2012 09:43:31 -0400
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <50378503.60007@rutgers.edu>

I'm all for this. I always thought this was standard policy for every 
mailing list. I don't see any legitimate reason to post to a list you 
don't subscribe to, since if you're not a subscriber, you're not going 
to see the replies to your post.

Prentice

On 08/23/2012 09:06 PM, Christopher Samuel wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> I'd like to solicit opinions from list members on changing the policy
> of the list on messages from non-subscribers from the current default
> of holding them for moderation to the position of rejecting them (with
> an explanatory message).
>
> The reason is that currently us new list admins get many dozens of
> spams a day being held for moderation and it would be very hard to
> spot a legitimate message (from either a new subscriber or from a new
> member who is having messages held for moderation) amongst them.
>
> It seems fairer to me for legitimate non-members to get an email
> saying that their post has been rejected because they're not a member,
> but pointing them to the web page to subscribe, rather than just a
> message saying it's been held for moderation and then nothing (if it
> gets missed).
>
> It's also fairer to new members who are under moderation (as spammers
> do sign up to lists too) as their messages won't get missed in the
> general flood of non-member spam.
>
> This is how all the other Mailman lists I run are configured and it's
> not caused issues or complaints there.
>
> Thoughts please!
>
> cheers,
> Chris
> - -- 
>   Christopher Samuel        Senior Systems Administrator
>   VLSCI - Victorian Life Sciences Computation Initiative
>   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>   http://www.vlsci.org.au/      http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> =RDNl
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From prentice.bisbal at rutgers.edu  Fri Aug 24 06:51:49 2012
From: prentice.bisbal at rutgers.edu (Prentice Bisbal)
Date: Fri, 24 Aug 2012 09:51:49 -0400
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <CAEiui-sXPVEUCoDjgRmBO-a72BBgTbSi6tp-35hvuCyRRKKALA@mail.gmail.com>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAEiui-sXPVEUCoDjgRmBO-a72BBgTbSi6tp-35hvuCyRRKKALA@mail.gmail.com>
Message-ID: <503786F5.9050701@rutgers.edu>

Doesn't the subscription confirmation e-mail kind of handle this 
already? I suspect spammers send from random or fictitious addresses and 
don't check (or even receive) email sent to those addresses, so it's 
unlikely they'll be confirmed as subscribers to the mailing list.

I think Greg raises an excellent point that we shouldn't reply to 
spammers. Perhaps we should just silently ignore posts from 
non-subscribers. If they are real humans, they should figure it out 
easily enough that they need to subscribe to post. Most mailing lists 
have a web page advertising the list, with instructions on how to 
subscribe to the list and list policy, such as needing to subscribe in 
order to post. I think that should be adequate to keep spam away but not 
real humans.

Besides, don't we hate when people post something that clearly indicates 
they didn't RTFM? ;)

--
Prentice


On 08/24/2012 06:47 AM, Andrew Holway wrote:
> Maybe we should have a
>
> 'Please send us a brief email explaining why you would like to join
> the list so that we know that your not a spammer"
>
>
> 2012/8/24 Christopher Samuel<samuel at unimelb.edu.au>:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi all,
>>
>> I'd like to solicit opinions from list members on changing the policy
>> of the list on messages from non-subscribers from the current default
>> of holding them for moderation to the position of rejecting them (with
>> an explanatory message).
>>
>> The reason is that currently us new list admins get many dozens of
>> spams a day being held for moderation and it would be very hard to
>> spot a legitimate message (from either a new subscriber or from a new
>> member who is having messages held for moderation) amongst them.
>>
>> It seems fairer to me for legitimate non-members to get an email
>> saying that their post has been rejected because they're not a member,
>> but pointing them to the web page to subscribe, rather than just a
>> message saying it's been held for moderation and then nothing (if it
>> gets missed).
>>
>> It's also fairer to new members who are under moderation (as spammers
>> do sign up to lists too) as their messages won't get missed in the
>> general flood of non-member spam.
>>
>> This is how all the other Mailman lists I run are configured and it's
>> not caused issues or complaints there.
>>
>> Thoughts please!
>>
>> cheers,
>> Chris
>> - --
>>   Christopher Samuel        Senior Systems Administrator
>>   VLSCI - Victorian Life Sciences Computation Initiative
>>   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>>   http://www.vlsci.org.au/      http://twitter.com/vlsci
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
>> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
>> =RDNl
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From graham.mullier at syngenta.com  Fri Aug 24 06:56:53 2012
From: graham.mullier at syngenta.com (graham.mullier at syngenta.com)
Date: Fri, 24 Aug 2012 15:56:53 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <503779B6.3070006@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
	<503756C3.9000907@diamond.ac.uk><201208241311.39808.j.sassmannshausen@ucl.ac.uk>
	<503779B6.3070006@unimelb.edu.au>
Message-ID: <3E91C69ADC46C4408C85991183A275F40BC8204827@FRGOCMSXMB04.EAME.SYNGENTA.ORG>

Gentlethings

While you're noting people who aren't spammers but who are currently locked out despite being subscribers, I'm another. I seem never to have been flagged as 'OK' so gradually (this is over some years) came to regard the list as an interesting thing to watch but not something to participate in.

Chris, while it would be helpful for me to be flagged as OK to post (for me), what about the several or many others who are stuck in the same sort of limbo? Is there a way to identify them, open up the list, and breathe a bit more life into it?

Regards
Graham

Graham Mullier | Head Information Connection & Design, R&DIS | R&D architecture CoE lead
graham.mullier at syngenta.com | Syngenta, Jealott's Hill International Research Station, Bracknell, RG42 6EY, UK



-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Christopher Samuel
Sent: 24 August 2012 08:55
To: beowulf at beowulf.org
Subject: Re: [Beowulf] RFC: Restrict posting to subscribers only ?

On 08/24/2012 10:11 PM, J?rg Sa?mannshausen wrote:

> Hi all,

Hiya!

> I am one of the many who cannot post to the list despite being a member of it
> for some years. :-/

Well I spotted your message in the queue and let it through, so welcome!

> I think the policy was that only members can post (good thing in my opinion)
> and new members are moderated.
> I am moderating an email list here as well and unfortunately the list's email
> address got to the outside world. I do get some spam but it is something I can
> cope with.
> Once a new member has established themselves as not being a spammer (and here
> I really mean the type of: buy vi online! I mean, I got vi installed on my
> machines so why shall I buy it? :D ) you can remove the moderation flag. If
> that member then becomes a spammer or troll you can them put back into
> moderation only.

That's precisely how the list works at the moment, in that new
subscribers have posts moderated at first and that flag gets cleared
after a while.  The only problem is that the list has been running on
autopilot for quite a while (I don't know how long) and so people
haven't been getting unmoderated or having posts approved.

> Does that sound like a good idea? We do not have to rule out the usual anti-
> spam mechanisms here.

Agreed, we need defence in depth on this. :-)

> My two cents from somebody who got put on the observer chair :D

You are now unmoderated, so you can get off that chair now. ;-)

cheers!
Chris
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



Syngenta Limited, Registered in England No 2710846
Registered Office : Syngenta Limited, European Regional Centre, Priestley Road, Surrey Research Park, Guildford, Surrey, GU2 7YH, United Kingdom



This message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the original and any copies. Any use of the message by you is prohibited. 



From jeff.johnson at aeoncomputing.com  Fri Aug 24 09:30:56 2012
From: jeff.johnson at aeoncomputing.com (Jeff Johnson)
Date: Fri, 24 Aug 2012 09:30:56 -0700
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5037A78E.5060400@scalableinformatics.com>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>
	<50377B11.3080608@unimelb.edu.au>
	<5037A78E.5060400@scalableinformatics.com>
Message-ID: <5037AC40.3090703@aeoncomputing.com>

I think I've seen another list or site do that very thing.

Oh wait, now I remember. It is inside the systemimager source somewhere.


On 8/24/12 9:10 AM, Joe Landman wrote:
> I am thinking we need a Monty Python Captcha
>
> "What is your name"
>
> "What is your quest"
>
> "What is the airspeed of an unladen swallow"
>
> ...
>

-- 
------------------------------
Jeff Johnson
Manager
Aeon Computing

jeff.johnson at aeoncomputing.com
www.aeoncomputing.com
t: 858-412-3810 x101   f: 858-412-3845
m: 619-204-9061

/* New Address */
4170 Morena Boulevard, Suite D - San Diego, CA 92117



From prentice.bisbal at rutgers.edu  Fri Aug 24 11:12:35 2012
From: prentice.bisbal at rutgers.edu (Prentice Bisbal)
Date: Fri, 24 Aug 2012 14:12:35 -0400
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <5037A78E.5060400@scalableinformatics.com>
References: <5036D37E.5030800@unimelb.edu.au>
	<CAH+WPdub64Z8r5df1PsjfTV-t+0PQfiFWF98d0ympxg-eLREuw@mail.gmail.com>
	<50377B11.3080608@unimelb.edu.au>
	<5037A78E.5060400@scalableinformatics.com>
Message-ID: <5037C413.1040407@rutgers.edu>

African or European?


On 08/24/2012 12:10 PM, Joe Landman wrote:
> I am thinking we need a Monty Python Captcha
>
> "What is your name"
>
> "What is your quest"
>
> "What is the airspeed of an unladen swallow"
>
> ...
>
>
> On 08/24/2012 09:01 AM, Christopher Samuel wrote:
>> On 08/24/2012 10:03 PM, Bogdan Costescu wrote:
>>
>>> Does someone have a figure of how many non-subscribers posts we are
>>> talking about ? I have the impression that lately I've seen only
>>> messages coming from "regulars". If non-subscribers post very little,
>>> then such a change would not be very annoying IMHO.
>> Well in the 6 hours since I last checked at work there have were over 30
>> new messages waiting to be processed, of which 1 was from a subscriber
>> still moderated and 1 was a legitimate post from a non-member
>> (presumably subscribed at a different address as they replied to my list
>> post).
>>
>> The other 30 plus were just spam.
>>
>>> OTOH, I hope that archives (wherever they are) remain publicly available.
>> I don't think anyone is planning to change those at all.
>>
>> http://www.beowulf.org/pipermail/beowulf/
>>
>>> We should be very thankful to Don Becker for doing a wonderful job
>>> with the list for many years :)
>> Oh indeed!  Very much so...
>>
>> All the best,
>> Chris
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>>
>


From holway at th.physik.uni-frankfurt.de  Sat Aug 25 15:22:16 2012
From: holway at th.physik.uni-frankfurt.de (holway at th.physik.uni-frankfurt.de)
Date: Sun, 26 Aug 2012 00:22:16 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
Message-ID: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>

Hello Beowulf 2.0

I've just started playing with NFSoIB to provide super fast backend
storage for a bunch of databases that we look after here. Oracle and
Nexenta are both sending me ZFS based boxes to test and I hope to compare
the performance and stability of these with the Netapp (formally lsi
engenio) E5400.

This will be the first time I will be getting into serious storage
benchmarking. Does anyone have any interesting tests they would like to
run or and experience performing these kinds of tests?

I have 4 HP G8 boxes with 96GB ram each QDR IB and 10G ethernet as consumers.

I will also be testing the performance of KVM and Virtuozzo (commercial
version of OpenVZ) which is a kernel sharing virtualization similar to BSD
Jails.

Thanks,

Andrew



From yong.chen at ttu.edu  Thu Aug 23 08:21:20 2012
From: yong.chen at ttu.edu (Chen, Yong)
Date: Thu, 23 Aug 2012 15:21:20 +0000
Subject: [Beowulf] [hpc-announce] CFP: DISCS-2012 Workshop at SC'12 with
 Cluster Computing journal special issue
Message-ID: <AC0D521357876C4AA01C144CD17C58D9012896B7@centaur09.ttu.edu>

[Apologies if you got multiple copies of this email. If you'd like to
opt out of these announcements, information on how to unsubscribe is
available at the bottom of this email.]
-------------- next part --------------

[Apologize for any duplicated posts]

                 ****************
                 CALL FOR PAPERS
                 ****************
          The International Workshop on
Data-Intensive Scalable Computing Systems (DISCS)
          http://data.cs.ttu.edu/discs/

          November 16th, 2012, Salt Lake City
              In conjunction with the
 2012 ACM/IEEE Supercomputing Conference (SC'12)

                 *********************
                 Scope of the Workshop
                 *********************
High-performance computing (HPC) is a major strategic tool
for science, engineering, and industry. Existing HPC systems,
however, are largely designed and developed for computation-
intensive applications with a computing-centric paradigm. With
the emerging and timely needs of supporting data-intensive
scientific discovery and innovations, there is a need of
rethinking the system architectures, programming models, runtime
systems, and tools available for data-intensive HPC. The DISCS
workshop provides a forum for researchers and developers in the
high-performance computing, data-intensive computing, and
parallel computing fields to take the Big Data challenges
together and present innovative ideas, experiences, and latest
developments that help address these challenges.

                   ******************
                   Topics of Interest
                   ******************
The focus areas of the DISCS-2012 workshop include, but are not
limited to:
    -> HPC system architectures for supporting data intensive
       applications
         # Data-centric system architectures
         # I/O systems and architectures
         # System area networks
         # Power efficient systems for data intensive applications
    -> Programming models for supporting data intensive
       applications
         # Data-centric programming models
         # MPI extensions for data intensive applications
         # GAS/PGAS/OpenMP programming model extensions for data-
           intensive applications
         # New programming languages/methodologies for data-
           intensive HPC
    -> Runtime systems for supporting data intensive applications
         # Communication systems for supporting data intensive
           applications
         # Data compression and deduplication
         # Caching and prefetching
         # Data integrity and consistency
    -> Tools for supporting data intensive applications
         # Data analytic, tracing, and trace analysis tools
         # Data mining and knowledge discovery tools
         # Computational, mathematical and statistical techniques
           and tools
         # Data visualization techniques and tools

                      ***************
                      Important Dates
                      ***************
    -> Paper Submission: September 8th, 2012
    -> Author Notification: October 22nd, 2012
    -> Workshop Date: November 16th, 2012
    -> Camera-Ready Copy: December 7th, 2012

                   ***********************
                   Submission Instructions
                   ***********************
Submissions should be in PDF format in U.S. letter size paper,
and should be formatted according to the IEEE standard double-
column single-space format. They should not exceed 5 pages (all-
inclusive). Please access the IEEE site for the manuscript template.

Please visit the following site to make paper submissions:
https://www.easychair.org/account/signin.cgi?conf=discs2012

All accepted papers will appear in the SC-2012 digital proceedings
in the IEEE digital library.

                   *********************
                   Journal Special Issue
                   *********************
DISCS-2012 workshop papers will be invited to extend the manuscripts
to be considered for a journal Special Issue on Data-Intensive High
Performance Computing of the Cluster Computing by Springer, edited
by Yong Chen and Xian-He Sun. The target journal special issue
submission deadline is Jan. 16th, 2013.

                     ***********
                     Organizers
                     ***********
STEERING COMMITTEE
William D. Gropp, University of Illinois Urbana-Champaign
Xian-He Sun, Illinois Institute of Technology
Rajeev Thakur, Argonne National Laboratory

PROGRAM CHAIRS
Yong Chen, Texas Tech University
Xian-He Sun, Illinois Institute of Technology

TECHNICAL PROGRAM COMMITTEE
Suren Byna, Lawrence Berkeley National Laboratory
Shane Canon, Lawrence Berkeley National Laboratory
Hui Jin, Oracle
Scott Klasky, Oak Ridge National Laboratory
Quincey Koziol, HDF5 Group
Wei-keng Liao, Northwestern University
Jay Lofstead, Sandia National Laboratory
Carlos Maltzahn, University of California at Santa Cruz
Ioan Raicu, Illinois Institute of Technology
Philip C. Roth, Oak Ridge National Laboratory
Weikuan Yu, Auburn University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120823/d32dae69/attachment.html>
-------------- next part --------------
********************************************************************************
The hpc-announce mailing list has been setup to have a common mailing
list to share information with respect to upcoming HPC related
events. You are included in this mailing list based on your
participation or interest in a previous HPC conference or other event.

The purpose for providing a single mailing list is to allow
participants to easily identify such emails, and handle them
appropriately. Some options include:

1. If you feel that the number of such emails is too many, filter them
to less-frequently-read folders in your email client.

2. Change your subscription to a digest option
(https://lists.mcs.anl.gov/mailman/listinfo/hpc-announce) which will
consolidate emails sent that week into a single summary email.

3. Finally, if you do not wish to receive any emails from hpc-announce,
you can unsubscribe from the mailing list
(https://lists.mcs.anl.gov/mailman/listinfo/hpc-announce). Once
unsubscribed, we guarantee that you will not be added back in through
participation in a different HPC related conference or event. You will
need to send an email to hpc-announce-owner at mcs.anl.gov to be added
back on.

If you need help with any of the above, please email us at
hpc-announce-owner at mcs.anl.gov.
********************************************************************************

From samuel at unimelb.edu.au  Sat Aug 25 20:15:11 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Sun, 26 Aug 2012 13:15:11 +1000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <3E91C69ADC46C4408C85991183A275F40BC8204827@FRGOCMSXMB04.EAME.SYNGENTA.ORG>
References: <5036D37E.5030800@unimelb.edu.au>
	<503756C3.9000907@diamond.ac.uk><201208241311.39808.j.sassmannshausen@ucl.ac.uk>
	<503779B6.3070006@unimelb.edu.au>
	<3E91C69ADC46C4408C85991183A275F40BC8204827@FRGOCMSXMB04.EAME.SYNGENTA.ORG>
Message-ID: <503994BF.1060309@unimelb.edu.au>

On 08/24/2012 11:56 PM, graham.mullier at syngenta.com wrote:

> Gentlethings

Hiya,

> While you're noting people who aren't spammers but who are currently
> locked out despite being subscribers, I'm another. I seem never to
> have been flagged as 'OK' so gradually (this is over some years)
> came to regard the list as an interesting thing to watch but not
> something to participate in.

You're now OK'd (as have a couple of other people who've got held up 
over this weekend.  Participation is the lifeblood of a mailing list.

> Chris, while it would be helpful for me to be flagged as OK to post
> (for me), what about the several or many others who are stuck in the
> same sort of limbo? Is there a way to identify them, open up the list,
> and breathe a bit more life into it?

The issue is that there are a lot of subscribers to the list, and a lot 
of them are from odd looking addresses, so I'd rather set the list to 
reject non-subscriber posts and encourage subscribers to post to the list.

That way when the moderation queue is checked all there will be will be 
messages from moderated members and (hopefully) they'll be almost 
entirely non-spam.  That way we'll be able to clear peoples moderated 
flags a lot more quickly (as you'll be easier to spot).

cheers,
Chris


From eagles051387 at gmail.com  Sat Aug 25 23:59:49 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Sun, 26 Aug 2012 08:59:49 +0200
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <50378424.6060000@scalableinformatics.com>
References: <CC5CCF3B.2466D%james.p.lux@jpl.nasa.gov>
	<50378424.6060000@scalableinformatics.com>
Message-ID: <CAHdpx62=NZHyxVV-RS0MBbWKDTb+GqVYu1nhuEg+zk3RYNNj6w@mail.gmail.com>

Hey guys I am looking at the admin interface, and it seems like there is a
way to set moderators to recieve emails would that be something we want.

Also on another note something we could setup which would be easier that
anyone can manage would be untangle security distro. It has a web based
interface and all.

Arend is the mailing list on a virtual server or a shared server? If a
virtual server it would be great if we could put together an untangled
virtual machine and all the mods would have a web based interface to manage
various aspects spam filtering included. its debian based and I have seen
it in use and the spam filter works wonders.

On Fri, Aug 24, 2012 at 3:39 PM, Joe Landman <
landman at scalableinformatics.com> wrote:

> On 08/24/2012 09:32 AM, Lux, Jim (337C) wrote:
> >
> > I'm one of the moderators on the (moderated) Tesla Coil Mailing List,
> > using Mailman, and Chip Atkinson (list owner) has set up something that
> > silently discards a lot of the spam.  I'd say we get maybe <10% posts
> from
> > non-subscribers coming through along with the rest of the traffic.  Of
> > that 10% maybe half or a third is something not worthy of response.  The
> > rest of the non-subscriber posts, we just bounce with a nice message
> > saying "please subscribe, and your post will go through".
>
> FWIW, I did offer to set up spam filtering for this.  Need to know how
> they process their mail, and we can set up an annotation pipeline
> (complete with Bayesian, heuristic, and other filters).  At the end of
> the process, we have a pretty good score and cutoff for spam vs
> not-spam.  We use that for our filtering.  Since we get a couple
> thousand emails running through the filters a day, we get maybe 3-4
> false positives, and a few false negatives that we correct by hand
> (training the Bayesian network).
>
>
>
> --
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics Inc.
> email: landman at scalableinformatics.com
> web  : http://scalableinformatics.com
>         http://scalableinformatics.com/sicluster
> phone: +1 734 786 8423 x121
> fax  : +1 866 888 3112
> cell : +1 734 612 4615
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120826/4766eec4/attachment.html>

From hearnsj at googlemail.com  Sun Aug 26 01:44:58 2012
From: hearnsj at googlemail.com (John Hearns)
Date: Sun, 26 Aug 2012 09:44:58 +0100
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
Message-ID: <CAPqNE2XKP1YzVjTZxrHRmUAfXr3kY+u12n==vb+_hLeH1aLCvA@mail.gmail.com>

I'm not familiar with NFS over IB
Do you have a good reference?
Is this uding native IB addressing - if so how do you configure the server
exoirts fike etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120826/b865e7d4/attachment.html>

From andrew.holway at gmail.com  Sun Aug 26 02:21:03 2012
From: andrew.holway at gmail.com (Andrew Holway)
Date: Sun, 26 Aug 2012 11:21:03 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <CAPqNE2XKP1YzVjTZxrHRmUAfXr3kY+u12n==vb+_hLeH1aLCvA@mail.gmail.com>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
	<CAPqNE2XKP1YzVjTZxrHRmUAfXr3kY+u12n==vb+_hLeH1aLCvA@mail.gmail.com>
Message-ID: <CAEiui-t=XmMiQHZvaOaHYFan-wxN6CW+VjzKsbz8=7vjNmQ3-g@mail.gmail.com>

Sorry, NFSoIB is more commonly known as NFSoRDMA.

This link explains it quite well.
http://www.opengridcomputing.com/nfs-rdma.html.

Currently my export looks like this.

/dev/shm				10.149.0.0/16(rw,fsid=1,no_root_squash,insecure)

Where 10.149.0.0 is the IPoIB interface. NFSoRDMA uses TCP/IP to
establish and tear down connections and then RDMA for data transfer.
Using TCP for connections enables high availability setups.


2012/8/26 John Hearns <hearnsj at googlemail.com>:
> I'm not familiar with NFS over IB
> Do you have a good reference?
> Is this uding native IB addressing - if so how do you configure the server
> exoirts fike etc.
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>


From fallsoffmotorcycle at gmail.com  Sun Aug 26 08:54:15 2012
From: fallsoffmotorcycle at gmail.com (RA Creacy)
Date: Sun, 26 Aug 2012 08:54:15 -0700
Subject: [Beowulf] Beowulf Digest, Vol 102, Issue 19
In-Reply-To: <mailman.41743.1345950295.2391.beowulf@beowulf.org>
References: <mailman.41743.1345950295.2391.beowulf@beowulf.org>
Message-ID: <CAKMW5LDzvzwXDQDcGDZX+i4vpXYRopo=TcaOiy_XLdmc9FjZ8Q@mail.gmail.com>

*Agreed.
Long term lurker/subscriber. *
*Bob** Creacy
**Don't poop where you eat.  (Stephen King)**
We're not in Wonderland anymore Alice. (C. Manson)*

 **
**

**
*
*
* *





On Sat, Aug 25, 2012 at 8:04 PM, <beowulf-request at beowulf.org> wrote:

> Send Beowulf mailing list submissions to
>         beowulf at beowulf.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://www.beowulf.org/mailman/listinfo/beowulf
> or, via email, send a message with subject or body 'help' to
>         beowulf-request at beowulf.org
>
> You can reach the person managing the list at
>         beowulf-owner at beowulf.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Beowulf digest..."
>
>
> Today's Topics:
>
>    1. Re: RFC: Restrict posting to subscribers only ? (Prentice Bisbal)
>    2. Re: RFC: Restrict posting to subscribers only ? (Prentice Bisbal)
>    3. Re: RFC: Restrict posting to subscribers only ?
>       (graham.mullier at syngenta.com)
>    4. Re: RFC: Restrict posting to subscribers only ? (Jeff Johnson)
>    5. Re: RFC: Restrict posting to subscribers only ? (Prentice Bisbal)
>    6. NFSoIB and SRP benchmarking (holway at th.physik.uni-frankfurt.de)
>    7. [hpc-announce] CFP: DISCS-2012 Workshop at SC'12 with Cluster
>       Computing journal special issue (Chen, Yong)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 24 Aug 2012 09:43:31 -0400
> From: Prentice Bisbal <prentice.bisbal at rutgers.edu>
> Subject: Re: [Beowulf] RFC: Restrict posting to subscribers only ?
> To: beowulf at beowulf.org
> Message-ID: <50378503.60007 at rutgers.edu>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> I'm all for this. I always thought this was standard policy for every
> mailing list. I don't see any legitimate reason to post to a list you
> don't subscribe to, since if you're not a subscriber, you're not going
> to see the replies to your post.
>
> Prentice
>
> On 08/23/2012 09:06 PM, Christopher Samuel wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Hi all,
> >
> > I'd like to solicit opinions from list members on changing the policy
> > of the list on messages from non-subscribers from the current default
> > of holding them for moderation to the position of rejecting them (with
> > an explanatory message).
> >
> > The reason is that currently us new list admins get many dozens of
> > spams a day being held for moderation and it would be very hard to
> > spot a legitimate message (from either a new subscriber or from a new
> > member who is having messages held for moderation) amongst them.
> >
> > It seems fairer to me for legitimate non-members to get an email
> > saying that their post has been rejected because they're not a member,
> > but pointing them to the web page to subscribe, rather than just a
> > message saying it's been held for moderation and then nothing (if it
> > gets missed).
> >
> > It's also fairer to new members who are under moderation (as spammers
> > do sign up to lists too) as their messages won't get missed in the
> > general flood of non-member spam.
> >
> > This is how all the other Mailman lists I run are configured and it's
> > not caused issues or complaints there.
> >
> > Thoughts please!
> >
> > cheers,
> > Chris
> > - --
> >   Christopher Samuel        Senior Systems Administrator
> >   VLSCI - Victorian Life Sciences Computation Initiative
> >   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
> >   http://www.vlsci.org.au/      http://twitter.com/vlsci
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.11 (GNU/Linux)
> > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> >
> > iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> > kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> > =RDNl
> > -----END PGP SIGNATURE-----
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
> ------------------------------
>
> Message: 2
> Date: Fri, 24 Aug 2012 09:51:49 -0400
> From: Prentice Bisbal <prentice.bisbal at rutgers.edu>
> Subject: Re: [Beowulf] RFC: Restrict posting to subscribers only ?
> To: beowulf at beowulf.org
> Message-ID: <503786F5.9050701 at rutgers.edu>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Doesn't the subscription confirmation e-mail kind of handle this
> already? I suspect spammers send from random or fictitious addresses and
> don't check (or even receive) email sent to those addresses, so it's
> unlikely they'll be confirmed as subscribers to the mailing list.
>
> I think Greg raises an excellent point that we shouldn't reply to
> spammers. Perhaps we should just silently ignore posts from
> non-subscribers. If they are real humans, they should figure it out
> easily enough that they need to subscribe to post. Most mailing lists
> have a web page advertising the list, with instructions on how to
> subscribe to the list and list policy, such as needing to subscribe in
> order to post. I think that should be adequate to keep spam away but not
> real humans.
>
> Besides, don't we hate when people post something that clearly indicates
> they didn't RTFM? ;)
>
> --
> Prentice
>
>
> On 08/24/2012 06:47 AM, Andrew Holway wrote:
> > Maybe we should have a
> >
> > 'Please send us a brief email explaining why you would like to join
> > the list so that we know that your not a spammer"
> >
> >
> > 2012/8/24 Christopher Samuel<samuel at unimelb.edu.au>:
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> Hi all,
> >>
> >> I'd like to solicit opinions from list members on changing the policy
> >> of the list on messages from non-subscribers from the current default
> >> of holding them for moderation to the position of rejecting them (with
> >> an explanatory message).
> >>
> >> The reason is that currently us new list admins get many dozens of
> >> spams a day being held for moderation and it would be very hard to
> >> spot a legitimate message (from either a new subscriber or from a new
> >> member who is having messages held for moderation) amongst them.
> >>
> >> It seems fairer to me for legitimate non-members to get an email
> >> saying that their post has been rejected because they're not a member,
> >> but pointing them to the web page to subscribe, rather than just a
> >> message saying it's been held for moderation and then nothing (if it
> >> gets missed).
> >>
> >> It's also fairer to new members who are under moderation (as spammers
> >> do sign up to lists too) as their messages won't get missed in the
> >> general flood of non-member spam.
> >>
> >> This is how all the other Mailman lists I run are configured and it's
> >> not caused issues or complaints there.
> >>
> >> Thoughts please!
> >>
> >> cheers,
> >> Chris
> >> - --
> >>   Christopher Samuel        Senior Systems Administrator
> >>   VLSCI - Victorian Life Sciences Computation Initiative
> >>   Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
> >>   http://www.vlsci.org.au/      http://twitter.com/vlsci
> >>
> >> -----BEGIN PGP SIGNATURE-----
> >> Version: GnuPG v1.4.11 (GNU/Linux)
> >> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> >>
> >> iEYEARECAAYFAlA2034ACgkQO2KABBYQAh+bKwCePy8HxXFL/cXH3HCvmcyNZcPB
> >> kYsAn3pa2/pWewv0wJ9Ee7pAJessq3iR
> >> =RDNl
> >> -----END PGP SIGNATURE-----
> >> _______________________________________________
> >> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> Computing
> >> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
> ------------------------------
>
> Message: 3
> Date: Fri, 24 Aug 2012 15:56:53 +0200
> From: <graham.mullier at syngenta.com>
> Subject: Re: [Beowulf] RFC: Restrict posting to subscribers only ?
> To: <samuel at unimelb.edu.au>, <beowulf at beowulf.org>
> Message-ID:
>         <
> 3E91C69ADC46C4408C85991183A275F40BC8204827 at FRGOCMSXMB04.EAME.SYNGENTA.ORG>
>
> Content-Type: text/plain; charset="Windows-1252"
>
> Gentlethings
>
> While you're noting people who aren't spammers but who are currently
> locked out despite being subscribers, I'm another. I seem never to have
> been flagged as 'OK' so gradually (this is over some years) came to regard
> the list as an interesting thing to watch but not something to participate
> in.
>
> Chris, while it would be helpful for me to be flagged as OK to post (for
> me), what about the several or many others who are stuck in the same sort
> of limbo? Is there a way to identify them, open up the list, and breathe a
> bit more life into it?
>
> Regards
> Graham
>
> Graham Mullier | Head Information Connection & Design, R&DIS | R&D
> architecture CoE lead
> graham.mullier at syngenta.com | Syngenta, Jealott's Hill International
> Research Station, Bracknell, RG42 6EY, UK
>
>
>
> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On
> Behalf Of Christopher Samuel
> Sent: 24 August 2012 08:55
> To: beowulf at beowulf.org
> Subject: Re: [Beowulf] RFC: Restrict posting to subscribers only ?
>
> On 08/24/2012 10:11 PM, J?rg Sa?mannshausen wrote:
>
> > Hi all,
>
> Hiya!
>
> > I am one of the many who cannot post to the list despite being a member
> of it
> > for some years. :-/
>
> Well I spotted your message in the queue and let it through, so welcome!
>
> > I think the policy was that only members can post (good thing in my
> opinion)
> > and new members are moderated.
> > I am moderating an email list here as well and unfortunately the list's
> email
> > address got to the outside world. I do get some spam but it is something
> I can
> > cope with.
> > Once a new member has established themselves as not being a spammer (and
> here
> > I really mean the type of: buy vi online! I mean, I got vi installed on
> my
> > machines so why shall I buy it? :D ) you can remove the moderation flag.
> If
> > that member then becomes a spammer or troll you can them put back into
> > moderation only.
>
> That's precisely how the list works at the moment, in that new
> subscribers have posts moderated at first and that flag gets cleared
> after a while.  The only problem is that the list has been running on
> autopilot for quite a while (I don't know how long) and so people
> haven't been getting unmoderated or having posts approved.
>
> > Does that sound like a good idea? We do not have to rule out the usual
> anti-
> > spam mechanisms here.
>
> Agreed, we need defence in depth on this. :-)
>
> > My two cents from somebody who got put on the observer chair :D
>
> You are now unmoderated, so you can get off that chair now. ;-)
>
> cheers!
> Chris
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
>
> Syngenta Limited, Registered in England No 2710846
> Registered Office : Syngenta Limited, European Regional Centre, Priestley
> Road, Surrey Research Park, Guildford, Surrey, GU2 7YH, United Kingdom
>
>
>
> This message may contain confidential information. If you are not the
> designated recipient, please notify the sender immediately, and delete the
> original and any copies. Any use of the message by you is prohibited.
>
>
>
> ------------------------------
>
> Message: 4
> Date: Fri, 24 Aug 2012 09:30:56 -0700
> From: Jeff Johnson <jeff.johnson at aeoncomputing.com>
> Subject: Re: [Beowulf] RFC: Restrict posting to subscribers only ?
> To: beowulf at beowulf.org
> Message-ID: <5037AC40.3090703 at aeoncomputing.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> I think I've seen another list or site do that very thing.
>
> Oh wait, now I remember. It is inside the systemimager source somewhere.
>
>
> On 8/24/12 9:10 AM, Joe Landman wrote:
> > I am thinking we need a Monty Python Captcha
> >
> > "What is your name"
> >
> > "What is your quest"
> >
> > "What is the airspeed of an unladen swallow"
> >
> > ...
> >
>
> --
> ------------------------------
> Jeff Johnson
> Manager
> Aeon Computing
>
> jeff.johnson at aeoncomputing.com
> www.aeoncomputing.com
> t: 858-412-3810 x101   f: 858-412-3845
> m: 619-204-9061
>
> /* New Address */
> 4170 Morena Boulevard, Suite D - San Diego, CA 92117
>
>
>
> ------------------------------
>
> Message: 5
> Date: Fri, 24 Aug 2012 14:12:35 -0400
> From: Prentice Bisbal <prentice.bisbal at rutgers.edu>
> Subject: Re: [Beowulf] RFC: Restrict posting to subscribers only ?
> To: beowulf at beowulf.org
> Message-ID: <5037C413.1040407 at rutgers.edu>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> African or European?
>
>
> On 08/24/2012 12:10 PM, Joe Landman wrote:
> > I am thinking we need a Monty Python Captcha
> >
> > "What is your name"
> >
> > "What is your quest"
> >
> > "What is the airspeed of an unladen swallow"
> >
> > ...
> >
> >
> > On 08/24/2012 09:01 AM, Christopher Samuel wrote:
> >> On 08/24/2012 10:03 PM, Bogdan Costescu wrote:
> >>
> >>> Does someone have a figure of how many non-subscribers posts we are
> >>> talking about ? I have the impression that lately I've seen only
> >>> messages coming from "regulars". If non-subscribers post very little,
> >>> then such a change would not be very annoying IMHO.
> >> Well in the 6 hours since I last checked at work there have were over 30
> >> new messages waiting to be processed, of which 1 was from a subscriber
> >> still moderated and 1 was a legitimate post from a non-member
> >> (presumably subscribed at a different address as they replied to my list
> >> post).
> >>
> >> The other 30 plus were just spam.
> >>
> >>> OTOH, I hope that archives (wherever they are) remain publicly
> available.
> >> I don't think anyone is planning to change those at all.
> >>
> >> http://www.beowulf.org/pipermail/beowulf/
> >>
> >>> We should be very thankful to Don Becker for doing a wonderful job
> >>> with the list for many years :)
> >> Oh indeed!  Very much so...
> >>
> >> All the best,
> >> Chris
> >> _______________________________________________
> >> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> Computing
> >> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> >>
> >
>
>
> ------------------------------
>
> Message: 6
> Date: Sun, 26 Aug 2012 00:22:16 +0200
> From: holway at th.physik.uni-frankfurt.de
> Subject: [Beowulf] NFSoIB and SRP benchmarking
> To: beowulf at beowulf.org
> Message-ID:
>         <
> 59905dbf784579608a180b56bb8cb8a3.squirrel at th.physik.uni-frankfurt.de>
> Content-Type: text/plain;charset=utf-8
>
> Hello Beowulf 2.0
>
> I've just started playing with NFSoIB to provide super fast backend
> storage for a bunch of databases that we look after here. Oracle and
> Nexenta are both sending me ZFS based boxes to test and I hope to compare
> the performance and stability of these with the Netapp (formally lsi
> engenio) E5400.
>
> This will be the first time I will be getting into serious storage
> benchmarking. Does anyone have any interesting tests they would like to
> run or and experience performing these kinds of tests?
>
> I have 4 HP G8 boxes with 96GB ram each QDR IB and 10G ethernet as
> consumers.
>
> I will also be testing the performance of KVM and Virtuozzo (commercial
> version of OpenVZ) which is a kernel sharing virtualization similar to BSD
> Jails.
>
> Thanks,
>
> Andrew
>
>
>
> ------------------------------
>
> Message: 7
> Date: Thu, 23 Aug 2012 15:21:20 +0000
> From: "Chen, Yong" <yong.chen at ttu.edu>
> Subject: [Beowulf] [hpc-announce] CFP: DISCS-2012 Workshop at SC'12
>         with Cluster Computing journal special issue
> To: "hpc-announce at mcs.anl.gov" <hpc-announce at mcs.anl.gov>,
>         "distributed-computing-announce at datasys.cs.iit.edu"
>         <distributed-computing-announce at datasys.cs.iit.edu>,
>         "cse-cfp at cse.stfx.ca"   <cse-cfp at cse.stfx.ca>, "
> cfp at grid.chu.edu.tw"
>         <cfp at grid.chu.edu.tw>
> Message-ID:
>         <AC0D521357876C4AA01C144CD17C58D9012896B7 at centaur09.ttu.edu>
> Content-Type: text/plain; charset="us-ascii"
>
> [Apologies if you got multiple copies of this email. If you'd like to
> opt out of these announcements, information on how to unsubscribe is
> available at the bottom of this email.]
> -------------- next part --------------
>
> [Apologize for any duplicated posts]
>
>                  ****************
>                  CALL FOR PAPERS
>                  ****************
>           The International Workshop on
> Data-Intensive Scalable Computing Systems (DISCS)
>           http://data.cs.ttu.edu/discs/
>
>           November 16th, 2012, Salt Lake City
>               In conjunction with the
>  2012 ACM/IEEE Supercomputing Conference (SC'12)
>
>                  *********************
>                  Scope of the Workshop
>                  *********************
> High-performance computing (HPC) is a major strategic tool
> for science, engineering, and industry. Existing HPC systems,
> however, are largely designed and developed for computation-
> intensive applications with a computing-centric paradigm. With
> the emerging and timely needs of supporting data-intensive
> scientific discovery and innovations, there is a need of
> rethinking the system architectures, programming models, runtime
> systems, and tools available for data-intensive HPC. The DISCS
> workshop provides a forum for researchers and developers in the
> high-performance computing, data-intensive computing, and
> parallel computing fields to take the Big Data challenges
> together and present innovative ideas, experiences, and latest
> developments that help address these challenges.
>
>                    ******************
>                    Topics of Interest
>                    ******************
> The focus areas of the DISCS-2012 workshop include, but are not
> limited to:
>     -> HPC system architectures for supporting data intensive
>        applications
>          # Data-centric system architectures
>          # I/O systems and architectures
>          # System area networks
>          # Power efficient systems for data intensive applications
>     -> Programming models for supporting data intensive
>        applications
>          # Data-centric programming models
>          # MPI extensions for data intensive applications
>          # GAS/PGAS/OpenMP programming model extensions for data-
>            intensive applications
>          # New programming languages/methodologies for data-
>            intensive HPC
>     -> Runtime systems for supporting data intensive applications
>          # Communication systems for supporting data intensive
>            applications
>          # Data compression and deduplication
>          # Caching and prefetching
>          # Data integrity and consistency
>     -> Tools for supporting data intensive applications
>          # Data analytic, tracing, and trace analysis tools
>          # Data mining and knowledge discovery tools
>          # Computational, mathematical and statistical techniques
>            and tools
>          # Data visualization techniques and tools
>
>                       ***************
>                       Important Dates
>                       ***************
>     -> Paper Submission: September 8th, 2012
>     -> Author Notification: October 22nd, 2012
>     -> Workshop Date: November 16th, 2012
>     -> Camera-Ready Copy: December 7th, 2012
>
>                    ***********************
>                    Submission Instructions
>                    ***********************
> Submissions should be in PDF format in U.S. letter size paper,
> and should be formatted according to the IEEE standard double-
> column single-space format. They should not exceed 5 pages (all-
> inclusive). Please access the IEEE site for the manuscript template.
>
> Please visit the following site to make paper submissions:
> https://www.easychair.org/account/signin.cgi?conf=discs2012
>
> All accepted papers will appear in the SC-2012 digital proceedings
> in the IEEE digital library.
>
>                    *********************
>                    Journal Special Issue
>                    *********************
> DISCS-2012 workshop papers will be invited to extend the manuscripts
> to be considered for a journal Special Issue on Data-Intensive High
> Performance Computing of the Cluster Computing by Springer, edited
> by Yong Chen and Xian-He Sun. The target journal special issue
> submission deadline is Jan. 16th, 2013.
>
>                      ***********
>                      Organizers
>                      ***********
> STEERING COMMITTEE
> William D. Gropp, University of Illinois Urbana-Champaign
> Xian-He Sun, Illinois Institute of Technology
> Rajeev Thakur, Argonne National Laboratory
>
> PROGRAM CHAIRS
> Yong Chen, Texas Tech University
> Xian-He Sun, Illinois Institute of Technology
>
> TECHNICAL PROGRAM COMMITTEE
> Suren Byna, Lawrence Berkeley National Laboratory
> Shane Canon, Lawrence Berkeley National Laboratory
> Hui Jin, Oracle
> Scott Klasky, Oak Ridge National Laboratory
> Quincey Koziol, HDF5 Group
> Wei-keng Liao, Northwestern University
> Jay Lofstead, Sandia National Laboratory
> Carlos Maltzahn, University of California at Santa Cruz
> Ioan Raicu, Illinois Institute of Technology
> Philip C. Roth, Oak Ridge National Laboratory
> Weikuan Yu, Auburn University
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://www.beowulf.org/pipermail/beowulf/attachments/20120823/d32dae69/attachment.html
> -------------- next part --------------
>
> ********************************************************************************
> The hpc-announce mailing list has been setup to have a common mailing
> list to share information with respect to upcoming HPC related
> events. You are included in this mailing list based on your
> participation or interest in a previous HPC conference or other event.
>
> The purpose for providing a single mailing list is to allow
> participants to easily identify such emails, and handle them
> appropriately. Some options include:
>
> 1. If you feel that the number of such emails is too many, filter them
> to less-frequently-read folders in your email client.
>
> 2. Change your subscription to a digest option
> (https://lists.mcs.anl.gov/mailman/listinfo/hpc-announce) which will
> consolidate emails sent that week into a single summary email.
>
> 3. Finally, if you do not wish to receive any emails from hpc-announce,
> you can unsubscribe from the mailing list
> (https://lists.mcs.anl.gov/mailman/listinfo/hpc-announce). Once
> unsubscribed, we guarantee that you will not be added back in through
> participation in a different HPC related conference or event. You will
> need to send an email to hpc-announce-owner at mcs.anl.gov to be added
> back on.
>
> If you need help with any of the above, please email us at
> hpc-announce-owner at mcs.anl.gov.
>
> ********************************************************************************
>
> ------------------------------
>
> _______________________________________________
> Beowulf mailing list
> Beowulf at beowulf.org
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
> End of Beowulf Digest, Vol 102, Issue 19
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120826/13f7e9c6/attachment.html>

From hustkjh at 126.com  Sun Aug 26 07:32:46 2012
From: hustkjh at 126.com (HUSTKJH)
Date: Sun, 26 Aug 2012 22:32:46 +0800 (CST)
Subject: [Beowulf] seamless failover
Message-ID: <6c2e3554.cde1.139635962d6.Coremail.hustkjh@126.com>

 Hi, all!
I am desiging a HA system for VoIP server cluster now.
My system should support fault tolerance with seamless failover feature.
I first begin with 2 nodes, and using Linux HA heartbeat package.
A lot of existing HA clusters are based on Linux HA heartbeat, but it doesn't support seamless failover/failback at present.
A lot of academic research papers address this problem (kernel-level, log-based...), but not in practice,and complicated and limited in the realization.
Linux HA is somewhat simple and easy-to-use, but don't know how to extend and change to support seamless failover.
Does anyone have related experience or good ideas...?
Or, are there any other open-source s/w for this?

Thanks.
John Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120826/c36e7e1d/attachment.html>

From reuti at staff.uni-marburg.de  Sun Aug 26 15:02:40 2012
From: reuti at staff.uni-marburg.de (Reuti)
Date: Mon, 27 Aug 2012 00:02:40 +0200
Subject: [Beowulf] seamless failover
In-Reply-To: <6c2e3554.cde1.139635962d6.Coremail.hustkjh@126.com>
References: <6c2e3554.cde1.139635962d6.Coremail.hustkjh@126.com>
Message-ID: <20DAE2AF-F5EB-4B7A-80DC-0F94ECF54DCF@staff.uni-marburg.de>

Hi,

Am 26.08.2012 um 16:32 schrieb HUSTKJH:

>  Hi, all!
> I am desiging a HA system for VoIP server cluster now.
> My system should support fault tolerance with seamless failover feature.
> I first begin with 2 nodes, and using Linux HA heartbeat package.
> A lot of existing HA clusters are based on Linux HA heartbeat, but it doesn't support seamless failover/failback at present. 
> A lot of academic research papers address this problem (kernel-level, log-based...), but not in practice,and complicated and limited in the realization.
> Linux HA is somewhat simple and easy-to-use, but don't know how to extend and change to support seamless failover.
> Does anyone have related experience or good ideas...?

Depends on the budget and importance of application. E.g. NEC offers fault tolerant servers where the HA is in mirrored hardware.

http://www.necam.com/servers/ft/

-- Reuti


> Or, are there any other open-source s/w for this?
> 
> Thanks.
> John Peter
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



From samuel at unimelb.edu.au  Sun Aug 26 17:22:26 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Mon, 27 Aug 2012 10:22:26 +1000
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <CAHdpx62=NZHyxVV-RS0MBbWKDTb+GqVYu1nhuEg+zk3RYNNj6w@mail.gmail.com>
References: <CC5CCF3B.2466D%james.p.lux@jpl.nasa.gov>
	<50378424.6060000@scalableinformatics.com>
	<CAHdpx62=NZHyxVV-RS0MBbWKDTb+GqVYu1nhuEg+zk3RYNNj6w@mail.gmail.com>
Message-ID: <503ABDC2.7000206@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 26/08/12 16:59, Jonathan Aquilina wrote:

> Hey guys I am looking at the admin interface, and it seems like
> there is a way to set moderators to recieve emails would that be
> something we want.

Not until we've disabled non-subscriber posts.  I've no desire to get
another 100 emails or so a day of notifications about spam.

> Also on another note something we could setup which would be easier
> that anyone can manage would be untangle security distro. It has a
> web based interface and all.

Please no.  Let's let Penguin run their servers the way that they wish
to.  We can offer to help with spam filtering configuration (which Joe
has already done) but that doesn't need a whole distribution change.

All the best,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlA6vcIACgkQO2KABBYQAh9ABwCbBDFWSAT8FMjpRX4qxiRu29Bc
xUEAnjZ2CkApCNQ4+yXROjbu6GdfHbyC
=by7x
-----END PGP SIGNATURE-----


From landman at scalableinformatics.com  Sun Aug 26 18:11:21 2012
From: landman at scalableinformatics.com (Joe Landman)
Date: Sun, 26 Aug 2012 21:11:21 -0400
Subject: [Beowulf] RFC: Restrict posting to subscribers only ?
In-Reply-To: <503ABDC2.7000206@unimelb.edu.au>
References: <CC5CCF3B.2466D%james.p.lux@jpl.nasa.gov>
	<50378424.6060000@scalableinformatics.com>
	<CAHdpx62=NZHyxVV-RS0MBbWKDTb+GqVYu1nhuEg+zk3RYNNj6w@mail.gmail.com>
	<503ABDC2.7000206@unimelb.edu.au>
Message-ID: <503AC939.5060204@scalableinformatics.com>

On 08/26/2012 08:22 PM, Christopher Samuel wrote:

>> Also on another note something we could setup which would be easier
>> that anyone can manage would be untangle security distro. It has a
>> web based interface and all.
>
> Please no.  Let's let Penguin run their servers the way that they wish
> to.  We can offer to help with spam filtering configuration (which Joe
> has already done) but that doesn't need a whole distribution change.

Absolutely.  We'd be happy to help set this up as a service in POD (or 
in a VM if they need, or running as a pipeline here, ... etc.)

Minimal changes are recommended.  If they use procmail or similar for 
their mail, we can inject most of the pipeline there.  I did re-work it 
for an updated postfix (circa 2.9).

BTW:  I still don't have admin access.  I'll bug Arend tomorrow.  I am 
guessing my email got lost in the interwebs ...  (sadly since the switch 
of our servers to EC2 for mail, this happens much more).

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615



From samuel at unimelb.edu.au  Sun Aug 26 19:11:36 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Mon, 27 Aug 2012 12:11:36 +1000
Subject: [Beowulf] Beowulf list now rejecting emails from non-subscribers
In-Reply-To: <5036D37E.5030800@unimelb.edu.au>
References: <5036D37E.5030800@unimelb.edu.au>
Message-ID: <503AD758.803@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 24/08/12 11:06, Christopher Samuel wrote:

> I'd like to solicit opinions from list members on changing the
> policy of the list on messages from non-subscribers from the
> current default of holding them for moderation to the position of
> rejecting them (with an explanatory message).

Thank you *all* for your valuable feedback, it was very much appreciated.

I have now set the list to reject non-subscriber postings now as the
feedback has been overwhelmingly for that, with the only reservation
being that the list would be a source of back-scatter.

However, the list is already a source of backscatter, we're just
replacing the message saying "Your email has been moderated" with one
saying "Your email has been rejected". :-)

If you post to the list with a different email address to that which
you receive email through (as I do, to avoid having to deal with
Exchange) then you can subscribe your other address to the list and
set it to "no mail" once it is added.   That will permit you to post
and stop you receiving duplicate emails.

We now return you to your regularly scheduled Beowulf coverage.. ;-)

All the best,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlA611gACgkQO2KABBYQAh9bhQCdE6coozp78P2a+Y77/p1m2YXU
gU0AnRXPHHvcEqGjPojTxEhd3SUmtKs0
=urRO
-----END PGP SIGNATURE-----


From james.p.lux at jpl.nasa.gov  Sun Aug 26 22:00:53 2012
From: james.p.lux at jpl.nasa.gov (Lux, Jim (337C))
Date: Mon, 27 Aug 2012 05:00:53 +0000
Subject: [Beowulf] Beowulf list now rejecting emails from non-subscribers
In-Reply-To: <503AD758.803@unimelb.edu.au>
Message-ID: <CC604BA2.247A5%james.p.lux@jpl.nasa.gov>

And here I was looking for a discussion of using a Beowulf to model the
airflow over a swallow (or pair of swallows) carrying a coconut, thereby
creating a fascinating loop of using a Beowulf to answer the proposed
challenge question used for authentication. I leave discussion of coconut
migration, origin of swallow species, and other such questions to others.

Half seriously, I have seen some papers on doing CFD of the flows between
fins of adjacent fish or penguins (I can't recall which, off hand).

And as far as coconut distribution about the world, I've used a cluster to
do a lagrangian model for floating objects (with small propulsion forces)
using a database of ocean currents and winds. Others have done so for
objects as diverse as tennis shoes, floating bottles, rubber ducks, etc.,
although interestingly, I don't recall having seen coconuts in the
literature.





On 8/26/12 7:11 PM, "Christopher Samuel" <samuel at unimelb.edu.au> wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On 24/08/12 11:06, Christopher Samuel wrote:
>
>> I'd like to solicit opinions from list members on changing the
>> policy of the list on messages from non-subscribers from the
>> current default of holding them for moderation to the position of
>> rejecting them (with an explanatory message).
>
>Thank you *all* for your valuable feedback, it was very much appreciated.
>
>I have now set the list to reject non-subscriber postings now as the
>feedback has been overwhelmingly for that, with the only reservation
>being that the list would be a source of back-scatter.
>
>However, the list is already a source of backscatter, we're just
>replacing the message saying "Your email has been moderated" with one
>saying "Your email has been rejected". :-)
>
>If you post to the list with a different email address to that which
>you receive email through (as I do, to avoid having to deal with
>Exchange) then you can subscribe your other address to the list and
>set it to "no mail" once it is added.   That will permit you to post
>and stop you receiving duplicate emails.
>
>We now return you to your regularly scheduled Beowulf coverage.. ;-)
>
>All the best,
>Chris
>- -- 
> Christopher Samuel        Senior Systems Administrator
> VLSCI - Victorian Life Sciences Computation Initiative
> Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
> http://www.vlsci.org.au/      http://twitter.com/vlsci
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.11 (GNU/Linux)
>Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
>iEYEARECAAYFAlA611gACgkQO2KABBYQAh9bhQCdE6coozp78P2a+Y77/p1m2YXU
>gU0AnRXPHHvcEqGjPojTxEhd3SUmtKs0
>=urRO
>-----END PGP SIGNATURE-----
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>To change your subscription (digest mode or unsubscribe) visit
>http://www.beowulf.org/mailman/listinfo/beowulf



From diep at xs4all.nl  Mon Aug 27 04:31:00 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Mon, 27 Aug 2012 13:31:00 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
Message-ID: <A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>

Yeah it will be a great machines. You get what you pay for of course.
Confirming that is not even interesting. If you pay big bucks  
obviously it'll be fast.

For such machines, just break the raid card during operation, then  
replace it with another
card and see whether your data still is there and whether it can work  
without too much problems
within the crucial 2 minutes that you've got normally at exchanges to  
get new equipment in
(not counting rebuild time), or you're fired.

So for rich financials i bet such machines are attractive.

As for HPC, scaling is important.

As for cheap scaling....

If i search for benchmarks of raid controllers, you can find  
everything as well.

What really misses is benchmarks of built in raid controllers.

If you want to scale cheap it's interesting to know how to do the i/o  
obviously.

The motherboards i've got have the ESB2 from intel built in as RAID  
controller.

If i google i see nothing on expected read/write speeds.

Nearly all raid controllers you always see have a limit of 1 GB/s,  
that really sucks
compared to even the bandwidth one can generate to the RAM.

Basically 1 GB/s with 8 cores Xeon 2.5Ghz means you've got 8 * 2.5Ghz  
= 20k cycles a byte, or 32k instructions you can
execute for reading or writing just 1 byte.

Most raid controllers that are cheap, say a 100-300 euro (130 - 400  
dollar), they're not managing more than a
write speed of a 600MB/s , that's in fact in raid-0, not really  
realistic, and 700-900MB/s read speed.

That's with around a drive or 8.

We know however that when the raid array is empty, those drives will  
handsdown get a write speed at outside of drives of
a 130MB/s, so the limitation is the raid cards CPU itself. Usually  
around a 800Mhz at the better cards.

Many benchmark with SAS drives. I see some fujitsu drives that are  
rated 188MB/s at outer side of drives, which benchmark to
900MB/s readspeed.

The limitation seems to be the raidcards in most cases.

So my simple question would be: is this better with the built in raid  
controllers that most motherboards have as they can use
the fast 2.5Ghz Xeon cpu's?

On Aug 26, 2012, at 12:22 AM, holway at th.physik.uni-frankfurt.de wrote:

> Hello Beowulf 2.0
>
> I've just started playing with NFSoIB to provide super fast backend
> storage for a bunch of databases that we look after here. Oracle and
> Nexenta are both sending me ZFS based boxes to test and I hope to  
> compare
> the performance and stability of these with the Netapp (formally lsi
> engenio) E5400.
>
> This will be the first time I will be getting into serious storage
> benchmarking. Does anyone have any interesting tests they would  
> like to
> run or and experience performing these kinds of tests?
>
> I have 4 HP G8 boxes with 96GB ram each QDR IB and 10G ethernet as  
> consumers.
>
> I will also be testing the performance of KVM and Virtuozzo  
> (commercial
> version of OpenVZ) which is a kernel sharing virtualization similar  
> to BSD
> Jails.
>
> Thanks,
>
> Andrew
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From diep at xs4all.nl  Mon Aug 27 04:38:20 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Mon, 27 Aug 2012 13:38:20 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
	<A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
Message-ID: <6034660F-9F1F-4540-B8D6-BEFCC6DCFAAF@xs4all.nl>


On Aug 27, 2012, at 1:31 PM, Vincent Diepeveen wrote:

> Yeah it will be a great machines. You get what you pay for of course.
> Confirming that is not even interesting. If you pay big bucks  
> obviously it'll be fast.
>
> For such machines, just break the raid card during operation, then  
> replace it with another
> card and see whether your data still is there and whether it can  
> work without too much problems
> within the crucial 2 minutes that you've got normally at exchanges  
> to get new equipment in
> (not counting rebuild time), or you're fired.
>
> So for rich financials i bet such machines are attractive.
>
> As for HPC, scaling is important.
>
> As for cheap scaling....
>
> If i search for benchmarks of raid controllers, you can find  
> everything as well.
>
> What really misses is benchmarks of built in raid controllers.
>
> If you want to scale cheap it's interesting to know how to do the i/ 
> o obviously.
>
> The motherboards i've got have the ESB2 from intel built in as RAID  
> controller.
>
> If i google i see nothing on expected read/write speeds.
>
> Nearly all raid controllers you always see have a limit of 1 GB/s,  
> that really sucks
> compared to even the bandwidth one can generate to the RAM.
>
> Basically 1 GB/s with 8 cores Xeon 2.5Ghz means you've got 8 *  
> 2.5Ghz = 20k cycles a byte, or 32k instructions you can
> execute for reading or writing just 1 byte.

Oh dear calculations on monday... ...make that 20Ghz * 4 instructions  
per cycle / 1 GB = 80 cycles a byte...

Or for each 'bitboard' which is 64 bits, so similar to a double, if  
you read and write it.

You read 2 and write 1.

That's 24 bytes of i/o.

So that's 80 cycles * 24 = 1920 instructions. I didn't factor in  
SSE4.2 yet...

For i/o intensive applications bandwidth is interesting to have to  
the i/o. I don't see how file servers deliver that for clusters.


>
> Most raid controllers that are cheap, say a 100-300 euro (130 - 400  
> dollar), they're not managing more than a
> write speed of a 600MB/s , that's in fact in raid-0, not really  
> realistic, and 700-900MB/s read speed.
>
> That's with around a drive or 8.
>
> We know however that when the raid array is empty, those drives  
> will handsdown get a write speed at outside of drives of
> a 130MB/s, so the limitation is the raid cards CPU itself. Usually  
> around a 800Mhz at the better cards.
>
> Many benchmark with SAS drives. I see some fujitsu drives that are  
> rated 188MB/s at outer side of drives, which benchmark to
> 900MB/s readspeed.
>
> The limitation seems to be the raidcards in most cases.
>
> So my simple question would be: is this better with the built in  
> raid controllers that most motherboards have as they can use
> the fast 2.5Ghz Xeon cpu's?
>
> On Aug 26, 2012, at 12:22 AM, holway at th.physik.uni-frankfurt.de wrote:
>
>> Hello Beowulf 2.0
>>
>> I've just started playing with NFSoIB to provide super fast backend
>> storage for a bunch of databases that we look after here. Oracle and
>> Nexenta are both sending me ZFS based boxes to test and I hope to  
>> compare
>> the performance and stability of these with the Netapp (formally lsi
>> engenio) E5400.
>>
>> This will be the first time I will be getting into serious storage
>> benchmarking. Does anyone have any interesting tests they would  
>> like to
>> run or and experience performing these kinds of tests?
>>
>> I have 4 HP G8 boxes with 96GB ram each QDR IB and 10G ethernet as  
>> consumers.
>>
>> I will also be testing the performance of KVM and Virtuozzo  
>> (commercial
>> version of OpenVZ) which is a kernel sharing virtualization  
>> similar to BSD
>> Jails.
>>
>> Thanks,
>>
>> Andrew
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit  
>> http://www.beowulf.org/mailman/listinfo/beowulf
>



From holway at th.physik.uni-frankfurt.de  Mon Aug 27 06:55:14 2012
From: holway at th.physik.uni-frankfurt.de (holway at th.physik.uni-frankfurt.de)
Date: Mon, 27 Aug 2012 15:55:14 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
	<A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
Message-ID: <034fa87f8bb9961eaacbe3e85a600bea.squirrel@th.physik.uni-frankfurt.de>


> So my simple question would be: is this better with the built in raid
> controllers that most motherboards have as they can use
> the fast 2.5Ghz Xeon cpu's?

Im not going to be doing any hardware RAID. ZFS does everything in software.







From diep at xs4all.nl  Mon Aug 27 07:12:48 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Mon, 27 Aug 2012 16:12:48 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <034fa87f8bb9961eaacbe3e85a600bea.squirrel@th.physik.uni-frankfurt.de>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
	<A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
	<034fa87f8bb9961eaacbe3e85a600bea.squirrel@th.physik.uni-frankfurt.de>
Message-ID: <8F1D7854-168C-4F35-8494-93474EEDBFB9@xs4all.nl>

Do the machines have built in controller in the motherboard,
or do they have a pci-e card that's hosting the disks?

 From those pci-e cards there is plenty of benchmarks around, and  
basically
they all struggle to get nearby 1GB/s bandwidth.

 From the motherboards i see 0 benchmarks putting disks on them and  
measuring aggregated bandwidth.

Which chip is on the motherboard doing that and how many disks can it  
host?

On Aug 27, 2012, at 3:55 PM, holway at th.physik.uni-frankfurt.de wrote:

>
>> So my simple question would be: is this better with the built in raid
>> controllers that most motherboards have as they can use
>> the fast 2.5Ghz Xeon cpu's?
>
> Im not going to be doing any hardware RAID. ZFS does everything in  
> software.
>
>
>
>
>



From holway at th.physik.uni-frankfurt.de  Mon Aug 27 08:16:04 2012
From: holway at th.physik.uni-frankfurt.de (holway at th.physik.uni-frankfurt.de)
Date: Mon, 27 Aug 2012 17:16:04 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <8F1D7854-168C-4F35-8494-93474EEDBFB9@xs4all.nl>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
	<A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
	<034fa87f8bb9961eaacbe3e85a600bea.squirrel@th.physik.uni-frankfurt.de>
	<8F1D7854-168C-4F35-8494-93474EEDBFB9@xs4all.nl>
Message-ID: <3404574f22488d6858cf78fc283d8581.squirrel@th.physik.uni-frankfurt.de>

> Do the machines have built in controller in the motherboard,
> or do they have a pci-e card that's hosting the disks?

We shall probably use a few of the MegaRAID 9280 controllers in
passthrough mode. These are capable of handling 2 or 3 GB/s.

It depends on what the vendor supports really.



>
>  From those pci-e cards there is plenty of benchmarks around, and
> basically
> they all struggle to get nearby 1GB/s bandwidth.
>
>  From the motherboards i see 0 benchmarks putting disks on them and
> measuring aggregated bandwidth.
>
> Which chip is on the motherboard doing that and how many disks can it
> host?
>
> On Aug 27, 2012, at 3:55 PM, holway at th.physik.uni-frankfurt.de wrote:
>
>>
>>> So my simple question would be: is this better with the built in raid
>>> controllers that most motherboards have as they can use
>>> the fast 2.5Ghz Xeon cpu's?
>>
>> Im not going to be doing any hardware RAID. ZFS does everything in
>> software.
>>
>>
>>
>>
>>
>
>



From diep at xs4all.nl  Mon Aug 27 10:13:21 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Mon, 27 Aug 2012 19:13:21 +0200
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <CAEiui-tJbZ6-zLy07LXve=g6OaAGU_xhCD3GDEWVEzNz18jcpg@mail.gmail.com>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
	<A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
	<034fa87f8bb9961eaacbe3e85a600bea.squirrel@th.physik.uni-frankfurt.de>
	<8F1D7854-168C-4F35-8494-93474EEDBFB9@xs4all.nl>
	<CAEiui-tJbZ6-zLy07LXve=g6OaAGU_xhCD3GDEWVEzNz18jcpg@mail.gmail.com>
Message-ID: <531047AD-F2D9-455A-A30D-00B5EDCDF8DF@xs4all.nl>


On Aug 27, 2012, at 4:50 PM, Andrew Holway wrote:

>> Which chip is on the motherboard doing that and how many disks can it
>> host?
>
> We shall probably use a few of the MegaRAID 9280 controllers in
> passthrough mode. These are easily capable of handling 2 or 3 GB/s.
>

With a few cards in it will be 2GB/s for readspeed. Which isn't bad.

With just 1 at the benchmark it achieves with 16 SSD's the 3GB/s  
readspeed though.
Which is pretty impressive.

The problem of course is the price of the drives there, 16 SSD's, if  
you want to store something on
them that is, that's not cheap :)

Cheap on ebay now are 36GB and 73GB U320 disks, yet i didn't see a  
cheap raid controller that can have a ton
of them AND get to 2GB/s or so.

My struggle is to get to a 60 TB raid array though, though seems i  
have to wait a few years for that :)

>>
>> On Aug 27, 2012, at 3:55 PM, holway at th.physik.uni-frankfurt.de wrote:
>>
>>>
>>>> So my simple question would be: is this better with the built in  
>>>> raid
>>>> controllers that most motherboards have as they can use
>>>> the fast 2.5Ghz Xeon cpu's?
>>>
>>> Im not going to be doing any hardware RAID. ZFS does everything in
>>> software.
>>>
>>>
>>>
>>>
>>>
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit  
>> http://www.beowulf.org/mailman/listinfo/beowulf



From john.hearns at mclaren.com  Tue Aug 28 02:40:42 2012
From: john.hearns at mclaren.com (Hearns, John)
Date: Tue, 28 Aug 2012 09:40:42 +0000
Subject: [Beowulf] NFSoIB and SRP benchmarking
In-Reply-To: <A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
References: <59905dbf784579608a180b56bb8cb8a3.squirrel@th.physik.uni-frankfurt.de>
	<A6C519AA-7B5A-4BDA-924A-68F2EA58C798@xs4all.nl>
Message-ID: <2791E54A864A8C4482B3F5F80EC59DF002D8A1@MGL-PWEXCH01.mgl.tagmclarengroup.com>



-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Vincent Diepeveen
Sent: 27 August 2012 12:31
To: holway at th.physik.uni-frankfurt.de
Cc: beowulf at beowulf.org
Subject: Re: [Beowulf] NFSoIB and SRP benchmarking

Yeah it will be a great machines. You get what you pay for of course.
Confirming that is not even interesting. If you pay big bucks  
obviously it'll be fast.

For such machines, just break the raid card during operation, then  
replace it with another
card and see whether your data still is there and whether it can work  
without too much problems
within the crucial 2 minutes that you've got normally at exchanges to  
get new equipment in
(not counting rebuild time), or you're fired.


I'm not really sure what you're referring to here.
In an environment where access to storage arrays is that critical you would have multiple controllers and more
Then one fibrechannel card per host, and use multipathing.
And a dual fabric of fibrechannel switches plus trunked links between them.

You wouldn't be replying on any one piece of hardware which would "have to be changed within two minutes".

That's not saying that downtime is impossible in a situation like that - just that you don't rely on one RAID card.



The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.


From rwindsor at alphalink.com.au  Tue Aug 28 04:00:33 2012
From: rwindsor at alphalink.com.au (Richard Windsor)
Date: Tue, 28 Aug 2012 21:00:33 +1000
Subject: [Beowulf] access to paper
Message-ID: <AF7686E55228444F80F3F7CD192F5A2E@ANDY>


Does anyone have a copy of this paper that they could pass on?
Regards
R

A Case of Parallel EEG Data Processing upon a Beowulf Cluster
ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5395346
by Y Yao - 2009 - Related articles
Electroencephalogram (EEG) data processing applications have become routine tasks in both bioscience and neuroscience research, which are usually highly ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120828/73128ffb/attachment.html>

From duke.lists at gmx.com  Wed Aug 29 02:57:15 2012
From: duke.lists at gmx.com (Duke Nguyen)
Date: Wed, 29 Aug 2012 16:57:15 +0700
Subject: [Beowulf] SuperMicro X7DWT-INF board - what connector? what cable?
Message-ID: <503DE77B.5090401@gmx.com>

Hi folks,

We plan to build a cluster using server built with SuperMicro X7DWT-INF 
motherboards. Unfortunately google does not give us much information 
about the type of connector that X7DWT-INF has.

Anybody using this motherboard in their cluster? Please give us advice 
about which connector the board has and what infiniband cable we should 
acquire to fit in the rack-mount servers built with these boards.

Thanks,

D.


From djholm at fnal.gov  Wed Aug 29 04:49:18 2012
From: djholm at fnal.gov (Donald J Holmgren)
Date: Wed, 29 Aug 2012 11:49:18 +0000
Subject: [Beowulf] SuperMicro X7DWT-INF board - what connector? what
 cable?
In-Reply-To: <503DE77B.5090401@gmx.com>
References: <503DE77B.5090401@gmx.com>
Message-ID: <1EC63CA9-C7D7-41B6-8C1C-7E8843D98216@fnal.gov>

We use the AMD version of this motherboard ("TwinU").  The landed Mellanox MT25204 HCA has a 4x microGiGaCN connector at the back of motherboard.  These are also often called CX4 connectors. 

Depending upon the switch you will connect the systems to, you need either microGiGaCN to microGiGaCN cables, or microGiGaCN to QSFP cables. Colfaxdirect.com lists many of each type. 

Supermicro.com has detailed specs and the manual for this motherboard. 

Don 



On Aug 29, 2012, at 5:05 AM, "Duke Nguyen" <duke.lists at gmx.com> wrote:

> Hi folks,
> 
> We plan to build a cluster using server built with SuperMicro X7DWT-INF 
> motherboards. Unfortunately google does not give us much information 
> about the type of connector that X7DWT-INF has.
> 
> Anybody using this motherboard in their cluster? Please give us advice 
> about which connector the board has and what infiniband cable we should 
> acquire to fit in the rack-mount servers built with these boards.
> 
> Thanks,
> 
> D.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From john.hearns at mclaren.com  Wed Aug 29 03:16:38 2012
From: john.hearns at mclaren.com (Hearns, John)
Date: Wed, 29 Aug 2012 10:16:38 +0000
Subject: [Beowulf] SuperMicro X7DWT-INF board - what connector? what
 cable?
In-Reply-To: <503DE77B.5090401@gmx.com>
References: <503DE77B.5090401@gmx.com>
Message-ID: <2791E54A864A8C4482B3F5F80EC59DF002E49A@MGL-PWEXCH01.mgl.tagmclarengroup.com>

Anybody using this motherboard in their cluster? Please give us advice 
about which connector the board has and what infiniband cable we should 
acquire to fit in the rack-mount servers built with these boards.


Should be a copper CX4 cable.

However, why not ask whoever is selling you these servers to give you one to test?

The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.


From holway at th.physik.uni-frankfurt.de  Wed Aug 29 08:26:44 2012
From: holway at th.physik.uni-frankfurt.de (holway at th.physik.uni-frankfurt.de)
Date: Wed, 29 Aug 2012 17:26:44 +0200
Subject: [Beowulf] SuperMicro X7DWT-INF board - what connector? what
 cable?
In-Reply-To: <2791E54A864A8C4482B3F5F80EC59DF002E49A@MGL-PWEXCH01.mgl.tagmclarengroup.com>
References: <503DE77B.5090401@gmx.com>
	<2791E54A864A8C4482B3F5F80EC59DF002E49A@MGL-PWEXCH01.mgl.tagmclarengroup.com>
Message-ID: <1b0c4246fa183187b2bc35b92ca0a386.squirrel@th.physik.uni-frankfurt.de>

> Anybody using this motherboard in their cluster? Please give us advice
> about which connector the board has and what infiniband cable we should
> acquire to fit in the rack-mount servers built with these boards.
>
>
> Should be a copper CX4 cable.

Yes. It is InfiniHost? III

>
> However, why not ask whoever is selling you these servers to give you one
> to test?
>
> The contents of this email are confidential and for the exclusive use of
> the intended recipient.  If you receive this email in error you should not
> copy it, retransmit it, use it or disclose its contents but should return
> it to the sender immediately and delete your copy.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



From djholm at fnal.gov  Wed Aug 29 07:56:14 2012
From: djholm at fnal.gov (Don Holmgren)
Date: Wed, 29 Aug 2012 09:56:14 -0500
Subject: [Beowulf] SuperMicro X7DWT-INF board - what connector? what
 cable?
In-Reply-To: <503E0AD1.1000006@gmx.com>
References: <503DE77B.5090401@gmx.com>
	<1EC63CA9-C7D7-41B6-8C1C-7E8843D98216@fnal.gov>
	<503E0AD1.1000006@gmx.com>
Message-ID: <alpine.LRH.2.00.1208290955320.14319@jpsi2.fnal.gov>


Yes, that's right, CX4 to CX4.

Don


On Wed, 29 Aug 2012, Duke Nguyen wrote:

> Hi Don,
>
> On 8/29/12 6:49 PM, Donald J Holmgren wrote:
>> We use the AMD version of this motherboard ("TwinU").  The landed Mellanox 
>> MT25204 HCA has a 4x microGiGaCN connector at the back of motherboard. 
>> These are also often called CX4 connectors.
>
> Excellent! Thanks so much for this info.
>
>> 
>> Depending upon the switch you will connect the systems to, you need either 
>> microGiGaCN to microGiGaCN cables, or microGiGaCN to QSFP cables. 
>> Colfaxdirect.com lists many of each type.
>
> The switch we are considering is QLogic SilverStorm 9024 (9024-FC24-ST1-DDR). 
> So I suppose we need microGiGaCN to microGiGaCN (CX4 to CX4) cable?
>
> Thanks,
>
> D.
>
>> 
>> Supermicro.com has detailed specs and the manual for this motherboard.
>> 
>> Don
>> 
>> 
>> 
>> On Aug 29, 2012, at 5:05 AM, "Duke Nguyen" <duke.lists at gmx.com> wrote:
>> 
>>> Hi folks,
>>> 
>>> We plan to build a cluster using server built with SuperMicro X7DWT-INF
>>> motherboards. Unfortunately google does not give us much information
>>> about the type of connector that X7DWT-INF has.
>>> 
>>> Anybody using this motherboard in their cluster? Please give us advice
>>> about which connector the board has and what infiniband cable we should
>>> acquire to fit in the rack-mount servers built with these boards.
>>> 
>>> Thanks,
>>> 
>>> D.
>>> _______________________________________________
>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>>> To change your subscription (digest mode or unsubscribe) visit 
>>> http://www.beowulf.org/mailman/listinfo/beowulf
>
>


From duke.lists at gmx.com  Wed Aug 29 05:28:01 2012
From: duke.lists at gmx.com (Duke Nguyen)
Date: Wed, 29 Aug 2012 19:28:01 +0700
Subject: [Beowulf] SuperMicro X7DWT-INF board - what connector? what
	cable?
In-Reply-To: <1EC63CA9-C7D7-41B6-8C1C-7E8843D98216@fnal.gov>
References: <503DE77B.5090401@gmx.com>
	<1EC63CA9-C7D7-41B6-8C1C-7E8843D98216@fnal.gov>
Message-ID: <503E0AD1.1000006@gmx.com>

Hi Don,

On 8/29/12 6:49 PM, Donald J Holmgren wrote:
> We use the AMD version of this motherboard ("TwinU").  The landed Mellanox MT25204 HCA has a 4x microGiGaCN connector at the back of motherboard.  These are also often called CX4 connectors.

Excellent! Thanks so much for this info.

>
> Depending upon the switch you will connect the systems to, you need either microGiGaCN to microGiGaCN cables, or microGiGaCN to QSFP cables. Colfaxdirect.com lists many of each type.

The switch we are considering is QLogic SilverStorm 9024 
(9024-FC24-ST1-DDR). So I suppose we need microGiGaCN to microGiGaCN 
(CX4 to CX4) cable?

Thanks,

D.

>
> Supermicro.com has detailed specs and the manual for this motherboard.
>
> Don
>
>
>
> On Aug 29, 2012, at 5:05 AM, "Duke Nguyen" <duke.lists at gmx.com> wrote:
>
>> Hi folks,
>>
>> We plan to build a cluster using server built with SuperMicro X7DWT-INF
>> motherboards. Unfortunately google does not give us much information
>> about the type of connector that X7DWT-INF has.
>>
>> Anybody using this motherboard in their cluster? Please give us advice
>> about which connector the board has and what infiniband cable we should
>> acquire to fit in the rack-mount servers built with these boards.
>>
>> Thanks,
>>
>> D.
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



From samuel at unimelb.edu.au  Wed Aug 29 23:02:50 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Thu, 30 Aug 2012 16:02:50 +1000
Subject: [Beowulf] Building Paraview on RHEL5/CentOS5 ?
Message-ID: <503F020A.8020602@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi folks,

Has anyone tried to build a recent version of Paraview (3.14.1) on
CentOS5/RHEL5 with offscreen rendering and MPI?

One of our folks here is trying to do this for one of our users and is
struggling as she is trying to avoid falling into the dependency hell
that would be compiling Mesa from source.

Has anyone succeeded with this and can shed some light?

cheers!
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlA/AgkACgkQO2KABBYQAh/UwgCfepS2IE651K3Tcp9P3V1ZKp6K
OwUAn0j3yT2yBX/3eLRiH4Y92UbtTN0k
=jO7T
-----END PGP SIGNATURE-----


From duke.lists at gmx.com  Thu Aug 30 02:53:30 2012
From: duke.lists at gmx.com (Duke Nguyen)
Date: Thu, 30 Aug 2012 16:53:30 +0700
Subject: [Beowulf] SuperMicro X7DWT-INF board - what connector? what
	cable?
In-Reply-To: <alpine.LRH.2.00.1208290955320.14319@jpsi2.fnal.gov>
References: <503DE77B.5090401@gmx.com>
	<1EC63CA9-C7D7-41B6-8C1C-7E8843D98216@fnal.gov>
	<503E0AD1.1000006@gmx.com>
	<alpine.LRH.2.00.1208290955320.14319@jpsi2.fnal.gov>
Message-ID: <503F381A.3080606@gmx.com>

Great! Thanks for all of your help guys.

D.

On 8/29/12 9:56 PM, Don Holmgren wrote:
>
> Yes, that's right, CX4 to CX4.
>
> Don
>
>
> On Wed, 29 Aug 2012, Duke Nguyen wrote:
>
>> Hi Don,
>>
>> On 8/29/12 6:49 PM, Donald J Holmgren wrote:
>>> We use the AMD version of this motherboard ("TwinU").  The landed 
>>> Mellanox MT25204 HCA has a 4x microGiGaCN connector at the back of 
>>> motherboard. These are also often called CX4 connectors.
>>
>> Excellent! Thanks so much for this info.
>>
>>>
>>> Depending upon the switch you will connect the systems to, you need 
>>> either microGiGaCN to microGiGaCN cables, or microGiGaCN to QSFP 
>>> cables. Colfaxdirect.com lists many of each type.
>>
>> The switch we are considering is QLogic SilverStorm 9024 
>> (9024-FC24-ST1-DDR). So I suppose we need microGiGaCN to microGiGaCN 
>> (CX4 to CX4) cable?
>>
>> Thanks,
>>
>> D.
>>
>>>
>>> Supermicro.com has detailed specs and the manual for this motherboard.
>>>
>>> Don
>>>
>>>
>>>
>>> On Aug 29, 2012, at 5:05 AM, "Duke Nguyen" <duke.lists at gmx.com> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> We plan to build a cluster using server built with SuperMicro 
>>>> X7DWT-INF
>>>> motherboards. Unfortunately google does not give us much information
>>>> about the type of connector that X7DWT-INF has.
>>>>
>>>> Anybody using this motherboard in their cluster? Please give us advice
>>>> about which connector the board has and what infiniband cable we 
>>>> should
>>>> acquire to fit in the rack-mount servers built with these boards.
>>>>
>>>> Thanks,
>>>>
>>>> D.
>>>> _______________________________________________
>>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin 
>>>> Computing
>>>> To change your subscription (digest mode or unsubscribe) visit 
>>>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>>
>



From eagles051387 at gmail.com  Thu Aug 30 03:15:44 2012
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Thu, 30 Aug 2012 12:15:44 +0200
Subject: [Beowulf] Current mailing list setup
Message-ID: <CAHdpx61eikQRUsKhRLV5j1j8f6WXMHBSfBOY7-NxaWqHtzXedA@mail.gmail.com>

I have to say that the way things are currently now we have drastically
reduced the amount of spam that is waiting moderation. Yesterday I checked
throughout the day and there was only one email which was caught for
moderation, and was let through as well as one spam message.

Correct me if I am wrong but is spamassassin necessary if this is working.

Also how can i set flags for users so that they done require moderation if
they are legitimate users?

-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20120830/5860430f/attachment.html>

From tegner at renget.se  Thu Aug 30 12:24:34 2012
From: tegner at renget.se (Jon Tegner)
Date: Thu, 30 Aug 2012 21:24:34 +0200
Subject: [Beowulf] Strange error, gluster/ext4/zone_reclaim_mode
Message-ID: <503FBDF2.9070104@renget.se>

Hi,

have this strange error. We run CFD calculations on a small cluster.  
Basically it consists of bunch of machines connected to a file system.  
The file system consists of 4 servers, CentOS-6.2, ext4 and glusterfs 
(3.2.7) on top. Infiniband is used for interconnect.

For scheduling/resource management we use torque/maui, and typically we 
submit job in a torque submit script like:

mpirun -machinefile bla bla bla

However, at one point one of the machines serving the file system went 
down, after spitting out error messages as indicated in

https://bugzilla.redhat.com/show_bug.cgi?id=770545

We used the advice indicated in that link ("sysctl -w 
vm.zone_reclaim_mode=1"), and after that the file servers seems to run 
OK. This happened in the middle of summer, and a few weeks later we 
noticed a few strange things:

1. We had to change the torque submit script like

ssh $(hostname) "mpirun -machinefile bla bla bla"

2. zone_reclaim_node were set to 1 on all computational nodes (on the 
file servers this was done explicitly, NOT so on the computational nodes).

3. We have seen particularly lousy performance on one of our applications.

4. The command "tail -f file" doesn't get updated properly.

Any help/hints would be greatly appreciated!

Regards,

/jon



From samuel at unimelb.edu.au  Thu Aug 30 16:32:11 2012
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Fri, 31 Aug 2012 09:32:11 +1000
Subject: [Beowulf] Current mailing list setup
In-Reply-To: <CAHdpx61eikQRUsKhRLV5j1j8f6WXMHBSfBOY7-NxaWqHtzXedA@mail.gmail.com>
References: <CAHdpx61eikQRUsKhRLV5j1j8f6WXMHBSfBOY7-NxaWqHtzXedA@mail.gmail.com>
Message-ID: <503FF7FB.1080607@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 30/08/12 20:15, Jonathan Aquilina wrote:

> I have to say that the way things are currently now we have
> drastically reduced the amount of spam that is waiting moderation.
> Yesterday I checked throughout the day and there was only one email
> which was caught for moderation, and was let through as well as one
> spam message.
> 
> Correct me if I am wrong but is spamassassin necessary if this is
> working.

Rejecting spam at the incoming SMTP conversation level is still
necessary as it will stop Mailman from producing backscatter to the
forged addresses used in spam.

> Also how can i set flags for users so that they done require
> moderation if they are legitimate users?

There's a checkbox by the email address when you're looking at
messages queued for moderation.

I've set reply-to back to just me as how to use Mailman isn't really
on-topic for the list.

cheers,
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlA/9/sACgkQO2KABBYQAh/uOQCfXvl3Ek/BU3Qc7FkVFRZTCLkB
lyIAn2akT+H0xtanTMzRjcN+rzg1GtcF
=X1aB
-----END PGP SIGNATURE-----


From tegner at renget.se  Thu Aug 30 22:11:50 2012
From: tegner at renget.se (Jon Tegner)
Date: Fri, 31 Aug 2012 07:11:50 +0200
Subject: [Beowulf] Strange error, gluster/ext4/zone_reclaim_mode
In-Reply-To: <alpine.LFD.2.02.1208301817040.15768@coffee.psychology.mcmaster.ca>
References: <503FBDF2.9070104@renget.se>
	<alpine.LFD.2.02.1208301817040.15768@coffee.psychology.mcmaster.ca>
Message-ID: <50404796.4020304@renget.se>

Hi! And thanks for answer, much appreciated!

On 08/31/2012 12:47 AM, Mark Hahn wrote:
>> However, at one point one of the machines serving the file system went
>> down, after spitting out error messages as indicated in
>> it
>> https://bugzilla.redhat.com/show_bug.cgi?id=770545
>>
>> We used the advice indicated in that link ("sysctl -w
>> vm.zone_reclaim_mode=1"), and after that the file servers seems to run
>> OK.
>
> this seems to be quite dependent on hardware, architecture, workload,
> kernel.  did you notice any performance problems?  or try the 
> vm.min_kbytes_free angle?  (also, does this server have swap?)
We use standard centos-6.2 kernel (2.6.32-220.17.1), and I didn't notice 
anything strange on the servers (except the error messages before 
changing zone_reclaim_mode). Regarding min_free_kbytes it is 225280 on 
computational nodes (132 G total memory) and 90112 on file servers (32 G 
total memory). Is this something I should try changing? The servers have 
swap (64 G).

>
>> 1. We had to change the torque submit script like
>>
>> ssh $(hostname) "mpirun -machinefile bla bla bla"
>
> I think this is unrelated.  are you sure nothing changed, torque-wise,
> even its qmgr-level config?  (or mpi versions/config.)
I agree, it seems unrelated, but I can't find anything else that have 
changed!
>
>
>> 3. We have seen particularly lousy performance on one of our 
>> applications.
>
> does it do a lot of file IO?
>
No, and when profiling this I noticed that one particular operation, 
computing the gradient of a vector field, took too long, and besides the 
time to complete this operation varies substantially over the 
iterations. However, when performing this operation a second time (an 
extra "dummy operation") that was NOT that slow. Could this indicate 
that it has something to do with how the memory is handled?

Also, we have used a very similar set up previously, but were all 
machines were running CentOS-5, and then we didn't see these strange 
behaviours.

Thanks again!

/jon


From atchley at tds.net  Thu Aug 30 05:15:10 2012
From: atchley at tds.net (atchley tds.net)
Date: Thu, 30 Aug 2012 08:15:10 -0400
Subject: [Beowulf] Current mailing list setup
In-Reply-To: <CAHdpx61eikQRUsKhRLV5j1j8f6WXMHBSfBOY7-NxaWqHtzXedA@mail.gmail.com>
References: <CAHdpx61eikQRUsKhRLV5j1j8f6WXMHBSfBOY7-NxaWqHtzXedA@mail.gmail.com>
Message-ID: <CAN6L_JwjzvyMs6FOVdyafk0i+SPcg62UeNYXv7+Cv9DE1WXA+Q@mail.gmail.com>

On Thu, Aug 30, 2012 at 6:15 AM, Jonathan Aquilina
<eagles051387 at gmail.com> wrote:
>
> I have to say that the way things are currently now we have drastically reduced the amount of spam that is waiting moderation. Yesterday I checked throughout the day and there was only one email which was caught for moderation, and was let through as well as one spam message.
>
> Correct me if I am wrong but is spamassassin necessary if this is working.
>
> Also how can i set flags for users so that they done require moderation if they are legitimate users?

If you are using mailman, under the admin interface, click on
Membership Management. You will see a list of members. Uncheck the mod
button net to their email address.

Scott


From diep at xs4all.nl  Fri Aug 31 02:47:49 2012
From: diep at xs4all.nl (Vincent Diepeveen)
Date: Fri, 31 Aug 2012 11:47:49 +0200
Subject: [Beowulf] Strange error, gluster/ext4/zone_reclaim_mode
In-Reply-To: <503FBDF2.9070104@renget.se>
References: <503FBDF2.9070104@renget.se>
Message-ID: <2936C812-0D46-4B46-B390-556FE65E8EF4@xs4all.nl>

hi Jon,

It seems a kernel page problem. Maybe somehow a file manager or other  
software had allocated too many shared memory pages?
This is easy to check by executing 'ipcs' at every node.

I saw some strange things there in kernel used by Scientific Linux  
6.2 - even after deletion of shared memory pages it kept remembering
them after a reboot.



On Aug 30, 2012, at 9:24 PM, Jon Tegner wrote:

> Hi,
>
> have this strange error. We run CFD calculations on a small cluster.
> Basically it consists of bunch of machines connected to a file system.
> The file system consists of 4 servers, CentOS-6.2, ext4 and glusterfs
> (3.2.7) on top. Infiniband is used for interconnect.
>
> For scheduling/resource management we use torque/maui, and  
> typically we
> submit job in a torque submit script like:
>
> mpirun -machinefile bla bla bla
>
> However, at one point one of the machines serving the file system went
> down, after spitting out error messages as indicated in
>
> https://bugzilla.redhat.com/show_bug.cgi?id=770545
>
> We used the advice indicated in that link ("sysctl -w
> vm.zone_reclaim_mode=1"), and after that the file servers seems to run
> OK. This happened in the middle of summer, and a few weeks later we
> noticed a few strange things:
>
> 1. We had to change the torque submit script like
>
> ssh $(hostname) "mpirun -machinefile bla bla bla"
>
> 2. zone_reclaim_node were set to 1 on all computational nodes (on the
> file servers this was done explicitly, NOT so on the computational  
> nodes).
>
> 3. We have seen particularly lousy performance on one of our  
> applications.
>
> 4. The command "tail -f file" doesn't get updated properly.
>
> Any help/hints would be greatly appreciated!
>
> Regards,
>
> /jon
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf



From eugen at leitl.org  Fri Aug 31 07:57:22 2012
From: eugen at leitl.org (Eugen Leitl)
Date: Fri, 31 Aug 2012 16:57:22 +0200
Subject: [Beowulf] FY;) GROMACS on the Raspberry Pi
Message-ID: <20120831145722.GU12615@leitl.org>


http://www.reddit.com/r/comp_chem/comments/z2rqr/gromacs_on_the_raspberry_pi/

GROMACS on the Raspberry Pi (self.comp_chem)

submitted 1 day ago by steelgargoyle

As we all know our discipline tends to involve rather a lot of waiting
around. In my downtime I decided, having heard that GROMACS can compile on
ARM CPUs, to attempt to compile it on my Model B RPi. I dug out an old
sequential Bash script I wrote which sorts the FFTW libraries out for you,
and it actually worked.

I haven't tested it yet with any simulations, because I'm actually an AMBER
user nowadays and I've forgotten how to use GROMACS so I need to brush up,
but if it works well I'm hoping to get another one to test the feasibility of
MPI GROMACS on RPis as part of a small low-power Beowulf cluster, though the
ethernet speed could limit data transfer rates.

Thought this is probably the most appropriate place to post this. Let me know
if you're interested and if you have any ideas.