[Beowulf] dedupe filesystem

[Rahul Nabar]

On Tue, Jun 2, 2009 at 11:56 PM, Matt Lawrence<matt at technoronin.com> wrote:
> I have found a great deaal of duplication in install trees, particularly
> when you just want to install the latest.  I've managed to get some massive
> savings with NIM on AIX and some lesser but stll very good savings with
> CentOS by building parallel trees and hard linking the files.

I'm jumping on this discussion late but I found a great number (but
not so much in size I suspect) of dupes when I ran fdupes on my drives
now. Some in quite unexpected places. eg. we have job submission shell
wrappers for PBS/torque. The number of dupes indicates that often
people are not removing these files at all after the job finishes. So,
more than an automated-dedup solution fdupes has helped me know what
are the filesystem cleanup opportunities I have!

The next time someone comes running for a disk quota increase I know
what to check! ;-)

-- 
Rahul