[Beowulf] ELK query "Elasticsearch, logstash and Kibana"

[Lux, Jim (337C)]

Nanoseconds?
You need something like a GPS (run of the mill is good to 10-50 ns) or IEEE-1588 timing distribution (Precision Time Protocol)

If you have accurate 1pps to appropriate hardware on your nodes, then getting (much better) than sub-microsecond timing is more a matter of software than the hardware.

James Lux, P.E.
Task Manager, FINDER – Finding Individuals for Disaster and Emergency Response
Co-Principal Investigator, SCaN Testbed (née CoNNeCT) Project
Jet Propulsion Laboratory
4800 Oak Grove Drive, MS 161-213
Pasadena CA 91109
+1(818)354-2075
+1(818)395-2714 (cell)


From: Matt Hurd <matthurd at acm.org<mailto:matthurd at acm.org>>
Date: Saturday, August 9, 2014 at 5:16 PM
To: Walid <walid.shaari at gmail.com<mailto:walid.shaari at gmail.com>>
Cc: "beowulf at beowulf.org<mailto:beowulf at beowulf.org>" <beowulf at beowulf.org<mailto:beowulf at beowulf.org>>
Subject: Re: [Beowulf] ELK query "Elasticsearch, logstash and Kibana"

Can it do better than millisecond time stamps yet? My network stuff needs nanoseconds or better...

--Matt.


On 7 August 2014 12:48, Walid <walid.shaari at gmail.com<mailto:walid.shaari at gmail.com>> wrote:
Hi,

I am interested to see if any one is already using ELK for HPC related logs. i would like to know more about what metrics, information, reports they are generating and logstash patterns, filters, any groks used. I have just started working on this starting with the configuration management side of systems, however i see huge potential for people who can not afford to have splunk, I am thinking scheduler, interconnect, GPFS, provisioning, system logs, job tractability, metrics from accounting file, profiling, better failure management and visualisation.

kind regards

Walid

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20140810/18b2948e/attachment.html>