[Beowulf] ELK query "Elasticsearch, logstash and Kibana"

[Matt Hurd]

Apologies, you may have misunderstood. I have nanoseconds via network
captures (in pcap-ng etc) but the ELK stack AFAIK only copes with
milliseconds which makes it problematic. ELK looks lovely, but without
nanosecond support it doesn't work for many people with similar needs to
myself. Though I may be missing something and it has nanosecond support
hidden in there somewhere which is what I was asking about.

Kind regards,

--Matt.



On 10 August 2014 14:19, Lux, Jim (337C) <james.p.lux at jpl.nasa.gov> wrote:

>   Nanoseconds?
> You need something like a GPS (run of the mill is good to 10-50 ns) or
> IEEE-1588 timing distribution (Precision Time Protocol)
>
>  If you have accurate 1pps to appropriate hardware on your nodes, then
> getting (much better) than sub-microsecond timing is more a matter of
> software than the hardware.
>
>  James Lux, P.E.
>
> Task Manager, FINDER – Finding Individuals for Disaster and Emergency
> Response
>
> Co-Principal Investigator, SCaN Testbed (*née* CoNNeCT) Project
>
> Jet Propulsion Laboratory
>
> 4800 Oak Grove Drive, MS 161-213
>
> Pasadena CA 91109
>
> +1(818)354-2075
>
> +1(818)395-2714 (cell)
>
>
>
>   From: Matt Hurd <matthurd at acm.org>
> Date: Saturday, August 9, 2014 at 5:16 PM
> To: Walid <walid.shaari at gmail.com>
> Cc: "beowulf at beowulf.org" <beowulf at beowulf.org>
> Subject: Re: [Beowulf] ELK query "Elasticsearch, logstash and Kibana"
>
>   Can it do better than millisecond time stamps yet? My network stuff
> needs nanoseconds or better...
>
>  --Matt.
>
>
> On 7 August 2014 12:48, Walid <walid.shaari at gmail.com> wrote:
>
>> Hi,
>>
>>  I am interested to see if any one is already using ELK for HPC related
>> logs. i would like to know more about what metrics, information, reports
>> they are generating and logstash patterns, filters, any groks used. I have
>> just started working on this starting with the configuration management
>> side of systems, however i see huge potential for people who can not afford
>> to have splunk, I am thinking scheduler, interconnect, GPFS, provisioning,
>> system logs, job tractability, metrics from accounting file, profiling,
>> better failure management and visualisation.
>>
>>  kind regards
>>
>>  Walid
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20140810/d33dd04a/attachment.html>