[lug] Modern log analysis?

[Bear Giles]

This isn't an answer to the original question but related. I've read
repeatedly that the best approach is to log everything then use filters to
take out the innocent things. Figuring out what's innocent can take some
time and that does seem like a place where ML could be useful.

What's left is the anomalies, they're the ones that deserve a second look.
That's where you should be applying the intelligence to look for patterns.

This is also why it helps tremendously to go through a logging mechanism
that ensures your log messages have a consistent appearance. It's more of a
pain to implement but it makes filtering much more reliable. At the extreme
the code doesn't even have the error messages, it always logs an error
code. The code is converted to a text message at another level, with the
error messages maintained by a group responsible to ensuring consistency,
completeness, etc.

On Thu, Jun 7, 2018 at 12:11 PM, Quentin Hartman <qhartman at gmail.com> wrote:

> Coincidentally I discovered this about an hour ago:
> https://www.elastic.co/guide/en/x-pack/current/ml-
> configuring-categories.html
>
> And since X-Pack is free now, I'm taking a real hard look at this over the
> next week or so.
>
> QH
>
> On Thu, Jun 7, 2018 at 10:29 AM Rob Nagler <nagler at bivio.biz> wrote:
>
>> On Thu, Jun 7, 2018 at 7:58 AM, Quentin Hartman wrote:
>>
>>>  https://www.moogsoft.com/lp/product-itoa
>>>
>>
>> Looks promising, and pricing is reasonable. BTW, I think it is now
>> (thanks to Gartner) called AIOps.  This is really funny (to me):
>>
>> *Our Intelligent Notifications automatically pull in the right people,
>> from the right teams, at the right times, while the Situation Room provides
>> operators with a virtual war room that unifies communication in a single
>> location, facilitated by a variety of ChatOps tools (including Slack). And
>> it can be accessed anywhere, including from your mobile device.*
>>
>>
>>  https://www.youtube.com/watch?v=cQXT3E8Anq8
>>>
>>
>> ROFL. Moog, no! No ssh attack! It's just a dumb bot, Moog.
>>
>> Rob
>>
>> _______________________________________________
>> Web Page:  http://lug.boulder.co.us
>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>
>
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20180609/bd821b71/attachment.html>