After the project presentation, we will start by making a HelloWorld test prototype. The purpose of a prototype machine learning model would be to identify an input value message based on which log type the value message is and return log level as an output value.
Goal
The log information is sorted into 0-7 different log levels for example. Hypervisor detected: “Hypervisor detected: KVM”,”PRIORITY”:6″ so, its informative level 6 log information which tells that KVM Hypervisor detected from the computer. We start by teaching the machine learning model a lot of this type of data from different log levels.
| 0 | KERN_EMERG | Emergency condition, system is probably dead |
| 1 | KERN_ALERT | Some problem has occurred, immediate attention is needed |
| 2 | KERN_CRIT | A critical condition |
| 3 | KERN_ERR | An error has occurred |
| 4 | KERN_WARNING | A warning |
| 5 | KERN_NOTICE | Normal message to take note of |
| 6 | KERN_INFO | Some information |
| 7 | KERN_DEBUG | Debug information related to the program |
The idea would be to have the final model identify the feed value from the message “Hypervisor detected: KVM” that this log belongs to informative level 6.
The model shown in the figure has an input value of 1, based on which the machine learning model calculates the most likely result and gives its output as 0-7. 1 input value of this model will be the informative message and 7 output values will be the log level for that message.
Sources:
Wikipedia, Artificial neural network: https://en.wikipedia.org/wiki/Artificial_neural_network
Wikipedia, printk https://en.wikipedia.org/wiki/Printk
Detecting log anomalies with machine learning 