An outlier is a data point outside the normal range of values. Q-Sensei Logs builds a model for normal data points for each facet and then detects outliers. Each outlier data point is assigned a score to indicate the severity of the anomaly.
Facet configuration: Grouping criteria
A certain combination of facets can be used to specify grouping criteria for data points. This grouping criteria is used for model building and detecting outliers within each group.
Example
Consider a system monitoring CPU utilization for EC2 machines across multiple AWS accounts. Q-Sensei Logs is ingesting log events with following 3 attributes:
- aws_account
- cpu_utilization
- ec2_instance_id
Option 1: No grouping
In this case, the model training will use all data points without grouping. The outliers detected may not be accurate because we are using values across multiple AWS accounts.
Option 2 : Group CPU utilization by AWS account
In this case, the model training will group data points by AWS account ID. This is useful if you want to detect outliers at account level.
Option 3 : Group CPU utilization by EC2 instance id
In this case, you will be able to detect outliers for each EC2 instance. This grouping criteria may be applicable in most scenarios.
Algorithms – Methods for outlier detection
Out of the box, Q-Sensei Logs supports 3 different algorithms for outlier detection: