Q-Sensei Logs Cluster Architecture

Q-Sensei Logs can be deployed either in cloud or on-prem. Our JSON based REST API endpoint makes it easy to upload events from any Log data source.


Basic Architecture


Log data source

Log data source can be any kind of streaming, machine and historical data source. A few examples of log data sources are:

  • Web server logs
  • Live application logs
  • Network events
  • Message Queues; Apache Kafka

Log data source can be on-prem, private cloud or public cloud. Q-Sensei Logs exposes an HTTP endpoint using which connectors can upload events to the deployment.


Connectors connect to the log data source, monitors for new events and forwards events to Q-Sensei Logs with minimal processing overhead. With an HTTP upload endpoint, there are multiple options:

  • Fluentd
  • Fluent Bit
  • Logstash
  • Q-Sensei’s official connectors

Use the following link for a complete list of integration options: https://www.qsensei.com/q-sensei-connectors


A highly available HTTP endpoint to upload events. It implements buffering to handle events generated at high volume.


Q-Sensei Fuse instance to handle database updates and index updates.

Search Head

Q-Sensei Fuse instance to handle user query requests.


AWS SaaS Architecture

Q-Sensei Logs’ AWS SaaS architecture extends the basic architecture to provide a fully managed, highly available and secure service. The table below lists all AWS services used.


Category List of AWS Services
Security, Identity and Compliance AWS Cognito
AWS Identity and Access Management
AWS Secrets Manager
AWS Key Management Service (KMS)
Network & Content delivery AWS PrivateLink
AWS Route53
AWS Virtual Private Cloud (VPC)
Elastic Load Balancing
Management & Governance AWS CloudFormation
AWS Organizations
AWS Systems Manager
Compute AWS Elastic Compute Cloud (EC2)
Application Integration Amazon API Gateway
AWS Simple Queue Service
AWS Simple Notification Service
Customer Engagement AWS Simple Email Service (SES)


To deploy Q-Sensei Logs on-prem a mapping of following services should be established in the on-prem environment.

User Identity and Access Management

  • AWS Cognito

Authentication flow

  • In Q-Sensei Logs, the authentication work is offloaded to the Internet facing Application Load Balancer.

Email Notifications

  • AWS Simple Email Service


Was this article helpful?
0 out of 0 found this helpful