Q-Sensei Logs can be deployed either in cloud or on-prem. Our JSON based REST API endpoint makes it easy to upload events from any Log data source.
Log data source
Log data source can be any kind of streaming, machine and historical data source. A few examples of log data sources are:
- Web server logs
- Live application logs
- Network events
- Message Queues; Apache Kafka
Log data source can be on-prem, private cloud or public cloud. Q-Sensei Logs exposes an HTTP endpoint using which connectors can upload events to the deployment.
Connectors connect to the log data source, monitors for new events and forwards events to Q-Sensei Logs with minimal processing overhead. With an HTTP upload endpoint, there are multiple options:
- Fluent Bit
- Q-Sensei’s official connectors
Use the following link for a complete list of integration options: https://www.qsensei.com/q-sensei-connectors
HTTP Upload API
A highly available HTTP endpoint to upload events. It implements buffering to handle events generated at high volume.
Q-Sensei Fuse instance to handle database updates and index updates.
Q-Sensei Fuse instance to handle user query requests.
AWS SaaS Architecture
Q-Sensei Logs’ AWS SaaS architecture extends the basic architecture to provide a fully managed, highly available and secure service. The table below lists all AWS services used.
|Category||List of AWS Services|
|Security, Identity and Compliance||AWS Cognito
AWS Identity and Access Management
AWS Secrets Manager
AWS Key Management Service (KMS)
|Network & Content delivery||AWS PrivateLink
AWS Virtual Private Cloud (VPC)
Elastic Load Balancing
|Management & Governance||AWS CloudFormation
AWS Systems Manager
|Compute||AWS Elastic Compute Cloud (EC2)|
|Application Integration||Amazon API Gateway
AWS Simple Queue Service
AWS Simple Notification Service
|Customer Engagement||AWS Simple Email Service (SES)|
To deploy Q-Sensei Logs on-prem a mapping of following services should be established in the on-prem environment.
User Identity and Access Management
- AWS Cognito
- In Q-Sensei Logs, the authentication work is offloaded to the Internet facing Application Load Balancer.
- AWS Simple Email Service