- Problem
- A long-lived evaluator carries idle overhead and failure state from one check cycle into the next.
- Decision
- Run the alert evaluator as a one-shot process per cycle and leave scheduling, retries and backfill to an external scheduler.
Tenant-scoped telemetry and alerting
Limnopulse
An API and evaluation runtime for authorized telemetry reads, alert rules, durable incidents, and notification delivery.
- Python
- FastAPI
- Go
- DynamoDB
- InfluxDB
- SQS
- OpenTofu
- Problem
- Telemetry becomes operationally useful only when access, alert evaluation, incident state, and notification delivery remain attributable to the correct tenant and device context.
- Contribution
- Built tenant-scoped telemetry access, versioned alert rules, a replayable Go evaluator and separate email/Telegram delivery.
- Key decision
- Treat durable domain state as the authority and queues as at-least-once transports, with a one-shot evaluator so replay stays observable.
- Repository
- Limnopulse public repository ↗
01 / Context
Context & contribution
Problem
Telemetry becomes operationally useful only when access, alert evaluation, incident state, and notification delivery remain attributable to the correct tenant and device context.
Context
Limnopulse is a software implementation with local integration scaffolding and a separately documented cloud-infrastructure path. Its device and broker examples are development contracts, not evidence of a deployed sensor fleet.
Contribution
- Implemented tenant membership checks and pond ownership checks around telemetry reads.
- Built versioned rule configuration, durable alert events, and role-aware incident transitions.
- Developed a one-shot Go evaluator with explicit replay and work-ownership behavior.
- Separated email and Telegram notification queues, workers, bindings, and delivery state.
- Documented cloud resources and operational prerequisites with OpenTofu modules.
02 / Technical choices
Engineering decisions
- Problem
- Asynchronous consumers need durable backpressure and retry isolation without reimplementing an event bus.
- Decision
- Treat SQS with dead-letter queues as the durable work boundary and keep domain state correct under at-least-once, out-of-order delivery.
- Problem
- Telegram and email have different limits, credentials and failure modes; a shared path lets one channel degrade the other.
- Decision
- Give Telegram its own queue, dead-letter queue, worker and least-privilege IAM policy, isolated from email delivery.
03 / Current state
Current state
Outcomes
- The implementation connects authorized readings, configurable rules, durable incidents, and delivery workers through explicit tenant boundaries.
- Channel isolation and replay-oriented operations make failure modes inspectable in code and documentation.
04 / Public record
Public evidence
Limnopulse public repository
Public API, evaluator, notification workers, local integration scaffolding, and tests.
Open public evidence ↗Alert evaluator operations
Public replay, sharding, scheduling, and failure-handling contract.
Open public evidence ↗Notification operations
Public delivery-state, retry, rollout, and recovery documentation.
Open public evidence ↗Limnopulse architecture
Public boundaries for authorized telemetry, one-shot evaluation, durable events, outbox relays, and channel-specific notification work.
Open public evidence ↗Telegram notification operations
Public Telegram binding, worker, delivery-state, and retry contracts, separated from email delivery.
Open public evidence ↗05 / Optional walkthrough
A paused, synthetic lifecycle of the system. Press Play or step through the chapters to explore states, failures and limits.
Systems Atlas / LifecycleSynthetic · illustrative
01 / 06
See the starting state
Uma pessoa precisa receber avisos e acompanhar o incidente.
Record
See the starting state
Incident: none. Email: idle; Telegram: idle. User: idle.
Paused · operation 1 / 24
Dispositivo e provedores são falsos; nenhuma mensagem real é enviada. Regra/leituras são sintéticas; não são orientação de aquicultura. Tela do usuário é mock proposto, não aplicativo auditado.
Read the complete lifecycle transcript
Calculated from the engine during the build. Available without JavaScript.
Da água à ação — e à recuperação.
Antes da leitura
Uma pessoa precisa receber avisos e acompanhar o incidente.
Incident: none. Email: idle; Telegram: idle. User: idle.
Definir regra
Limiar, janela, duração e canal são configurados para este exemplo.
Incident: none. Email: idle; Telegram: idle. User: idle.
Vincular destinatário
Binding prova a associação. Preferência continua desligada.
Incident: none. Email: idle; Telegram: idle. User: idle.
Habilitar canal
Usuário com membership ativa habilita a preferência.
Incident: none. Email: idle; Telegram: idle. User: idle.
Publicar amostra
A boia ilustrativa publica um ponto; ainda não há janela persistida.
Incident: none. Email: idle; Telegram: idle. User: idle.
Persistir janela
Quatro pontos cobrem três ticks e confirmam a condição do exemplo.
Incident: none. Email: idle; Telegram: idle. User: idle.
Avaliar e registrar
Scheduler invoca evaluator. Incidente e dois trabalhos de outbox ficam duráveis.
Incident: open. Email: idle; Telegram: idle. User: idle.
Publicar por canal
Relay cria Delivery e enfileira IDs; isso ainda não é entrega.
Incident: open. Email: queued; Telegram: queued. User: idle.
Tentativa de email
Worker reclama a entrega de email e chama o provedor falso.
Incident: open. Email: attempting; Telegram: queued. User: idle.
Email aceito
Aceite do provedor não é recibo de leitura humana.
Incident: open. Email: accepted; Telegram: queued. User: idle.
Tentativa Telegram
O canal possui worker e fila próprios.
Incident: open. Email: accepted; Telegram: attempting. User: idle.
Aguardar retry_after
O provedor responde 429. Não reenviar antes do tick 6.
Incident: open. Email: accepted; Telegram: retry wait. User: idle.
Espera cumprida
O relógio lógico alcançou o vencimento; a mesma Delivery pode tentar de novo.
Incident: open. Email: accepted; Telegram: retry wait. User: idle.
Segunda tentativa
Nova Attempt, mesma Delivery e mesmo incidente.
Incident: open. Email: accepted; Telegram: attempting. User: idle.
Telegram aceito
A confirmação falsa registra message ID; não comprova leitura.
Incident: open. Email: accepted; Telegram: accepted. User: idle.
Mensagem no dispositivo
O mock representa o resultado para o usuário. É uma cena, não callback de leitura.
Incident: open. Email: accepted; Telegram: accepted. User: message shown.
Abrir incidente
Usuário autenticado abre detalhe pela API; não acessa InfluxDB diretamente.
Incident: open. Email: accepted; Telegram: accepted. User: incident opened.
Reconhecer, não resolver
ACK com versão correta registra ação humana. A condição baixa continua.
Incident: acknowledged. Email: accepted; Telegram: accepted. User: acknowledged.
Nova janela limpa
Novas leituras válidas, frescas e cobertas mostram condição normal.
Incident: acknowledged. Email: accepted; Telegram: accepted. User: acknowledged.
Recuperação durável
Evaluator recupera o incidente e cria trabalhos correlacionados de recuperação.
Incident: recovered. Email: accepted; Telegram: accepted. User: acknowledged.
Publicar recuperação
Elegibilidade é revalidada; canais recebem novos trabalhos da mesma linhagem.
Incident: recovered. Email: accepted; Telegram: accepted. User: acknowledged.
Enviar recuperação
Workers fazem tentativas independentes nos provedores falsos.
Incident: recovered. Email: accepted; Telegram: accepted. User: acknowledged.
Recuperação aceita
Respostas falsas confirmam aceite; o resultado ambiental e o de entrega são distintos.
Incident: recovered. Email: accepted; Telegram: accepted. User: acknowledged.
Ciclo até o destinatário
A recuperação aparece no mock. O histórico mantém abertura e reconhecimento.
Incident: recovered. Email: accepted; Telegram: accepted. User: recovery shown.
Current state and operation
{
"type": "INIT"
}06 / Limits & details
Limitations & details
Limitations
- No production sensor fleet or hardware deployment is verified.
- MQTT ingestion is demonstrated only as a local scaffold; production broker TLS, device credentials, and access controls remain planned.
- Cloud infrastructure is documented as code, but this page does not claim a live hosted environment or notification volume.
Reliability
- Optimistic versions and idempotency keys make conflicting rule changes and replacement replays explicit.
- Durable outboxes connect alert transitions to notification work without treating queue delivery as authoritative state.
- Local fakes and emulators exercise integration boundaries without being represented as hosted services.
- Telemetry is authorized through the API, so clients never receive direct time-series database access.
System View
Architecture and component details
Conceptual operational map, not live telemetry. It traces observation, telemetry, and event responsibilities across local and external boundaries without representing a real environment.
Operational map
Observations move through telemetry and event bands, with operational boundaries framing the flow.
Authorized telemetry
Versioned rules
Reading windows
Durable events
Delivery work
Configuration
Production boundary
Observations
Telemetry
Events
Component details
FastAPI authorizes membership and resource ownership before querying time-series readings.
Receives from
- Makes local readings available through InfluxDB for authorized API queries
Versioned rules guide evaluation; DynamoDB stores durable alert events and outboxes for opening, acknowledgement, resolution, and recovery workflows.
Receives from
- Persists durable alert events and outboxes in DynamoDB
Sends to
- Defines versioned rules for one-shot evaluation
- An outbox relay publishes per-channel notification work to SQS
A Go runtime queries reading windows in InfluxDB, evaluates owned work once per call, and persists events and outboxes in DynamoDB before exiting. Scheduling remains external.
Receives from
- Defines versioned rules for one-shot evaluation
- Maps managed resources and external scheduling prerequisites
Sends to
- Queries reading windows in InfluxDB populated by the local ingestion scaffold
- Persists durable alert events and outboxes in DynamoDB
An outbox relay publishes per-channel work to SQS. Separate email and Telegram queues and workers preserve authoritative delivery, attempt, and idempotency state.
Receives from
- An outbox relay publishes per-channel notification work to SQS
MQTT, Telegraf, and sample publishers write readings to InfluxDB in a loopback-only local ingestion scaffold.
Receives from
- Queries reading windows in InfluxDB populated by the local ingestion scaffold
- Marks the production-device boundary beyond the local scaffold
Sends to
- Makes local readings available through InfluxDB for authorized API queries
OpenTofu describes the intended managed resources and their operational prerequisites.
Sends to
- Maps managed resources and external scheduling prerequisites
A hardened broker, device credentials, fleet operations, and production sensor hardware remain outside the verified implementation.
Sends to
- Marks the production-device boundary beyond the local scaffold