Knowledge Base

Security Monitoring Architecture and Data Flows

Map source-to-alert dependencies before relying on monitoring during an investigation.

Monitoring is a data path

Logs and events pass through collectors, forwarders, queues, parsers, normalization, enrichment, correlation, detection engines, storage, case management, dashboards, and alert routing. Each layer has owners, permissions, timestamps, retention, and failure modes. Collecting logs does not guarantee useful detection: records can be missing, delayed, misparsed, inaccessible, or lack identity and asset context.

Architecture inventory

LayerEvidence to retain
SourcesOwner, event types, clock quality, volume, collection method, and health signal.
PipelineTransport, parser version, queue capacity, loss handling, and administrative access.
Detection and responseRule owner, alert route, case path, retention tier, and recovery procedure.

Review workflow

  1. Trace one important source from generation through alert and case handling.
  2. Identify trust boundaries, multi-tenant separation, enrichment dependencies, and hot, warm, or archive retention.
  3. Test a bounded health failure such as a disabled forwarder, clock drift, or queue backlog.
  4. Record data loss, recovery, monitoring-of-monitoring, and escalation ownership.

Fictional example

A hybrid identity event reaches a collector but its asset enrichment dependency is unavailable. The alert still arrives, yet the triage team cannot prioritize it reliably. The team preserves the raw event, records the enrichment gap, and adds health monitoring rather than treating the alert as complete context.

Common mistakes

Common mistakes include centralizing logs without access for responders, retaining data without a retrieval plan, or assuming a dashboard proves pipeline health.

Related Vuln Signal content

See Cloud Logging, Detection Readiness, Evidence Preservation, and Detection Rule Review.