Knowledge Base

Log Source Onboarding and Data Quality

Validate source ownership, event quality, and pipeline health before depending on a new data source.

Ingestion is not validation

A successful connection does not prove events are complete, correctly parsed, or useful. Onboarding needs a business purpose, source owner, expected event types, collection method, transport security, parser version, field mapping, identity and asset mapping, retention, access control, sensitive-field handling, and decommissioning plan.

Data-quality checklist

  • Review sample events for timestamps, time zones, clock drift, event identifiers, expected fields, and sensitive values.
  • Test normalization and parsing with representative permitted data, including missing, truncated, duplicate, and version-changed events.
  • Set health monitoring for volume spikes, unexpected silence, parser errors, delivery failure, and schema drift.
  • Record source scope and known gaps. Missing fields should create a follow-up question, not a conclusion about safety.

Fictional example

A cloud service begins forwarding audit records, but a parser maps the actor field inconsistently after a version update. The team detects the change through field-quality monitoring, preserves samples, corrects the mapping, and documents the affected window before using the source in a high-confidence detection.

Related Vuln Signal content

See Monitoring Architecture, Log Tool, Structured Data Tool, and Detection Rule Review.