Correlation Engine v3

The Correlation Engine transforms raw security events into actionable incidents by detecting multi-stage attacks, cross-sensor patterns, and temporal anomalies.

Overview

Metric	Value
Built-in rules	15
Correlation window	5 minutes (configurable)
Alert clustering	DBSCAN + MiniLM embeddings
Noise reduction	100+ alerts/hour → 5-10 incidents
Lattice integration	TSA, GPS, MIRE primitives

How correlation works

Raw events (100+/hour)
    │
    ├── 1. Group by time window ──────────── 5-minute windows
    │
    ├── 2. Match against rules ───────────── 15 rules in parallel
    │
    ├── 3. Map to the kill chain ─────────── MITRE ATT&CK stages
    │
    ├── 4. Cluster alerts ────────────────── DBSCAN removes duplicates
    │
    ├── 5. Compute severity ──────────────── Max(severity) + escalation
    │
    └── 6. Create the incident ───────────── Unique incident + links

Correlated incidents (5-10/hour)

Built-in rules

Multi-stage attack rules

#	Rule ID	Trigger events	Window	Severity
1	`MULTI_STAGE_JAILBREAK`	jailbreak → tool_abuse	5m	HIGH
2	`EXFILTRATION_CHAIN`	pii_detected → exfiltration → network_alert	5m	CRITICAL
3	`INJECTION_TO_EXEC`	injection → tool_abuse (file/exec)	3m	CRITICAL
4	`CRESCENDO_ATTACK`	5+ low-severity events from one source	10m	HIGH
5	`DORMANT_PAYLOAD_ACTIVATION`	dormant_payload → any_high_severity	30m	CRITICAL

Cross-sensor rules

#	Rule ID	Trigger	Window	Severity
6	`BYPASS_SHIELD_THEN_INJECT`	shield_block → core_detect	30s	HIGH
7	`CROSS_SENSOR_ESCALATION`	shield_bypass + core_detect	1m	CRITICAL
8	`IMMUNE_KERNEL_PLUS_CORE`	immune_alert + core_alert	2m	HIGH

Temporal rules

#	Rule ID	Trigger	Window	Severity
9	`REPEATED_TOOL_ABUSE`	3+ tool_abuse from one IP	10m	HIGH
10	`ALERT_FLOOD`	100+ events from one sensor	60s	MEDIUM
11	`BRUTE_FORCE_PATTERN`	10+ auth_bypass attempts	5m	HIGH

Lattice rules

#	Rule ID	Trigger	Primitive	Severity
12	`TSA_VIOLATION`	Temporal Safety Automata violation	TSA	CRITICAL
13	`GPS_HIGH_DANGER`	Goal Predictability Score > 0.7	GPS	HIGH
14	`MIRE_CONTAINMENT`	Model containment triggered	MIRE	CRITICAL
15	`IRM_HIDDEN_INTENT`	Intent Revelation divergence > 0.5	IRM	HIGH

Rule definition format

Rules are defined in YAML:

YAML

rules:
  - id: MULTI_STAGE_JAILBREAK
    name: "Multi-Stage Jailbreak Attack"
    description: "Jailbreak followed by tool_abuse within 5 minutes"
    enabled: true

    conditions:
      - sequence:
          - category: jailbreak
            min_confidence: 0.7
          - category: tool_abuse
            min_confidence: 0.5
        within: "5m"
        same_field: "source_ip"

    action:
      create_incident: true
      severity: HIGH
      kill_chain_stage: "exploitation"
      playbook: "auto_block_jailbreak"

    metadata:
      mitre_atlas: ["AML.T0054", "AML.T0040"]
      owasp_llm: ["LLM01", "LLM07"]

Condition types

Type	Description	Example
`sequence`	Events in order	jailbreak → tool_abuse
`count`	N events of a type	3+ tool_abuse
`absence`	An expected event is missing	No auth within 5m of access
`threshold`	A metric exceeds a value	GPS > 0.7

Matching parameters

Parameter	Description
`within`	Time window for correlation
`same_field`	Events must agree on this field
`different_source`	Events come from different sensors
`min_confidence`	Minimum confidence per event

Alert clustering

The problem

Every engine × incoming requests = hundreds of raw alerts per hour. Most of them are about the same attack.

The solution: DBSCAN + MiniLM

100+ raw alerts
    │
    ├── 1. Embed alert descriptions (MiniLM-L6)
    │       → 384-dimensional vectors
    │
    ├── 2. DBSCAN clustering (ε=0.3, min_samples=3)
    │       → Group the similar ones
    │
    ├── 3. Select a representative
    │       → The highest-confidence alert in the cluster
    │
    └── 4. Filter noise
            → Isolated alerts are kept as standalone events

5-10 incident-worthy clusters

Configuration

YAML

correlation:
  clustering:
    enabled: true
    algorithm: "dbscan"
    min_cluster_size: 3        # Minimum events per cluster
    similarity_threshold: 0.7  # Cosine similarity threshold
    embedding_model: "all-MiniLM-L6-v2"

Kill chain mapping

Correlated incidents are mapped to attack stages:

┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐
│   Recon    │→ │  Delivery  │→ │Exploitation│→ │   Action   │
│            │  │            │  │            │  │            │
│ injection  │  │ shield_blk │  │ jailbreak  │  │exfiltration│
│ probe      │  │ evasion    │  │ tool_abuse │  │ pii_leak   │
└────────────┘  └────────────┘  └────────────┘  └────────────┘

Each incident shows:

Kill chain stage: the current phase of the attack
Progression: a visual timeline of the stages
Forecast: the GPS prediction of the likely next stage

Incident lifecycle

┌────────────────────────────────────────────────────┐
│                                                    │
│   ┌──────┐     ┌───────────────┐     ┌──────────┐  │
│   │ OPEN │────▶│ INVESTIGATING │────▶│ RESOLVED │  │
│   └──┬───┘     └───────┬───────┘     └──────────┘  │
│      │                 │                           │
│      │         ┌───────▼───────┐                   │
│      └────────▶│   ESCALATED   │                   │
│                │ (Zero-G mode) │                   │
│                └───────────────┘                   │
│                                                    │
└────────────────────────────────────────────────────┘

Status	Description	Available actions
OPEN	New, unacknowledged	Acknowledge, escalate
INVESTIGATING	An analyst is working on it	Run playbook, update, resolve
ESCALATED	Needs a senior analyst	Zero-G actions
RESOLVED	Handled and documented	Reopen (if needed)

Performance

Metric	Value
Rule evaluation	<5ms for all 15 rules
Clustering	<50ms for 100 alerts
End-to-end correlation	<100ms
Memory usage	~50MB for the rule engine
Max concurrent correlations	1000

Custom rules

See the Custom Correlation Rules tutorial for a step-by-step guide to writing your own.

Next steps

Lattice Primitives — mathematical safety guarantees
Tutorial: custom rules — creating your own rules
Clustering configuration — fine-tuning