Skip to content

Documentation

Correlation

Finding events across two providers that belong together — what a correlation rule is, the three kinds, how confidence is calculated, and how to act on a correlation.

A trigger answers a question about one event. Correlation answers a question about two, from systems that know nothing about each other.

HR marked someone as terminated on Tuesday. Did Active Directory actually disable their account?

Neither system can answer that. HR does not know what the directory did, and the directory does not know why. What ties the two events together is a shared value — an employee id — and a plausible gap in time. That is what a correlation rule describes.

What it is for

The useful question is usually about the gap between two systems, and it comes in two shapes:

  • Did the second thing happen? An offboarding that never reached the directory is an account still live for someone who left.
  • What happened around this? When something goes wrong, the events either side of it across other providers are the context you would otherwise assemble by hand from several tabs.

Correlation records those pairings so they can be read later, rather than reconstructed each time, and publishes each new one so a trigger can act on it: alert someone when the directory didn't follow HR, or open a ticket.

A rule

A rule names two providers, a window, and what "related" means:

Setting Meaning
Source and target provider Which two providers the events come from
Time window How long after a source event a target may arrive and still count
Type field_match, temporal or sequence — below
Minimum confidence Correlations scoring below this are not recorded

Rules are managed on the Insights page, under Correlation rules: add, edit, enable, disable and delete. The same operations exist on the API (createCorrelationRule and friends) if you would rather script them.

Deleting a rule deletes the correlations it found, because a correlation only means anything as the output of the rule that produced it. The interface says so before it does it.

The three kinds

Field match

Two events whose payloads share a value. This is the offboarding case, and the one that produces specific, defensible results.

sourceField: employee_id   →   targetField: user_id

Both fields are dot-paths into the provider payload, compared with the same operators triggers use, so equals and contains mean exactly what they mean in a rule tree. A rule can configure several pairs; how many of them matched is what drives confidence.

Temporal

Two events that merely occurred close together, with no shared value. Useful when the systems have no identifier in common, and deliberately the weakest: every source pairs with every target inside the window, so on a busy provider it produces a great deal that is technically true and rarely interesting.

Its confidence is capped accordingly — see below.

Sequence

An ordered chain across more than two events: A, then B, then C, all inside the window. Every step must be present. A chain that does not complete is not a weak match, it is an absence — the thing the rule describes did not happen — so nothing is recorded.

The recorded correlation keeps the first and last events as its source and target, with the whole chain alongside.

Confidence

Every correlation carries a score between 0 and 1. It is evidence × proximity, and both halves are deliberately simple, because a number nobody can account for is worse than no number:

Evidence — how much of the rule matched.

Type Evidence
Field match the fraction of configured field pairs that matched
Sequence 1.0 — a partial chain is not recorded at all
Temporal 0.5 — co-occurrence alone is weak

Proximity — how close in time. 1.0 when the events are simultaneous, falling to 0.5 at the edge of the window.

So two of three fields matching at the same instant scores 0.5. Three of three an hour apart in a one-hour window also scores 0.5. A temporal rule can never exceed 0.5, which means the default minimum confidence admits it only for near-simultaneous events — that is intentional, and lowering the threshold for a temporal rule should be a deliberate act.

The score is arithmetic over the rule you configured. It is not a judgement, a model, or a learned weighting, which is why the Insights page shows what matched next to it.

Seeing the shape of it

The Insights page draws the relationships as a graph (on a phone, as a list of provider pairs with their count and average confidence): each provider is a node, and a line between two providers means a rule has been correlating their events. The line carries how many correlations and their average confidence, and heavier lines mean stronger relationships.

It aggregates deliberately. A hundred correlations between the same two providers is one relationship, not a hundred lines — the detail is in the list beneath it.

When it runs

Both on demand and on a schedule.

  • On demand through runCorrelationEngine, or the button on the Insights page, over an explicit range.
  • On a schedule, every few minutes, over a rolling window sized to cover the widest rule — so a pair whose second event arrives late is still seen.

Re-running is safe. A correlation already recorded is not recorded again, which is what lets the schedule overlap its windows deliberately rather than risk missing pairs at the boundaries.

Acting on a correlation

Every organisation has a built-in provider called Correlations. Each new correlation arrives there as a message, which makes it an event like any other. A trigger attached to the Correlations provider evaluates it, and the workflows attached to that trigger run, so notifying someone is a workflow with a Slack or email step — or, for the people who installed RunDis on a phone and turned the alert on, a notification that opens Insights the moment the correlation is published. The correlation's row on Insights links to the runs it fired, so "what did we do about it" is one click from "what happened".

A trigger on the Correlations provider sees a message shaped like this:

{
  "correlationId": 42,
  "correlationType": "field_match",
  "confidence": 0.9417,
  "summary": "1 of 1 field(s) matched, 7m0s apart",
  "timeDeltaSeconds": 420,
  "rule": { "id": 7, "name": "offboarding", "windowMinutes": 60, "minConfidence": 0.5 },
  "matchedFields": [ { "source": "employee_id", "target": "user_id", "reason": "…" } ],
  "source": { "eventId": 101, "providerId": 3, "provider": "Workday", "description": "provider_received_message", "time": "2026-09-14T09:00:00Z", "data": { "employee_id": "E-2231" } },
  "target": { "eventId": 108, "providerId": 5, "provider": "Active Directory", "description": "provider_received_message", "time": "2026-09-14T09:07:00Z", "data": { "user_id": "E-2231" } },
  "sequence": []
}

source.data and target.data are the two events' own payloads, and sequence lists every step of a sequence correlation. The trigger builder suggests these fields even before the first correlation has arrived.

A trigger that acts on confident offboarding correlations, with a Slack step that says which person:

confidence  GTE     0.8
rule.name   EQUALS  offboarding
{{ .trigger.rule.name }}: {{ .trigger.source.data.employee_id }} was disabled in {{ .trigger.target.provider }}

Templates are covered in Actions.

What to rely on:

  • Each correlation is published once. Re-running over a range, or a scheduled pass overlapping a manual one, records nothing new and fires nothing.
  • Only recent correlations fire. A correlation is published if its later event is less than 24 hours old. Scheduled passes always are. Running correlation by hand over a long range records older matches but doesn't act on them, so a new rule doesn't set off alerts for last month.
  • Firing is at most once. If the service stops between recording a correlation and evaluating its triggers, that correlation won't fire. It is never fired twice.
  • Correlations don't correlate with correlations. The Correlations provider can't be a rule's source, target or sequence step, and its events are never matched.
  • The Correlations provider is managed by RunDis. It can't be deleted, renamed, disabled or posted to; you can describe it and attach triggers to it. If your organisation already had a provider called "Correlations", the built-in one is named "Correlations (built-in)".
  • Correlation messages count as events, so they appear in event volume on the Insights page.

Limits worth knowing

  • Field matching needs a payload. Events raised by RunDis itself — a workflow completing, an action running — carry no provider payload, so only events ingested from a provider can be matched on fields.
  • Providers are named, not linked. A rule names its providers as text, so renaming a provider stops its rules matching until they are updated.
  • Deleting a rule deletes its correlations, because a correlation only means anything as the output of the rule that found it.

For measuring one provider rather than relating two, see Insights.