Skip to content

feat(datadog): add Datadog incident management data source plugin - #9406

Open
stigi-work wants to merge 12 commits into
apache:mainfrom
pitch-io:feat/datadog-incident-plugin
Open

stigi-work wants to merge 12 commits into
apache:mainfrom
pitch-io:feat/datadog-incident-plugin

Conversation

@stigi-work

Copy link
Copy Markdown
Contributor

Summary

Adds a data source plugin for Datadog Incident Management. It is in the same category as the PagerDuty, Opsgenie, Rootly and incident.io plugins, and it feeds the existing DORA incident metrics.

  • Connection: API key and application key (DD-API-KEY / DD-APPLICATION-KEY). The endpoint is configurable for each Datadog site (US1, EU1, …). A separate web URL builds the incident links.
  • Scopes: incident types (/api/v2/incidents/config/types), with remote-scope listing.
  • Collection: incidents from /api/v2/incidents with page[size]/page[offset] pagination and include=users for the commander.
  • Extraction: test incidents (is_test) are skipped.
  • Conversion to the ticket domain:
    • issues with type INCIDENT, plus boards and board_issues for each incident type.
    • issue_labels from select-type custom fields, written as field:value.
    • issue_assignees from the incident commander.
    • lead_time_minutes uses Datadog's time_to_repair and falls back to created → resolved. A zero value or a negative duration does not count as an instant restore.
  • Scope config: componentField maps a custom field (for example services) to issues.component. The optional severityField replaces the native SEV severity with a custom field.
  • config-ui: connection form, scope selection, transformation form and a placeholder icon. The icon is not the Datadog logo, which is a trademark.

Closes #9124

Tests

  • Unit tests for the collector, extractor, converter and incident-type extractor.
  • An e2e data-flow test with snapshot tables for issues, boards, board_issues, issue_labels and issue_assignees.
  • We have run the plugin since September 2026 against a production Datadog org in our self-hosted DevLake. The DORA dashboards read its incidents next to CircleCI, GitHub and Linear.
Notes for reviewers
  • The plugin has two migration scripts (20260907 init and 20261103 commander columns), not one init script. Our deployment already applied both, so I did not merge them.
  • The docLink is empty until a Datadog page exists on the DevLake website. I can open that docs PR, and a Grafana dashboard, as follow-ups.
  • I could not add the pr-type/feature-development label myself. A maintainer has to add it.

🤖 Generated with Claude Code

stigi and others added 11 commits November 17, 2026 16:11
Collect incidents from Datadog Incident Management (`/api/v2/incidents`)
into the domain layer as issues of type INCIDENT, so teams whose on-call
runs on Datadog keep change failure rate and time to restore service.
DevLake already covers PagerDuty, Opsgenie, Rootly and incident.io;
Datadog is a common destination for teams consolidating monitoring,
paging and incident response, and until now the only path was the generic
webhook plugin, which carries no severity, detection timestamps or
custom fields.

Structure follows the incidentio plugin: connection, incident-type scope,
and collector/extractor/converter subtasks per entity.

Notes on the mapping:

- Auth is Datadog's key pair, `DD-API-KEY` plus `DD-APPLICATION-KEY`,
  sent as headers rather than a bearer token. The endpoint is
  region-specific (api.datadoghq.com, api.datadoghq.eu, ...).
- Scope is the incident type (`/api/v2/incidents/config/types`). Its
  `prefix` reconstructs the key Datadog shows in its UI (`IR-22`).
- Organization-specific vocabulary stays out of the plugin: the scope
  config names which custom field maps onto the domain Component
  (`componentField`) and which, if any, overrides the native SEV severity
  (`severityField`). The raw `fields` object is stored verbatim, so
  remapping does not require re-collection.
- Lead time prefers Datadog's own `time_to_repair`, which measures
  service restoration; created-to-resolved also counts post-incident
  work and overstates MTTR. A retrospectively declared incident can
  resolve before it was created, so that duration is dropped rather than
  wrapped into a bogus unsigned value.
- `is_test` incidents are drills and are not collected.
- The API returns no incident permalink, so the connection takes an
  optional `webUrl` to build one.

Refs: apache#9124

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the connection form and doc links so a Datadog connection can be
created from the UI rather than only through the API.

The connection form forwards a whitelist of keys to the test-connection
endpoints, so `apiKey`, `applicationKey` and `webUrl` are added there
too; without that, testing a new connection would post no credentials.

The bundled icon is a placeholder mark original to this contribution.
Datadog's own logo is a trademark and is deliberately not vendored here.

Refs: apache#9124

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Unit tests over the pure functions, with payloads shaped like real
Datadog responses:

- custom fields resolve through the scope config, a multi-select keeps
  its first value, and an empty configured field leaves the native
  severity alone
- test incidents and incidents of other types are skipped; an empty
  scope collects every type
- restore time prefers `time_to_repair` over created-to-resolved, and
  yields nothing for a retrospectively declared incident, whose negative
  duration would otherwise wrap into a huge unsigned value and corrupt
  MTTR
- an unknown incident state reports itself as unknown instead of failing

Both guards were checked by removing them: the matching tests fail.

Refs: apache#9124

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Verified the model against a real Datadog organization. Four changes:

- `time_to_repair` of 0 means "not recorded", not an instant restore. An
  incident that never passes through `stable` reports 0 while still
  carrying a real `time_to_resolve`, so taking the value literally
  reported an MTTR of zero for every such incident. Zero now falls
  through to the timestamps; a genuinely fast restore is reported in
  seconds and survives.
- Datadog mirrors its own attributes into the `fields` object, `slug`
  among them, so the issue key is the slug the UI shows (`IR-22`) with
  the incident type's prefix as the fallback.
- Pagination follows `meta.pagination.next_offset` rather than assuming
  offset arithmetic, and stops when it is absent or fails to advance.
  `page[size]` is the parameter Datadog honours; an unknown `page[limit]`
  is ignored, so the name matters.
- Collect the remaining attributes the API returns: `visibility`,
  `customer_impact_duration` and `time_to_internal_response`.

Custom-field values are commonly null (`root_cause`, `teams`), which must
read as absent rather than as an empty override; the tests now use
payloads shaped like the real responses, including the `autocomplete`
field type.

Refs: apache#9124

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Raw fixtures are synthetic but shaped like real responses, and cover the
cases where the mapping has to make a decision:

- restore time from `time_to_repair` (30 minutes) rather than the 95
  until resolution
- `time_to_repair` of 0, which means "not recorded", falling back to the
  timestamps
- an unresolved incident, which has neither resolution date nor lead time
- a retrospectively declared incident, resolved before it was created,
  which keeps the date and drops the duration
- a test incident and an incident of another type, neither collected
- an incident with no `slug` field, keyed from the type's prefix instead
- an incident whose custom fields are all null, so no component

Boards now carry the incident type's real creation date from Datadog.
`ticket.NewBoard` stamps CreatedDate with the row's insert time, which
is neither stable across collections nor true, so the board is built by
hand and the type's `createdAt` is collected to fill it.

Refs: apache#9124

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The connection form dispatches on a fixed set of field keys and returns
null for anything else, so declaring `apiKey`, `applicationKey` and
`webUrl` as plain field objects rendered nothing at all: the dialog asked
only for a name, a proxy and a rate limit, and a connection could not be
created from the UI.

Pass them as render functions, the pattern the opsgenie plugin already
uses for its custom endpoint field:

- both keys use a password input, and start empty on update so the
  sanitized value the form receives cannot overwrite the stored secret
- the UI URL is a plain optional input, since the API returns no incident
  permalink and it only affects whether issues carry a link

Also hide the "check out this doc" banner when a plugin declares no
docLink, rather than linking a page that does not exist, and drop the
placeholder DOC_URL entry that pointed at one.

Scopes listed from the remote-scopes endpoint now carry the incident
type's creation date on the tool model too, so the row is complete before
the first collection.

Verified against a local DevLake running the fork image: the form shows
API Key, Application Key and Datadog UI URL; a connection test with real
keys returns 200; and a collection of a production Datadog organization
produced 24 incidents through the raw, tool and domain layers, keyed by
slug (IR-1 … IR-24), with components mapped from the `services` custom
field and restore times taken from time_to_repair where recorded.

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The connections page orders tiles by each config's sort value, so the
default of appending at the end puts a new plugin far from its
alphabetical neighbours. Place Datadog beside ClickUp and CircleCI, where
a reader scanning the A-N group expects to find it.

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The scope config carries the whole point of the custom-field mapping —
which Datadog field becomes the domain component, and which may override
the SEV severity — but the plugin registered no scope-config routes, so
there was no way to set either one on a running instance. The value could
only be passed by hand-writing pipeline options.

Register the standard scope-config resources, as the circleci plugin does,
and declare the scope config in the config-ui plugin config so the Data
Scope Config form offers the two fields.

Generated-by: Claude Code (Claude Opus 5)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The plugin declared componentField/severityField defaults but registered no
transformation component, so config-ui showed an empty Transformations step.
Saving it created a scope config with an empty componentField that silently
overrode an API-set one, and the next collection wiped the mapped component.

Add a DatadogTransformation with inputs for the component field (maps a Datadog
incident field onto the domain component) and an optional severity override,
and wire it into the scope-config form.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
extractIncidentTypes writes the incident type, which is also the scope
row. The extractor deletes and re-inserts that row on every run without
ScopeConfigId, so the scope lost its scope config each night. The next
blueprint plan then carried an empty scope config, and incidents lost
their component.

Load the existing ScopeConfigId of each incident type before the
extractor replaces the rows, and set it on the new row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Grafana IRM and PagerDuty emit issue labels and assignees next to the
incident issue; the Datadog plugin emitted neither.

- Labels: every select-type custom field (dropdown, multiselect,
  autocomplete) becomes a `field:value` label, e.g. `services:api`.
  severity, state and state_category are skipped because the issue
  already carries them, and textbox fields hold free text.
- Assignee: the incident commander. The collector requests
  `include=users` and copies the commander's attributes from the page's
  `included` users into the incident's relationship, so the extractor
  needs no request per incident. The name falls back to the email for
  users without one.

The commander is stored on _tool_datadog_incidents in two new columns,
added by a migration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestAllGoPluginsListed compares the plugin directories that have an impl
package with the hand-kept allGoPlugins() list, and failed because the
Datadog plugin was missing from it. With the plugin in the list, the
schema test also runs its migrations on MySQL and PostgreSQL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][Plugin] Datadog Incident Management integration

2 participants