Repository navigation
feat(datadog): add Datadog incident management data source plugin - #9406
Open
stigi-work wants to merge 12 commits into
Open
stigi-work wants to merge 12 commits into
stigi-work wants to merge 12 commits into
Conversation
Collect incidents from Datadog Incident Management (`/api/v2/incidents`) into the domain layer as issues of type INCIDENT, so teams whose on-call runs on Datadog keep change failure rate and time to restore service. DevLake already covers PagerDuty, Opsgenie, Rootly and incident.io; Datadog is a common destination for teams consolidating monitoring, paging and incident response, and until now the only path was the generic webhook plugin, which carries no severity, detection timestamps or custom fields. Structure follows the incidentio plugin: connection, incident-type scope, and collector/extractor/converter subtasks per entity. Notes on the mapping: - Auth is Datadog's key pair, `DD-API-KEY` plus `DD-APPLICATION-KEY`, sent as headers rather than a bearer token. The endpoint is region-specific (api.datadoghq.com, api.datadoghq.eu, ...). - Scope is the incident type (`/api/v2/incidents/config/types`). Its `prefix` reconstructs the key Datadog shows in its UI (`IR-22`). - Organization-specific vocabulary stays out of the plugin: the scope config names which custom field maps onto the domain Component (`componentField`) and which, if any, overrides the native SEV severity (`severityField`). The raw `fields` object is stored verbatim, so remapping does not require re-collection. - Lead time prefers Datadog's own `time_to_repair`, which measures service restoration; created-to-resolved also counts post-incident work and overstates MTTR. A retrospectively declared incident can resolve before it was created, so that duration is dropped rather than wrapped into a bogus unsigned value. - `is_test` incidents are drills and are not collected. - The API returns no incident permalink, so the connection takes an optional `webUrl` to build one. Refs: apache#9124 Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the connection form and doc links so a Datadog connection can be created from the UI rather than only through the API. The connection form forwards a whitelist of keys to the test-connection endpoints, so `apiKey`, `applicationKey` and `webUrl` are added there too; without that, testing a new connection would post no credentials. The bundled icon is a placeholder mark original to this contribution. Datadog's own logo is a trademark and is deliberately not vendored here. Refs: apache#9124 Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Unit tests over the pure functions, with payloads shaped like real Datadog responses: - custom fields resolve through the scope config, a multi-select keeps its first value, and an empty configured field leaves the native severity alone - test incidents and incidents of other types are skipped; an empty scope collects every type - restore time prefers `time_to_repair` over created-to-resolved, and yields nothing for a retrospectively declared incident, whose negative duration would otherwise wrap into a huge unsigned value and corrupt MTTR - an unknown incident state reports itself as unknown instead of failing Both guards were checked by removing them: the matching tests fail. Refs: apache#9124 Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Verified the model against a real Datadog organization. Four changes: - `time_to_repair` of 0 means "not recorded", not an instant restore. An incident that never passes through `stable` reports 0 while still carrying a real `time_to_resolve`, so taking the value literally reported an MTTR of zero for every such incident. Zero now falls through to the timestamps; a genuinely fast restore is reported in seconds and survives. - Datadog mirrors its own attributes into the `fields` object, `slug` among them, so the issue key is the slug the UI shows (`IR-22`) with the incident type's prefix as the fallback. - Pagination follows `meta.pagination.next_offset` rather than assuming offset arithmetic, and stops when it is absent or fails to advance. `page[size]` is the parameter Datadog honours; an unknown `page[limit]` is ignored, so the name matters. - Collect the remaining attributes the API returns: `visibility`, `customer_impact_duration` and `time_to_internal_response`. Custom-field values are commonly null (`root_cause`, `teams`), which must read as absent rather than as an empty override; the tests now use payloads shaped like the real responses, including the `autocomplete` field type. Refs: apache#9124 Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Raw fixtures are synthetic but shaped like real responses, and cover the cases where the mapping has to make a decision: - restore time from `time_to_repair` (30 minutes) rather than the 95 until resolution - `time_to_repair` of 0, which means "not recorded", falling back to the timestamps - an unresolved incident, which has neither resolution date nor lead time - a retrospectively declared incident, resolved before it was created, which keeps the date and drops the duration - a test incident and an incident of another type, neither collected - an incident with no `slug` field, keyed from the type's prefix instead - an incident whose custom fields are all null, so no component Boards now carry the incident type's real creation date from Datadog. `ticket.NewBoard` stamps CreatedDate with the row's insert time, which is neither stable across collections nor true, so the board is built by hand and the type's `createdAt` is collected to fill it. Refs: apache#9124 Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The connection form dispatches on a fixed set of field keys and returns null for anything else, so declaring `apiKey`, `applicationKey` and `webUrl` as plain field objects rendered nothing at all: the dialog asked only for a name, a proxy and a rate limit, and a connection could not be created from the UI. Pass them as render functions, the pattern the opsgenie plugin already uses for its custom endpoint field: - both keys use a password input, and start empty on update so the sanitized value the form receives cannot overwrite the stored secret - the UI URL is a plain optional input, since the API returns no incident permalink and it only affects whether issues carry a link Also hide the "check out this doc" banner when a plugin declares no docLink, rather than linking a page that does not exist, and drop the placeholder DOC_URL entry that pointed at one. Scopes listed from the remote-scopes endpoint now carry the incident type's creation date on the tool model too, so the row is complete before the first collection. Verified against a local DevLake running the fork image: the form shows API Key, Application Key and Datadog UI URL; a connection test with real keys returns 200; and a collection of a production Datadog organization produced 24 incidents through the raw, tool and domain layers, keyed by slug (IR-1 … IR-24), with components mapped from the `services` custom field and restore times taken from time_to_repair where recorded. Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The connections page orders tiles by each config's sort value, so the default of appending at the end puts a new plugin far from its alphabetical neighbours. Place Datadog beside ClickUp and CircleCI, where a reader scanning the A-N group expects to find it. Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The scope config carries the whole point of the custom-field mapping — which Datadog field becomes the domain component, and which may override the SEV severity — but the plugin registered no scope-config routes, so there was no way to set either one on a running instance. The value could only be passed by hand-writing pipeline options. Register the standard scope-config resources, as the circleci plugin does, and declare the scope config in the config-ui plugin config so the Data Scope Config form offers the two fields. Generated-by: Claude Code (Claude Opus 5) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The plugin declared componentField/severityField defaults but registered no transformation component, so config-ui showed an empty Transformations step. Saving it created a scope config with an empty componentField that silently overrode an API-set one, and the next collection wiped the mapped component. Add a DatadogTransformation with inputs for the component field (maps a Datadog incident field onto the domain component) and an optional severity override, and wire it into the scope-config form. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
extractIncidentTypes writes the incident type, which is also the scope row. The extractor deletes and re-inserts that row on every run without ScopeConfigId, so the scope lost its scope config each night. The next blueprint plan then carried an empty scope config, and incidents lost their component. Load the existing ScopeConfigId of each incident type before the extractor replaces the rows, and set it on the new row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Grafana IRM and PagerDuty emit issue labels and assignees next to the incident issue; the Datadog plugin emitted neither. - Labels: every select-type custom field (dropdown, multiselect, autocomplete) becomes a `field:value` label, e.g. `services:api`. severity, state and state_category are skipped because the issue already carries them, and textbox fields hold free text. - Assignee: the incident commander. The collector requests `include=users` and copies the commander's attributes from the page's `included` users into the incident's relationship, so the extractor needs no request per incident. The name falls back to the email for users without one. The commander is stored on _tool_datadog_incidents in two new columns, added by a migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3 tasks done
TestAllGoPluginsListed compares the plugin directories that have an impl package with the hand-kept allGoPlugins() list, and failed because the Datadog plugin was missing from it. With the plugin in the list, the schema test also runs its migrations on MySQL and PostgreSQL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a data source plugin for Datadog Incident Management. It is in the same category as the PagerDuty, Opsgenie, Rootly and incident.io plugins, and it feeds the existing DORA incident metrics.
DD-API-KEY/DD-APPLICATION-KEY). The endpoint is configurable for each Datadog site (US1, EU1, …). A separate web URL builds the incident links./api/v2/incidents/config/types), with remote-scope listing./api/v2/incidentswithpage[size]/page[offset]pagination andinclude=usersfor the commander.is_test) are skipped.issueswith typeINCIDENT, plusboardsandboard_issuesfor each incident type.issue_labelsfrom select-type custom fields, written asfield:value.issue_assigneesfrom the incident commander.lead_time_minutesuses Datadog'stime_to_repairand falls back to created → resolved. A zero value or a negative duration does not count as an instant restore.componentFieldmaps a custom field (for exampleservices) toissues.component. The optionalseverityFieldreplaces the native SEV severity with a custom field.Closes #9124
Tests
issues,boards,board_issues,issue_labelsandissue_assignees.Notes for reviewers
20260907init and20261103commander columns), not one init script. Our deployment already applied both, so I did not merge them.docLinkis empty until a Datadog page exists on the DevLake website. I can open that docs PR, and a Grafana dashboard, as follow-ups.pr-type/feature-developmentlabel myself. A maintainer has to add it.🤖 Generated with Claude Code