CI and Automation

Agents that run in CI, such as PR review bots and pipeline automation, make MCP tool calls with no person at the keyboard. Running them through Ultra gives those calls the same traces, audit events, and governance rules as a developer laptop. This page uses GitHub Actions as the worked example. The same pattern applies to any CI system that can run a shell step and hold a secret.

Before you start

Create a workspace for CI

In Ultra Hub, create a workspace that holds only CI devices, separate from the ones your laptops and cloud dev environments link to. Policy and traces are scoped per workspace, so a CI-only workspace lets you govern pipeline agents without touching anyone’s laptop. See Workspaces.

Provision an identity for the pipeline

Decide the address this pipeline’s activity will be attributed to, and make sure it already exists as a member of the organization the deploy key belongs to. A device whose address does not resolve to a member stays ownerless, and its events are attributed to the machine rather than to that address. See Identities.

Create a deploy key

Go to Settings > Security and create a key in the Deploy Keys section, scoped to that workspace. Interactive ultra login is not available in CI, so the deploy key is how the runner authenticates. Copy the dk_... value immediately, it is shown only once. See Deploy keys.

Store the credentials as CI secrets

Add the deploy key as ULTRA_DEPLOY_KEY. Add any connector credentials the agent needs as their own secrets. Note that GitHub rejects secret names beginning with GITHUB_.

Quick start

Add the install and connector steps to your workflow before the step that runs your agent:

- name: Install Ultra and link to the ci workspace
  env:
    ULTRA_DEPLOY_KEY: ${{ secrets.ULTRA_DEPLOY_KEY }}
    ULTRA_DEVICE_EMAIL: ${{ vars.ULTRA_CI_IDENTITY }}
  run: |
    curl -sSL https://get.ultra.security/install.sh | bash
    ultra version

- name: Register connectors
  env:
    GH_MCP_TOKEN: ${{ secrets.GH_MCP_TOKEN }}
  run: |
    ultra connectors add filesystem \
      --command npx \
      --args "-y @modelcontextprotocol/server-filesystem ${{ github.workspace }}"

    ultra connectors add github --url https://api.githubcopilot.com/mcp/
    ultra config set-token github "$GH_MCP_TOKEN"

- name: Confirm the device linked
  run: ultra doctor -v

Pipe the install script to bash, not sh. The script checks for bash and exits if it does not find it, and /bin/sh is not bash on Debian and Ubuntu, which covers most Linux CI runners and container images.

Remote connectors authenticate with their own credentials, not with your CI system’s built-in token. The remote GitHub MCP endpoint above needs a personal access token or a GitHub App token; a GitHub Actions GITHUB_TOKEN is not accepted. Give the pipeline a dedicated token with the narrowest scopes the agent needs, and fail the job early when it is missing rather than letting the agent fail partway through a run.

Both connector forms work in CI without a prompt. ultra connectors add takes the transport and any environment variables as flags, and ultra config set-token takes the token as an argument, so nothing waits on input. Prefer that pair over the interactive ultra add, which prompts for credentials during setup. The runner is a fresh machine on every run, so it installs Ultra, links a device, and registers its connectors each time. Each run appears in Hub as a device in the workspace the deploy key is scoped to.

Pointing the agent at Ultra

A CI agent usually reads an MCP config from a path you control rather than from a user config directory. Commit a config that names Ultra as the only MCP server:

{
  "mcpServers": {
    "ultra": {
      "command": "ultra",
      "args": ["start"]
    }
  }
}

Then point your agent at that file. For Claude Code, pass it with --mcp-config, and add --strict-mcp-config so no other MCP configuration on the runner is picked up. For agents that read a config path Ultra already knows, ultra install --agent <name> --yes writes the entry for you. See Configure agents for the supported list and their paths.

An agent with other tools can route around Ultra

Ultra governs MCP tool calls. An agent that reaches the same service another way, for example by calling gh or curl from a shell tool, or by reading the checkout with a built-in file tool, produces no MCP traffic, and Ultra has nothing to trace or govern. This matters more in CI than on a laptop, because CI agents are often written to shell out. If you want the traces and the governance rules to reflect what the agent actually did, the agent needs the connector and no alternate route.

Check what your agent’s flags actually do before relying on them. In Claude Code, --allowed-tools only pre-approves matching tools so they run without a prompt. It does not restrict what is available. Restricting the available tools is a different flag, and removing a tool from the agent’s context entirely is --disallowed-tools. An agent configured with only an allow-list still has its built-in shell and file tools.

claude_args: >-
  --mcp-config .github/ultra-mcp.json
  --strict-mcp-config
  --tools ""
  --disallowed-tools "Bash"
  --allowed-tools "mcp__ultra"

Two different controls are at work here, and it is worth keeping them apart:

Which tools the agent has at all is your agent’s own configuration, not Ultra. Nothing in Ultra prevents someone adding a shell tool back later.
Which of Ultra’s tools the agent may actually call belongs in a governance rule scoped to the CI workspace. That is the half that survives a change to the workflow file, and it is the half Ultra enforces.

Relying on the agent’s flags alone gives you a configuration promise. Pairing them with a governance rule gives you an enforced one.

Identity

There is no person behind a CI run, so set the identity explicitly. Keep the value in a CI variable rather than hardcoding it in the workflow file:

env:
  ULTRA_DEVICE_EMAIL: ${{ vars.ULTRA_CI_IDENTITY }}

Set ULTRA_CI_IDENTITY to an address you reserve for that pipeline, and use a distinct address per pipeline rather than a shared one, so a noisy or misbehaving agent can be told apart from the rest of your automation.

The address only attributes activity when it resolves to an existing member of the deploy key’s organization. If it does not resolve, the device is left without an owner, its identity status is reported as unresolved, and its events are attributed to the machine rather than to the address you set. Provision the member or service identity before the first run, and check the device in Hub afterwards rather than assuming attribution worked.

Moving from monitor to enforcement

CI is usually the first surface where enforcement is worth turning on. The agents are non-interactive, their tool use is narrow and repeatable, and nobody is blocked waiting on an approval.

Run in monitor first

Leave the workspace on its inherited posture and let real pipeline runs build the tool catalog. You need to see what the agent actually calls before you can allowlist it.

Write the allow rules

On the Governance page, scope rules to the CI workspace and allow the connectors and tools the pipeline needs. This is also where you narrow a broad connector down to the operations the agent should be making, rather than leaving every tool it exposes reachable.

Set the workspace posture to default_deny

With the allow rules in place, anything the pipeline was not approved for is denied at the device before it reaches the upstream connector, and the denial is recorded in the audit log.

A denied call surfaces to the agent as a failed tool call. Whether the job then fails, retries, or carries on depends on how your agent handles tool errors, so run at least one deliberate deny against a non-blocking pipeline before you rely on this in a required check.

Verification

# Ultra installed on the runner
ultra version

# Device linked, identity resolved, connectors reachable
ultra doctor -v

# Connectors registered for this run
ultra connectors

In Hub, the run appears as a device in the CI workspace, and traces appear within about 60 seconds of the agent’s first MCP tool call. A green check is not the signal to trust here. The trace list is. A job can pass having done nothing, so confirm the workspace filled before concluding the pipeline is wired correctly.

Make the job prove the agent worked

Build that check into the pipeline rather than leaving it to habit. Have the agent emit something the job can look for, such as a marker line in the comment it posts, and add a final step that fails when the marker is absent. Otherwise a denied tool call leaves the agent exiting cleanly and the job going green with no work done.

The marker has to be unique to the run. A fixed string is satisfied by anything that happens to contain it, including an earlier comment, a template, or someone describing how the check works, so the job passes without the agent having done anything. Build the run identifier into the marker and match it exactly.

env:
  REVIEW_MARKER: <!-- review:${{ github.run_id }} -->

# ...then, after the agent step
gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" --paginate --jq '.[].body' \
  | grep -qF "$REVIEW_MARKER" || {
      echo "::error::The agent did not post. Check the workspace traces."
      exit 1
    }

A verification step that can be satisfied by something other than the work is decorative. It is worth testing it against a run you know did nothing, so you find that out deliberately rather than the first time it matters.

Troubleshooting

The install step exits with 'This script requires bash'

The script was piped to sh. On Debian and Ubuntu, including the standard Linux CI runners, /bin/sh is dash rather than bash. Pipe to bash instead. Every later step then fails with ultra: command not found, which is a symptom of this rather than a separate problem.

The job passes but the agent never reviewed anything

Some agent actions refuse to run when the workflow file on the branch differs from the copy on your default branch, a guard against a pull request modifying the workflow that reviews it. The agent step is skipped and the job still succeeds, so the check goes green while nothing was reviewed. Expect this on the pull request that first adds the workflow, and treat it as resolved once the file is merged to the default branch. It is also a reason not to make a run like this a required check without also checking what it produced.

No device appears in Hub

Confirm ULTRA_DEPLOY_KEY is set on the step that runs the install script, and that the key has not been revoked. ultra doctor -v reports link status and identity resolution.

The device appears but its activity is not attributed to your CI identity

The address did not resolve to a member of the deploy key’s organization, so the device has no owner and its events are machine-attributed. Provision that identity in the organization, then re-run. ultra doctor -v reports identity resolution on the runner.

The device links to the wrong workspace

A deploy key carries its own workspace, so the device links wherever the key points. Check which workspace the key was created under rather than looking for a workspace setting on the runner.

The job runs but Hub shows no traces

The agent is most likely not calling MCP at all. Confirm it is reading the config that names Ultra, and that it still has a shell or file tool it can reach the same service with. An allow-list alone does not remove those.

A remote connector returns an authentication error

Confirm the token was set for the connector name you registered, since ultra config set-token takes that name as its first argument. Check that the token itself carries the scopes the connector needs, and that it is a credential the upstream accepts. A CI system’s built-in token is often not one.

Secrets are missing on automated PRs

Some CI systems run bot-authored builds against a separate secret store. On GitHub Actions, Dependabot runs do not see Actions secrets, so guard the job rather than letting it fail at the install step.

Getting Started

Ultra Hub

Observability

Concepts

CI and Automation

Before you start

Quick start

Pointing the agent at Ultra

An agent with other tools can route around Ultra

Identity

Moving from monitor to enforcement

Verification

Make the job prove the agent worked

Troubleshooting

​Before you start

​Quick start

​Pointing the agent at Ultra

​An agent with other tools can route around Ultra

​Identity

​Moving from monitor to enforcement

​Verification

​Make the job prove the agent worked

​Troubleshooting

Before you start

Quick start

Pointing the agent at Ultra

An agent with other tools can route around Ultra

Identity

Moving from monitor to enforcement

Verification

Make the job prove the agent worked

Troubleshooting