# Test in CI

Run your firmware on twins from your CI job, publish the results where your team already looks, and fail the job when a case does not pass.

One command, `ct cloud suite run`, sends a test suite to the hosted service, waits for it and writes the reports. You can run exactly what CI runs from your own terminal. Results describe the digital behavior your firmware observes on the modeled boards; they say nothing about analog, radio, thermal or cycle-level behavior.

## What you need

1. A CircuitTwin account, and in its workspace the systems your firmware runs on.
2. An [access token](https://circuittwin.ai/docs/tokens) with the `runs` and `suites` scopes, stored as a masked CI secret, for example `CIRCUITTWIN_TOKEN`.
3. A CI runner with `ct` installed (see [ct CLI](https://circuittwin.ai/docs/cli)). A Linux runner installs it with one command.
4. A suite file in your repository.

## The suite file

Firmware paths are relative to the suite file, so CI can point at the image the build just produced. Systems are referenced by id and must exist in the token’s workspace. Board and scenario files cannot be referenced on the server; use a bundled scenario by its id.

ct/suite.yaml

```
schemaVersion: ct.test-suite/1
id: firmware-smoke
name: Firmware smoke test
defaults:
  board:
    id: sample-nucleo-f767zi
  limits:
    maxVirtualTimeNs: 30000000000
    maxWallMs: 900000
cases:
  - id: boots
    name: Firmware boots and the timer interrupt fires
    firmware:
      path: ../build/firmware.elf
      name: firmware
    assertions:
      - kind: marker
        pass: CT-BAREMARKER-DONE
        fail: CT-BAREMARKER-TIMER-IRQ-FAIL
      - kind: no_fault
    tags: [smoke]
```

Try the assertions locally first with `ct test run ct/suite.yaml`.

## Run it in CI

`ct` needs two environment variables, so the same step works in GitHub Actions, GitLab CI or any other CI:

shell

```
export CT_SERVER=https://app.circuittwin.ai
export CT_TOKEN="$CIRCUITTWIN_TOKEN"
ct cloud suite run ct/suite.yaml --wait \
  --junit out/junit.xml --report out/ --markdown out/summary.md
```

- Each firmware image is uploaded once; an image the workspace already holds (same SHA-256) is reused.
- `--wait` waits until every case has ended, up to `--timeout` seconds (1800 by default). On timeout the queued cases are cancelled.
- `--junit` writes JUnit XML, which GitHub and GitLab can show as test results. `--report` writes the JSON report, `--markdown` a short summary for the job log or a comment.
- Without `--wait` the suite is submitted and the command returns; `ct cloud suite wait <ID>` collects the results later.

## Exit codes

| Code | Meaning                                                                                    |
| ---- | ------------------------------------------------------------------------------------------ |
| 0    | Every case passed                                                                          |
| 1    | A case failed, errored or was inconclusive (it ran on peripherals the twin does not model) |
| 2    | Invalid suite, input or credentials, including a token without the runs or suites scope    |
| 3    | The service could not run it, or the wait timed out                                        |

> **Gate on regressions, not only failures**
>
> Locally, `ct test run --baseline main-report.json --fail-on-regression` fails when a case that passed on the baseline now fails, or when a case’s report digest drifts.