Build an Agent Feedback Loop

This guide builds a job where an agent fixes failing tests and lint errors. The job runs two checks: npm test and npm run lint. Each check has its own recovery step. When a check fails, Shipfox runs the recovery step for that check. Shipfox also tells the recovery step what failed.

Before you beginCopy Anchor Link

You need:

• A repository with npm test and npm run lint commands.
• Configured agent defaults for the project.
• A safe branch where an agent can edit test and lint failures.

Add the workflowCopy Anchor Link

Create .shipfox/workflows/fix-tests.yml. This is a complete workflow:

# yaml-language-server: $schema=https://www.shipfox.io/docs/workflow.schema.json
name: Fix failing tests
runner: shipfox

triggers:
  manual:
    source: manual

jobs:
  improve:
    execution_timeout: "30m"
    steps:
      - key: implement
        prompt: >
          Fix the failing tests and lint errors in this repository.
          Run both checks before finishing.

      - key: test_failure_handler
        if: '${{ has(step.restart) && has(step.restart.from.key) && step.restart.from.key == "test" }}'
        prompt: Fix the failing tests and correct their cause.

      - key: test
        run: npm test
        gate:
          success: step.exit_code == 0
          on_failure:
            restart_from: test_failure_handler
            feedback: The tests still fail. Run them again and fix the cause.
            max_attempts: 8

      - key: lint_failure_handler
        if: '${{ has(step.restart) && has(step.restart.from.key) && step.restart.from.key == "lint" }}'
        prompt: Fix the lint errors without weakening the rules.

      - key: lint
        run: npm run lint
        gate:
          success: step.exit_code == 0
          on_failure:
            restart_from: lint_failure_handler
            feedback: Lint still fails. Run it again and fix the reported issues.

Replace npm test and npm run lint with the commands that check your work. Keep a key on each checking step. Each recovery step uses that key to know which check failed.

Run and inspect itCopy Anchor Link

1. Commit and push the workflow to the project's default branch.
2. Wait for its definition to sync.
3. Open the workflow and select Run.
4. Open the improve job in the run detail.
5. Confirm both recovery steps skip before their matching check fails.
6. If test fails, confirm only test_failure_handler runs.
7. After test passes, confirm lint_failure_handler stays skipped.
8. If lint fails, confirm only lint_failure_handler runs.
9. Confirm the job finishes after both checks pass.

Each recovery step runs only for its own check, because its if condition compares the key of the failed step. step.is_retry cannot do this. It tells a step that a restart occurred, but not which check failed.

Gate fields lists the gate fields and their rules. The Contexts reference lists the step.restart fields and when each one is available.

Set the loop limitsCopy Anchor Link

Two limits stop a loop: the attempt limit and the job timeout.

Attempt limitCopy Anchor Link

gate.on_failure.max_attempts sets how many times the gating step can run. The first run counts as attempt 1.

• Use a whole number from 1 through 1,000.
• If you omit the field, the step can run five times.
• You cannot remove the limit.
• With a list of rules, each rule has its own value. The value counts every run of the step. See Restart or fail.

When the step fails on its last attempt, the job fails with the status restart_exhausted.

Job timeoutCopy Anchor Link

The job timeout sets how long the job can run. The time includes every attempt. The maximum is 24 hours.

When the time ends, the job fails with the status timed_out. This can happen before the step uses all its attempts.

LogsCopy Anchor Link

Each job can keep a limited amount of log output. A long loop can go over this amount. The job continues to run, but Shipfox does not keep the new log output. Later attempts can have incomplete logs or no logs.

Steps that writeCopy Anchor Link

A restart runs the steps in the loop again. A step that writes outside the job, such as a push or a comment, can write more than one time. Put such a step after the gate. If it must stay in the loop, make sure that it is safe to run again.

Limits lists the exact values. Feedback loops explains how a loop works.