Skip to content

Resolving CI Issues with Goose: A Practical Walkthrough

CI

Running into Continuous Integration (CI) failures in pull requests can be quite frustrating but they happen very often. In this post, we leverage the GitHub CLI (gh) using Goose to simplify your CI debugging process, fetch detailed information about failed CI runs & annotations directly from GitHub, and even apply fixes directly.

Getting Started

Before diving in, ensure you have the necessary tools set up.

1. Install and Authenticate GitHub CLI (gh)

You'll need the GitHub CLI gh to enable Goose's access to CI check run details. 

brew install gh
gh auth login

Follow the prompts to authenticate your account.

2. Configure Goose

Ensure Goose is configured and ready to interact with your repository and local tools. Specifically, you will need to configure a goose profile with the GitHub toolkit.

Update your profiles.yaml file in ~/.config/goose with the necessary toolkits:

my-profile:
  provider: anthropic
  processor: claude-3-5-sonnet
  accelerator: claude-3-5-sonnet
  moderator: truncate
  toolkits:
    - name: developer
      requires: {}
    - name: github
      requires: {}

Start a session with the profile:

goose session start --profile my-profile

If the profile’s toolkits are not correctly configured, Goose might fail to access gh commands.

Resolving CI Failures with Goose

Once everything is set up, we can now use Goose to tackle CI issues with the following steps:

Step 1: Fetch CI Failure Details

Ask Goose to retrieve information about the CI failures for a specific PR.

Example Request:

Can you tell me about the CI failures on PR #11111 in the organization/repo repository? The check run ID is 123456.

You can access a check run ID by clicking on a PR’s checks tab and selecting a failure. The ID will be available in the URL.

Goose's Workflow:

Goose will use the gh CLI to check run information and provide a detailed summary of the issues. Goose uses the GitHub CLI to:

  • Fetch details about failed CI runs, including logs and annotations.
  • Provide a clear and actionable summary of issues.

Example Command Goose Runs:

gh api -H "Accept: application/vnd.github+json" /repos/<owner>/<repo>/check-runs/<check-run-id>/annotations

Example Goose Output:

Here are the details for Check Run ID 123456:

**Name**: CI-Build-All  
**Status**: completed  
**Conclusion**: failure  

**Annotations Summary:**  
1. **ModuleA Test Failure**  
   File: `ModuleATests`  
   Line: 35  
   Error: Mismatched parameter order in function call.  

2. **ModuleB Test Failure**  
   File: `ModuleBTests`  
   Line: 64  
   Error: Unexpected nil value encountered during initialization.  

3. **ModuleC Test Failure**  
   File: `ModuleCTests`  
   Line: 98  
   Error: Type mismatch in returned value. Expected 'String', found 'Int'.

Step 2: Request Fixes from Goose

Once Goose provides the failure details, ask it to resolve the issues directly.

Example Request:

Can you fix these issues and stage the changes so I can review them? Do not add unnecessary code comments.

What Goose Does:

  • Applies fixes based on error messages.
  • Stages changes for review.

Example Goose Response:

I have fixed the following files and staged them:

- `ModuleATests`: Corrected the order of parameters in the function call.  
- `ModuleBTests`: Resolved the unexpected nil value encountered during initialization.  
- `ModuleCTests`: Adjusted the type mismatch in the returned value.  

You can now review the staged changes.

Step 3: Review the Changes

Carefully review the staged changes to ensure they address the issues correctly and avoid unintended modifications.

Things to Check:

  • No unnecessary comments (e.g., // Fix xyz).
  • No alterations to unrelated parts of the code.

If needed, clean up or re-stage the changes before committing.

Benefits of Using Goose

Using Goose to resolve CI issues streamlines your workflow by:

  • identifying issues and applying fixes with minimal manual effort.
  • integrating with tools like the GitHub CLI to validate changes.
  • handling repetitive CI debugging tasks while you focus on code quality.

Goose allows you to resolve CI failures efficiently, ensuring confidence in your workflow while reducing the effort required for debugging and testing.

Try it out, and let Goose handle the heavy lifting of CI debugging for you!