> ## Documentation Index
> Fetch the complete documentation index at: https://docs.autonoma.app/llms.txt
> Use this file to discover all available pages before exploring further.
# CI/CD Integration Guide
> Learn how to integrate Autonoma with your CI/CD pipeline to automatically run tests on code changes using GitHub Actions, GitLab CI, Bitbucket Pipelines, or cURL.
## Integration Options
Autonoma supports multiple CI/CD platforms. Choose the one your team uses:
## What is CI/CD and Why Does It Matter?
**CI/CD** stands for **Continuous Integration / Continuous Deployment**—it's the practice of automatically testing and deploying code changes.
Let's break down what this means in plain language:
### Continuous Integration (CI)
**Traditional approach**: Developers work on features for weeks, then try to merge everything together. Often, things break because changes conflict with each other.
**Continuous Integration**: Developers integrate (merge) their code changes frequently—multiple times per day. Every time code is merged, automated tests run to catch problems immediately.
**Analogy**: Instead of writing an entire book and checking for typos at the end (overwhelming!), you check each page as you write it (manageable!).
### Continuous Deployment (CD)
**Traditional approach**: After testing, someone manually deploys the code to production—copying files, restarting servers, hoping nothing goes wrong.
**Continuous Deployment**: Code that passes all tests automatically deploys to production. No manual steps, no human error.
**Analogy**: Instead of manually mailing each customer their order, you have an automated system that ships orders as soon as they're processed.
### Why This Matters for Testing
Without CI/CD:
* Tests are run manually (slow, easy to forget)
* Broken code can sit undetected for days
* Deployments are risky and stressful
With CI/CD:
* Tests run automatically on every code change
* Problems are caught within minutes
* Deployments become boring and routine (which is good!)
## How Autonoma Fits Into CI/CD
Your development workflow probably looks something like this:
1. Developer writes code
2. Developer creates a Pull Request (PR) / Merge Request
3. **← Autonoma runs tests here**
4. Code review happens
5. Code merges to main branch
6. **← Autonoma runs tests here**
7. Code deploys to production
8. **← Autonoma runs tests here**
By integrating Autonoma at these checkpoints, you catch issues before they reach users.
## Prerequisites
### Generate your API Key
Go to the API Keys section in your Autonoma dashboard and generate a key. You will see something like this once done.
You'll need:
* **Client ID**: Your unique identifier
* **Client Secret**: Your secret key (keep this secure!)
## GitHub Actions
GitHub Actions is GitHub's built-in automation platform. If your code is on GitHub, this is the easiest option.
#### Step 1: Go to Settings > [Integrations](https://autonoma.app/settings/integrations) in your Autonoma dashboard.
#### Step 2: Select the tests or folders that you want to run in your CI, copy the generated action job!
#### Step 3: Add to GitHub Workflow
Copy the action job configuration from the Integrations page and paste it into your GitHub Actions workflow file (`.github/workflows/deploy.yml`):
#### Step 4: Set up GitHub Secrets
1. In your GitHub repository, go to Settings → Secrets and variables → Actions
2. Add your API credentials as secrets:
* `AUTONOMA_CLIENT_ID`: Your client ID from the API Keys section
* `AUTONOMA_CLIENT_SECRET`: Your client secret from the API Keys section
```yaml deploy.yml theme={null}
jobs:
run_autonoma_tests:
runs-on: ubuntu-latest
name: Run Autonoma Tests
steps:
- name: Run Single Test (cmbi807au0172xv01dqu4drhi)
id: step-1
uses: autonoma-ai/actions/test-runner@v1
with:
test-id: 'cmbi807au0172xv01dqu4drhi'
client-id: ${{ secrets.AUTONOMA_CLIENT_ID }}
client-secret: ${{ secrets.AUTONOMA_CLIENT_SECRET }}
max-wait-time: '10'
- name: Show cmbi807au0172xv01dqu4drhi results
if: always()
run: |
echo "Test status: ${{ steps.step-1.outputs.final-status }}"
echo "Message: ${{ steps.step-1.outputs.message }}"
echo "View results at: ${{ steps.step-1.outputs.url }}"
```
#### Step 5: Customize When Tests Run
The default configuration runs on every push. You can customize this:
**Run on every Pull Request**:
```yaml theme={null}
on:
pull_request:
branches: [ main, develop ]
```
**Run on every push to main**:
```yaml theme={null}
on:
push:
branches: [ main ]
```
**Run on both PR and push**:
```yaml theme={null}
on:
pull_request:
branches: [ main ]
push:
branches: [ main ]
```
**Run on schedule** (e.g., nightly):
```yaml theme={null}
on:
schedule:
- cron: '0 2 * * *' # Runs at 2 AM UTC every day
```
### How It Works
Once configured:
1. Developer creates a Pull Request
2. GitHub automatically runs your workflow
3. Autonoma receives the request and runs your tests
4. Results appear in the GitHub PR:
* ✅ Green checkmark if tests pass
* ❌ Red X if tests fail
5. Team can see test results before merging
**Result**: No broken code reaches your main branch!
## GitLab CI
GitLab CI is GitLab's built-in automation platform. If your code is on GitLab, use this option.
#### Step 1: Go to Settings > [Integrations](https://autonoma.app/settings/integrations) in your Autonoma dashboard.
#### Step 2: Add Credentials to GitLab CI/CD Variables
1. Go to your GitLab project
2. Navigate to **Settings** → **CI/CD**
3. Expand **Variables**
4. Click **Add variable**
5. Key: `CLIENT_ID`, Value: Your Autonoma client ID
6. Key: `CLIENT_SECRET`, Value: Your Autonoma client secret
7. Check **Mask variable** (hides it in logs)
8. Click **Add variable**
#### Step 2: Create GitLab CI Configuration
In your repository, create or edit `.gitlab-ci.yml`:
```yaml .gitlab-ci.yml theme={null}
stages:
- test
autonoma_tests:
stage: test
script:
- curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location "https://api.prod.autonoma.app/v1/run/folder/$FOLDER_ID" \
--header "autonoma-client-id: $CLIENT_ID" \
--header "autonoma-client-secret: $CLIENT_SECRET" \
--header "Content-Type: application/json" || true
only:
- merge_requests
- main
```
👉 Set `FOLDER_ID` as an environment variable in GitLab → Settings → CI/CD → Variables.
### Customizing GitLab CI
**Run only on merge requests**:
```yaml theme={null}
only:
- merge_requests
```
**Run different tests on different branches**:
```yaml theme={null}
smoke_tests:
stage: test
script:
- curl -X POST ... --location "https://api.prod.autonoma.app/v1/run/folder/$SMOKE_FOLDER_ID" ...
only:
- merge_requests
regression_tests:
stage: test
script:
- curl -X POST ... --location "https://api.prod.autonoma.app/v1/run/folder/$REGRESSION_FOLDER_ID" ...
only:
- main
```
## Bitbucket Pipelines
Bitbucket Pipelines is Bitbucket's built-in automation platform.
#### Step 1: Go to Settings > [Integrations](https://autonoma.app/settings/integrations) in your Autonoma dashboard.
#### Step 2: Add Credentials to Bitbucket Variables
1. Go to your Bitbucket repository
2. Navigate to **Repository settings**
3. Click **Repository variables** (under Pipelines)
4. Click **Add variable**
5. Name: `CLIENT_ID`, Value: Your Autonoma client ID
6. Name: `CLIENT_SECRET`, Value: Your Autonoma client secret
7. Check **Secured** (encrypts the variable)
8. Click **Add**
#### Step 2: Create Bitbucket Pipeline
In your repository, create or edit `bitbucket-pipelines.yml`:
```yaml bitbucket-pipelines.yml theme={null}
pipelines:
default:
- step:
name: Deploy
script:
- curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location "https://api.prod.autonoma.app/v1/run/folder/$FOLDER_ID" \
--header "autonoma-client-id: $CLIENT_ID" \
--header "autonoma-client-secret: $CLIENT_SECRET" \
--header "Content-Type: application/json" || true
```
👉 Set `FOLDER_ID` in Bitbucket Repository Variables under Repository settings → Pipelines → Repository variables.
## cURL (Universal)
If you use Jenkins, CircleCI, Travis CI, or any other CI/CD system, you can use cURL commands directly.
### What is cURL?
**cURL** is a command-line tool that makes HTTP requests. It's pre-installed on most systems and works everywhere.
Think of it like this: instead of clicking buttons in a web browser, you're telling your computer to visit a URL and perform an action.
### Getting Your cURL Command
You can build the cURL command manually or find it in Autonoma:
1. Log into Autonoma
2. Go to a test or folder
3. Click the three dots menu
4. Select "Get cURL command"
5. Copy the generated command
### cURL Command Structure
Here's a basic cURL command for running tests:
```bash theme={null}
curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location 'https://api.prod.autonoma.app/v1/run/folder/' \
--header 'autonoma-client-id: ' \
--header 'autonoma-client-secret: ' \
--header 'Content-Type: application/json' || true
```
Let's break it down:
* `curl` - The command tool
* `-X POST` - Type of request (POST means we're sending data)
* `--silent` - Don't show progress meter
* `--retry 3` - Retry up to 3 times on failure
* `--retry-connrefused` - Retry on connection refused
* `--location` - Follow redirects
* `--header` - Authentication headers
* `|| true` - Don't fail the pipeline if the command fails
### Running Different Test Groups
**Run tests by folder**:
```bash theme={null}
curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location 'https://api.prod.autonoma.app/v1/run/folder/' \
--header 'autonoma-client-id: ' \
--header 'autonoma-client-secret: ' \
--header 'Content-Type: application/json' || true
```
**Run a single test**:
```bash theme={null}
curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location 'https://api.prod.autonoma.app/v1/run/test/' \
--header 'autonoma-client-id: ' \
--header 'autonoma-client-secret: ' \
--header 'Content-Type: application/json' || true
```
### Using cURL in Different CI Systems
#### Jenkins
Add an Execute Shell build step:
```bash theme={null}
curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location "https://api.prod.autonoma.app/v1/run/folder/$FOLDER_ID" \
--header "autonoma-client-id: $CLIENT_ID" \
--header "autonoma-client-secret: $CLIENT_SECRET" \
--header "Content-Type: application/json" || true
```
#### CircleCI
Add to `.circleci/config.yml`:
```yaml theme={null}
jobs:
test:
steps:
- run:
name: Run Autonoma Tests
command: |
curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location "https://api.prod.autonoma.app/v1/run/folder/$FOLDER_ID" \
--header "autonoma-client-id: $CLIENT_ID" \
--header "autonoma-client-secret: $CLIENT_SECRET" \
--header "Content-Type: application/json" || true
```
#### Travis CI
Add to `.travis.yml`:
```yaml theme={null}
script:
- curl -X POST \
--silent \
--retry 3 \
--retry-connrefused \
--location "https://api.prod.autonoma.app/v1/run/folder/$FOLDER_ID" \
--header "autonoma-client-id: $CLIENT_ID" \
--header "autonoma-client-secret: $CLIENT_SECRET" \
--header "Content-Type: application/json" || true
```
## Best Practices for CI/CD Integration
### 1. Run Different Tests at Different Stages
**On Pull Requests** → Run smoke tests (fast feedback)
```
Tags: smoke
Time: ~5 minutes
Purpose: Catch critical breaks quickly
```
**Before Merging** → Run regression tests (thorough check)
```
Tags: regression
Time: ~20 minutes
Purpose: Comprehensive verification
```
**After Deployment** → Run smoke tests in production
```
Tags: smoke, production-safe
Time: ~5 minutes
Purpose: Verify deployment succeeded
```
### 2. Fail Fast
Configure your CI/CD to stop if Autonoma tests fail:
```yaml theme={null}
# Example for GitHub Actions
- name: Run Tests
run: curl ...
# If curl returns non-zero (test failed), pipeline stops
```
This prevents broken code from being deployed.
### 3. Set Up Notifications
Configure notifications so the team knows when tests fail:
**In Autonoma**:
1. Go to Integrations
2. Set up Slack notifications
3. Choose which failures to notify about
**In Your CI/CD System**:
Most platforms (GitHub, GitLab, etc.) automatically notify on pipeline failures.
### 4. Use Descriptive Job Names
**Bad**:
```yaml theme={null}
- name: Tests
```
**Good**:
```yaml theme={null}
- name: Autonoma Smoke Tests - Authentication and Checkout
```
When tests fail, the job name tells you what broke.
### 5. Don't Over-Test
You don't need to run your entire test suite on every single change.
**Smart approach**:
* Every PR: Smoke tests (5-10 critical tests)
* Before merge: Regression tests (all tests)
* Nightly: Regression tests (catches accumulated issues)
* After deploy: Smoke tests (verification)
## Troubleshooting CI/CD Integration
### Tests aren't running
**Check**:
* Is your API token correct?
* Is the token properly stored as a secret/variable?
* Is the workflow/pipeline file in the correct location?
* Are you triggering the right events (PR, push, etc.)?
### Tests run but always fail
**Check**:
* Do tests pass when run manually in Autonoma?
* Is the test environment accessible from CI/CD?
* Are there environment-specific issues (URLs, credentials)?
### CI/CD times out waiting for tests
**Check**:
* Are you running too many tests? Run fewer tests more frequently
* Is there a timeout configuration in your CI/CD? Increase it
* Are tests stuck on a loading state? Add better waits
### Don't see test results in CI/CD
**Check**:
* Is the cURL command configured to show output?
* Are you checking the right logs/build output?
* Is Autonoma receiving the request? (Check Autonoma dashboard)
## Key Takeaways
1. **CI/CD automates test execution** - no manual test running required
2. **Catch bugs before they reach users** - tests run automatically on code changes
3. **Multiple integration options** - GitHub Actions, GitLab CI, Bitbucket, cURL
4. **Run different tests at different stages** - smoke tests for speed, regression for thoroughness
5. **Fail fast** - stop the pipeline if critical tests fail
6. **Smart testing strategy** - don't run all tests everywhere, be strategic
This ensures your CI/CD pipeline can **trigger the API** reliably. 🚀