dbt Core Sync

Automate your documentation deployment to Colibri Cloud using continuous integration and deployment workflows.

Overview

The colibri-cli tool allows you to automatically upload your dbt documentation to Colibri Cloud whenever you push changes to your repository. This ensures your documentation is always up-to-date with your latest dbt models and configurations.

Prerequisites

Before setting up CI/CD, ensure you have:

A Colibri Cloud account
A dbt project configured with your models
Access to your repository’s CI/CD settings (e.g., GitHub Actions)

Setting Up GitHub Actions

Install colibri-cli

The colibri-cli package needs to be installed in your CI/CD environment. You can do this using pip:


pip install colibri-cli

Configure Secrets

You’ll need to configure the following secrets in your GitHub repository settings:

COLIBRI_API_URL - The API URL for Colibri Cloud
COLIBRI_API_KEY - Your Colibri API key (generate this from your Colibri Cloud account)
COLIBRI_PROJECT_ID - Your Colibri project ID

To add secrets in GitHub:

Go to your repository settings
Navigate to Secrets and variables → Actions
Click New repository secret
Add each of the above secrets

Create Workflow File

Create a new file at .github/workflows/publish-to-colibri.yml in your repository:


name: Publish to Colibri
 
on:
  push:
    branches: [master]
  workflow_dispatch:
 
jobs:
  upload-lineage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
 
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.13'
 
      - name: Install dependencies
        run: |
          pip install dbt-duckdb colibri-cli
 
      - name: Run dbt
        run: |
          dbt deps
          dbt build
          dbt docs generate
        env:
          DBT_PROFILES_DIR: .
 
      - name: Upload to Colibri
        run: colibri-pro upload
        env:
          COLIBRI_API_URL: ${{ secrets.COLIBRI_API_URL }}
          COLIBRI_API_KEY: ${{ secrets.COLIBRI_API_KEY }}
          COLIBRI_PROJECT_ID: ${{ secrets.COLIBRI_PROJECT_ID }}

Note

This example uses dbt-duckdb, but the workflow works for any database supported by Colibri. The CI environment must be able to compile the manifest and run dbt docs generate, which requires access to the data warehouse where your tables are materialized.

Verification

After setting up your workflow:

Push a commit to your configured branch
Go to the Actions tab in your GitHub repository
You should see the “Publish to Colibri” workflow running
Once complete, verify your documentation is updated in Colibri Cloud

Troubleshooting

Upload Failures

If the upload step fails:

Check that dbt docs generate completed successfully
Verify the manifest.json and catalog.json files were generated