Skip to content

CI/CD Getting Started

Automate data validation in your development workflow. Catch data issues before they reach production with continuous integration and delivery built specifically for dbt projects.

What you'll achieve

Set up automated workflows that:

  • Save time on reviews - Eliminate manual validation steps for every change
  • Run data validations on every pull request/merge request - Run data validation checks automatically when changes are proposed
  • Prevent regressions - Catch data quality issues before they reach production

Note

CI/CD automation requires a Cloud Plan. Get started for free here.

What is CI/CD?

Recce uses both continuous integration (CI) and continuous delivery (CD) to automate data validation:

Continuous Integration (CI)

  • When: Runs when you open a new or update a Pull Request/Merge Request
  • Purpose: Validates proposed changes against baseline (typically this mean production)
  • Benefit: Catches issues before merge, with results in your PR/MR

Continuous Delivery (CD)

  • When: Runs after merge to main branch
  • Purpose: Updates baseline artifacts Recce uses to with latest production state
  • Benefit: Ensures future comparisons use current baseline

What does look like with Recce?

Both CI and CD workflows follow the same pattern:

  1. Trigger event (merge to main, or PR/MR opened/updated)
  2. Generate dbt artifacts (dbt docs generate or external source)
  3. Upload to Recce Cloud (automatic via workflow action)
  4. Validation results appear in Recce dashboard and PR/MR

Recce CI/CD architecture

Automated validation workflow for pull requests

Getting Started with your CI/CD

Recce currently integrates with both GitHub Actions and GitLab CI/CD. If you use another CI/CD product and interested in Recce, let us know.

Prerequisites

Before setting up, ensure you have:

  • Recce Cloud account You can signup and start your free trial here
  • Repository connected to Recce Cloud (setup guide)
  • dbt artifacts generated (manifest.json and catalog.json) from your project

GitHub

If your dbt project uses GitHub:

  1. Setup CD - Auto-update baseline on merge to main
  2. Setup CI - Auto-validate changes in every PR

GitLab

If your dbt project uses GitLab:

  1. Setup CD - Auto-update baseline on merge to main
  2. Setup CI - Auto-validate changes in every MR
  3. GitLab Personal Access Token Guide - Required for GitLab integration

Next steps

  1. Start with relevant CD setup (Gitlab or Github) to establish automatic baseline (production artifacts) updates.
  2. Configure CI setup (Gitlab or Github) to enable PR/MR validation
  3. Review best practices for environment preparation

After setting up CI/CD automation, explore these workflow guides: