Skip to content

CI/CD Getting Started

Automate data validation in your development workflow. Catch data issues before they reach production with continuous integration and delivery built specifically for dbt projects.

What you'll achieve

Set up automated workflows that:

  • Save time on reviews - Eliminate manual validation steps for every change
  • Run data validations on every pull request/merge request - Run data validation checks automatically when changes are proposed
  • Prevent regressions - Catch data quality issues before they reach production

Note

CI/CD automation requires a Cloud Plan. Get started for free here.

What is CI/CD?

Recce uses both continuous integration (CI) and continuous delivery (CD) to automate data validation:

Continuous Integration (CI)

  • When: Runs when you open a new or update a Pull Request/Merge Request
  • Purpose: Validates proposed changes against baseline (typically this mean production)
  • Benefit: Catches issues before merge, with results in your PR/MR

Continuous Delivery (CD)

  • When: Runs after merge to main branch
  • Purpose: Updates baseline artifacts Recce uses to with latest production state
  • Benefit: Ensures future comparisons use current baseline

What does look like with Recce?

Both CI and CD workflows follow the same pattern:

  1. Trigger event (merge to main, or PR/MR opened/updated)
  2. Generate dbt artifacts (dbt docs generate or external source)
  3. Upload to Recce Cloud (automatic via workflow action)
  4. Validation results appear in Recce dashboard and PR/MR

Recce CI/CD architecture

Automated validation workflow for pull requests

Getting Started with your CI/CD

Recce integrates with both GitHub Actions and GitLab CI/CD using the lightweight recce-cloud CLI. If you use another CI/CD platform and are interested in Recce, let us know.

Prerequisites

Before setting up, ensure you have:

Setup Steps

Both GitHub and GitLab follow the same simple pattern:

1. Setup CD - Auto-update baseline

Setup CD Guide - Configure automatic baseline updates when you merge to main

  • Updates your production baseline artifacts automatically
  • Runs on merge to main + optional scheduled updates
  • Works with both GitHub Actions and GitLab CI/CD

2. Setup CI - Auto-validate PRs/MRs

Setup CI Guide - Enable automatic validation for every PR/MR

  • Validates data changes in every pull request or merge request
  • Catches issues before they reach production
  • Works with both GitHub Actions and GitLab CI/CD

Why This Order?

Start with CD first to establish your baseline (production artifacts), then add CI for PR/MR validation. CI validation compares your PR/MR changes against the baseline created by CD.

Next Steps

  1. Setup CD - Establish automatic baseline updates
  2. Setup CI - Enable PR/MR validation
  3. Review best practices for environment preparation

After setting up CI/CD automation, explore these workflow guides: