Skip to content

Data Workflow in OSS

Validate data changes locally using Recce open source (OSS). This guide covers saving your work, managing checklists, and sharing results with reviewers.

Goal: Validate data changes locally and share evidence with PR reviewers.

Prerequisites

  • Recce OSS installed
  • dbt project with base and current environments configured
  • Access to your data warehouse

Development Workflow

1. Run Validation Checks

Start the Recce server and validate your changes:

recce server

Use lineage diff, schema diff, and data diffs to validate your changes. Add important checks to your checklist with descriptions explaining what reviewers should verify.

2. Iterate as You Develop

When you update your dbt models locally, Recce automatically detects changes to your target/ artifacts. The lineage diff updates to reflect your latest changes, so you can keep validating as you develop.

  1. Make changes to your dbt models
  2. Run dbt build or dbt run to update artifacts
  3. Recce refreshes automatically—check the updated lineage and re-run validations

3. Save Your State

Switching branches is often unavoidable during development. Save your current state for future use:

  1. Click Export in the Recce UI
  2. Save the state file (e.g., recce_issue_123.json)

To resume later, start Recce with the state file:

recce server recce_issue_123.json

Save state file

4. Import Checklist

Reuse checks from previous sessions:

  1. Go to the Checklist page
  2. Click the Import icon
  3. Select a state file to import checks from

This preserves your favorite checks across branches.

5. Share with Reviewers

When ready for PR review, share your validation results.

As the submitter:

  1. Export your state file
  2. Attach the state file to your PR comment
  3. Use Copy to Clipboard to paste screenshots in PR comments

As the reviewer:

  1. Download the state file from the PR
  2. Run Recce in review mode:
recce server --review <state_file>

The --review option uses artifacts from the state file to connect to both base and PR environments.

Required files

You still need profiles.yml and dbt_project.yml so Recce knows which credentials to use for the data warehouse connection.

Validation Techniques

See Data Diffing for available validation methods:

  • Schema diff - Column additions, removals, type changes
  • Row count diff - Record count comparison
  • Value diff - Column-level match percentage
  • Profile diff - Statistical comparison
  • Query diff - Custom SQL validation

Verification

Confirm your workflow works:

  1. Make a model change and run dbt build && dbt docs generate
  2. Start Recce: recce server
  3. Add a validation check to your checklist
  4. Export the state file
  5. Start a new Recce session and import the checklist
  6. Verify checks imported correctly

Next Steps

  • Share - More sharing options including Cloud upload
  • Checklist - Managing validation checks
  • State File - State file reference