Debug dlthub Transformation

Trust Score 88/100

Diagnose and fix dlthub transformation failures: SQL dialect incompatibility, pipeline state errors, and missing columns after deployment.

triggers:debug transformationsql dialectpipeline recoverymissing columnsdlthubtransformation failure

GitHub SKILL.md

What it does

Provides step-by-step diagnostics and remediation for dlthub transformation failures. It helps detect SQL dialect portability issues, recover pipelines stuck in failed states, and identify/repair missing or silently dropped columns caused by type inference. The skill documents static compatibility checks, escalation steps for pipeline recovery, and destination-specific workarounds (e.g., DuckDB).

When to use it

Use this skill when a transformation works locally but fails on another destination (BigQuery, Snowflake, Postgres), when pipelines exhibit syntax/unsupported-function errors, when runs are stuck in retry loops, or when expected columns are absent from the output. Also use proactively to run static SQL compatibility checks before deployment.

What's included

Scripts: none bundled (has_scripts=false) but references a dialect checker and CLI commands to inspect and recover pipelines.
References: inline guidance for SQLGlot, dlt CLI commands, and a DuckDB workaround.
Instructions: clear, ordered remediation steps: static SQL checks, clearing stale packages, reconciling local state, selective drops (with safety rules), and validation after fixes.

Compatible agents

Best used by agents with shell/Bash and dlt CLI access; compatible with tooling that can run static SQL checks (SQLGlot) and invoke pipeline commands.

Audit Summary

Well-written skill for debugging dlthub transformation failures, covering SQL dialect incompatibility, pipeline state recovery, and missing column diagnosis. No bundled scripts — purely instructional. Strong safety rules around destructive operations like drop commands. Clear escalation order and practical examples make it immediately usable for data engineers working with dlt pipelines.

Watch Out

DuckDB-specific workaround requires direct SQL access to the database file
Relies on dlthub CLI and SQLGlot being available in the agent's environment

Notes

Single-file SKILL.md with no scripts or references directory. Monolithic but well-organized internally with clear section numbering and escalation paths. Would benefit from splitting into scripts/ for the dialect checker and references/ for external docs.

Information

Repository: dlthub-ai-workbench
Stars: 41

Trust Score

Overall88

Security95

Code Quality88

Architecture68

Usefulness72

Related Skills

Dune Analytics (MoonPay Integration)

Run Dune SQL and saved queries via the Dune REST API for on-chain analytics; pairs with MoonPay CLI to create/fund wallets for analysis.

MetricFlow: Interactive Metric Definition

Interactively define MetricFlow metrics from natural-language business descriptions; proposes, validates, and dry-runs metric YAML for semantic modeling.

Visualization Best Practices

Provides step-by-step guidance and production-ready patterns for creating clear, accurate, and effective data visualizations in analytics workflows.

ABAP Performance — HANA

ABAP performance best-practices for SAP S/4HANA: push data-intensive work to HANA, use CDS/AMDP, and prefer set-based SQL over ABAP loops.

SF Data Cloud — Connect

Tools and workflows to discover, inspect, test, and create Salesforce Data Cloud connector connections and their schemas.

Event Deduplication

Group and deduplicate multi-source events/articles using ID- and content-based grouping, reputation scoring, and canonical selection to pick the best version.

Data Triage

Quickly profile, triage, and summarise local data files (CSV/TSV/JSON/JSONL) and logs with safe defaults and reversible cleaning steps.

Statistical Significance Calculator

Guides data-analytics workflows to compute and interpret statistical significance, with step-by-step calculations, code snippets, and validation checks.

Back to Skills