Our data team of four was drowning in ad-hoc requests. “What was the revenue last month?” “How many active customers do we have in Germany?” “What’s the conversion rate for Q4?” Each question meant someone opening a SQL editor, writing a query, formatting the results, and posting them in Slack. So I built an AI-powered data assistant: a Slack bot backed by Claude and BigQuery that lets anyone on the team ask business questions in natural language and get SQL-backed answers in seconds.
Category:
Data Engineering
How two duplicate webhook events, arriving 2ms apart, broke 12 dbt tests in production — and the surprisingly simple fix that made the sync idempotent.
We had been paying roughly 400 euros a month for a managed ETL connector to sync our CRM data into BigQuery. Six tables, a few thousand records each, updated a handful of times per day. The connector worked, but it had problems: data landed in a US-region dataset (we needed EU for GDPR compliance), the CDC implementation had quirks that caused phantom duplicates, and every time we needed to debug something we were staring at a black box. So I replaced it with a custom Cloud Run service. The whole thing runs for under 5 euros a month.
Older Posts