In Analytics the SQL Was Always the Easy Part. Anthropic's Data Team Put a Figure on It.
Published: September 4, 2026
In analytics the SQL was always the easy part. Anthropic's data team have published two posts on how they run self-service analytics with Claude, and their numbers put a figure on that.
Without skills, accuracy on their own test questions did not exceed 21 percent. With them it sits above 95 percent in aggregate. Same model, same warehouse.
A month after launch their offline accuracy had gone from about 95 percent to about 65 percent, because the data model kept moving and the notes did not.
The hard parts in analytics were always the same. Validating a number against the governed one, finding knowledge that sits in silos, and getting to something someone can act on. AI has not changed any of that, it has made it obvious where the hard part was sitting all along. However, and from my experience, it has definitely made it easier to solve for.
I have read the two Anthropic posts together and put them into one deck, with my own read at the end on what has worked in my own work.
The two Anthropic posts
The first is on getting the answers right. Claude Code as the surface for their data scientists, a governed warehouse underneath, and skills on top, which are folders of plain markdown describing what each table holds, the joins, and the traps a senior analyst would warn you about. That is where the 21 to 95 comes from.
claude.com/blog/how-anthropic-enables-self-service-data-analytics-with-claude
The second is on putting it where the rest of the company works, which for them is Slack. Their own line is that if you have already done the work in the first post, this part is mostly plumbing, though the order is important.
On what has worked in my own analytics work
A semantic layer, where the definitions live, what each table holds and which number is the governed one. These sit as markdown files with a clear directory structure, or as skills, depending on the business case.
Then a separate skill for each kind of work. For one business I keep one for building the propensity model and another for the regular reporting, and each carries the judgment and the tolerance levels.
Then progressive disclosure protocols, because as the documents and tables grow the agent should not be reading a hundred of them to answer one question.
The one doing the most work is continuous update. After every milestone and every project, the skill documents and the semantic layer both get updated. Without that the rest decays, which is what happened to them.
It becomes repeatable. Point it at new data a month later and it knows the validations to run, the format to produce and the tolerances to hold.
Is it perfect? Nope. Businesses move very fast and the various layers do go stale, and it is extra effort to maintain them. But far, far easier than the pre-AI era.
In Analytics the SQL Was Always the Easy Part
Browse the slides or download the PDF
Full Deck Content (Text Format)
Text below was extracted from the source deck. Chart visuals stay in the PDF and as slide images above the post.
Slide 1
TIGZIG · SELF-SERVICE DATA ANALYTICS WITH AI Anthropic · Aug 2026
Self-service data analytics
How Anthropic built it
The SQL was never the hard problem
95% of their business analytics questions are now answered by Claude, at about 95% accuracy in aggregate
Two posts from Anthropic's Data Science and Data Engineering team, August 2026. Read together here.
the analyst's tool shed · AI-agent first
Amar Harolikar
Decision Sciences & Applied AI
Slide 2
1 · Why this never worked before
Self-service analytics has always had two routes. Both fail, and the new one has its own trap
Route 1
Build wide, flat tables for everyone
Definitions drift as the business grows, so you end up with overlapping views that disagree. And people who do not want to learn SQL still cannot use it.
Route 2
Build fenced dashboards instead
Misses every question nobody thought of in advance, and the dashboards multiply until nobody knows which to trust.
Route 3
Point an AI at the warehouse
Looks like the answer. Their warning: it can create a false sense of precision, because the person asking is cut off from the documentation and people who used to steer them to the right table.
Their words: "The initial elation of liberation from ad-hoc requests turns into dread."
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 2 / 14
Slide 3
2 · The insight everything rests on
Answering a data question is not like writing code, and treating it like code is the mistake
Writing code
Many good answers exist
Creativity helps you
Tests catch a wrong answer
You find out when it breaks
Answering with data
Usually one right answer
From one right source
Nothing proves it is correct
A wrong number looks fine
"The central problem comes down to our ability to map a user's question to specific and up-to-date entities in our data model and know the correct way of working with them. If we can do that, then the resulting execution and SQL becomes trivial."
The hard part is turning the words weekly active users into the one governed thing in your warehouse. After that, the query writes itself.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 3 / 14
Slide 4
3 · Three ways it gets the answer wrong
Nearly every wrong answer comes from one of three things
1
The word means too many things
Revenue matches forty plausible tables. If it resolves to one governed dataset, the problem disappears before the agent even starts looking.
2
It cannot find the right one
The correct table exists and the agent never reaches it, because it is searching a warehouse with a million fields.
3
The notes are out of date
A column was renamed, a metric was corrected, a table was retired. The agent reads the old note and answers with full confidence.
These three account for the overwhelming majority of wrong answers, and everything they built is aimed at them.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 4 / 14
Slide 5
4 · The number that makes the case
A skill is a folder of plain notes the agent reads when it needs them. Here is what adding them did
21%
No skills
Their own words: did not exceed 21%
95%
With skills
in aggregate, and around 99% in some areas
Accuracy on their own test questions
Same model, same warehouse. Though this sits on top of a governed warehouse, which the post ranks as the most important part of all.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 5 / 14
Slide 6
5 · What they actually wrote down
They write skills in pairs. One says where things are, the other says how an analyst would work
The router
They call it the knowledge skill. Try the semantic layer first. If it does not cover this, here are about thirty reference files for this area, with the tables, the joins and the traps.
The runbook skill
The process a senior analyst follows. Clarify the question, find the source, run it, then send the result through review agents whose job is to attack it.
Reference notes
What one row means, what is excluded, the filter every query applies, and the wrong-answer traps a senior person would warn you about.
Stock patterns
A dozen ready analyses. Retention curves, funnels, rate decomposition. So common requests are not rebuilt from scratch each time.
The router is the real trick. It narrows a million-field warehouse to a few dozen curated files before a single query is written.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 6 / 14
Slide 7
6 · And then it rotted
The notes describe a data model that changes daily. Nobody was updating them
~95%
At launch
~65%
One month later
Their offline eval accuracy. The data model kept moving, the notes did not
How they stopped it
The notes now live in the same place as the data models, so the change that renames a column and the change that updates the note are the same change. A review check flags any model edit that leaves the notes untouched.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 7 / 14
Slide 8
7 · Now hand it to everybody
Getting an agent accurate and getting it used by non-analysts are, in their words, quite different motions
Anthropic put it in Slack
That is the shared room the whole company is already in. Two things change the moment it lands there. You go from one expert using their own login to many non-experts sharing one machine login, and the reader loses every clue that used to tell them an answer looked wrong.
What is lost
The sniff test
On a dashboard you see the trend line and the metrics around it, so a wrong number looks wrong. In a chat you get one number and nothing else. If it is not something you look at daily, you accept it.
So they
Reload the notes every conversation
The data model changes several times a day. Reading last Tuesday's copy gives last Tuesday's wrong answer, confidently.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 8 / 14
Slide 9
9 · Correct is not the same as useful
Their first instinct was to teach it where the tables are, and stop there
Why that was not enough
It gave correct numbers and stopped short of useful answers. People do not ask for a figure. They ask what is driving this dip, or where this lands at month end. That needs the agent to know how an analyst would work, not just where the data sits.
Before
Sign-ups dropped 12% on Tuesday.
After
Sign-ups dropped 12% Tuesday. There was a payment-service incident open 9 to 11am that morning, and the dip is concentrated in the affected region.
The difference is a second feed. Incidents, releases, and what people said in chat. They call it the highest-leverage thing you can add after the warehouse itself.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 9 / 14
Slide 10
9 · The part that is easy to get wrong
The agent queries as one shared account. Not as the person asking
What that means in plain terms
No per-person permission. Anyone who can call the bot has whatever access the bot has, so adding it to a room gives that room read access to everything the bot can query.
Governed data only
Curated tables only. No raw event streams, no staging, no sandboxes.
No clearance for personal data
Claude flags likely candidates, a person decides. The account holds no clearance, so those columns are invisible to it.
Channels are a grant
Only the data team adds the bot to a room, and owns the list of rooms.
Label every query
Enforces nothing at the time. It lets you find out afterwards who ran the query that scanned four terabytes.
Their framing: a shared read copy of your governed warehouse. Scope it that way.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 10 / 14
Slide 11
10 · How they know it is still working
Whether the answer was good enough is not something you can eyeball
~90% of data model changes now carry a notes change in the same edit
75% of questions answered without being tagged, in one channel over one month
Metric 1
Adoption, and it was the useful one
What share of questions go through the governed layer. When it dips, it almost always means the notes have drifted, or a new kind of question has appeared that the governed layer does not cover.
Metric 2
Corrections, counted
Thumbs down and corrections by area. Every time someone corrects the agent in a thread, that correction becomes a candidate test question.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 11 / 14
Slide 12
11 · The problem they have not solved
The failure none of this fully catches is the silent one
In their own words
The failure mode none of this fully catches is the silent one. The answer is wrong, but looks plausible and is used without objection. They say plainly that they do not have a robust solution yet.
What helps
Show where the number came from
Every answer carries a note saying which source it used, so the reader can go and check.
And
A person signs off on anything going up
Nothing reaches leadership without a human in the loop, and the top numbers in each area are checked against the trusted dashboard daily.
If you start from nothing: a handful of trusted datasets, a few dozen test questions, and one thin router. That is most of the gain.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 12 / 14
Slide 13
My read
In analytics the SQL was always the easy part
The hard parts were the same three, every time. Validating a number against the governed one. Finding knowledge that sits in silos. And getting to an insight someone can actually act on.
AI has not changed that. It has made it obvious where the hard part was sitting all along.
Knowledge sits in silos because companies are built that way, by division and by department. That is normal, not a failure. Consolidating it has been hard for decades. Single source of truth and knowledge sharing are old buzzwords, and most attempts never got there.
The documentation going stale in this post is the same story. Somebody has to update it and nobody does. And it is not only analytics knowledge. Product knowledge sits with product, campaign knowledge with marketing, process knowledge with operations. Pulling all of it together is the work.
What is different now is that there is a realistic hope of tackling it in reasonable time. That is new.
tigzig.com · analytics, macro signals & AI tools · Amar HarolikarData Analytics with Claude · 13 / 14
Slide 14
What works in my own practice
Four things made the difference
1A semantic layer
Where the definitions live. What each table holds, what each metric means, and which number is the governed one.
2One skill per kind of work
For one business, a skill for the propensity model and another for regular reporting. Each carries the judgment and the tolerances.
3Progressive disclosure
As documents and tables grow it turns into a mess. The agent should not read a hundred to answer one.
4Continuous update
The big one. After every milestone and project, the skill documents and the semantic layer both get updated. Skip it and the rest decays.
It becomes repeatable. Point it at new data a month later and it knows the validations, the format and the tolerances.
Sources. How Anthropic enables self-service data analytics with Claude and Self-service data analytics in Slack, both by Anthropic's Data Science and Data Engineering team, August 2026. These pages summarise the two read together.
the analyst's tool shed · AI-agent first
Amar Harolikar
Decision Sciences & Applied AI
Working on something similar? How I work covers the rates, the availability and what I take on.