Semantic Layer Solutions in Modern Data Architecture
Summary
I went down a semantic layer rabbit hole recently, so here are the notes before they evaporate. It's the translation layer between your raw data and the people asking questions of it: define "revenue" once and it means the same thing in SQL, MDX, DAX, an API, and now your AI agents, which kills most of the governance drama before it starts. Risk is the real draw. Every extract into a cube or data mart leaves a stale copy to govern and one more surface to breach, which is why the vendors that build aggregates in place, like AtScale on Databricks, are the ones worth watching.
I went down a semantic layer rabbit hole recently, and I'm dumping my notes here before they evaporate. Gartner's definition is decent for once: the semantic layer is the "business translation" layer that sits between raw data and business users, taking technical data structures and turning them into terms the business already uses so that non-technical people can query and analyze data without knowing what's underneath. (The data kind of semantics, not the Derrida kind.)
The idea that hooked me is the shared metric contract. It's the missing link between your data stores and your BI tools. Lately your AI agents too. If "revenue" means the same thing in SQL, MDX, DAX, and an API, you've killed 80% of the governance drama before it starts (agreeing on what a "customer" is remains an open research problem).
"No movement" sounds like a cost pitch, and it does save money, but the part I care about is risk. Every extract into a cube or a data mart leaves behind a stale copy of your data. That copy needs its own governance. It's one more surface for a breach. If a vendor can build aggregates in place, like AtScale on Databricks, and cache them intelligently, you get the cube-like user experience without duplicating your data topology.
40% of Databricks users haven't started using dbt yet. Every BI tool defines its metrics its own way, which means every BI tool tells a slightly different truth. Business users can't reach the warehouse or the lakehouse directly, so the data sits locked in formats only engineers can read. A semantic layer gives you one place where "revenue" gets defined. It opens complicated data up to the business side, takes over aggregate management for performance, and keeps definitions consistent across whatever BI tools your org has accumulated over the years.
So who sells this? AtScale is the big standalone player, with automated aggregates and a unified access layer. Stardog and Timbr both come at it from the knowledge-graph side, Stardog to show how complex relationships work and Timbr with semantic SQL knowledge graphs. Then there are the platform-native options: Power BI ships a built-in semantic model, and Databricks Unity Catalog Metrics is a new feature that lets AI reach metrics through SQL.
If you're building a practice around this, I count three ways in. Partnering with AtScale or Stardog to stand up their tools on client systems is the fastest path to market, though it's the hardest to differentiate. Custom metric stores with your own interfaces take product thinking, and they pay for it in value and differentiation. The third door is AI: semantic layers built just for AI/ML use cases, where consistent metric definitions matter for training and testing models.
AI QA is what got me here. It's a small slice of the market. Every Databricks or Snowflake customer needs a semantic layer, which opens up high-value consulting work: projects worth $500,000 or more.
The AtScale-Databricks "Semantic Lakehouse"
AtScale's "Semantic Lakehouse" puts a semantic layer between Databricks Lakehouse storage and your business intelligence tools. Business users ask questions about complicated data in terms they already know. The data never leaves Databricks.
AtScale runs against both Databricks SQL and Apache Spark engines, integrates with Unity Catalog for governance and lineage, connects directly to SQL warehouses and compute clusters, and sets up through Databricks Partner Connect. That's a lot of plumbing. On top of Databricks' native SQL it also speaks MDX for OLAP cube work, DAX for native Power BI integration, LookML so Looker can join the party, and REST and Python APIs for programmatic access.
The "no movement" promise holds up in the details. Nothing gets copied out into separate OLAP cubes or BI-specific data marts. AtScale queries the data where it lives in Databricks, creates and manages its aggregates inside the lakehouse, leans on smart caching for query performance, and keeps a single source of truth with no duplication. That's the fix for the 40% problem above: wrap the technical structures in a business-friendly interface, and business users run their own analytics while the data team keeps control.
A few follow-ups worth reading after this one. Headless BI and metric store patterns cover semantics that work across more than one query dialect. Knowledge-graph semantics are the Stardog and Timbr angle, complicated relationship logic. And platform-native semantics (Power BI datasets and Unity Catalog Metrics) suit teams that can work together on one surface.