RelatedWhat I read
before I decide.

Claude discovers a novel enzyme system with CRISPR-like repeats

We’re announcing a new life sciences research group and laboratory at Anthropic. This post introduces the team behind this work and shares early results in which Claude discovered a novel enzyme system with properties reminiscent of CRISPR, with only high-level direction from our…

How I built agent-based security reviews on Databricks

We already had automation in parts of our security review process. It was useful,...

‘Batches’ in Scrum

Image If you consider each of Scrum’s events, you’ll notice batches of various kinds: Sprint: The team conducts all their work in the context of Sprints and each Sprint is an opportunity to focus on, and get feedback about, a batch of new work. Sprint Planning: The team is consid…

Fabric CLI Explained: Install, Use, and Connect It to GitHub Copilot

There are multiple ways you can connect AI to Power BI and Microsoft Fabric. You could use MCP servers, you could use Fabric Skills, but there’s also another way: Fabric CLI, the command line interface. It’s not used as much as the other methods, but it’s quite powerful. In this…

The quality of capital allocation in Europe

Europe's investment debate tends to ask how much capital can be mobilised and how quickly. Allocation quality also depends on who receives capital and on the spillovers their expansion creates. This column argues that for public allocation, Europe should combine simpler administr…

Unraveling the cobweb of global imbalances: drivers, vulnerabilities, and adjustment scenarios

This paper analyses the evolution and drivers of the stock of global imbalances in order to assess associated vulnerabilities and potential adjustment scenarios. These imbalances have recently increased sharply, with a deterioration in the US net international investment position…

Two years of OpenAI Academy

Marking two years of OpenAI Academy and bringing AI skills to even more communities.

How Concurrence governs clinical AI at a trillion-token scale with Unity Gateway

Healthcare AI has little margin for error. AI agents helping coordinate patient care...

Your Team Isn't Slow. Your Handoffs Are.

Image Watch where the work actually spends its time. Not the hours someone is heads-down building. The days it sits, waiting. Waiting for the one person who knows that system. Waiting for the other team to finish their sprint. Waiting for a review, a sign-off, an environment, an…

Start using DAX with AI

The time has come: in 2026, using AI to write DAX has started to be effective. Latest news and the new courses are at https://www.sqlbi.com/ai/ We also created three new video courses that should not become obsolete in two weeks!…

How data centres affect electricity prices and for whom

Increases in US electricity prices since 2020 have coincided with a sharp rise in demand from data centres. This column analyses the impact of the surge in data centre growth since 2021 on electricity prices across US utility service territories. It finds that data centres raise…

Maritime chokepoints and the global economy: evidence from the Strait of Hormuz

The conflict in the Middle East and the subsequent disruption of traffic in the Strait of Hormuz have triggered an unprecedented shock to the global economy. Using historical data from the IMF/University of Oxford Portwatch database on maritime traffic, this paper empirically exa…

OpenAI extends cyber access to Ukraine for civilian defense

OpenAI is extending access to its Daybreak program to the Government of Ukraine to support the cyber defense of civilian infrastructure.

Genie One MCP: Give any AI Agent the Right Business Context

Business leaders often have access to plenty of data, but still can’t get a reliable...

Finding AI Gold With Lean Startup Techniques

Originally published on yuvalyeret.com. How do we deliver valuable real-world AI impact through solutions that solve problems, using some of what we've learned by building products over the last couple of decades? We're not seeing the value yet Unless you've been in a cave, you'r…

The hidden complexity of a daily average in DAX

A simple daily average can hide complexities and return different values. We explore several edge-case scenarios to build the desired calculation using AI to accelerate the development.

The energy transition: Progress and challenges

Solar and wind have become the cheapest sources of new electricity almost everywhere, yet Europe is now switching them off in record numbers. This column argues that the binding constraint on the energy transition is no longer technology but market design — and that fixing it mea…

Hidden by complexity? Measuring stablecoin, crypto and decentralised finance ecosystems

Decentralised finance data presents a distinctive paradox: while every data point is publicly recorded and accessible, deriving meaningful insights is obscured by the scale, fragmentation and complexity of the ecosystem. Key metrics illustrate that the rapidly evolving DeFi ecosy…

Sam Altman’s remarks at the United Nations Security Council

OpenAI CEO Sam Altman discusses AI safety, human control, and international cooperation in remarks to the United Nations Security Council.

The Genie One MCP is now Generally Available

AI coworkers and coding agents are spreading fast across organizations, and each...

Cognitive Trap: Planning Certainty

Planning Certainty Trap Most mistakes in Scrum aren’t because people don’t understand the framework—they come from applying reasonable thinking in the wrong context. Cognitive traps happen when decisions favor efficiency, control, or comfort over transparency, inspection, and ada…

PBIP, PBIR, and Making Power BI AI-Addressable: A Conversation with Rui Romano (Fabric Insider Ep. 14)

PBIX isn’t going away, but the code behind it is finally becoming something you can version, script, and hand to an AI agent without things blowing up. In this episode of Fabric Insider, I sit down with Rui Romano, Product Manager on the Power BI team within Fabric, the person be…

Pourquoi passons-nous autant de temps en réunions ?

Peu d’aspects du monde du travail moderne sont aussi largement critiqués que les réunions. À partir de données recueillies auprès de plus de 9 000 salariés en Norvège, cet article examine la manière dont les réunions sont organisées et leurs effets sur les résultats des entrepris…

BIS Quarterly Review, September 2026

Harvey turns legal context into stronger drafts with GPT-6 Astra

GPT-6 Astra produces more structured, context-aware legal documents, freeing lawyers to focus on strategy.

Unity Catalog Pages: a governed home for your business knowledge in Genie Ontology

Every AI agent is only as good as the context it is grounded in. Ask an agent a question...

Does Your AI Know the Scrum Guide? Twelve Questions to Find Out

Image More and more participants in my Professional Scrum Master classes arrive with Scrum "rules" they picked up from an AI chatbot. Some of those rules are accurate. Some stopped being true in November 2020. And a few were never part of Scrum at all. The trouble is that from th…

Where we stand on AI and DAX in 2026

The time has come: in 2026, using AI to write DAX has started to be effective. Let me recap what has changed between 2023 and 2026, and what we are doing with our three new courses to best assist you…

Building resilience to global financial shocks in emerging markets

Emerging markets are vulnerable to sudden shifts in investor sentiment, which can lead to pressure on exchange rates and financing conditions. Using panel evidence from 23 emerging markets, this column shows that the sensitivity of portfolio flows to global shocks has weakened si…

When machines attack: frontier AI cyber threats and policy responses in the financial sector

Frontier artificial intelligence (AI) models are a game changer in the cyber threat landscape. Unlike earlier generations of AI models, they can autonomously identify critical vulnerabilities, develop effective exploits and conduct increasingly complex multi-step cyber operations…

How invideo improves color grading 3x with GPT‑6 Astra

With GPT‑6 Astra, invideo plans edits with greater precision, improves color correction and grading threefold, and produces 50 custom effects in one day.

DuckDB Now Ships inside dbt v2

dbt is the tool many data teams use to manage their SQL transformations: you write each model as a SELECT statement, and dbt works out the order to run them in from the references between models, builds the resulting tables and views in your database, and can test them along the…

AI on Top of a Dysfunctional System (1): The Product Backlog

TL;DR: Polished Artifacts, Unchanged Decisions, Or Ten Backlog Anti-Patterns AI Makes Worse Add AI to a Product Backlog process that already struggles, and everything seems to improve within an afternoon. The problem is that polishing artifacts with AI doesn’t fix the root cause:…

The hidden complexity of a daily average in DAX

A simple daily average masks a level of complexity that may not be immediately apparent. In this article, we use AI to create a daily average, and then to analyze several edge-case scenarios to build a sound calculation. Computing an…

Productive firms can better spot and deploy their internal talent

When a job vacancy opens, the manager faces a choice: to hire from outside the organisation, or to fill the position with an existing employee through a promotion or horizontal move. This column uses data from Finland to show that more productive firms are more likely to hire fro…

What determines banks' excess demand for reserves?

The amount of central bank reserves to supply has become a highly contentious policy issue for many central banks that shrink their balance sheets which were built up during quantitative easing.

Ringg’s AI agents resolve up to 65% of customer calls with OpenAI

Using GPT-5.6, Ringg powers multilingual agents across voice, chat, WhatsApp, and web for 90% less cost vs. GPT-4.1.

Querying the entire internet (100 billion rows!) with MotherDuck

Common Crawl publishes petabytes of web crawl data on S3. With DuckDB and MotherDuck you can query the Common Crawl dataset directly, no cluster and no download, and measure how fast the vibe-coded web is growing.

Scrum's Meetings Are Capped at Five Hours a Week. Something Else Filled Your Calendar.

Image Scrum has too many meetings. I have heard that sentence in training rooms since 2010 and for years I argued with it. Now I do the arithmetic on the whiteboard instead, because the Scrum Guide caps its own events and the cap is far below what anyone in the room guesses. Add…

Find Unused Objects In Power BI Semantic Models With Semantic Link Labs

A few weeks ago I wrote about how Semantic Link Labs now has tools with interactive UIs and showed how you could use this to view lineage and so Vertipaq-Analyzer-stuff in a notebook. I didn’t show what I think is the coolest new feature though: the ability to find the tables, co…

Anatomy of a rise: Monetary policy and the post-Covid surge in long-term interest rates

The sharp rise in long-term interest rates since 2020 is difficult to explain from slow-moving fundamentals. This column shows that narrow windows around nonfarm payroll releases and speeches by prominent Federal Reserve policymakers capture 80-90% of the observed increase in lon…

Simple, resilient and proportional: revisiting regulation for small banks

Small banks are the backbone of local economies but have distinct business models and risk profiles compared to larger banks. Recognising these differences, the Basel Committee on Banking Supervision (BCBS) promotes proportionality in global banking standards, encouraging tailore…

Introducing MentalHealthBench

MentalHealthBench is an expert-informed benchmark for evaluating helpful and safe AI responses across realistic mental health conversations.

Data Engineering Weekly #288

The Weekly Data Engineering Newsletter

Psychological Safety is not Impunity

You can find him in the retro, though he rarely says much. He doesn't need to. The team has learned to write around him. Watch what happens to a good idea in this room. Someone proposes a real improvement, and before it's even finished, you can see a couple of people glance, just…

Fabric Core MCP Server: Scan and Configure OneLake Security with AI

With the appearance of AI tools, LLMs, MCP servers, and skills, everyone these days is interested in what tool can do what. I’ve already explained the Power BI Modeling MCP server and Fabric Skills in previous videos. In this one, I’ll talk about the Fabric Core MCP server, an MC…

Beyond trade diversion: How the US-China trade war reshaped global production

Trade wars do not simply redirect exports; they also reshape the costs of production, disrupt supplier-customer relationships, and trigger adjustments within multinational firms’ global affiliate networks. This column uses data covering millions of manufacturing plants in 50 majo…

Zombie firms in emerging Asia: domestic and cross border implications

Using firm-bank linked data for 10 Asian emerging market economies (EMEs) over 2005–2021, we study the domestic and cross-border implications of zombie firms. We document three main findings.

ChatGPT Ads expands to Southeast Asia and Taiwan

ChatGPT Ads is expanding to Southeast Asia and Taiwan, giving eligible businesses new ways to reach people across more than 60 countries.

dbt Charts on MotherDuck: Declarative Dashboards

Define your dashboards in YAML with dbt Charts, then the same dbt build that runs your pipeline deploys them to MotherDuck Dives. Just 2 config changes! Your Dives will refresh live whenever your stakeholders take a look. See how we constructed the dbt package as well.

I don't like LLMs

I have a lot of mixed feelings about AI and LLM technology. I’m fascinated by its effect on our profession, excited by the potential gains in productivity - and thus the products we could rapidly build. On the other hand, I’m fearful of the damage AI might cause: agent swarms tak…

DAX UDFs, Arrow REST API, and Outbound Access Protection: A Conversation with Kay Unkroth (Fabric Insider Ep. 13)

DAX UDFs just went GA, there’s a new REST API for running DAX queries at scale, and a network security feature most people haven’t even heard of yet is quietly changing how enterprises govern their semantic models. In this episode of Fabric Insider, I sit down with Kay Unkroth, P…

Mortgage Lock-in: A Review of the Literature

Federal Reserve Bank of Atlanta Working Papers by Kristopher Gerardi, David Hao Zhang and Franklin Qian

The PRA will hold a captive insurance industry roundtable

On 17 September 2026, the PRA will host a roundtable in relation to CP11/26 – A tailored regime for captive insurance.

Claude Opus 5.5, GPT-6 Sol, GPT-6 Luna, and a new price war

Introducing prompt_jev(): bringing Jev to Motherduck SQL

Classify text in SQL with prompt_jev(), powered by TypeSafe's Jev. 100,000 rows labeled in 40 seconds for $0.50 at frontier-LLM accuracy. Live on paid plans.

Scaling Product Orgs with Portfolio Agility

Originally published on yuvalyeret.com. Your portfolio process isn’t slowing you down by accident You built empowered product teams. You wrote down a product operating model. And somehow decisions still stall, competitors still pivot faster than you do, and every initiative that…

The query finished…Why is my Fabric SQL database still consuming CUs?

Two minutes of database activity can mean 17 minutes of compute billing. And a CPU chart doesn't tell the whole story. Let's work through SQL database in Fabric billing with examples from application workloads, development databases, and everyday troubleshooting. The post The que…

The transmission of shocks across sectors and the dynamics of sectoral prices

European Central Bank Working Papers by Francesca Monti and Leïla Van Keirsbilck

PRA Regulatory Digest – July 2026

The PRA Regulatory Digest is for people working in the UK financial services industry and highlights key regulatory news and publications delivered for the month.

Jev introduces a new shape of LLM - System One, aka Decision Models

RADAR: Catch gray failures with anomaly detection

Some of the most damaging outages are the ones your monitoring never flags: a slice...

Fragments: September 16

Reports of agentic hacking continue, in this case it happened back in May and it seems OpenAI did not disclose that they were responsible. Simon Willison sees two options: After the Hugging Face and Wiki attacks OpenAI were still unable to review their previous logs and determine…

OneLake Security: Setting Up Row Level Security From Lakehouse to Report

If you’ve set up row level security in Power BI before, you know the usual drill: open the semantic model, add a role, write a DAX filter. But once your data lives inside a Microsoft Fabric OneLake structure, there’s a better way to do this. You can implement it directly in OneLa…

Commercial real estate debt financing in Europe: evidence from a network analysis

European Systemic Risk Board Working Papers by Frauke Skudelny, Mihai Aliman, Martin Regnér, Cecilia Sarchi and Matteo Valetto

PS18/26 – Solvency UK: Post-implementation reporting and disclosure amendments and Own Funds permissions update

Policy statement 18/26

MilleMiglia: A realistic instance generator for middle-mile logistics

Algorithms & Theory

Enabling secure, productive work on personal devices

At Databricks IT, our vision is to empower people to work from anywhere without putting...

AI Tool Introduction: Promptly AI

Image Imagine you are using ChatGPT and writing a prompt. But you are not sure if it is a good prompt or not. Here, Promptly AI comes to life to help you enhance your prompt with just one click. Click here to open its website. Image How to add Promptly AI to your AI chatbots? Pro…

Find What Changed And When In Your Power BI Semantic Model

Here’s a situation I’ve found myself in many times. You’re called to help someone whose Power BI report is suddenly very slow. You ask whether any changes were made to the semantic model around the time performance got worse and the answer is no, absolutely not. You then spend an…

How Firms Form Beliefs and the Implications for Inflation

Board of Governors of the Federal Reserve System Finance and Economics Discussion Series by Robert Minton and Hugo Monnery

LIAF02/26 – Low Impact Amendments Finalisation July 2026

Low Impact Amendments Finalisation July 2026

The future of practice: Enabling teachers to create learning interactives with generative UI

Education Innovation

Database Branching: A Developer's Guide to Git-Style Workflows

Git made isolated development a baseline for software teams. Each developer can create...

The Real Meeting Happens in the Hallway

Image The real meeting happens in the hallway. The meeting ends. Everyone agreed, or at least nobody disagreed. Then, two minutes later, the real conversation starts. In the corridor. In a side chat. In the small group that lingers after the call drops. "I didn't want to derail i…

Writing DAX at the correct granularity

A DAX calculation can produce convincing values at the detail level and fail at the total. The problem is often that the business rule was evaluated at the wrong granularity. This video introduces four grains, or levels of granularity, to…

Credit Surfaces and Economic Uncertainty

Board of Governors of the Federal Reserve System Finance and Economics Discussion Series by David E. Rappoport and John Geanakoplos

LIAC02/26 – Low Impact Amendments Consultation July 2026

Low Impact Amendments Consultation July 2026

How Claude is uplifting biomolecular modeling

Claude made the open-source models that scientists use to predict and design biomolecules faster and more memory-efficient. Claude optimized more than 30 of these models in just under four weeks, speeding them up roughly 4x on average. It also created a low-memory mode that enabl…

Persistent Databases in the Browser with DuckDB-Wasm and OPFS

When DuckDB-Wasm was launched in 2021, databases could not be persisted: everything lived in the Wasm heap and vanished when the tab closed. Keeping data meant serializing tables to Parquet, storing the bytes in IndexedDB, and re-registering them on the next page load. This was d…

Nail the Narrative

Sumeet Gayathri Moghe finds many folks building presentations get tangled in building slides without a coherent narrative. He advises distilling the big idea, visualizing the audience, and building a structured storyline. more…

Writing DAX at the correct granularity

A DAX calculation can produce convincing values at the detail level and still fail at the total. The problem is often not the total itself: the business rule was evaluated at the wrong granularity. This article introduces four grains, or…

Structural Estimation with Unstructured Data

Federal Reserve Bank of Dallas Working Papers by Jesús Fernández-Villaverde, Minchul Shin, Stephen Hansen, Ryohei Oishi and Sara Casella

CP12/26 – Insurance friendly societies, amalgamations and transfers

Consultation paper 12/26

Measurements for understanding the pace of AI development inside frontier labs

Today, the world can’t see what’s going on inside AI labs. Anthropic is proposing new metrics that would give the public visibility into frontier AI development.

Database for AI Agents: 5 Evaluation Criteria

The five criteria for evaluating a database for AI agents are branch isolation, serverless...

AI Can Write Backlog Items. It Can’t Create Shared Understanding

AI can generate polished user stories and acceptance criteria in seconds. Learn how to use it without skipping product judgment, user evidence, or team conversations.

Choosing A Power BI Storage Mode For Reporting On Data From Azure Databricks

In case you haven’t already seen the blog post on the Power BI blog or the discussion on LinkedIn, we at Microsoft published a new white paper last week to help you decide which storage mode to use when you’re using Power BI to create semantic models and reports on data stored in…

Inference Based on Scale, Label, and Economic Restrictions

Federal Reserve Bank of Philadelphia Working Papers by Daniel F. Waggoner, Juan Francisco Rubio-Ramírez and Jonas E. Arias

Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-Train

Algorithms & Theory

The Web Search Your Agent Inherited Isn't Good Enough

An agent that needs the outside worldAn engineer at a software company is building...

Na era da IA, ainda está a medir trabalho e esforço? ou finalmente a medir valor?

Image Durante décadas habituámo-nos a medir trabalho através daquilo que conseguimos ver: Horas trabalhadas Número de tarefas concluídas Tickets fechados Linhas de código Story Points entregues Número de funcionalidades lançadas Projetos terminados dentro do prazo São números con…

OneLake Updates: A Conversation with Josh Caplan (Fabric Insider Ep 12)

In this episode of Fabric Insider, I sat down with Josh Caplan, Director of Product for OneLake at Microsoft. Josh has been around this product family for a long time, starting as an intern on the Analysis Services team back in 2007, working on Project Gemini (which became Power…

Prices and Monetary Policy: The Role of Financial Constraints

Federal Reserve Bank of San Francisco Working Papers by Michael D. Bauer, Alexander Czarnota and Mathias Klein

Introducing Gemini 3.8 Live and 3.8 Live Extended Thinking

DuckDB Skills for Claude Code

More likely than not, you've been using AI tools such as Claude Code for day-to-day work. You may have noticed that when AI needs to look at a data file, it makes use of Python, writes a small script, runs it, and then reads the output. This works, although it is slow, and the ag…

Gallery Walk to IT Roadmap

How do you get meaningful input from an entire IT department without getting everyone in the same meeting or hearing only a few of the loudest voices? A few years ago, I faced that problem with a well-known consumer products company. The leadership team was creating a five-year I…

Implementing SCD Type 2 in Microsoft Fabric – The Definitive Guide!

Slowly Changing Dimensions (SCD) are one of the fundamental concepts in dimensional modeling. If you are not sure what dimensional modeling is, I suggest you first check the series of articles I wrote on data modeling some time ago. And, when it comes to SCD in particular, SCD Ty…

Generating running routes with GPT-6 Astra and ChatGPT Work

DuckDB Ecosystem Newsletter : September 2026

The September 2026 DuckDB Ecosystem Newsletter: DuckLabs joins AWS and MotherDuck acquires Tower, DuckDB v2.0-alpha lands, table functions in pure Java, bulk loads into SQL Server, Zarr stores as SQL tables, an infinite canvas for your data, plus a community spotlight on Vladimir…

How to Measure AI Impact Beyond Token Caps

Originally published on yuvalyeret.com. The token cap is the least interesting number in your AI budget Somewhere in your company right now, someone is trying to decide whether an employee should get $100, $300, or $1,000 a month of AI spend. It is a real question with a real inv…

Teach Claude Your Power BI Standards, Custom Skills for the Power BI Modeling MCP Server

Can you teach Claude, or any LLM, to follow your standards and your guidelines, so that when it connects to a Power BI model, when it wants to build a semantic model or a report, it does it exactly as you do, following all the instructions that you would normally provide? Yes, it…

OpenAI agents attacked RubyGems back in May

They All Write, Store, and Read Data: How to Pick The Right Database

SQLite, Postgres, DuckDB and 20 others: they all write, store and read data. The questions you ask every day tell you which one you need.

Hi Scrum Masters, if your goal is to make yourself obsolete, then you're doing it all wrong.

Image I keep hearing it at meetups, in interviews, on LinkedIn. A Scrum Master says, with a straight face and a bit of pride: "my job is to make myself obsolete." And every time I hear it, I think: so you've decided when you're done growing? The three levels of serving The Scrum…

Microsoft Fabric in GCC High: Building the data foundation for AI

Government organizations are ready to move beyond using AI to answer questions and toward intelligent systems and agents that can take action. The post Microsoft Fabric in GCC High: Building the data foundation for AI appeared first on Microsoft Fabric Blog.

ToolGrad: Efficient tool-use dataset generation with textual "gradients"

Machine Intelligence

Data Engineering Weekly #287

The Weekly Data Engineering Newsletter

Coffee and Scrum - is there a connection?

Years ago when I was a Scrum Master for a great team we had a ritual. Right before the Daily Scrum, we would walk to our tea/coffee station and fill up our cups with our choice of drink and then walk back to our Jira board to inspect our progress. And I assume, a lot of other tea…

Matrix Totals – The Whiteboard #13

Let’s talk about matrix totals in this episode of The Whiteboard! Special guest: the SQLBI Whiteboard! Learn abstract DAX concepts in a more interactive way with “The Whiteboard” series. Read more: https://www.sqlbi.com/blog/marco/2022/07/14/the-whiteboard-video-series-on-sqlbi-y…

Measuring tactical intelligence targeting and conventional weapons capabilities of AI models

Anthropic’s Frontier Red Team developed new evaluations to measure AI capabilities in tactical intelligence targeting and conventional weapons development.

Your DuckDB Flock Deserves A Mother: MotherDuck Now Offers Open Source DuckDB Support

We're launching commercial support for open-source DuckDB, so every organization can get the power of a modern data engine, with direct access to database experts.

Can Your Team Name the Work It Already Runs With AI?

TL;DR: The AI Workflow Inventory Finalizes the A3 Delegation System You probably know your own AI shortcuts: the colleague who drafts the stakeholder update, the nightly job somebody set up before they left, the interview notes that go through a model on demand. However, I want t…

Referencing Images And Other Files In Power BI Reports Using OneLake URLs

Several years ago I wrote a very popular blog post about how to store images for your reports inside your Power BI semantic model. It solved the problem of how you could use display images (for example of products) inside your reports without making those images available via a p…

An alignment assessment of recent cybersecurity incidents

We present an alignment assessment of four incidents in which Claude models gained unauthorized access to real third-party systems.

Why DuckDB 2.0 is faster

DuckDB 2.0 alpha vs 1.5.5 on one laptop: recursive CTEs up to 90x, VARIANT 6x faster than JSON text, async I/O 2.4x on S3. What changes, and how to model data to get the speedup.

Deep Market Research with AI for Product Owners

Doing market research generally demands a lot of time, effort, and budget. But with the advent of AI, you can leverage it as a highly productive assistant to do deep market research. Let’s see how. Image I want to show you the process with an example. Imagine I want to start a bu…

Fabric Apps & Rayfin Deep Dive: A Conversation with Sachin Patney | Fabric Insider Ep. 11

I have written and recorded about Rayfin before, what it is, how the Data App template works, and the cost story that makes it genuinely significant. But this conversation goes much deeper. I sat down with Sachin Patney, who leads Fabric App Development at Microsoft, and by exten…

Some thoughts on the Navier–Stokes Millennium Prize Problem

Data Engineering Weekly #286

The Weekly Data Engineering Newsletter

Social Media Engagement: summer 2026

A quick survey of recent engagement of my posts on social media, indicating which service has by far the most engagement, and which service has seen a precipitous decline since early 2025. more…

SQLBI Whiteboard

SQLBI Whiteboard is a free, open-source whiteboard for Windows 11 designed for teaching DAX, SQL, and data modeling, with pen ink, live application capture, and syntax-highlighted code on the same canvas. Pen-first ink — Low-latency, pressure-aware strokes with palm rejection,…

AlphaGenome Atlas: A predictive map of every possible DNA letter change in the human genome

AlphaGenome Atlas maps the molecular effects of 9 billion single-letter DNA variants across the human genome.

Try DuckDB v2.0-alpha

The State of DuckDB v2.0 A few weeks ago, we published a preview of DuckDB v2.0. Today, we branched off v2.0-cyanoptera from the main development branch in the duckdb/duckdb repository. This marks a feature freeze – in the coming weeks, development work will focus on testing, bug…

Fragments: September 8

Christian Catalini says we’re in a situation where we are vastly reducing the cost of generating things, but not the cost of verifying them:. This explains why the first major AI products appeared in chat, image generation, and code assistance. Not because these were the hardest…

No-Code Power BI Report Authoring — GitHub Copilot + Fabric Skills + Power BI Desktop Bridge

AI can perform a lot of interesting tasks these days. It can check your email. It can draft an email response. And it can do many actions within the Power BI environment — including creating a complete Power BI report entirely from a prompt window, without touching anything insid…

The Pelican comparison grid for Astra is pretty interesting

Context, Semantics, and Ontology: A Primer for the Agentic Era

A primer on the context layer, semantic layer, and ontology: what each term means, how they fit together, and why AI agents still need a human to decide what's true.

Do you even need a presentation?

Like me, Sumeet Gayathri Moghe is tired of poor presentations with bad slide decks. He's started to write a series of posts on how to avoid these calamities, beginning with a post that questions whether a presentation is needed at all. more…

Testing DAX measures by using AI

How to use AI to identify relevant test cases in a semantic model and create repeatable DAX tests for a measure.

OpenAI's rogue agents were caught communicating via public wikis

Agents Don’t Query Like Humans Do

We analyzed a slice of MotherDuck query history and found agents ran 29 times more queries than humans last month, a gap that has roughly doubled every month. The average organization now has twice as many agents as human users. Agents are the dominant consumers of the data platf…

Bliki: Paracelsus Maxim

The difference between a medicine and a poison is dosage. Often we talk about certain habits, in programming or life, are good or bad. But few things are simple binaries. Some vary with context: reading a book is a good thing sitting in my garden, but not while driving my car. Bu…

Testing DAX measures by using AI

This article describes how to use AI to identify relevant test cases in a semantic model and create repeatable DAX tests for a measure. A DAX measure is correct only when it produces the expected result in every situation of…

Formalizing Fermat's Last Theorem

We are sharing the first complete computer-checked proof of Fermat’s Last Theorem. Claude worked largely autonomously over 11 days to write the proof in the Lean programming language.

Data Engineering Weekly #285

The Weekly Data Engineering Newsletter

Fostering Agency in Daily Leadership

This is the third and final part of the series on agency. Part one explained what agency actually is, and part two looked at what a lack of agency looks like in practice. This part could almost be short. Most of it is already captured in the fifth principle of the Agile Manifesto…

Tools With Interactive UIs In Fabric Notebooks With Semantic Link Labs

There’s so much going on in the Fabric community that it can be hard to keep up with it all. Semantic Link Labs is a great example: in the six months or so since I last had a proper look at it my colleague Michael Kovalsky has done a whole load of cool things and … Continue readi…

Transfer learning for genomic prediction in underrepresented populations

General Science

Agentic SQL for Free with Qwen3.8 27B and DuckDB

If your laptop has 16GB of RAM, your agent can write SQL locally for free with Qwen3.8 27B. On the DABstep SQL benchmark, Qwen beat GPT 5.6 Luna and cost under 50 cents in electricity, over 17x less. Add DuckDB and you have a data analyst that runs on your own machine. This post…

An Accidental Blackboard

Giles Edwards-Alexander reports that during an experiment to see how productive a team could be using fully agentic engineering practices, the team accidentally prompted the agents into creating a blackboard coordination system inside the git repository. more…

Displaying The Output Of Detail Rows Expressions In Power BI Reports Using The Paginated Report Visual

In last week’s post I mentioned that while Power BI reports (unlike Excel PivotTables) do not support the Detail Rows Expression feature, it is possible to partially work around this limitation by using the paginated report visual. In this post I’ll show you how I was able to do…

A connectomics milestone: Mapping the complete male fruit fly brain

General Science

MotherDuck CLI: query, pipelines, and dashboards from your terminal

Meet the MotherDuck CLI: one install to query data, publish Python pipelines, and host dashboards from your terminal, for coding agents and CI alike.

Maybe We Shouldn't Be Reviewing All This Code

TL;DR Or, perhaps the problem isn't that AI has broken code review, maybe it’s that we've been using code review to solve the wrong problems I was on a panel recently with Brian Houck from DX at Code Remix, hosted by Moderne. It was one of the more interesting panels I’ve done, l…

The Next Evolution of Power BI — A Conversation with Mohammad Ali | Fabric Insider Ep. 10

What is the next evolution of Power BI? Will Power BI Desktop ever become a prompt-first UI? And how do you actually decide whether you need a semantic model, an ontology, or both? I sat down with Mohammad Ali — Director of Product Management for Power BI at Microsoft, overseeing…

Introducing WeatherNext 3, our most advanced and accurate global weather AI model

Building an Operational Ontology: An E-Commerce Walkthrough

The write side of the ontology conversation: named actions, business rules, and write-back to the systems of record — a pattern already running at enterprise scale.

Fragments: September 1

Like many readers, I’m wary of AI generated prose. Simon Wilison has written an LLM cliché highlighter - paste in some text, or a URL, and it will flag various patterns common to LLMs. It references a wikipedia page of signs of AI writing. That page points out that: Humans are no…

Creating DAX functions with AI to remove duplicated code

How to leverage AI to refactor existing DAX code, remove duplicated parts, and centralize the common code in user-defined functions.

Proactive cyber defense for governments and enterprises

DuckLabs to Join AWS, Projects to Remain Open Source

DuckLabs today has announced that they will be joining Amazon Web Services as a new subsidiary. The big news for DuckDB and related projects like DuckLake is that there are no changes for our projects’ roadmap, licensing, and governance model! DuckDB, DuckLake, Quack, and all the…

What Leaders Do That Keep Scrum Teams Working Like Individuals

Scrum teams often work like individuals because the organization staffs, measures, interrupts, and rewards them that way. Leaders can change the conditions that make real teamwork possible.

Creating DAX functions with AI to remove duplicated code

Code written by AI can contain many duplicate parts. User-defined functions are a great tool to simplify complex measures; this article shows how to leverage AI to build better DAX code. AI is getting better and better at generating DAX…

Introducing Gemini 3.8 Flash and 3.8 Flash Cyber

DuckDB outgrows its nest

Today Duck Labs, the developers of DuckDB, announced they are being acquired by Amazon. This is big news in the duck-iverse, and many people are wondering what this will mean for everyone’s favorite duck-powered database, MotherDuck.

Making Your Data Ready for Agentic AI

Lots of organizations are excited about what AI can do to streamline their processes, save money, and juice margins. But AI's capabilities are founded on the data that AI accesses, and for many organizations that foundation is little more than sand. Pramod Sadalage and Prem Chand…

Restoring alternating rows in Power BI matrices with Fluent 2

Microsoft recently introduced a new default visual design style for Power BI named Fluent 2. I personally disagree with a few choices in the new defaults, including the removal of alternating row colors from Table and Matrix visuals. However, the…

Claude's new system prompt really doesn't want to reproduce song lyrics

How DuckDB Runs Recursive CTEs Faster

When I implemented DuckDB's first recursive CTE operator in 2020, correctness determined the design: evaluate the non-recursive term once, then evaluate the recursive term until the next working table is empty. That established the right semantic contract, but reusable runtime st…

Fragments: August 24

I was listening to Ezra Klein’s interview with Helen Toner about the recent OpenAI hack of Hugging Face and the subsequent discovery that there were swarms of agents inside OpenAI doing unsanctioned activities. One of the points Klein made was that at no point did any of these (t…

Using Detail Rows Expressions To Drill To A Different Fact Table In Power BI

If you have a DirectQuery fact table in Power BI you can use user-defined aggregations to improve query performance; querying a smaller, summarised copy of your data in an Import mode aggregation table is always going to be faster than querying a large fact table containing all y…

DuckDB Table Functions in Java

In a large organization, data is spread across many systems, including relational databases, document stores, message queues, data lakes, and cloud data warehouses. Some of these systems can be reached only through a vendor SDK, usually provided in Java, or a SOAP endpoint, and m…

Citizens Build, Agents Execute, Experts Govern

TL;DR Why building an app over the weekend isn't the same as building enterprise software I’ve noticed an interesting gap opening up over the last six months. It isn’t really a gap in technology. It’s a gap in what different people think software engineering actually is. The conv…

How to check table sizes in a Microsoft Fabric warehouse

If you've tried to check how much storage your Fabric warehouse tables consume, you've probably discovered that the usual SQL Server approaches don't work. This post explains why, what else doesn't work, and the one approach that does The post How to check table sizes in a Micros…

MotherDuck Acquires Tower: Agents Can Answer. Now They Can Build.

Practitioner Voice: The Writing Category Nobody has Named Yet

Jim Highsmith recognizes that effective writing from a practitioner is a style distinct from academic writing or thought-leadership content. It's a style that I advocate, and my contributors mostly follow. Jim decided it was important to give it a name, and identify what makes it…

Analysing Power BI Data Using Copilot In Excel

This week there was another big Copilot-related announcement for Power BI – but on the Excel blog. You can now analyse data from Power BI semantic models using Copilot in Excel. Full details on how to enable this are in the docs here, but what does it look like? Using the same se…

Data Engineering Weekly #284

The Weekly Data Engineering Newsletter

Fragments: August 18

Part of the reason why I’m at Thoughtworks is because I’d like to see a software development organization founded on technical excellence as an example for the rest of the industry. The trouble is that I have little aptitude or inclination for the hard work of building such an or…

(Don’t) Stop materializing your silver layer!

Why is your silver layer a set of tables? Challenging the fully materialized medallion pattern - with benchmark numbers and reproducible demo scripts. The post (Don’t) Stop materializing your silver layer! appeared first on Data Mozart.

Chunked Query Results in the DuckDB Java Driver

DuckDB is a columnar, vectorized database. Every operator inside the engine works on data chunks: batches of column vectors, 2,048 rows at a time. That is a big part of why DuckDB is fast: the engine amortizes interpretation overhead over thousands of values instead of paying it…

Why Your Scrum Team Still Works Like Individuals

Learn how handoffs, individual task ownership, and status-report daily scrums keep Scrum teams from collaborating—and what to change next sprint.

Analyzing the performance impact of visual calculations

Visual calculations can either improve or degrade the performance of a report. Let’s see how we ensure visual calculations have a positive impact on performance.

DuckDB v2.0: Your Database Deserves a Better Parser

At DuckDB, one of our goals is to make working with a database system as easy as possible. Users interact with the system through the widely understood Structured Query Language (SQL). Previous blog posts have covered DuckDB’s friendly SQL, including GROUP BY ALL and column selec…

TDD inside the agent loop - theater or actual value?

My colleagues at Thoughtworks tend to be big fans of Test-Driven Development, and many people in the industry advocate telling LLM agents to use TDD when building software. Birgitta Böckeler was curious if this really makes a difference, so conducted a few experiments. more…

Analyzing the performance impact of visual calculations

Visual calculations can either improve or degrade the performance of a report. In this article, we outline how we ensure visual calculations have a positive impact on performance. The goal of visual calculations is to simplify some reports and calculations,…

AI Writes the Semantic Layer

We proved that AI agents could build a semantic layer. The more interesting result was that the semantic layer itself became disposable. Q&A pairs are unreasonably effective at capturing business intent and give agents something concrete to iterate with. If code can be regenerate…

Lack of Agency – When the system swallows the human

In my previous article, I explored the concept of agency — the capacity and freedom of individuals to act independently, make choices, and meaningfully influence their circumstances. But what does it look like when agency is entirely absent? I learned the hard way when I tried to…

Power BI, M365 Copilot And The Importance Of DAX UDFs

This was a big week for Power BI Copilot with the announcement of a new direction for the feature. If you haven’t done so already, read the two announcement blog posts here and here and check out this demo video. There are some pros and cons to this change (yes, you’re going to n…

Reconciling JSON in DuckDB, One Patch at a Time

Guest blog post by Mustafa Khan (Atlan). DuckDB's JSON extension already covers reading JSON, extracting paths, applying RFC 7396 merge patches, and the usual scalar accessors. The upcoming v2.0 release will extend it with four new scalar functions: json_merge_patch_diff computes…

Fragments: August 4

There’s been a fair bit of publicity of the Open AI “rogue agent” that hacked into Hugging Face. This prompted Anthropic to check what their models were up to and, to my complete lack of surprise, discovered three incidents where models had gained unauthorized access to data in o…

Using Fabric Operations Agents And Workspace Monitoring With Power BI

This week, in the announcement about support for Fabric Pipelines in Workspace Monitoring, I noticed that it came with an Operations Agent that actively monitors and analyses Pipeline activity. And that got me thinking, since Workspace Monitoring also contains Power BI activity d…

Scalable Data Apps with MotherDuck & Replit

AI agents make building data apps easy, but what about deploying them or making sure they scale? Replit provides a way for just about anyone to build real applications, and MotherDuck is the easiest way to help them scale to larger data sizes. Load data with MotherDuck Flights an…

The Conductor Developer

TL;DR Why I think software development is starting to feel a little more like conducting an orchestra. There’s a shift happening in software development that I don’t think we’re talking about clearly enough. For the last couple of years we’ve framed AI as a productivity tool. How…

Dynamic formatting by hierarchy level with ISINSCOPE and ISATLEVEL

How to apply a different formatting rule at each level of a hierarchy using ISINSCOPE in a measure or ISATLEVEL in a visual calculation.

Data Engineering Weekly #283

The Weekly Data Engineering Newsletter

The Economic Benefit of Refactoring

Giles Edwards-Alexander does an experiment to see if decomposing a large function helps reduce token costs, suggesting that is may now be possible to measure the economic benefit of refactoring more…

Dynamic formatting by hierarchy level with ISINSCOPE and ISATLEVEL

This article describes how to apply different formatting rules at each level of a hierarchy (one rule at the year level, another at the quarter level, another at the month level) using ISINSCOPE in a measure or ISATLEVEL in a…

What an Ontology for AI Agents Actually Needs

How to think about the semantic system that makes an agent coherent, governable, and useful.

Leading Agile Initiatives: How Leaders Help Agile Change Succeed

An agile initiative is not mainly a one-time rollout of Scrum, Kanban, SAFe, Jira, new job titles, or a different meeting calendar.

Generative AI guidelines at SQLBI (2026 update)

3 years ago, we wrote our “Generative AI guidelines at SQLBI”. We think it’s time to update them. Over the past 3 years, the world has changed: it is simply no longer possible to avoid using generative AI services for…

Declarative Automation in Dagster: Smarter Data Orchestration Without Complex Schedules

Learn how Dagster's Declarative Automation simplifies data orchestration by defining desired asset state instead of managing schedules, sensors, and custom triggers.

The Orchestrator's Tax

Subagents get justified by time saved and parallel execution, but Rahul Garg explains that's not what matters most. Every token in the orchestrator's context is competing for its attention, and the real value of a subagent is what it keeps out of that context. Subagents should be…

Liar, liar! What to do when AI writes your pipelines confidently wrong?

Should we let AI write our pipelines? YES...But not the way you probably think. The post Liar, liar! What to do when AI writes your pipelines confidently wrong? appeared first on Data Mozart.

Creative Dagster Projects from the Community

From public dataset explorers to infrastructure monitoring and research automation, the Dagster community is building projects we never could have predicted. Here are a few creative community-built use cases that caught our attention.

Focusing Where We Can Have the Most Impact

Mountain Goat Software is shifting its training and coaching focus to private client work, where teams and leaders can apply agile to their real challenges.

One-minute trap! How new Fabric Warehouse CU metering affect your workloads

Microsoft is moving Fabric Warehouse CU metering from per-query CPU time to per-workspace virtual-node time. That brings Fabric closer to Snowflake and Databricks, but it also changes the cost equation: dense workloads may benefit, while sparse, chatty workloads could get a surpr…

Data Engineering Weekly #282

The Weekly Data Engineering Newsletter

We Tried Agile and It Didn’t Work

“We tried agile and it didn’t work” is one of the most revealing things a leader can hear.

Using REMOVEFILTERS in DAX user defined functions

How to implement a DAX function that removes filter-keep column filters from a calendar, using REMOVEFILTERS as the return value of the function.

On Benchmarking

Why a throughput number is not an architecture decision.

Why Scrum Isn’t Working Even Though You’re Doing Scrum

Story splitting helps teams turn large user stories into smaller, valuable pieces they can finish within a sprint without turning them into tasks.

Using REMOVEFILTERS in DAX user-defined functions

In this article, we implement a function that removes filter-keep column filters from a calendar, using REMOVEFILTERS as the return value of the function. A DAX user-defined function, also known as a UDF, is expected to return a scalar or…

Data Engineering Weekly #281

The Weekly Data Engineering Newsletter

MobAI: From Copilot to Team Pilot

AI can make individuals faster. For leaders, the bigger question is whether it helps teams build shared context, judgment, and better decisions. AI can make individuals faster. It can also make teams more fragmented. That is one of the risks I see in how many organizations approa…

Why data teams are emerging as leaders in AI agent adoption

New research from Microsoft and MIT Technology Review Insights reveals how organizations are building trust in AI agents—starting with structured data workflows and scaling through shared context and real-time intelligence. The post Why data teams are emerging as leaders in AI ag…

Creative Dagster Projects from the Community

From public dataset explorers to infrastructure monitoring and research automation, the Dagster community is building projects we never could have predicted. Here are a few creative community-built use cases that caught our attention.

Beyond “Taking Responsibility”: Why Leading for Agency is the Future of Modern Management

In recent years, a new word has steadily climbed the ranks of leadership literature and corporate strategy sessions: Agency. While many managers intuitively grasp that it has something to do with employees taking ownership, the concept goes far deeper than simply “taking responsi…

FabCon Europe 2026: The sessions we’re most excited to bring to Barcelona

Get a first look at FabCon Europe 2026 in Barcelona, with curated session picks across Microsoft Fabric, SQL, AI, real-time intelligence, and data engineering—plus insights into the innovations shaping the future of data. The post FabCon Europe 2026: The sessions we’re most excit…

Data Engineering Weekly #280

The Weekly Data Engineering Newsletter

Catalyst Leadership: You Cannot Optimize the Whole System if Your Inner System Optimizes for Safety

Most leaders are not held back by a lack of competence. They are held back by an inner system that was built for a different kind of world. According to Bob Anderson and Bill Adams at Leadership Circle, their research suggests that 70-80% of leaders lead reactively. That does not…

Variable libraries in Microsoft Fabric: One value to rule them all!

Hardcoded values are the cockroaches of a data platform: you think you got them all, then Test starts reading from Prod and nobody notices for three days. Variable libraries in Microsoft Fabric put your configuration in one versioned place and let each stage resolve its own value…

Data Engineering Weekly #279

The Weekly Data Engineering Newsletter

When Planning Should Become A Shared Problem

Turn planning pressure into a shared conversation about options and tradeoffs.

Data, AI, and the case for continuous learning

Microsoft Data Days brings together training, certification pathways, and more—helping you build practical skills and stay ahead in a rapidly evolving field. The post Data, AI, and the case for continuous learning appeared first on Microsoft Fabric Blog.

Data Engineering Weekly #278

The Weekly Data Engineering Newsletter

How the Story Critic AI Skill Helps Teams Write Better Backlog Items

See how AI coaching can help teams write clearer, smaller, more testable backlog items.

Microsoft Build 2026: Building agentic apps with Microsoft Fabric and Microsoft Databases

Microsoft Build 2026 highlights advancements in app development with Microsoft Fabric and Microsoft Databases, emphasizing a unified data and AI platform for scalable, agentic applications. The post Microsoft Build 2026: Building agentic apps with Microsoft Fabric and Microsoft D…

Beyond Redaction: Anatomy of a Privacy-Safe Data Platform

Why privacy engineering is about governing data in use—not simply hiding it.

Why Smart Teams Overcommit And How Leaders Make It Worse

Understand how leader pressure can push smart teams into unrealistic commitments.

CLUSTER BY in Fabric Warehouse: when the magic shows up and when it doesn’t!

Microsoft's announcement post for data clustering in Fabric Warehouse promised 24x speedups. So I built a 100-million-row clickstream table and tested it myself. The result is more nuanced than the marketing - and more useful for anyone deciding whether to actually adopt this fea…

Data Engineering Weekly #277

The Weekly Data Engineering Newsletter

Leading Complexity 2025 — Just Two Weeks Away!

We’re just two weeks away from the launch of Leading Complexity 2025, and we couldn’t be more excited to welcome the amazing group of participants who’ve signed up from around the world. This year marks the fourth season of the program — and the theme feels more urgent and releva…

Hot, Cold, Gone: Caching and Retention in Microsoft Fabric Eventhouse

Two policies, three storage states, one rule connecting them. A newcomer's guide to caching, retention, and tiered storage in Microsoft Fabric Eventhouse. The post Hot, Cold, Gone: Caching and Retention in Microsoft Fabric Eventhouse appeared first on Data Mozart.

Announcing leetdata.ai — A Practice Ground for Data Engineers

Software engineers got a gym. Data engineers got production. It's time to fix that.

How to Lead—and Not Be Led—by AI in Complexity

A conversation with long-time Crisp collaborator Jonathan ReamsSummary (Abstract) AI is accelerating our tools—and our context. In this Leading Complexity Podcast episode, Jonathan Reams argues that leaders must grow their inner capacity at least as fast as AI evolves, or risk be…

Materialized Lake Views in Microsoft Fabric: When your medallion fits in a SELECT statement

Five surfaces collapsed into one declarative layer. Here's the full story of Materialized Lake Views in Microsoft Fabric - from syntax to the new GA capabilities announced at FabCon Atlanta The post Materialized Lake Views in Microsoft Fabric: When your medallion fits in a SELECT…

Creative Dagster Projects from the Community

From public dataset explorers to infrastructure monitoring and research automation, the Dagster community is building projects we never could have predicted. Here are a few creative community-built use cases that caught our attention.

Trying Out Lovable – A New Way to Build Websites with AI

Recently, I explored a new tool called Lovable. It’s an AI-powered platform that helps you build websites and web apps just by describing what you want. You write a short description (called a “prompt”), and Lovable builds the code and design for you. As someone who works with Ag…

Highlights from FabCon and SQLCon 2026: Where databases and Fabric come together

From technical sessions to hallway conversations, FabCon and SQLCon 2026 showcased the momentum behind the Fabric and SQL ecosystems. The post Highlights from FabCon and SQLCon 2026: Where databases and Fabric come together appeared first on Microsoft Fabric Blog.

Data Engineering Weekly #276

The Weekly Data Engineering Newsletter

Leveling Up Leadership: A 5-Year Journey Boosting Engagement at a Game Studio

How a long-term leadership program helped build stronger leaders, increase engagement, and boost collaboration across teams. The Challenge: When Engagement Slipped, Leadership Stepped Up When the executive team at a large game studio noticed a dip in employee engagement in their…

FabCon and SQLCon 2026: Unifying databases and Fabric on a single data platform

We're bring attendees together to share real experiences and solve challenges side-by-side. Only together can we move into meaningful results. The post FabCon and SQLCon 2026: Unifying databases and Fabric on a single data platform appeared first on Microsoft Fabric Blog.

Operationalizing Data Orchestration: Best Practices for DevOps, Infra, and Code Locations

Leading Complexity – Year 4: Navigating the Age of AI (and Everything Else)

Three years ago, we launched the Leading Complexity Program with a simple but urgent question: How do we lead when the world is changing at an exponential pace? Organizations everywhere were feeling the pressure of complexity—messy systems, rapid change, and no playbook for how t…

Advancing agentic AI with Microsoft databases across a unified data estate

Built on a consistent Microsoft SQL foundation from on premises to the cloud, Azure SQL brings AI capabilities directly into your database experience. The post Advancing agentic AI with Microsoft databases across a unified data estate appeared first on Microsoft Fabric Blog.

The Dagster Almanack: From Complexity to Composability

Mob Programming with AI: Inside a High-Performing Team’s Journey

Introduction What happens when you combine a passionate development team, a dash of Tesla-inspired thinking, and the principles of Mob Programming, with AI as an active participant? At Traton-Scania, the AI-Enablement team is doing just that, transforming how they build software…

Unpacking your top questions on agentic AI: The Shift podcast

We're launching a new podcast called "The Shift," which does exactly that. Grounded in questions we heard from you after Ignite news, we're releasing eight episodes that bring engineering, product, and strategy perspectives together. The post Unpacking your top questions on agent…

You Don't Graduate From Data Engineering: Why We Built aide for Continuous Learning

Introducing aidataengineer.io — an AI-powered learning platform built on 275+ editions of Data Engineering Weekly.

Three Strategies for Leading in Complexity

Summary of my talk from Agile Sverige 2025 A few years ago, I heard the quote: That felt true then—and it’s even more true now. After co-hosting the LeadingComplexity.com program for several years with over 30 world-class thought leaders on the topic of leading in complex environ…

Top sessions you won’t want to miss at the Microsoft Fabric Community Conference 2026

The Microsoft Fabric Community Conference (FabCon) is returning stateside in just a few weeks, taking place at the Georgia World Congress Center in Atlanta, March 16 - 20, and it’s shaping up to be our biggest event yet. The post Top sessions you won’t want to miss at the Microso…

Data Engineering Weekly #275

The Weekly Data Engineering Newsletter

How we deploy confidently with Canary Deployment

When you push to production multiple times a day, the stakes are high — and manual rollbacks slow. Here’s my experiences from deploying services using Canary Deployments for an important backend service for a large global streaming service. Why Canary Deployments? We adopted Cana…

The Orchestration Maturity Model: Why Teams Move from Jobs to Assets

Most orchestration systems answer questions about jobs, but organizations need answers about data. This post introduces the Orchestration Maturity Model and explains why enterprises adopt Dagster to move from job-centric orchestration to asset-aware data platforms.

1-2-4-All and Meetings People Actually Want to Attend

Have you ever found yourself stuck in a meeting where one person does all the talking… and everyone else just tunes out? Or where it’s so open-ended that nothing really gets done? Even with smart people in the room, the best ideas often never get heard. That’s precisely the probl…

How to Orchestrate dbt with Dagster

With Dagster’s dbt integration, run and monitor dbt models as part of a larger, asset-driven pipeline for improved lineage and scheduling.

Unlock Your Team’s Full Potential: Dive into a Liberating Structures Immersion Workshop

Are you ready to revolutionize the way your team collaborates, innovates, and solves problems? Welcome to the world of Liberating Structures, where conventional meeting formats are transformed into dynamic, inclusive, and highly productive sessions. ## What Are Liberating Structu…

How Brooklyn Data Uses Compass for Self-Service Analytics in Slack

Brooklyn Data shares how it uses Compass to make operational and financial data accessible in Slack, reducing time-to-answer, improving data discoverability, and enabling governed self-service analytics across the organization.

Align Your Strategies

In the past few weeks, I’ve received similar questions from two different companies, which I’d like to reflect on here. Both organizations were in the process of crafting strategies for critical areas: one for AI and the other for cybersecurity. Their question to me was whether i…

Snowflake Runs Your Data. Dagster Runs Everything Else

Snowflake powers modern data platforms, but orchestration still matters. Learn how Dagster complements Snowflake with SQL-native assets, Dynamic Table lineage, virtual assets, downstream automation, and Snowflake cost attribution.

Why your teams aren’t really empowered (and how to fix It)

What actually empowers a team? Despite all the talk about empowered teams in tech, there's often confusion about what real empowerment looks like. At its core, an empowered product team is given significant customer or business problems to solve, rather than being handed features…

We Tried ty for Performance. It Found Real Bugs

Dagster replaced Pyright with Astral’s new Python type checker, ty, and saw CI type checking drop from minutes to seconds. Along the way, ty surfaced real runtime bugs Pyright missed and improved developer feedback loops across a large Python monorepo.

If You Lived in Germany in 1939, What Would You Do?

Imagine the following: You live in Germany in 1939. Or perhaps it’s 1938, or even as early as 1935. The Nazis are in power. If you’re an average German, your material conditions are probably quite good. Unemployment is low, and the Nazis have introduced extensive “welfare program…

Announcing the Dagster+ Terraform Provider

Manage Dagster+ deployments, permissions, and alerting as code with the new Terraform provider. Bring consistency, reviewability, and automation to your data platform.

Why AI Agents Need Operational Context — Not Just Semantic Definitions

Semantic context tells AI agents what data means. Operational context tells them whether to trust it. Learn how Dagster and Atlan combine both halves of the context layer.

Databricks Cross-Workspace Orchestration with Dagster

Lakeflow Jobs can't see across workspace boundaries. Learn how Dagster unifies multiple Databricks workspaces into a single asset graph with real dependencies.

Dagster 1.13: AI Skills, Partitioned Asset Checks, Virtual Assets & 20+ New Components

Dagster 1.13 makes the framework easier to adopt with AI coding skills, partition-aware asset checks, virtual assets for database views, and 20+ new integration components.

Monorepos, the hub-and-spoke model, and Copybara

How we use Copybara to build a hub-and-spoke model of public and private git repositories

Making Dagster Easier to Contribute to in an AI-Driven World

As AI makes it easier to generate pull requests, reviewing them has become the bottleneck. Learn how Dagster is improving contributor experience with better tooling, clearer guidelines, and a focus on high-impact contributions.