Microsoft Fabric has been moving fast — and nowhere faster than in data integration. Between Microsoft Fabric Mirroring, Copy Job, Open Mirroring, and SAP integration, there is a lot to unpack.
I recently sat down with Wilson Lee, Principal Product Manager at Microsoft, who leads the entire data movement area — including Mirroring, Copy Job, SAP connectivity, and all data integration in Microsoft Fabric. We covered a lot of ground: what Mirroring really is, how it is priced, what Copy Job is and how it differs, the latest FabCon announcements, and the big question — is traditional ETL dead?
Watch the full conversation here:
Who Is Wilson Lee?
Reza: Before we start talking about this and considering we had a lot of announcements recently in FabCon, would you introduce yourself — who you are and what part of the product you are working on?
Wilson: “My name is Wilson. I’m a principal product manager in the Microsoft data integration team. I lead all the data movement including Mirroring, Copy Job, SAP area, and connectivity in Microsoft. Very happy to chat with you today.”
What Is Microsoft Fabric Mirroring?
Reza: Let’s start with the Mirroring subject. Looking at it from a high-level point of view — what is Mirroring, from your perspective?
Wilson: “Mirroring — it’s basically in the name, right? We mirror your data in your proprietary database and bring it into Fabric OneLake. Typically when we are talking about how is the best way to bring data into Fabric — if you have files, you will use Shortcut because you virtualize your files and don’t have to move them into Fabric OneLake. But if you have proprietary data like in a database — imagine you have a SQL database or Oracle database — you won’t be able to Shortcut that data. So in order for you to bring that data into Fabric OneLake, the easiest way is using Mirroring, where we first capture your snapshot from your entire database, bring it to Fabric OneLake, convert it into our Delta Parquet format, and then help you bring change data into Fabric OneLake as well — making sure everything is merged, everything is efficient. And the best thing is that once your data is in OneLake — no matter if it’s from SQL, from Oracle, from SAP — all your workloads, whether you want to run a notebook or Power BI on top of that, will work very seamlessly.”
Reza: So in short — correct me if I’m wrong — Mirroring is similar to if I went and built that ETL myself. I connect to Snowflake, Databricks, Azure SQL Database, and I write my Python code, SQL scripts, Dataflows, or pipelines to integrate data from that source and bring it into a Fabric Warehouse. Instead of doing that, I use Mirroring and Mirroring does all of that?
Wilson: “That’s right. And one thing to note — many of our customers maybe already have gold data that they’ve processed somewhere else. They just want to bring it to Fabric OneLake easily using Mirroring. Another pattern is folks who just need to bring their bronze data into Fabric OneLake as-is, without changes. And one great thing we announced at FabCon is the extended capability — we now support Change Delta Feed as well. So once you bring the data into Fabric OneLake, you can actually turn Change Delta Feed on and capture the changes as you process the data downstream to your silver and gold layers.”
Reza: So it is not only a one-off copy of the data. It is also using the change data capture or any mechanism available in the source system — it is aware of changes happening there and incrementally brings those changes into Fabric without us doing anything.
Wilson: “That’s right. And what’s great about Mirroring is that we made it all simple. We help you connect to the source, get the changes through CDC and many other mechanisms, get the change feed into Fabric OneLake, and we just make it all work for you so you don’t have to worry about it.”
Reza: And this happens in near real-time — not just once a day?
Wilson: “That’s right. Mirroring does it for you. So you don’t even need to figure out when to bring data in. If your use case can use that capability where Mirroring just takes care of everything and brings data as-is, Mirroring is the best tool to use.”
How Much Does Mirroring Cost?
Reza: Microsoft promotes Mirroring as being free — included in your Fabric capacity. That includes the compute used for mirroring and the storage for the Delta Parquet files in OneLake. Is this different depending on which F SKU I use?
Wilson: “For Mirroring, you will get free storage based on the capacity — it is tiered. If you have F64, you have 64 terabytes for free. If you have F128, you will have 128 terabytes for free. But the compute itself — no matter which capacity you provision — it is all on us. It will not count against your Fabric capacity and you won’t get throttled on that either. So no matter if you are provisioning an F2 or an F64 or F128, the efficiency and the speed that we replicate your data in near real-time will be exactly the same.”
Reza: So if I have F2, I can mirror my Snowflake data warehouse and the compute for that mirroring is not dependent on my F2. I shouldn’t be worried about having low compute capability, and I have up to two terabytes of storage available without paying anything?
Wilson: “That’s right. And that’s the best thing about Mirroring.”
Reza: That is great to know. So in summary — the compute for data transfer is free and separate from your Fabric capacity, and the storage is free up to a tier that matches your SKU.
Extended Capabilities: What Is Free and What Is Not
Reza: At FabCon you announced extended capabilities for Mirroring. Can you help us understand what remains free and what carries a charge?
Wilson: “Our core Mirroring capabilities — bringing data in, near real-time replication, the free compute, the tiered storage — those remain free. But because of the extended capabilities that we announced at FabCon, those will require additional compute and will have an additional charge. Things like Change Delta Feed — the ability for downstream processes to listen to changes in the mirror database — those will be charged as you are processing those changes. We also started supporting view replication with Snowflake as part of extended capabilities.”
Wilson: “The easy way to think about it: bringing data into Fabric — that is free. The compute to do so is free. The storage to store data is also free within the tier. The extended capability is for very unique use cases where you need to listen to those changes downstream. It might not be needed by everybody.”
Reza: One clarification for the audience — if I use standard Mirroring without the extended Change Delta Feed capabilities, does it only bring data once or does it still keep updating?
Wilson: “Bringing data in — and we do it in near real-time — those still remain free. Nothing changed in that part. It is only for folks who say they need to actually process the data downstream and listen to the changes coming into the mirror database. That is when you turn on CDF and that is where the charges start to happen.”
What Is Running Under the Hood?
Reza: Can you tell us a little about what happens behind the scenes of Mirroring? Is it creating a Dataflow or a pipeline under the hood? Python code?
Wilson: “When we launched Mirroring about two and a half years ago, Mirroring was completely built brand new from scratch using everything that we can use with the capabilities in Fabric. The engine we built was tailor-made for Fabric itself. This is why we can bring data for free and make it so efficient that we can do it in near real-time. Underneath the hood it is not using pipelines, not using Dataflows — it is actually an engine that is purposely built with the platform layer inside Fabric itself.”
Reza: So it is a proprietary engine that Microsoft has built specifically for integrating that data?
Wilson: “That’s right. That’s right.”
Reza: Nice. And that also explains why the compute can be offered separately — it is their own dedicated infrastructure, not consuming your Fabric capacity.
Mirroring vs. Shortcut: What Is the Difference?
Reza: Sometimes customers ask — should I use Mirroring or should I use Shortcut? From your point of view, what would you say?
Wilson: “The way that we think about it — Mirroring now represents the easiest way to bring data into Fabric OneLake. Underneath the hood, the technology is different. Shortcut is for files and replication is for databases. In Shortcut, you virtualize files — no matter if the file is in ADLS Gen2, AWS S3, GCP, or wherever — Shortcut virtualizes that file and brings it into Fabric OneLake. But you won’t be able to Shortcut a proprietary format like a SQL database. It is not files underneath the hood. Which is why we also have Mirroring — the replication piece — that brings both your snapshot and your changes into Fabric OneLake.”
Reza: So to summarize — if I want to bring files, Shortcut is the right tool. If I want to bring structured data from a database, Mirroring is the right tool. These are different tools that do different jobs.
Wilson: “Exactly right.”
Open Mirroring: When You Need Custom Sources
Reza: We also have the concept of Open Mirroring. How is that different from standard Mirroring?
Wilson: “In database Mirroring, we help you connect to the source and bring the snapshot and change data into Fabric OneLake. With Open Mirroring, you still enjoy the engine that we run in our mirror database — that helps you merge all the files, convert into the right format, make it super efficient inside Fabric OneLake. But you decide how you want to bring the data in. No matter if you are a developer who wants to write code to bring data from an API that we do not support — you have the ability to write your own code to bring data into Open Mirroring. Or you can leverage our vast number of partners who can actually help bring that data in.”
Reza: So to shorten that in my way of saying — I would use standard Mirroring when the source is already on the supported list. If I want to customize the ingestion — for example, I have an API as a data source — but I still want to use the free Fabric Mirroring engine, I would use Open Mirroring. Is that right?
Wilson: “That’s right. And at FabCon we were super excited to announce Informatica as a new Open Mirroring partner. If you already have pipelines running with Informatica in production, you can choose Open Mirroring as your destination and bring that into Fabric as well.”
It is not recommended to rebuild existing ETL pipelines from scratch if you already have partner tools like Informatica in production. Open Mirroring lets you keep what you have and simply redirect the destination to Fabric.
What Is on the Mirroring Roadmap?
Reza: Do you think Mirroring would ever get to a stage where it also replicates security rules — for example, row-level security configurations from the source system?
Wilson: “That’s a great question. For Mirroring, we are always finding ways to make it easier to use and support more scenarios. One of the big demands is for us to be able to bring security rows and metadata into Fabric OneLake as well. This is something that is on the roadmap. I’m sure that folks will be very happy to see that we will roll out something similar to that later in the year. Watch out for our next set of announcements coming later in the year.”
Has Mirroring Reliability Improved?
Reza: Early adopters of Mirroring reported reliability challenges. Have those been addressed?
Wilson: “Data integration is hard. Every customer has a production system that is a little bit different. But one thing we have noticed — and we are really grateful to the community — is that Mirroring has generated tons of excitement. Since we went GA more than a year ago on the initial set of sources, Mirroring is getting better and better. The engine that is running — the compute I mentioned for free — that engine is the same for every single connector. So what that means is that our engine has been continuously improving and becoming more and more efficient every day. Mirroring is stabilized and better and better for customers to use in production. And if any customer is facing issues, please do reach out. We want to understand your system and continue to make Mirroring better.”
Reza: My advice to customers on this: software always has a lifecycle. Power BI that we use today is not the Power BI from 2015. There were a lot of challenges and bugs that got fixed over time. I would not say wait until something is ten years old before using it — by then it is probably too late. I would say go and use a product that has a strong growth rate, where the product team is watching feedback and acting on it. And everywhere in Fabric we have that kind of communication channel, which is great.
What Is Copy Job?
Reza: We also have something in data integration called Copy Job. How does that work and how does it compare to Mirroring?
Wilson: “Copy Job gives you full flexibility to bring data from any source to any destination. You can think of it as more of an M-to-N pattern — many sources to many destinations. The sources include files, databases, REST APIs, and you can write to any destination including Fabric OneLake. Copy Job has three different modes: batch mode, incremental mode based on timestamps, or CDC mode — similar to what Mirroring does. And we also added advanced capabilities like column selection, column mapping from source to destination, and SCD Type 2 — keeping full history of changes in the destination. We also announced at FabCon that Copy Job is now available as an activity inside a Fabric Pipeline, so you can orchestrate it alongside notebooks, email notifications, and many more activities.”
Reza: Let me use an analogy. With Mirroring, it is like taking an Uber — I get in, I give my destination, and it takes me there the best way automatically. With Copy Job, it is more like renting a car — I am still using the same powerful engine, but I control the route, the speed, every decision along the way. I can map columns, choose incremental or CDC, and route to destinations outside of Fabric. Am I saying that right?
Wilson: “I love the analogy. You decide how you want to bring data in — it is highly configurable, highly orchestratable.”
Reza: And one more point I want to emphasize — with Copy Job I can also set my destination to be outside of Fabric entirely, whereas Mirroring always lands inside Fabric OneLake. Is that right?
Wilson: “That’s right. We support 170-plus connectors — no matter from source to destination. So if you need to write data outside of Fabric OneLake, Copy Job is the tool to use.”
Copy Job vs. Copy Activity: Are They the Same?
Reza: How is Copy Job different from Copy Activity in a Pipeline? Is Copy Job going to replace it?
Wilson: “The engine itself is exactly the same between the two — our petabyte-scale engine that has been proven to move data from one place to another for many years. The way we look at it: with Copy Job we are focusing on simplicity. Copy Activity gives you more flexibility — folks who want to run metadata-driven scenarios, variables, and those kinds of things. Those are still things you will use with Copy Activity. But one thing to note for everybody is that the engine itself is exactly the same. We are providing different options for you to use depending on what you need to do.”
Copy Job is not replacing Copy Activity. They serve different scenarios with the same proven underlying engine.
Is Traditional ETL Dead?
Reza: From a data engineer’s point of view — with Mirroring and Copy Job available, should we think of traditional ETL as dead? Is this the new way of doing data integration?
Wilson: “I don’t think that the traditional way is dead. It is just a different way of approaching how you want to process your data. We have seen both patterns — customers who load everything into a lake and then process it, and customers who do all the ETL beforehand before writing to a lake. To me right now it is really 50/50. We have customers using the so-called traditional pattern that works for them, and customers coming to a newer pattern where they load bronze data into a lake first and process it afterwards. Our goal is to provide all these solutions that fit you best into your production workload.”
Reza: I would also add this: Mirroring and Copy Job are excellent at solving the ingestion problem. But even with Mirroring, the data still comes in as it is from the transactional database — 200 normalized tables. You still need to transform it into a star schema, dimension tables, and fact tables for analytics. That transformation work — whether done through Dataflows, SQL scripts, notebooks, or any other tool — still requires the same data engineering skills it always has. You might get AI to help you with some parts of it, but the skill and the need remain.
SAP Integration in Microsoft Fabric
Reza: One other big topic I want to cover — SAP integration. This is something that comes up in pretty much every enterprise customer conversation I have. Can you walk us through what was announced at FabCon and what is coming?
Wilson: “In every customer conversation, the topic of SAP always comes up because we understand there is a lot of data in SAP and many customers want to use that in their analytics. A couple of things that are coming soon and a couple that we have already announced at FabCon.”
SAP Datasphere Mirroring — Now Generally Available
Wilson: “At FabCon we announced Mirroring for SAP Datasphere going to GA. For any customer who has a license to use SAP Datasphere, you can now leverage their premium outbound integration to write data into our landing zone. Mirroring will pick up both the snapshot and the delta and bring that into Fabric OneLake — making it all efficient, just like any other mirror database.”
Reza: So if you have SAP Datasphere, Mirroring is now your fastest path to getting that data into Microsoft Fabric — and it is GA.
Copy Job with BAPI Add-On — Private Preview
Wilson: “For customers who do not yet have a Datasphere license and want a way to get SAP data into Fabric — we are building a BAPI add-on that an SAP admin can install directly in their SAP system. Copy Job will then be able to incrementally copy that data using a watermark into Fabric OneLake or any other destination. This is currently in private preview. If anybody is interested, please do reach out — I would love to onboard you to that private preview and get your feedback.”
Reza: So this is for enterprise customers who have SAP but not SAP Datasphere — a different integration path through Copy Job. And if it is in private preview, now is the time to reach out to Wilson directly to join.
Microsoft-SAP Bidirectional Data Sharing — Coming in 2025
Wilson: “We also have the partnership between Microsoft and SAP — announced at Ignite last November — which will provide bidirectional data sharing through SAP BDC inside Fabric, where you can share data products into Fabric and vice versa, with zero copy of the data. This is coming later this year.”
Reza: The whole SAP integration is one of the major improvements in the entire Fabric platform. Pretty much every enterprise customer I work with is using SAP in some way. The fact that we have it covered in Mirroring, in Copy Job, and now with a bidirectional zero-copy partnership — that is removing a lot of friction for enterprise BI. And I would expect a lot more to be written about this once things move to public preview.
How to Connect with Wilson and Give Feedback
Reza: If people have ideas or feedback to share, what is the best way to reach you?
Wilson: “There are a couple of ways. One is via Fabric Ideas — we continuously monitor that forum. It is a great way to share ideas, look at similar use cases from others, and we do use it as part of our feedback mechanism for our backlog. Another way is through Reddit — we have a Microsoft Fabric subreddit and we host AMAs there as well. And of course feel free to reach out on LinkedIn directly. Our only goal is to provide products that you can use in production and make sure you are successful. We are actively listening to all this feedback and we welcome any feedback you have.”
Summary
This was a wide-ranging conversation covering three of the most important areas of Microsoft Fabric data integration.
Microsoft Fabric Mirroring is the simplest, cheapest way to bring database data into Fabric OneLake. The core compute and tiered storage are free with your Fabric capacity. It uses a purpose-built proprietary engine — not pipelines or Dataflows — and handles snapshots, CDC, and near real-time replication automatically. Extended capabilities like Change Delta Feed carry additional charges, but the core replication remains free.
Open Mirroring extends Mirroring to custom sources and certified partners like Informatica, where you control how data gets in but still benefit from the Mirroring engine for processing and storage.
Copy Job provides full flexibility — 170-plus connectors, batch, incremental, and CDC modes, column mapping, SCD Type 2, and destinations inside or outside of Fabric. It shares the same proven engine as Copy Activity but with a simpler experience, and is now available as a pipeline activity for full orchestration.
SAP Integration is moving fast: SAP Datasphere Mirroring is GA, BAPI-based Copy Job integration is in private preview, and bidirectional zero-copy sharing with SAP BDC is coming later in 2025.
And traditional ETL is not dead — Mirroring and Copy Job solve the ingestion problem brilliantly, but the transformation work for analytics still requires data engineering skill.
▶️ Watch the full conversation: Microsoft Fabric Mirroring, Copy Job, and SAP Integration
To connect with Wilson Lee or share feedback with the Microsoft Fabric data integration team, reach out on LinkedIn or through the Microsoft Fabric Ideas forum. The team also hosts regular AMAs on the Microsoft Fabric subreddit.
Have questions about Microsoft Fabric, Power BI, or data integration? At RADACAD, we help organizations through training, consulting, and content. Visit radacad.com to learn more.
Reza is author of more than 14 books on Microsoft Business Intelligence, most of these books are published under Power BI category. Among these are books such as Power BI DAX Simplified, Pro Power BI Architecture, Power BI from Rookie to Rock Star, Power Query books series, Row-Level Security in Power BI and etc.
He is an International Speaker in Microsoft Ignite, Microsoft Business Applications Summit, Data Insight Summit, PASS Summit, SQL Saturday and SQL user groups. And He is a Microsoft Certified Trainer.
Reza’s passion is to help you find the best data solution, he is Data enthusiast.
His articles on different aspects of technologies, especially on MS BI, can be found on his blog: https://radacad.com/blog.