Microsoft Fabric uses OneLake as the single logical layer for storage management across the Fabric. OneLake offers Domains and workspaces to work with Fabric Items, and amid all those structures, you may require a table or file from one Domain to be used in another. OneLake’s Shortcut feature will give you this option without duplicating the data. In this article, I will explain what Shortcuts are and their importance in building an analytics solution.
Video
Microsoft Fabric
Microsoft Fabric is an end-to-end Data Analytics software-as-a-service offering from Microsoft. Microsoft Fabric combined some products and services to cover an end-to-end and easy-to-use platform for data analytics. Here are the components (also called workloads) of Microsoft Fabric.
To learn more about Microsoft Fabric and enable it in your organization, I recommend reading the articles below;
OneLake and Domains
OneLake is the logical layer for the data foundation and storage used in the Microsoft Fabric. Underneath OneLake, the data is stored in ADLS Gen2 (Azure Data Lake Storage Gen2). OneLake is one per tenant; it can be spread across regions and split the data using Domains and Workspaces. Here is an example of how Domains and Workspaces can have the Fabric items structured.
To learn more about Domains, read my article here;
Shortcut: Linking Domains with OneCopy
When the data is spread out in multiple Domains, the data item (let’s say, table or file) from one Domain might be needed in another. Copying the data from one Domain to another is possible using Data Factory Dataflow Gen2, Data Pipeline‘s Copy Activity, etc. However, data duplicates are the last thing you want in an analytics solution. Data duplication comes with the cost of maintenance and the risk of the trustworthiness of the data. Shortcuts are links to the data entities that give you such functionality without creating a duplicate of the data.
A Shortcut in Microsoft Fabric OneLake is a link to another storage location. This location can be Internal (inside Fabric) or External (outside, such as ADLS Gen2 or Amazon S3). The Shortcut items can be created in Fabric items such as Lakehouse and KQL databases. The Shortcut item will be accessible like a standard data item from there. For example, a Shortcut table inside a Lakehouse will be like a regular table in that Lakehouse and can be used for querying the data using typical SQL commands.
The screenshot below shows the Customer Shortcut created inside the LHwithShortcut Lakehouse, which points to the Customer table inside the TestLH Lakehouse.
Shortcuts appear with a link icon in the Lakehouse Explorer, as shown in the screenshot above.
Consider Shortcuts like Windows Shortcuts; They create a link to a file or folder somewhere else.
The two tables above can be used in an SQL statement like the one below. As you can see, a Shortcut table will be referred to in the SQL statement using the name of the Shortcut.
Path for the Shortcut
There are two paths for every shortcut, and they are defined below;
- Target Path: This is the path where the shortcut points to.
- Shortcut Path: This is the path where the shortcut resides.
For example, if we create the Shortcut inside Lakehouse1, pointing to the Customer table in Lakehouse 2, then the Target path is the Customer table in Lakehouse 2, and the Shortcut path is Lakehouse1.
Where can you create the Shortcut?
Where you can create the Shortcut, or where the Shortcut path is, can be (at the time of writing this article);
- Lakehouse
- KQL database.
Where can the Shortcut point to?
The Target path, or where the Shortcut points to, can be (at the time of writing this article);
- OneLake locations (or you can say Fabric locations)
- ADLS Gen2
- Amazon S3 storage accounts
- Dataverse
Shortcut Type depends on the source; when the source is OneLake (or Fabric), it is an internal Shortcut. Otherwise, it is external.
Getting Started with Shortcuts
Let’s go through creating a Shortcut together. In a sample Lakehouse that you have opened through Lakehouse Explorer, Under the Tables, click on New Shortcut.
Choose the Shortcut type. For this example, I create an internal source from my Fabric objects.
Choose the location of the source.
The shortcut can point to files or tables. Because the Shortcut points to the folder, including the parquet files.
Once the Shortcut is created, it will have a link icon so that you can distinguish it from other tables or files.
The Shortcut can be used like an internal object in the Lakehouse. If it is a table, it can be easily used in a SQL statement like the one below;
The link to the target path in the Shortcut can also be visible in the Lineage View of the Fabric item;
You can also use Notebook or other objects to access the Shortcut items.
Permissions and Roles
To create a Shortcut, the user must have edit access on the workspace, which the Contributor, Member, or Admin attain. The same access can be used to read and write to the target path of the Shortcut, but the user then needs to have the required privilege on the target workspace (for example, for write, it needs to be Contributor, Member, or Admin in the target workspace).
Summary
Shortcuts are links between Fabric items without duplicating or moving the data. The data will stay in one place and can be read or written to from other places using Shortcuts. Shortcuts can link data entities between Domains and Workspaces. For example, a Shortcut can be created in a Lakehouse that points to a table in a Warehouse in another Domain. The Shortcut table can be used like an internal table in querying the data. Shortcuts can also be created for objects outside of Fabric, such as ADLS Gen2 or Amazon S3. Data duplication is a big challenge in many organizations that should be avoided to reduce maintenance costs and overhead. Shortcuts are a big step towards this goal.
Reza is author of more than 14 books on Microsoft Business Intelligence, most of these books are published under Power BI category. Among these are books such as Power BI DAX Simplified, Pro Power BI Architecture, Power BI from Rookie to Rock Star, Power Query books series, Row-Level Security in Power BI and etc.
He is an International Speaker in Microsoft Ignite, Microsoft Business Applications Summit, Data Insight Summit, PASS Summit, SQL Saturday and SQL user groups. And He is a Microsoft Certified Trainer.
Reza’s passion is to help you find the best data solution, he is Data enthusiast.
His articles on different aspects of technologies, especially on MS BI, can be found on his blog: https://radacad.com/blog.