As the year winds down, I find myself reflecting on what a journey it’s been. Honestly, it’s hard to believe we’re already stepping into December! This year, for me, was all about diving deep into Microsoft Fabric—really getting to know how it works and figuring out how to make the most of it.

Let me tell you, it’s been a bumpy road. So many unknowns, so many late nights staring at dashboards wondering, “Is this really the best way to do this?”

A Few Questions That Kept Me Up at Night

• How much can you really do with an F2 capacity?
  I mean, it looks great on paper, but when you’re actually trying to build something scalable and efficient, you quickly discover the limits. I’ve learned to get creative and maximize every ounce of that F2 capacity—like fine-tuning usage patterns and prioritizing workloads. But wow, it’s a balancing act!
• What’s the best implementation template for my clients?
  This one was a constant puzzle. Every client has unique needs, and while Microsoft Fabric offers flexibility, that same flexibility can feel overwhelming. Over time, I’ve found a few go-to templates that strike the right balance between simplicity and functionality. And yes, these took a lot of trial and error!
• How do I keep up with all the new features?
  If you’ve worked with Microsoft Fabric, you know they release updates constantly. (I’m looking at you, monthly feature drops! 😅) Staying on top of them can feel like a full-time job. My trick? Focus on the updates that directly impact your projects first. The rest? I bookmark them for a slow Sunday read.

Lessons Learned

This year has been about more than just technical knowledge. It’s been a lesson in patience, adaptability, and leaning on the community when I felt stuck. If there’s one thing I’ve learned, it’s that you’re never alone in your struggles. Whether it’s a blog, a webinar, or a quick post on LinkedIn, there’s always someone out there ready to share their insights or lend a helping hand.

Looking Ahead

As 2024 approaches, I’m excited to keep building on this foundation. I still have so much to learn, but now I feel more confident about tackling whatever Microsoft Fabric throws my way. (Let’s be real, there’s probably another feature drop coming any second now!)

What about you? How has your year been with Microsoft Fabric—or any other platform you’ve been deep-diving into? I’d love to hear about your challenges, wins, and lessons learned. Let’s keep the conversation going as we head into the new year!

-Martin Rojze

1. Is Microsoft Fabric included with Power BI Premium?

Yes, Microsoft has enabled Power BI Premium capacities to automatically become Fabric capacities. This means that in addition to creating and sharing Power BI content you can also create and share Fabric workloads with no extra cost:

Both options provide access to advanced analytics and data visualization features, Fabric SKUs offer a more flexible and cost-effective solution, especially for organizations that need to scale their usage dynamically. Power BI Premium per capacity provides a more comprehensive solution at a higher cost. The choice between the two would depend on the specific needs and budget of the organization. 

For more detailed information, you can visit the Power BI pricing page here https://powerbi.microsoft.com/en-us/pricing/ 

2. We currently have a Premium capacity. Are we going to start being billed for OneLake storage? How much will it cost?

The storage included in the Premium capacities is for Power BI objects. For Fabric objects, the storage costs will be calculated based on the OneLake licensing:

3. F64 is equivalent to P1 License, Why is the F64 license more expensive than the P1 license?

Fabric capacities offer a Pay-as-you-go basis, which means users can pause and restart the service, scale them up and down dynamically with no usage commitment, optimizing resource utilization and ensuring cost-effectiveness.

Whereas Power BI Premium licenses have a specific amount of dedicated storage and require a monthly/annual contract.

Por example, if you compare the Fabric F64 with the Premium P1 plan, you will see that the annual cost of F64 amounts to $100,000, compared to $60,000 for P1. The reasons for these differences are as follows:

– Pay-as-you-go basis: Users can pause and restart the service, scale them up and down dynamically with no usage commitment , optimizing resource utilization and ensuring cost-effectiveness.

– Contractual Commitment: Fabric’s F64 plan provides flexibility with no mandatory minimum contract duration while in Power BI premium there is a monthly/annual contract.

– Storage: Power BI premium licenses offer a certain amount of dedicated storage for Power BI items while in Fabric it is calculated based on the amount of data stored within OneLake.

4. Can I embed Power BI reports into applications and external websites using Microsoft Fabric?

Yes, Microsoft Fabric capacities (F SKUs) support the embedding of Power BI artifacts into applications or external websites. The pricing for F-SKUs is determined regionally, and billing occurs on a per-second basis, with a minimum billing period of one minute.

Additionally, Fabric capacities offer a lower entry level which can be ideal for Independent Software Vendors (ISVs) and smaller organizations, providing a cost-effective starting point for those looking to integrate Power BI embedding into their applications.

More info: https://powerbi.microsoft.com/en-us/blog/power-bi-embedded-with-microsoft-fabric/

5. How can I purchase Microsoft Fabric capacity?

Microsoft Fabric offers a range of capabilities that you can purchase, these are categorized into Stock Keeping Units (SKUs), each offering a distinct amount of computing power, denoted in Capacity Units (CU).

There are two primary types of Microsoft Fabric SKUs:

1. Azure SKUs (F SKUs): These are the recommended capacities for Microsoft Fabric. They offer flexibility as there’s no long-term commitment. The pricing structure is region-specific, and charges are incurred per second, with a base billing duration of one minute.

Steps to buy an Azure SKU:

1.1. Ensure you have owner or contributor rights on an Azure subscription.

1.2. Navigate to the Microsoft Fabric (preview) service on Azure.

1.3. Opt for “Create Fabric Capacity” and under the ‘Basics’ tab, provide:

• Subscription: The desired Azure subscription (all Azure subscriptions are consolidated for billing).
• Resource Group: Specify the resource group for your capacity.
• Capacity Name: Assign a unique name to your capacity.
• Region: Define the geographical region for your capacity.
• Size: Choose the capacity size, denoted by SKUs and measured in CUs.

• Fabric Capacity Administrator: Designate an admin for this capacity.

1.4. Click on “Next: Tags”. If required, input a name and value for your capacity.

1.5. Finally, select “Review + create” to finalize.

2. Microsoft 365 SKU (P SKUs): Billed monthly or yearly, with a monthly commitment, Basically, they are Power BI SKUs that can support Fabric when enabled in the Power BI subscription.

6. If I am a small company, currently I have 10 users with PPU licenses, would I benefit from Fabric and what would be the cost?

For a small company with 10 users on PPU licenses, Fabric integration can provide enhanced data analytics, streamlined workflows and versatile data management. 

Fabric offers advanced features such as integrated data engineering, real-time analytics and a unified data lake offering simplicity, integrity, and cost savings compared to paying for individual service licenses.

These features can be invaluable to companies that require extensive data operations. Before making a decision, it is crucial to evaluate whether these advanced capabilities align with your business objectives and whether the investment will produce a positive long-term return. This would be an estimated Cost Breakdown:

• PPU Licenses: You will continue paying for the 10 PPU licenses.
• Fabric F License: You will need to pay for the respective F license. The cost will depend on the specific F SKU you choose, and you will only pay for the hours that you use.
• OneLake Storage: The storage costs for Fabric objects will be calculated based on OneLake storage licensing, which is $0.023/GB/month.

Considerations:

– Cost vs. Benefit: Evaluate whether the advanced features provided by Fabric justify the additional costs, especially for a small user base.

– Learning Curve: Ensure that your team is ready to leverage the new capabilities offered by Fabric and that the transition will be smooth.

7. How can an organization estimate the pricing for Microsoft Fabric?

The first recommendation is to use the Microsoft Fabric Capacity Metrics app to monitor CU consumption and make an informed decision on the most suitable license type.

In addition, organizations can use the following list of questions to gather information for an initial cost estimate:

This app is designed to provide information about storage capacity and compute usage, helping you to more accurately assess the type of license you will need.

In addition, this app is especially useful for identifying periods of high demand and ensuring that the system scales appropriately to handle the increased load.

Fabric capacity Metrics

The Capacity Metrics app offers a plethora of metrics to help users understand their data usage. Here’s a breakdown of some of the key metrics.

1. Overview Page:

At the top of each page, the Capacity Name field allows you to select the capacity the app shows results for.

Multi metric ribbon chart

• Provides an hourly view of capacity usage.
• Allows users to drill down to specific days to identify patterns.
• Displays values for Capacity Units (CU) processing time, processing duration, number of operations, and number of users.

Capacity utilization over time

Utilization Tab

• Shows CU usage over time.
• Users can toggle between linear and logarithmic scales.
• Displays CU consumption during background and interactive operations, including billable and non-billable operations.
• Highlights when the capacity is overloaded with dotted lines.

Throttling Tab:

• Displays delay and rejection over time.
• Highlights when operations are delayed or rejected if values exceed 100%.
• Users can toggle between interactive delay, interactive rejection, and background rejection.

Overages Tab:

• Shows the add, burndown, and cumulative carryforward over time.
• Highlights the percent of carryforward added or burned down during a 30-second period.
• Users can select a column to drill through to the timepoint page.

Matrix by item and operation

• Displays metrics for each item on the capacity.
• Users can sort and filter the table based on various parameters.
• Displays values for CU processing time, processing duration, number of users, item size, overloaded minutes, performance delta, and billing type.
• The performance delta indicates the difference in performance compared to seven days ago, helping users assess performance improvements or declines.
• Items can be billable, non-billable, or both.

2. OneLake Page:

The OneLake page of the Microsoft Fabric Capacity Metrics app offers insights into capacity storage. Here’s a breakdown of its features:

Filters:

• Capacity Name: Choose a specific capacity to view its data.
• Date Range: Display results for a particular date range.
• Top: Adjust the number of top workspaces shown based on storage volume.

Cards:

• Workspaces: Total count of workspaces using storage.
• Current storage (GB): Shows the latest storage data.
• Billed storage (GB): Indicates the storage amount billed. Note that billed storage can sometimes be less than the current storage.

Table Visual:

Displays storage details for the top workspaces.  Fields include:

• Workspace name
• Workspace ID
• Current storage
• Billed storage
• Billed storage percentage.

Column Charts:

• Storage (GB) by date: Illustrates average storage over the past 30 days, with an option to view hourly data.
• Cumulative billed storage (GB) by date: Depicts the total billed storage over time.

Export Data:

Users can export detailed storage data for further analysis.

3. Timepoint page

This page is accessed from the ‘Explore’ button located on the Overview page. It offers insights into the compute impact of all activities within a capacity. It helps users identify which operations, both interactive and background, have the most significant impact on Capacity Unit (CU) usage.

Top Row Visuals:

• Timepoint: Shows the specific timepoint being analyzed.
• Heartbeat Line Chart: Displays a 60-minute CU activity window, highlighting the duration of activity peaks and troughs.
• Cards:
  • Interative Operations: Total number of interactive operations that contributed to CU activity during this timepoint.
  • Background Operations: Total number of background operations that influenced the CU’s activity during this particular timepoint.
  • SKU: Current SKU – Capacity.
  • CU (s): CU seconds availble for this capacity within a designated 30-second timepoint window.

Interactive Operations Table:

Lists every interactive operation impacting CU usage during the timepoint. Displays details like: 

• Item name
• Operation type
• Start and end times
• Status: Indicates whether the operation was successful or failed.
• It shows if the operation succeeded or failed.
• User
• Duration: Seconds it took to complete the interactive operation
• Total CU(s): Represents the seconds consumed by the interactive operation. This metric is crucial to determine if the capacity exceeds the total number of CU seconds allowed for the capacity.
• Timepoint CU(s): Seconds assigned to the interactive operation in the current timepoint.
• Throttling (s): Seconds the operation was throttled due to previous timepoint’s capacity overload.
• % Of Base Capacity: (Interactive CU Operations / Base Capacity Allowance) x 100
• Billing type: Specifies if the operation is billable. or not.
• Operation ID.

Background Operations Table:

This table shows background operations contributing to CU usage. It’s similar to the interactive operations table but omits the user column.

Burndown Table:

Presents the add, burndown, and cumulative percentages by experiences for the last 30 seconds.

Overages Visual:

Users can adjust the visual scale to show data for 10 minutes, 60 minutes, or 24 hours. It focuses on billable operations and provides insights into carryforward percentages.

Crucial Takeaways:

• Overloading the capacity can trigger throttling. When autoscale is on, an additional CU is provisioned for the next 24 hours upon reaching capacity. Without autoscale, throttling affects all ensuing interactive operations.
• CU activity can be influenced by background operations from the preceding 24 hours, attributed to performance smoothing.

How to Install

Installing the Microsoft Fabric Capacity Metrics app is straightforward. Users can access the app through the app from AppSource:

Go to AppSource > Power BI Premium Capacity Utilization and Metrics and select Get it now.

Then, go to Power BI Service:
Select Apps → Get apps → Search for Microsoft Fabric → Select the Microsoft Fabric Capacity Metrics app →  Select Get it now.

For running the app for the first time you need to include:

– CapacityID: To get the capacity ID go to Settings → Admin Portal → Trial or Fabric Capacity → Click on Actions:

– UTC offset:

Learn more about monitoring capacity usage: https://learn.microsoft.com/en-us/fabric/enterprise/metrics-app-overview-page

In conclusion, determining the precise capacity size necessary for your needs can be a complex task, as there is no exact formula available for this purpose. However, starting with the Fabric trial, which provides a performance equivalent to 64 CUs, or opting for a pay-as-you-go Fabric capacity size for initial testing, can be a wise approach.

Utilizing the Microsoft Fabric Capacity Metrics app is highly recommended, as it enables users to closely monitor their CU consumption, providing valuable insights into storage capacity and compute usage. This, in turn, aids in making an informed decision regarding the type of license required for optimal performance.

Microsoft Fabric provides a diverse range of licensing options, categorized primarily into capacity-based and individual user licenses.

1. Capacity Licenses -F SKUs (Dedicated capacity)

– Known as Premium licenses in Power BI.

– They provide the infrastructure for Microsoft Fabric. Without a capacity license, users can’t work in Fabric.  

– Capacity licenses are divided into Stock Keeping Units (SKUs) that deliver a set of Fabric resource levels. These levels are measured in Capacity Units (CU) which is a new concept used to measure the compute power available for each SKU.

– Each license offers double the computational power of its predecessor, for instance, F4 has twice the power of F2 and half the power of F8. 

– This license introduces a “pay as you go” model, it means:

      ✅ Users pay for their actual usage, there are no minimum terms, the SKUs are available on an hourly or monthly basis.

      ✅ Unlike Premium Capacity, Fabric Capacity can be paused, offering cost flexibility.

      ✅ Charges are incurred only when the capacity is actively used.

      ✅ Prices start at a competitive point of $8 per day, making it accessible for organizations.

Microsoft Fabric Pricing

Notes: 

• Billing is per second with a one minute minimum.
• SKUs that are smaller than F64 require a Pro or Premium Per User (PPU) license to consume Power BI content.

2. Individual licenses

– Free: This license allows users with access to Fabric Capacity to create and share content within Microsoft Fabric. It covers functionality such as pipelines, data warehousing, notebook usage and capacity management. However, sharing Power BI content requires a Pro license.

– Pro: Essential for sharing Power BI content with other users. Every organization intending to work with Power BI requires at least one Pro license. Thus, if you’re purchasing a Microsoft Fabric license, ensure you include at least one Pro license.

– Premium Per user (PPU): Operating on a shared capacity, this license grants limited access to Microsoft Fabric, and will continue to be suitable for organizations that want to leverage the premium features of Power BI without extending to any Fabric content. PPU can be more cost effective when Power BI Premium features are needed for less than 250 users.

In conclusion, the individual licenses work in Microsoft fabric only if they are combined with a Capacity License. Starting from the F64, you can share Power BI content with Free Licenses. For lower licenses, You will need Pro licenses.

Additional Licensing Points

– Power BI Embedded: also known as A SKUs, remains available for embedding purposes. However, it is important to note that Power BI Embedded does not provide access to the new Fabric workloads.

 – Microsoft Fabric trial: Include access to the Fabric product experiences and the resources to create and host Fabric items, the trial lasts for a period of 60 days, but may be extended by Microsoft discretion. It is crucial that users transition to a paid Fabric capacity after the trial in order to retain data and continue usage. Otherwise, non-Power BI Fabric items will be removed in accordance with the stipulated retention policy.

Variables to Consider for Cost Estimation

1. CAPACITY UNITS (CU): It’s the primary billing unit used in Microsoft Fabric, it combines CPU, Memory, Disk IO, Network bandwidth, and more. Each operation in Fabric consumes CUs, higher CUs equals higher performance.

2. NUMBER OF USERS THAT WILL SHARE AND CONSUME POWER BI CONTENTS: This is one of the variables that most impacts the price to be paid. The price of each license is $10 per user/month and $ 20 per user for Premium per user licenses (PPU).

Typically if you have more than 500 users (250 users in PPU license) consuming and sharing Power BI content, it is recommended to purchase an F64 license or higher.

3. INTENSIVE WORKLOADS THAT COULD OVERLOAD THE CAPACITY: If you have important capacity peaks you should consider scale your capacity, it means at any time, you can temporarily increase your capacity and reduce it again when you’re done. And since F-SKUs are billed on a usage basis and managed as an Azure resource, you only pay for the time you consume.

At the time of writing this article, auto-scaling is not currently available, but it is planned to be integrated in the future. So you will have to apply manual scaling in periods with intensive workloads.

Discover how to scale Fabric capacity: https://learn.microsoft.com/en-us/fabric/enterprise/scale-capacity

4. OPERATIONAL HOURS: Microsoft Fabric allows you to pause and resume the capacity when it is not operational, to save costs for your organization. Later, when you want to resume work on your capacity, you can reactivate it. 

So if your company isn’t doing anything with data or analytics during the weekend, you can just pause the capacity and save approximately 25% of the monthly license cost.

Learn more about how to pause and resume Fabric capacity: https://learn.microsoft.com/en-us/fabric/enterprise/pause-resume

This article explains how to automatically pause and resume Fabric capacity: https://github.com/nocsi-zz/fabric-capacity-management/

5. ONELAKE STORAGE: It is the data lake integrated into Microsoft Fabric that serves as a centralized repository for all of the organizational data and is billed on a pay-as-you-go rate. Microsoft Fabric storage is OneLake, this is the only place where all data is stored.

This cost is not included in the capacity licensing, so you will need to pay for it separately. Note that OneLake storage pricing follows the same pricing as Azure Data Lake Storage gen2:

Benefits of Fabric storage:

• Efficiency: Streamlined purchasing with auto-provisioned storage for all tasks.
• Cost-Effective: Unified data copies eliminate redundant data transfers and reduce duplication costs.
• Integration: Seamlessly connect third-party storage systems to Microsoft Fabric analytics.
• Accessibility: Open data formats facilitate access to multiple analytics engines, minimizing configuration resources.
• Security: Centralized access to all data and governance tools ensures consistent security and simplifies maintenance.

6. BANDWIDTH: Bandwidth refers to the data moving in and out of Azure data centers and between Azure data centers. While it’s a crucial component, its cost impact is often minimal compared to Capacity Units (CU) consumption and Power BI licenses:

• Data Transfer In: Free for data moving into Azure data centers.
• Within an Availability Zone: Free
  Availability Zone is an isolated location within an Azure region, equipped with its own power, cooling, and networking to protect applications and data from data center failures. It ensures high availability and resilience by allowing resources to be distributed across multiple independent physical locations within a specific region.
• Between Availability Zones: The price is around $0.01 per GB. So, even if a company transfers a substantial 500GB of data between zones in a month, the cost would be just $5.$0.01 per GB
• Inter-Region Transfer: Ranges from $0.02 to $0.16 per GB
• Inter-Continental Transfer: Ranges from $0.05 to $0.16 per GB. For a company transferring 1TB of data inter-continentally, the cost would range from $50 to $160.
• Internet Egress: First 100GB/month free; subsequent costs vary by region.

Learn more at Bandwidth pricing:  https://azure.microsoft.com/en-us/pricing/details/bandwidth/

7. SMOOTHING: This mechanism distributes CU consumption over time, allowing users to run more workloads in the same capacity without restrictions and helping them avoid being penalized with additional costs. 

It means that you only pay for the average rather than the peak performance, averaging capacity usage over short periods of time for interactive operations and over a 24-hour period for background operations.

Learn more about this feature: https://learn.microsoft.com/en-us/power-bi/enterprise/service-premium-smoothing

Power BI Licensing vs Fabric

You have probably noticed that the prices of the Fabric capacities are higher than those of the equivalent Power BI Premium capacities. 

A P1 Power BI Premium capacity provides the same power as an F64 Fabric capacity, a P2 Power BI Premium capacity provides the power of an F128 Fabric capacity, and so on. However, if you compare the Fabric F64 with the Premium P1 plan, you will see that the annual cost of F64 amounts to $100,000, compared to $60,000 for P1. The reasons for these differences are as follows:

✅ Pay-as-you-go basis: Users can pause and restart the service, scale them up and down dynamically with no usage commitment , optimizing resource utilization and ensuring cost-effectiveness.

✅ Contractual Commitment: Fabric’s F64 plan provides flexibility with no mandatory minimum contract duration while in Power BI premium there is a monthly/annual contract.

✅ Service Scope: The F64 plan grants comprehensive access to all Fabric workloads. In contrast, the P1 plan is tailored specifically for Power BI items.

✅ Storage: Power BI premium licenses offer a certain amount of dedicated storage for Power BI items while in Fabric it is calculated based on the amount of data stored within OneLake.

In a few months, Microsoft will introduce an Azure Reservation commitment model for Fabric Capacities  – RI SKUs, which will make their price comparable to the equivalent Power BI Premium capacities.

Example Cost Estimation – Choosing the Right Scenario

Understanding the costs associated with Microsoft Fabric is crucial for effective budgeting. Below, we will show different hypothetical scenarios for helping you to understand how you can estimate the licensing more fit for your organization.

However it is very important monitoring your Microsoft Fabric Capacity Metrics to understand better your spending across workloads and optimize your capacity usage. 

Scenario 1) Small Business with moderate workloads

• Number of users that will consume Power BI content: 25 users
• Operational hours: 8 hours/day (pauses capacity during off-hours and weekends)
• Workload: Moderate 
• Storage: 100GB on OneLake

Recommended License: F2 SKU 

Cost Calculations:

Scenario 2) Small business with 50 users that wants to use premium features in Power BI

• Number of users: 50 users
• Operational hours: 8 hours/day (pauses capacity during off-hours and weekends)
• Workload: Moderate 
• Storage: 100GB on OneLake

Recommended License: F2 SKU 

Cost Calculations:

Scenario 3) Medium Organization with 250 users and high-demand periods

• Number of users: 250
• Operational hours: 10 hours/day, 6 days a week (Monday to Saturday)
• Workload: They have peak intensive data processing, especially during specific projects or when handling large datasets (20 hours approx.)
• Storage: 1TB on OneLake

Recommended License: Starts with F2 SKU, but scales up to F8 during high-demand periods.

Cost Calculations:

Scenario 4) Large Corporation with Month-End Reporting Peaks

• Number of users: 600
• Operational hours: 10 hours/day, 7 days a week
• Workload: Regular operations with an F64 SKU, but scales up to F128 during month-end reporting (30 hours approx.)
• Storage: 1TB on OneLake
• Bandwidth: Inter-region transfer within North America of 1TB.

Recommended License: Starts with F64 SKU, but scales up to F128 during high-demand periods.

Cost Calculations:

Scenario 5) Global Financial Institution with Real-Time Analytics

• Number of users: 1,000
• Operational hours: 24/7 (Always on)
• Workload: Highly intensive due to real-time financial analytics. They start with the F128 SKU but scale up during peak market hours (50 hours approx.)
• Storage: 6TB on OneLake
• Bandwidth: 10TB inter-continental data transfer from North America to other continents.

Recommended License: Starts with F128 SKU, but scales up during high-demand periods.

Cost Calculations:

Navigating through Microsoft Fabric’s licensing and cost estimation can be a challenge, but understanding these key variables and scenarios can help you make informed decisions. Whether you’re a small business or a global corporation, Microsoft Fabric offers flexible options that cater to your specific needs.

As you explore the world of Fabric, remember to monitor your capacity metrics using the Microsoft Fabric Capacity Metrics app. In the next article of this series, we will go deeper into this app, which will be your great ally to help you optimize your consumption and manage your capacity effectively.

As organizations consider implementing Microsoft Fabric, understanding its licensing and cost estimation becomes crucial.

Based on our experience assisting our clients in navigating the intricacies of Microsoft Fabric’s licensing and pricing, I’ve written a series of articles on the subject, with the goal of making it easier to understand. We’ll delve into the factors to consider when choosing the most suitable license and provide examples exploring various licensing scenarios and their associated costs.

The article series is structured as follows:

1. Key Components of a Microsoft Fabric Subscription

2. Microsoft Fabric License Types

3. Understanding Microsoft Fabric Licensing and Cost Estimation

    ✔️ Variables to Consider for Cost Estimation

    ✔️ Power BI Licensing vs Fabric

    ✔️ Example cost estimation where we will explore different licensing scenarios

   ✔️ Checklist of questions to gather the information needed to estimate Microsoft Fabric pricing

4. Microsoft Fabric Capacity Metrics app

5. How to Acquire Microsoft Fabric Capacity

Part 1. Key Components of a Microsoft Fabric Subscription

A Microsoft Fabric subscription is composed of three fundamental elements: tenants, capacities, and workspaces.

1. Tenant

A Microsoft Fabric subscription starts with a “tenant.” Think of a “tenant” as a powerful computer server in a large data center. This server is dedicated to a specific organization, ensuring that all the data and applications within it are isolated and secure from other organizations.

A tenant usually represents an organization or domain and serves as the highest-level container that is linked to a unique internet address, called a Domain Name System (DNS). It ensures that the organization’s data is safe, organized, and easily accessible. Larger organizations may have multiple tenants for various departments or regions.

2. Capacity

A Microsoft Fabric capacity resides on a tenant and is a distinct pool of resources allocated to Microsoft Fabric. The size of the capacity determines the amount of computation power available to the organization.

The “capacity” can be thought of as the server’s hardware resources, like its RAM (memory) and CPU (processing power). The greater its capacity, the more computing power it will have for its tasks.

Capacity types:

    – Dedicated Capacity: A reserved set of computational resources allocated exclusively for a specific organization or task within Microsoft Fabric. This means that the memory, processing power, and other resources in this capacity are the exclusive use of that specific organization or task, ensuring consistent performance and preventing potential slowdowns from other users or tasks.

    – Shared Capacity: A pool of computational resources which are distributed among all the users accessing it. While it’s cost-effective, there might be fluctuations in performance, especially during peak usage times, as multiple users tap into the same pool of resources.

3. Workspace 

Workspaces are specialized containers for Microsoft Fabric items located within capacities. As in a server (tenant) and its resources (capacity), there are several applications and programs running. Each of these can be compared to a “workspace” in Microsoft Fabric. These workspaces use the server’s resources (capacity) to run, store data and perform tasks.

By default, workspaces are placed in  shared capacity.  But as the organization grows and has additional capacities available, these workspaces can be strategically placed in any of the existing shared or dedicated capabilities to better suit their organizational needs.

In essence, the server (tenant) provides the environment, its hardware resources (capacity) power the operations, and the applications (workspaces) are where the actual tasks and storage occur.

Solution: Relate every dimension with their respective fact tables.

• Identify the shared dimensions.
• Relate them with every fact.

Sometimes this model is called constellation schema, where each fact table individually is a perfect star schema design because each one is related with a dimension.

Advantages

• Star schemas are easy to read, understand and use.
• Best performance.
• You can filter both fact tables using the shared dimensions.

Disadvantages:

• As the fact tables are not related to each other if you try to add columns from both tables in the same visual that do not have a shared dimension you will get the following error:

Bad Practices:

• Duplicate dimensions in the model:
  • You will increase the size of your model which could impact the refresh time.
  • You can’t use the same dimension to filter the data of both tables.
• Consolidate the fact tables: Join fact tables through a union (append in Power Query) will create a huge table that can take a long time to refresh, in addition to a lot of blank values in the unmatched columns.

 Scenario 2: Multiple fact tables related to each other with different granularities

Remember that the granularity represents the level of detail in each row of your fact table. In the header/detail models you must be careful to use the correct granularity in your calculations and try to maintain a good performance considering that they are not a perfect star schema.

An example of header/detail models is when you have an order (header) and order details. The order table has an order number, date, a customer, and much more useful information at the order level.  The detail table contains the product, the price, the quantity, and detailed information related to the individual order line.

Solution 1: Flatten the header table into the detail table: With this solution you will get a denormalized model, where you locate the data from the header table to the detail table increasing the granularity.

Advantages:

• You will get a perfect star schema that offers all its advantages.
• The values will get the correct granularity.
• The model will be easier to use because it can be summed and sliced by any dimension.

Disadvantages:

• Increase the data model size.
• Categorical values repeated in the denormalized model.

Solution 2: Relate the header and detail table

Be careful with this solution because you could get incorrect results due to the granularity differences between both tables. You can use it in the following scenario:

• You need to use columns from the different fact tables in a same visual table.
• The header detail table had a big number of column and flattening it into the detail table would create a huge table.
• You need to make few calculations over the header.
• The reports just needed to be sliced using common dimensions.

Considering these points, apply the following solution:

• Identify the shared dimensions.
• Relate them with every fact to get a star schema model for every fact.
• Relate the header and detail table through an inactive relationship:

• Use a Many to one Cardinality and a “Both” cross filter direction:

• Finally, to summarize or bring the columns from the header table, use the USERELATIONSHIP DAX formula:

Contract Amount =
CALCULATE (
SUM ( ‘Fact Header'[Contract Amount] ),
USERELATIONSHIP ( ‘Fact Detail'[WIP PK], ‘Fact Header'[PK] )
)

In this solution, linking the header and the detail table breaks the rules of the star schema because both tables act as a dimension and a fact table at the same time.

It’s a dimension when you need to slice the detail and it’s a fact table when you summarize values at the header granularity.

So, you will get a snowflake when the header table acts as dimension, although you should avoid it, sometimes as in this specific scenario could be a good solution to representing the data.

Advantages:

• Avoid increase the size the data model having fact tables separated.
• You can use the star schemas to create reports that don’t need to include in a same visual, columns from different fact tables

Disadvantages:

• This solution only works if you slice the data using the common dimensions, because otherwise you will get incorrect results due to the granularity differences between both tables.
• Possible performance degradation.

What are the DMVs

The DMVs queries are an interface to schema rowsets based on SQL and use a SELECT statement and the $System schema with an XML/A schema rowset, however, they do not support the full syntax of a SELECT statement as: JOIN, GROUP BY, LIKE, CAST, and CONVERT.

Apply to: SQL Server Analysis Services, Azure Analysis Services, Power BI Desktop and Power BI Premium

Example of a DMV query:

SELECT * FROM $System.DISCOVER_CALC_DEPENDENCY

WHERE OBJECT_TYPE = ‘MEASURE’

ORDER BY TABLE ASC

For more information visit the official documentation: https://docs.microsoft.com/en-us/analysis-services/instances/use-dynamic-management-views-dmvs-to-monitor-analysis-services?view=asallproducts-allversions

What are they for?

• The DMVs provide information about the active sessions and connections, objects that consume the most CPU or memory at a specific point in time, such as:

• To document your Power BI or SSAS data model, the main queries DMVs available for model documentation are:

• • TMSCHEMA_TABLES: List the Column objects in each table.
  • TMSCHEMA_COLUMNS: List the Columns objects in each table.
  • TMSCHEMA_MEASURES: List the Measure objects in each table.
  • TMSCHEMA_RELATIONSHIPS: List the Relationship objects in the model.
  • TMSCHEMA_ROLES: Shows information about the Role objects in the model.
  • TMSCHEMA_PARTITIONS: Provides information about the Partition objects in each table.

In the past I used to create my own documenters using the DMVs, but I recently discovered the excellent and very complete tool “Model Documenter” created by Marc Lelijveld (MVP).

This tool generates the documentation about your data model in an easy and quick way, you can get more information and download the tool in the page: https://data-marc.com/model-documenter/

DISCOVER_CALC_DEPENDENCY

With this DMV you can obtain the list of the dependencies among the objects in your model and extract the DAX expressions. This allows you to have more control over your data models as you can measure the impact of modify or remove any object.

I created the tool “Tabular model cleaner” using this DMV, which helps you to obtain all the dependences of the data model in an efficient and automated way and the usability of the model objects in the reports connected to the Power BI or SSAS Data model.

For more information visit: Tabular model cleaner

How to run the DMVs

1. DAX Studio: Open the Power BI Dataset that you want to document. Open DAX Studio and choose the “PBI / SSDT Model” data source option, then click on Connect:

If you have Power BI Premium you can connect to the dataset, to do this select the workspace, go to “Workspace settings” and copy the Workspace Connection and paste it into the “Tabular Server” option in the DAX Studio.

Go to DMV pane and click on the query you want consult:

2. From Power BI Desktop: You can run DMVs queries from DAX Studio but that will return you a table with the data, you should connect to the DMVs using Power BI, to view and analyze the information more easily.

– Open Power BI Desktop.

– Choose the “SQL Server Analysis Services database” source.

– Include the Server and Database.

Consider if you are connected to a local data model, the database name will change every time you open the Power BI Desktop file, so you will have to edit the connection parameter when you need to refresh the data.

For datasets with incremental refresh policy applied, you must avoid publishing and replacing them with a version of a PBIX file from Power BI Desktop, because you will have to refresh all the historical data which could take hours and result in the system’s downtime for users.

Instead, it’s better to perform a metadata-only deployment using the ALM Toolkit tool which is an open-source schema compare tool for Power BI datasets. You can download it at: http://alm-toolkit.com/

This allows the deployment of new objects without losing historical data. For example, if you have added a few measures, you can deploy only the new measures without needing to refresh the data, saving a lot of time.

To perform a metadata only deployment:

1. Select the running Power BI Desktop instance as the source, and the existing dataset in the service as the target:

Both Source and Target you have the following options to connect to your datasets:

– Dataset: You have to indicate the workspace connection URL, to get it in workspace go to Settings > Premium > Workspace Connection, select Copy.

• Power BI Desktop: The tool scans open Power BI files and allows you to select the file you want to use.

• File: You can select a file from your computer.

2. To get only the objects with differences, go to Home > Select Actions > Hide Skip Objects with Same Definition:

In this case the difference is that the “Test Measure” is missed in the target:

You have the option to create it or select skip to don’t create it.

3. Finally click on “Validate Selection” to ensure the integrity of the target model and then click on “Update”:

4. If the deployment was successful, the following screen will be displayed:

– Use filters in your visualizations to reduce the amount of data that display: The more data a visual must display, the slower it will load, so apply filters in the Filters pane, or use Top N, to reduce the number of items displayed.

So, if your report is connected to a large data model and you do not apply any filtering, all the data will be loaded into memory and decompressed on each update, creating a huge memory demand.

– Only include the necessary number of visuals in your report: This will decrease the number of calculations that Power BI is performing when rendering your report. For example, you should use a multi-row card instead of using multiple single cards.

– Limit the use of slicers: Because they are visualizations, the underlying data will be updated every time the dashboard is refreshed, which could affect the performance.

– Use Microsoft-certified custom visuals: These have the Microsoft’s guarantee that these visual elements perform at their best, in addition have more options than non-certified ones. 

– Avoid unnecessary interactions between visuals: Power BI automatically creates interactions between all the visuals on a page, which require high loading capacity that could slow down the loading of reports.

Therefore, make sure to eliminate the unnecessary interactions that Power BI creates by default.

– Use Performance Analyzer to Analyze the performance of your report elements: such as visuals and DAX formulas when users interact with them, and which consume the most resources.

✅ Excel Checklist

Download the Excel checklist at the following link: https://github.com/NuricBI/Checklist

Conclusions

In this article we reviewed how to improve and analyze the performance of your Power BI solutions in the different development layers:

✔ Data source.

✔ Data model.

✔ DAX.

✔ Data visualization.

You must be careful in each of them, because in whatever could be the cause that affects the final performance of the report.

Keep handy the excel file with the summarized list of best practices and recommended tools to analyze the performance of each phase of your solution.