As the year winds down, I find myself reflecting on what a journey it’s been. Honestly, it’s hard to believe we’re already stepping into December! This year, for me, was all about diving deep into Microsoft Fabric—really getting to know how it works and figuring out how to make the most of it.

Let me tell you, it’s been a bumpy road. So many unknowns, so many late nights staring at dashboards wondering, “Is this really the best way to do this?”

A Few Questions That Kept Me Up at Night

• How much can you really do with an F2 capacity?
  I mean, it looks great on paper, but when you’re actually trying to build something scalable and efficient, you quickly discover the limits. I’ve learned to get creative and maximize every ounce of that F2 capacity—like fine-tuning usage patterns and prioritizing workloads. But wow, it’s a balancing act!
• What’s the best implementation template for my clients?
  This one was a constant puzzle. Every client has unique needs, and while Microsoft Fabric offers flexibility, that same flexibility can feel overwhelming. Over time, I’ve found a few go-to templates that strike the right balance between simplicity and functionality. And yes, these took a lot of trial and error!
• How do I keep up with all the new features?
  If you’ve worked with Microsoft Fabric, you know they release updates constantly. (I’m looking at you, monthly feature drops! 😅) Staying on top of them can feel like a full-time job. My trick? Focus on the updates that directly impact your projects first. The rest? I bookmark them for a slow Sunday read.

Lessons Learned

This year has been about more than just technical knowledge. It’s been a lesson in patience, adaptability, and leaning on the community when I felt stuck. If there’s one thing I’ve learned, it’s that you’re never alone in your struggles. Whether it’s a blog, a webinar, or a quick post on LinkedIn, there’s always someone out there ready to share their insights or lend a helping hand.

Looking Ahead

As 2024 approaches, I’m excited to keep building on this foundation. I still have so much to learn, but now I feel more confident about tackling whatever Microsoft Fabric throws my way. (Let’s be real, there’s probably another feature drop coming any second now!)

What about you? How has your year been with Microsoft Fabric—or any other platform you’ve been deep-diving into? I’d love to hear about your challenges, wins, and lessons learned. Let’s keep the conversation going as we head into the new year!

-Martin Rojze

1. Is Microsoft Fabric included with Power BI Premium?

Yes, Microsoft has enabled Power BI Premium capacities to automatically become Fabric capacities. This means that in addition to creating and sharing Power BI content you can also create and share Fabric workloads with no extra cost:

Both options provide access to advanced analytics and data visualization features, Fabric SKUs offer a more flexible and cost-effective solution, especially for organizations that need to scale their usage dynamically. Power BI Premium per capacity provides a more comprehensive solution at a higher cost. The choice between the two would depend on the specific needs and budget of the organization. 

For more detailed information, you can visit the Power BI pricing page here https://powerbi.microsoft.com/en-us/pricing/ 

2. We currently have a Premium capacity. Are we going to start being billed for OneLake storage? How much will it cost?

The storage included in the Premium capacities is for Power BI objects. For Fabric objects, the storage costs will be calculated based on the OneLake licensing:

3. F64 is equivalent to P1 License, Why is the F64 license more expensive than the P1 license?

Fabric capacities offer a Pay-as-you-go basis, which means users can pause and restart the service, scale them up and down dynamically with no usage commitment, optimizing resource utilization and ensuring cost-effectiveness.

Whereas Power BI Premium licenses have a specific amount of dedicated storage and require a monthly/annual contract.

Por example, if you compare the Fabric F64 with the Premium P1 plan, you will see that the annual cost of F64 amounts to $100,000, compared to $60,000 for P1. The reasons for these differences are as follows:

– Pay-as-you-go basis: Users can pause and restart the service, scale them up and down dynamically with no usage commitment , optimizing resource utilization and ensuring cost-effectiveness.

– Contractual Commitment: Fabric’s F64 plan provides flexibility with no mandatory minimum contract duration while in Power BI premium there is a monthly/annual contract.

– Storage: Power BI premium licenses offer a certain amount of dedicated storage for Power BI items while in Fabric it is calculated based on the amount of data stored within OneLake.

4. Can I embed Power BI reports into applications and external websites using Microsoft Fabric?

Yes, Microsoft Fabric capacities (F SKUs) support the embedding of Power BI artifacts into applications or external websites. The pricing for F-SKUs is determined regionally, and billing occurs on a per-second basis, with a minimum billing period of one minute.

Additionally, Fabric capacities offer a lower entry level which can be ideal for Independent Software Vendors (ISVs) and smaller organizations, providing a cost-effective starting point for those looking to integrate Power BI embedding into their applications.

More info: https://powerbi.microsoft.com/en-us/blog/power-bi-embedded-with-microsoft-fabric/

5. How can I purchase Microsoft Fabric capacity?

Microsoft Fabric offers a range of capabilities that you can purchase, these are categorized into Stock Keeping Units (SKUs), each offering a distinct amount of computing power, denoted in Capacity Units (CU).

There are two primary types of Microsoft Fabric SKUs:

1. Azure SKUs (F SKUs): These are the recommended capacities for Microsoft Fabric. They offer flexibility as there’s no long-term commitment. The pricing structure is region-specific, and charges are incurred per second, with a base billing duration of one minute.

Steps to buy an Azure SKU:

1.1. Ensure you have owner or contributor rights on an Azure subscription.

1.2. Navigate to the Microsoft Fabric (preview) service on Azure.

1.3. Opt for “Create Fabric Capacity” and under the ‘Basics’ tab, provide:

• Subscription: The desired Azure subscription (all Azure subscriptions are consolidated for billing).
• Resource Group: Specify the resource group for your capacity.
• Capacity Name: Assign a unique name to your capacity.
• Region: Define the geographical region for your capacity.
• Size: Choose the capacity size, denoted by SKUs and measured in CUs.

• Fabric Capacity Administrator: Designate an admin for this capacity.

1.4. Click on “Next: Tags”. If required, input a name and value for your capacity.

1.5. Finally, select “Review + create” to finalize.

2. Microsoft 365 SKU (P SKUs): Billed monthly or yearly, with a monthly commitment, Basically, they are Power BI SKUs that can support Fabric when enabled in the Power BI subscription.

6. If I am a small company, currently I have 10 users with PPU licenses, would I benefit from Fabric and what would be the cost?

For a small company with 10 users on PPU licenses, Fabric integration can provide enhanced data analytics, streamlined workflows and versatile data management. 

Fabric offers advanced features such as integrated data engineering, real-time analytics and a unified data lake offering simplicity, integrity, and cost savings compared to paying for individual service licenses.

These features can be invaluable to companies that require extensive data operations. Before making a decision, it is crucial to evaluate whether these advanced capabilities align with your business objectives and whether the investment will produce a positive long-term return. This would be an estimated Cost Breakdown:

• PPU Licenses: You will continue paying for the 10 PPU licenses.
• Fabric F License: You will need to pay for the respective F license. The cost will depend on the specific F SKU you choose, and you will only pay for the hours that you use.
• OneLake Storage: The storage costs for Fabric objects will be calculated based on OneLake storage licensing, which is $0.023/GB/month.

Considerations:

– Cost vs. Benefit: Evaluate whether the advanced features provided by Fabric justify the additional costs, especially for a small user base.

– Learning Curve: Ensure that your team is ready to leverage the new capabilities offered by Fabric and that the transition will be smooth.

7. How can an organization estimate the pricing for Microsoft Fabric?

The first recommendation is to use the Microsoft Fabric Capacity Metrics app to monitor CU consumption and make an informed decision on the most suitable license type.

In addition, organizations can use the following list of questions to gather information for an initial cost estimate:

This app is designed to provide information about storage capacity and compute usage, helping you to more accurately assess the type of license you will need.

In addition, this app is especially useful for identifying periods of high demand and ensuring that the system scales appropriately to handle the increased load.

Fabric capacity Metrics

The Capacity Metrics app offers a plethora of metrics to help users understand their data usage. Here’s a breakdown of some of the key metrics.

1. Overview Page:

At the top of each page, the Capacity Name field allows you to select the capacity the app shows results for.

Multi metric ribbon chart

• Provides an hourly view of capacity usage.
• Allows users to drill down to specific days to identify patterns.
• Displays values for Capacity Units (CU) processing time, processing duration, number of operations, and number of users.

Capacity utilization over time

Utilization Tab

• Shows CU usage over time.
• Users can toggle between linear and logarithmic scales.
• Displays CU consumption during background and interactive operations, including billable and non-billable operations.
• Highlights when the capacity is overloaded with dotted lines.

Throttling Tab:

• Displays delay and rejection over time.
• Highlights when operations are delayed or rejected if values exceed 100%.
• Users can toggle between interactive delay, interactive rejection, and background rejection.

Overages Tab:

• Shows the add, burndown, and cumulative carryforward over time.
• Highlights the percent of carryforward added or burned down during a 30-second period.
• Users can select a column to drill through to the timepoint page.

Matrix by item and operation

• Displays metrics for each item on the capacity.
• Users can sort and filter the table based on various parameters.
• Displays values for CU processing time, processing duration, number of users, item size, overloaded minutes, performance delta, and billing type.
• The performance delta indicates the difference in performance compared to seven days ago, helping users assess performance improvements or declines.
• Items can be billable, non-billable, or both.

2. OneLake Page:

The OneLake page of the Microsoft Fabric Capacity Metrics app offers insights into capacity storage. Here’s a breakdown of its features:

Filters:

• Capacity Name: Choose a specific capacity to view its data.
• Date Range: Display results for a particular date range.
• Top: Adjust the number of top workspaces shown based on storage volume.

Cards:

• Workspaces: Total count of workspaces using storage.
• Current storage (GB): Shows the latest storage data.
• Billed storage (GB): Indicates the storage amount billed. Note that billed storage can sometimes be less than the current storage.

Table Visual:

Displays storage details for the top workspaces.  Fields include:

• Workspace name
• Workspace ID
• Current storage
• Billed storage
• Billed storage percentage.

Column Charts:

• Storage (GB) by date: Illustrates average storage over the past 30 days, with an option to view hourly data.
• Cumulative billed storage (GB) by date: Depicts the total billed storage over time.

Export Data:

Users can export detailed storage data for further analysis.

3. Timepoint page

This page is accessed from the ‘Explore’ button located on the Overview page. It offers insights into the compute impact of all activities within a capacity. It helps users identify which operations, both interactive and background, have the most significant impact on Capacity Unit (CU) usage.

Top Row Visuals:

• Timepoint: Shows the specific timepoint being analyzed.
• Heartbeat Line Chart: Displays a 60-minute CU activity window, highlighting the duration of activity peaks and troughs.
• Cards:
  • Interative Operations: Total number of interactive operations that contributed to CU activity during this timepoint.
  • Background Operations: Total number of background operations that influenced the CU’s activity during this particular timepoint.
  • SKU: Current SKU – Capacity.
  • CU (s): CU seconds availble for this capacity within a designated 30-second timepoint window.

Interactive Operations Table:

Lists every interactive operation impacting CU usage during the timepoint. Displays details like: 

• Item name
• Operation type
• Start and end times
• Status: Indicates whether the operation was successful or failed.
• It shows if the operation succeeded or failed.
• User
• Duration: Seconds it took to complete the interactive operation
• Total CU(s): Represents the seconds consumed by the interactive operation. This metric is crucial to determine if the capacity exceeds the total number of CU seconds allowed for the capacity.
• Timepoint CU(s): Seconds assigned to the interactive operation in the current timepoint.
• Throttling (s): Seconds the operation was throttled due to previous timepoint’s capacity overload.
• % Of Base Capacity: (Interactive CU Operations / Base Capacity Allowance) x 100
• Billing type: Specifies if the operation is billable. or not.
• Operation ID.

Background Operations Table:

This table shows background operations contributing to CU usage. It’s similar to the interactive operations table but omits the user column.

Burndown Table:

Presents the add, burndown, and cumulative percentages by experiences for the last 30 seconds.

Overages Visual:

Users can adjust the visual scale to show data for 10 minutes, 60 minutes, or 24 hours. It focuses on billable operations and provides insights into carryforward percentages.

Crucial Takeaways:

• Overloading the capacity can trigger throttling. When autoscale is on, an additional CU is provisioned for the next 24 hours upon reaching capacity. Without autoscale, throttling affects all ensuing interactive operations.
• CU activity can be influenced by background operations from the preceding 24 hours, attributed to performance smoothing.

How to Install

Installing the Microsoft Fabric Capacity Metrics app is straightforward. Users can access the app through the app from AppSource:

Go to AppSource > Power BI Premium Capacity Utilization and Metrics and select Get it now.

Then, go to Power BI Service:
Select Apps → Get apps → Search for Microsoft Fabric → Select the Microsoft Fabric Capacity Metrics app →  Select Get it now.

For running the app for the first time you need to include:

– CapacityID: To get the capacity ID go to Settings → Admin Portal → Trial or Fabric Capacity → Click on Actions:

– UTC offset:

Learn more about monitoring capacity usage: https://learn.microsoft.com/en-us/fabric/enterprise/metrics-app-overview-page

In conclusion, determining the precise capacity size necessary for your needs can be a complex task, as there is no exact formula available for this purpose. However, starting with the Fabric trial, which provides a performance equivalent to 64 CUs, or opting for a pay-as-you-go Fabric capacity size for initial testing, can be a wise approach.

Utilizing the Microsoft Fabric Capacity Metrics app is highly recommended, as it enables users to closely monitor their CU consumption, providing valuable insights into storage capacity and compute usage. This, in turn, aids in making an informed decision regarding the type of license required for optimal performance.

Microsoft Fabric provides a diverse range of licensing options, categorized primarily into capacity-based and individual user licenses.

1. Capacity Licenses -F SKUs (Dedicated capacity)

– Known as Premium licenses in Power BI.

– They provide the infrastructure for Microsoft Fabric. Without a capacity license, users can’t work in Fabric.  

– Capacity licenses are divided into Stock Keeping Units (SKUs) that deliver a set of Fabric resource levels. These levels are measured in Capacity Units (CU) which is a new concept used to measure the compute power available for each SKU.

– Each license offers double the computational power of its predecessor, for instance, F4 has twice the power of F2 and half the power of F8. 

– This license introduces a “pay as you go” model, it means:

      ✅ Users pay for their actual usage, there are no minimum terms, the SKUs are available on an hourly or monthly basis.

      ✅ Unlike Premium Capacity, Fabric Capacity can be paused, offering cost flexibility.

      ✅ Charges are incurred only when the capacity is actively used.

      ✅ Prices start at a competitive point of $8 per day, making it accessible for organizations.

Microsoft Fabric Pricing

Notes: 

• Billing is per second with a one minute minimum.
• SKUs that are smaller than F64 require a Pro or Premium Per User (PPU) license to consume Power BI content.

2. Individual licenses

– Free: This license allows users with access to Fabric Capacity to create and share content within Microsoft Fabric. It covers functionality such as pipelines, data warehousing, notebook usage and capacity management. However, sharing Power BI content requires a Pro license.

– Pro: Essential for sharing Power BI content with other users. Every organization intending to work with Power BI requires at least one Pro license. Thus, if you’re purchasing a Microsoft Fabric license, ensure you include at least one Pro license.

– Premium Per user (PPU): Operating on a shared capacity, this license grants limited access to Microsoft Fabric, and will continue to be suitable for organizations that want to leverage the premium features of Power BI without extending to any Fabric content. PPU can be more cost effective when Power BI Premium features are needed for less than 250 users.

In conclusion, the individual licenses work in Microsoft fabric only if they are combined with a Capacity License. Starting from the F64, you can share Power BI content with Free Licenses. For lower licenses, You will need Pro licenses.

Additional Licensing Points

– Power BI Embedded: also known as A SKUs, remains available for embedding purposes. However, it is important to note that Power BI Embedded does not provide access to the new Fabric workloads.

 – Microsoft Fabric trial: Include access to the Fabric product experiences and the resources to create and host Fabric items, the trial lasts for a period of 60 days, but may be extended by Microsoft discretion. It is crucial that users transition to a paid Fabric capacity after the trial in order to retain data and continue usage. Otherwise, non-Power BI Fabric items will be removed in accordance with the stipulated retention policy.

Variables to Consider for Cost Estimation

1. CAPACITY UNITS (CU): It’s the primary billing unit used in Microsoft Fabric, it combines CPU, Memory, Disk IO, Network bandwidth, and more. Each operation in Fabric consumes CUs, higher CUs equals higher performance.

2. NUMBER OF USERS THAT WILL SHARE AND CONSUME POWER BI CONTENTS: This is one of the variables that most impacts the price to be paid. The price of each license is $10 per user/month and $ 20 per user for Premium per user licenses (PPU).

Typically if you have more than 500 users (250 users in PPU license) consuming and sharing Power BI content, it is recommended to purchase an F64 license or higher.

3. INTENSIVE WORKLOADS THAT COULD OVERLOAD THE CAPACITY: If you have important capacity peaks you should consider scale your capacity, it means at any time, you can temporarily increase your capacity and reduce it again when you’re done. And since F-SKUs are billed on a usage basis and managed as an Azure resource, you only pay for the time you consume.

At the time of writing this article, auto-scaling is not currently available, but it is planned to be integrated in the future. So you will have to apply manual scaling in periods with intensive workloads.

Discover how to scale Fabric capacity: https://learn.microsoft.com/en-us/fabric/enterprise/scale-capacity

4. OPERATIONAL HOURS: Microsoft Fabric allows you to pause and resume the capacity when it is not operational, to save costs for your organization. Later, when you want to resume work on your capacity, you can reactivate it. 

So if your company isn’t doing anything with data or analytics during the weekend, you can just pause the capacity and save approximately 25% of the monthly license cost.

Learn more about how to pause and resume Fabric capacity: https://learn.microsoft.com/en-us/fabric/enterprise/pause-resume

This article explains how to automatically pause and resume Fabric capacity: https://github.com/nocsi-zz/fabric-capacity-management/

5. ONELAKE STORAGE: It is the data lake integrated into Microsoft Fabric that serves as a centralized repository for all of the organizational data and is billed on a pay-as-you-go rate. Microsoft Fabric storage is OneLake, this is the only place where all data is stored.

This cost is not included in the capacity licensing, so you will need to pay for it separately. Note that OneLake storage pricing follows the same pricing as Azure Data Lake Storage gen2:

Benefits of Fabric storage:

• Efficiency: Streamlined purchasing with auto-provisioned storage for all tasks.
• Cost-Effective: Unified data copies eliminate redundant data transfers and reduce duplication costs.
• Integration: Seamlessly connect third-party storage systems to Microsoft Fabric analytics.
• Accessibility: Open data formats facilitate access to multiple analytics engines, minimizing configuration resources.
• Security: Centralized access to all data and governance tools ensures consistent security and simplifies maintenance.

6. BANDWIDTH: Bandwidth refers to the data moving in and out of Azure data centers and between Azure data centers. While it’s a crucial component, its cost impact is often minimal compared to Capacity Units (CU) consumption and Power BI licenses:

• Data Transfer In: Free for data moving into Azure data centers.
• Within an Availability Zone: Free
  Availability Zone is an isolated location within an Azure region, equipped with its own power, cooling, and networking to protect applications and data from data center failures. It ensures high availability and resilience by allowing resources to be distributed across multiple independent physical locations within a specific region.
• Between Availability Zones: The price is around $0.01 per GB. So, even if a company transfers a substantial 500GB of data between zones in a month, the cost would be just $5.$0.01 per GB
• Inter-Region Transfer: Ranges from $0.02 to $0.16 per GB
• Inter-Continental Transfer: Ranges from $0.05 to $0.16 per GB. For a company transferring 1TB of data inter-continentally, the cost would range from $50 to $160.
• Internet Egress: First 100GB/month free; subsequent costs vary by region.

Learn more at Bandwidth pricing:  https://azure.microsoft.com/en-us/pricing/details/bandwidth/

7. SMOOTHING: This mechanism distributes CU consumption over time, allowing users to run more workloads in the same capacity without restrictions and helping them avoid being penalized with additional costs. 

It means that you only pay for the average rather than the peak performance, averaging capacity usage over short periods of time for interactive operations and over a 24-hour period for background operations.

Learn more about this feature: https://learn.microsoft.com/en-us/power-bi/enterprise/service-premium-smoothing

Power BI Licensing vs Fabric

You have probably noticed that the prices of the Fabric capacities are higher than those of the equivalent Power BI Premium capacities. 

A P1 Power BI Premium capacity provides the same power as an F64 Fabric capacity, a P2 Power BI Premium capacity provides the power of an F128 Fabric capacity, and so on. However, if you compare the Fabric F64 with the Premium P1 plan, you will see that the annual cost of F64 amounts to $100,000, compared to $60,000 for P1. The reasons for these differences are as follows:

✅ Pay-as-you-go basis: Users can pause and restart the service, scale them up and down dynamically with no usage commitment , optimizing resource utilization and ensuring cost-effectiveness.

✅ Contractual Commitment: Fabric’s F64 plan provides flexibility with no mandatory minimum contract duration while in Power BI premium there is a monthly/annual contract.

✅ Service Scope: The F64 plan grants comprehensive access to all Fabric workloads. In contrast, the P1 plan is tailored specifically for Power BI items.

✅ Storage: Power BI premium licenses offer a certain amount of dedicated storage for Power BI items while in Fabric it is calculated based on the amount of data stored within OneLake.

In a few months, Microsoft will introduce an Azure Reservation commitment model for Fabric Capacities  – RI SKUs, which will make their price comparable to the equivalent Power BI Premium capacities.

Example Cost Estimation – Choosing the Right Scenario

Understanding the costs associated with Microsoft Fabric is crucial for effective budgeting. Below, we will show different hypothetical scenarios for helping you to understand how you can estimate the licensing more fit for your organization.

However it is very important monitoring your Microsoft Fabric Capacity Metrics to understand better your spending across workloads and optimize your capacity usage. 

Scenario 1) Small Business with moderate workloads

• Number of users that will consume Power BI content: 25 users
• Operational hours: 8 hours/day (pauses capacity during off-hours and weekends)
• Workload: Moderate 
• Storage: 100GB on OneLake

Recommended License: F2 SKU 

Cost Calculations:

Scenario 2) Small business with 50 users that wants to use premium features in Power BI

• Number of users: 50 users
• Operational hours: 8 hours/day (pauses capacity during off-hours and weekends)
• Workload: Moderate 
• Storage: 100GB on OneLake

Recommended License: F2 SKU 

Cost Calculations:

Scenario 3) Medium Organization with 250 users and high-demand periods

• Number of users: 250
• Operational hours: 10 hours/day, 6 days a week (Monday to Saturday)
• Workload: They have peak intensive data processing, especially during specific projects or when handling large datasets (20 hours approx.)
• Storage: 1TB on OneLake

Recommended License: Starts with F2 SKU, but scales up to F8 during high-demand periods.

Cost Calculations:

Scenario 4) Large Corporation with Month-End Reporting Peaks

• Number of users: 600
• Operational hours: 10 hours/day, 7 days a week
• Workload: Regular operations with an F64 SKU, but scales up to F128 during month-end reporting (30 hours approx.)
• Storage: 1TB on OneLake
• Bandwidth: Inter-region transfer within North America of 1TB.

Recommended License: Starts with F64 SKU, but scales up to F128 during high-demand periods.

Cost Calculations:

Scenario 5) Global Financial Institution with Real-Time Analytics

• Number of users: 1,000
• Operational hours: 24/7 (Always on)
• Workload: Highly intensive due to real-time financial analytics. They start with the F128 SKU but scale up during peak market hours (50 hours approx.)
• Storage: 6TB on OneLake
• Bandwidth: 10TB inter-continental data transfer from North America to other continents.

Recommended License: Starts with F128 SKU, but scales up during high-demand periods.

Cost Calculations:

Navigating through Microsoft Fabric’s licensing and cost estimation can be a challenge, but understanding these key variables and scenarios can help you make informed decisions. Whether you’re a small business or a global corporation, Microsoft Fabric offers flexible options that cater to your specific needs.

As you explore the world of Fabric, remember to monitor your capacity metrics using the Microsoft Fabric Capacity Metrics app. In the next article of this series, we will go deeper into this app, which will be your great ally to help you optimize your consumption and manage your capacity effectively.

As organizations consider implementing Microsoft Fabric, understanding its licensing and cost estimation becomes crucial.

Based on our experience assisting our clients in navigating the intricacies of Microsoft Fabric’s licensing and pricing, I’ve written a series of articles on the subject, with the goal of making it easier to understand. We’ll delve into the factors to consider when choosing the most suitable license and provide examples exploring various licensing scenarios and their associated costs.

The article series is structured as follows:

1. Key Components of a Microsoft Fabric Subscription

2. Microsoft Fabric License Types

3. Understanding Microsoft Fabric Licensing and Cost Estimation

    ✔️ Variables to Consider for Cost Estimation

    ✔️ Power BI Licensing vs Fabric

    ✔️ Example cost estimation where we will explore different licensing scenarios

   ✔️ Checklist of questions to gather the information needed to estimate Microsoft Fabric pricing

4. Microsoft Fabric Capacity Metrics app

5. How to Acquire Microsoft Fabric Capacity

Part 1. Key Components of a Microsoft Fabric Subscription

A Microsoft Fabric subscription is composed of three fundamental elements: tenants, capacities, and workspaces.

1. Tenant

A Microsoft Fabric subscription starts with a “tenant.” Think of a “tenant” as a powerful computer server in a large data center. This server is dedicated to a specific organization, ensuring that all the data and applications within it are isolated and secure from other organizations.

A tenant usually represents an organization or domain and serves as the highest-level container that is linked to a unique internet address, called a Domain Name System (DNS). It ensures that the organization’s data is safe, organized, and easily accessible. Larger organizations may have multiple tenants for various departments or regions.

2. Capacity

A Microsoft Fabric capacity resides on a tenant and is a distinct pool of resources allocated to Microsoft Fabric. The size of the capacity determines the amount of computation power available to the organization.

The “capacity” can be thought of as the server’s hardware resources, like its RAM (memory) and CPU (processing power). The greater its capacity, the more computing power it will have for its tasks.

Capacity types:

    – Dedicated Capacity: A reserved set of computational resources allocated exclusively for a specific organization or task within Microsoft Fabric. This means that the memory, processing power, and other resources in this capacity are the exclusive use of that specific organization or task, ensuring consistent performance and preventing potential slowdowns from other users or tasks.

    – Shared Capacity: A pool of computational resources which are distributed among all the users accessing it. While it’s cost-effective, there might be fluctuations in performance, especially during peak usage times, as multiple users tap into the same pool of resources.

3. Workspace 

Workspaces are specialized containers for Microsoft Fabric items located within capacities. As in a server (tenant) and its resources (capacity), there are several applications and programs running. Each of these can be compared to a “workspace” in Microsoft Fabric. These workspaces use the server’s resources (capacity) to run, store data and perform tasks.

By default, workspaces are placed in  shared capacity.  But as the organization grows and has additional capacities available, these workspaces can be strategically placed in any of the existing shared or dedicated capabilities to better suit their organizational needs.

In essence, the server (tenant) provides the environment, its hardware resources (capacity) power the operations, and the applications (workspaces) are where the actual tasks and storage occur.

In this checklist you will find the tools you can use at every step of the way:

1. DATA SOURCE

Query Diagnostic

 This Power BI Desktop feature will help you understand what happens inside the Power Query when a query is executed, the parts of the query that are being performed locally or remotely, and the slowdowns you may encounter during:

– Loading and transforming your data.

– Refreshing your data in Power Query.

Start a Query Diagnostic:

• Go to the ‘Tools’ tab in the Power Query Editor ribbon. You will have the following options Diagnose Step and Start Diagnostics / Stop Diagnostics:

Start Diagnostics / Stop Diagnostics is more used because it provides complete information about each query while Diagnose Step analyzes the performance of a specific step without showing you the diagnostic of the other steps of the query.

• In Power Query Editor, select the query you want to analyze.
• Go to Tools in the Home ribbon, click “Start Diagnostics”.
• Perform the evaluations you want analyze, as: authoring, refreshing a preview, full refresh, etc.
• When you are finished, make sure that you select “Stop Diagnostics”.

This generates two new queries with the diagnostics results:

In my opinion the most important column is the Exclusive Duration, you should convert it to seconds in the Power Query Editor and then summarize it to get an idea of where time is spent and identify how you can optimize the queries in your power bi models:

2. DATA MODEL

DAX Studio

– Launch DAX Studio. Select the model that you want to analyze:

– Go to the advanced tab and click on “View Metrics”:

This will show metrics that help you quickly detect the performance issues.

Cardinality: This is the number of unique values in a column. High cardinality columns can be particularly expensive, especially when have a string or date/time format.

If possible, remove columns such as GUID and timestamp from a SQL Server table from your model. In addition, I recommend splitting Datetime columns into two columns, one for date and one for time.

Size information: The other important thing you can see is the space that is taking up each table and column of the model, for example:

In this case, the Work Order table is taking up almost 20% of the entire model, which helps you analyze where your model consumes the most memory and choose which columns to remove to optimize it.

Summary information: In the Summary tab you can see the total size of this model, in this example the analyzed PBIX is consuming 198.27 MB. You can also see the number of tables and columns:

Bravo for Power BI

This is a free open-source tool for users who do not need a high level of complexity. In case you need more advanced functionalities you should use other more advanced tools such as DAX Studio and Tabular Editor.

After downloading Bravo, go to the “Analyze Model” and connect to the data model:

You will get information about the tables and columns in your model that are consuming the most memory:

In addition, you can drill down to see more detail for any table:

To analyze the unreferenced columns, I recommend my tool Tabular Editor, which can help you identify the reports connected to the dataset where these columns are used.

3. DAX FORMULAS / VISUALIZATION

Performance Analyzer: To Analyze the performance of your report elements, such as visuals and DAX formulas when users interact with them, and which consume the most resources.

In the View ribbon you can find the Performance Analyzer. By opening this pane and clicking Record, this will show you how fast your report renders when performing different functions in your report.

You can review the results of your performance test in the Performance analyzer pane. To review the tasks in order of duration, longest to shortest, right-click the Sort icon next to the Duration (ms) column header:

The log information for each visual shows the time it has taken (duration) to complete the following categories of tasks, which will help you detect what is causing performance delays so you can work on improving those sections.

• DAX query– Time taken by the to send the query and the time it took Analysis Services to return the results.
• Visual display– Rendering time on the screen, including the time required to retrieve web images or geocoding.
• Other– The time it took the visual to prepare queries, wait for other visuals to complete, or perform other background processing tasks.

The only real way to reduce the long duration is to optimize DAX queries for other visuals or reduce the number of visuals in the report.

4. DATA MODEL / DAX FORMULAS / VISUALIZATION

Best Practice Analyzer within Tabular Editor.

Best Practice Analyzer is an external tool that scans your data model and easily validates if your data model complies with all best practices at formatting level, performance, naming conventions and DAX expressions.

I analyzed a test data model getting the following information which shows the potential issues and improvements that can be made in the data model in each category:

This can help you save a lot of time by avoiding having to validate all these recommendations manually.

In this article 👉 Best practice rules to improve your model’s performance you can read the rules that analyze which are segmented into categories. Some rules are more aesthetic-oriented while others are more focused on performance optimization.

Or after cleaning up your data model, do you find that connected reports break so you manually go through each one to find the cause?

In this post I will introduce you to the first version of a tool I created to help you in the process of clean your data models, which can become a nightmare if you don’t have an effective and fast way to do it.

Challenge

When developers remove unnecessary objects from large data models, connected reports break a and the validation process is manual, tedious and time-consuming.

Solution:

When I tried to clean my data models I couldn’t find any tool with all the functionalities I needed, so decided to create my own using Power BI, which allows:

– Obtain all the dependencies of the data model in an efficient and automated way.

– Identify the list of objects (measures, columns, calculated columns, tables) in use and those that are not, in the reports connected to the Power BI or SSAS Data model.

I hope it can help you in the process of keeping your data models optimized 😊.

How It Works

This tool contains two parts:

1. Get the dependencies between all objects in the data model: This tool identifies how all the objects in your data model are connected to each other. For example, if you want to remove a measure, you will be able to detect the impact on your reports.

To get the dependencies I used Dynamic Management Views (DMV) which are queries that return information about model objects, server operations and server status. If you want to know more that DMVs, check our article What are Dynamic Management Views (DMVs) and what are they for?.

The main DMV that I used is the DISCOVER_CALC_DEPENDENCY, which Returns information about the calculation dependency for an object that is specified in a Tabular database or in a DAX query that is executed against a Tabular database.

2. Identify the objects used in the data model: For that I used the tool “Report Analyzer” explained in detail in this article: Report Analyzer

Finally, I related the information provided for the DMVs and I merged it with the information generated for the Report Analyzer in a Power BI report.

How to Use the Tool

You can use this tool in any Power BI data model with the following simple steps:

1. Get the connection details of a Power BI Dataset

Local Power BI Dataset:

Open a Power BI Dataset that you want to document. Open DAX Studio and choose the “PBI / SSDT Model” data source option, then click on Connect:

In the lower right-hand corner of the screen is the name of the server, and the port number you have connected to. In this case it is localhost:52223

Run the following DMV query in a DAX query window. This will give you the GUID, the name of the only database in the Power BI data model:

SELECT [CATALOG_NAME] FROM $SYSTEM.DBSCHEMA_CATALOGS

Power BI Premium

– Server: Go to “Workspace settings” and copy the Workspace Connection.

– Database: Is the dataset name.

2. Update the Dataset parameters in the Data model cleaner tool:

3. Get all the objects used in the Power BI reports connected to the data model:

– Save all the PBI report files connected to the golden dataset in a folder.

– Open External tool: Report Analyzer:

– Click on “Select folder”:

– Select the respective folder, and select “Export Report Metadata”:

– You will see the following message:

The files in .txt format will be saved in the same folder where the reports are.

4. Update the “LocalSoucePBITemplates” parameter in Power BI:

Type the location of the folder with the .txt files generated by the Report Analyzer:

Information provided by the Data Model Cleaner tool

– Object dependencies: On this tab you can consult the list of calculated objects affected by the selected object:

– Calculated Objects Breakdown: This tab shows the breakdown of the queried calculated object.

– Objects Used / Unused List: On this tab you can easily consult the Used / Unused list of the objects that will help you make decisions about hiding or removing them from the model.

If you consult by table additionally you will see the “M” or DAX expression of the table.

To determine the used list of objects this tool consider:

• Visuals and custom visuals
• Report-Level measures
• Filters added to the visuals, pages, and reports
• Bookmarks

You can also drill through on any object used and consult the reports, pages and in which of the visualizations this object is being used:

– Used Objects in the Reports: List of objects used by report.

Download Tabular Model Cleaner tool here 👇:

Tabular Model Cleaner tool.zip

This is the second exam that includes Power BI and in this article you will find the comparison between it and the PL-300 Exam, the skills required, study resources and the benefits of obtaining the certification.

The Exam DP-500 was released on April 19, 2022 and is the second one that includes Power BI, the main difference with the PL-300 exam is that it focuses on self-service BI while DP-500 is the enterprise data analyst certification.

Role Comparison between PL-300 and DP-500 certifications

Benefits of obtaining the certification

Preparing for a certification gives you the opportunity to study in an orderly way, reinforce your knowledge and validate those aspects you need to deepen. After obtaining my certifications, I have felt a strong sense of accomplishment, more confidence and it has opened new doors in my career.

I know that getting certified requires an investment of time, effort and money, but I assure you it will be well worth it. These are some of the benefits of getting certified, I hope these will motivate you to invest time and effort in getting it:

• Demonstrate your knowledge and skills.
• Grow professionally.
• New professional oportunities.
• Boost your confidence.
• Increase you salary.
• If you work as a freelancer you improve your chances of finding new clients.
• Be more influential in your work projects.

Here are some statistics provided by Microsoft on the career impact of certified IT professionals:

kills measured

This exam contains the following 4 sections:

• Implement and manage a data analytics environment (25–30%)
• Query and transform data (20–25%)
• Implement and manage data models (25–30%)
• Explore and visualize data (20–25%)

In the following link: Exam DP-500 Skills Measured you can find a downloadable document that contains the complete list of skills tested on the exam. I recommend that you check it from time to time because it has updates.

 Microsoft Resources: Here you will find the resources provided by Microsoft for exam preparation.

• Microsoft Power BI data analyst (PL-300 Exam): This is the learning path to prepare for the PL-300 exam, however, it covers some of the topics included in the DP-500 exam.
• DP-500 Microsoft learning Path: These resources were recently added to Microsoft DP-500 web page, you will see them If you scroll down to the end of the page:
  • Introduction to data analytics on Azure
  • Govern data across an enterprise
  • Model, query, and explore data in Azure Synapse
  • Design and build tabular models
  • Implement advanced data visualization techniques by using Power BI
• Master the basics of Azure: data fundamentals
• Level up: Azure enterprise data analyst collection on Learn.

Other Resources:

– Nikola Ilic  

– Serverless SQL