What are the DMVs

The DMVs queries are an interface to schema rowsets based on SQL and use a SELECT statement and the $System schema with an XML/A schema rowset, however, they do not support the full syntax of a SELECT statement as: JOIN, GROUP BY, LIKE, CAST, and CONVERT.

Apply to: SQL Server Analysis Services, Azure Analysis Services, Power BI Desktop and Power BI Premium

Example of a DMV query:

SELECT * FROM $System.DISCOVER_CALC_DEPENDENCY

WHERE OBJECT_TYPE = ‘MEASURE’

ORDER BY TABLE ASC

For more information visit the official documentation: https://docs.microsoft.com/en-us/analysis-services/instances/use-dynamic-management-views-dmvs-to-monitor-analysis-services?view=asallproducts-allversions

What are they for?

• The DMVs provide information about the active sessions and connections, objects that consume the most CPU or memory at a specific point in time, such as:

• To document your Power BI or SSAS data model, the main queries DMVs available for model documentation are:

• • TMSCHEMA_TABLES: List the Column objects in each table.
  • TMSCHEMA_COLUMNS: List the Columns objects in each table.
  • TMSCHEMA_MEASURES: List the Measure objects in each table.
  • TMSCHEMA_RELATIONSHIPS: List the Relationship objects in the model.
  • TMSCHEMA_ROLES: Shows information about the Role objects in the model.
  • TMSCHEMA_PARTITIONS: Provides information about the Partition objects in each table.

In the past I used to create my own documenters using the DMVs, but I recently discovered the excellent and very complete tool “Model Documenter” created by Marc Lelijveld (MVP).

This tool generates the documentation about your data model in an easy and quick way, you can get more information and download the tool in the page: https://data-marc.com/model-documenter/

DISCOVER_CALC_DEPENDENCY

With this DMV you can obtain the list of the dependencies among the objects in your model and extract the DAX expressions. This allows you to have more control over your data models as you can measure the impact of modify or remove any object.

I created the tool “Tabular model cleaner” using this DMV, which helps you to obtain all the dependences of the data model in an efficient and automated way and the usability of the model objects in the reports connected to the Power BI or SSAS Data model.

For more information visit: Tabular model cleaner

How to run the DMVs

1. DAX Studio: Open the Power BI Dataset that you want to document. Open DAX Studio and choose the “PBI / SSDT Model” data source option, then click on Connect:

If you have Power BI Premium you can connect to the dataset, to do this select the workspace, go to “Workspace settings” and copy the Workspace Connection and paste it into the “Tabular Server” option in the DAX Studio.

Go to DMV pane and click on the query you want consult:

2. From Power BI Desktop: You can run DMVs queries from DAX Studio but that will return you a table with the data, you should connect to the DMVs using Power BI, to view and analyze the information more easily.

– Open Power BI Desktop.

– Choose the “SQL Server Analysis Services database” source.

– Include the Server and Database.

Consider if you are connected to a local data model, the database name will change every time you open the Power BI Desktop file, so you will have to edit the connection parameter when you need to refresh the data.

Benefits.

• Reduce space and processing time.
• Faster refresh times.
• Lower latency reports.
• Reduce the time development.
• More user-friendly data models because the user will not navigate through a bunch of useless objects that are not needed to build the reports.

– Choose the appropriate dataset mode for your solution: You can use Import, DirectQuery, and Composite model.

There are a variety of factors to consider when deciding which method to use, so I recommend that you review the advantages, disadvantages, and limitations of each in the Microsoft documentation so that you can select the correct one.

– Build your data model using a star schema architecture avoiding snowflake schema:

Benefits

• More user friendly.
• The DAX formulas are simpler.
• The time refreshes are faster.
• Reduces redundant data

Star schema are specialized data models used when designing data warehouses, the tables are classified as either dimension or fact. In this schema the fact sits in the middle and the dimensions are all around related directly to the fact which gives it the shape of a star.

– Only include the essential columns to create the reports: You do not want a report larger than necessary. However, removing the unnecessary columns could be too complicated when you have a lot of reports connected to the dataset because they can break.

I created a tool to help you to clean up your data models easily and quickly, you can download it for free, visit the link: Tabular model cleaner.

– Load in your data model the necessary tables: Probably you need to import a table in Power Query to make intermediate calculations but if you won’t use this table in your reports remember unchecking the “enable load” option in Power Query:

– Apply an Incremental Refresh policy to reduce the amount of data: Power BI desktop is limited for your computer resources, if you are working with very large models will be necessary configure the Incremental refresh.

Benefits

• Optimize your time development because you work over a smaller data model.
• Shorter data refresh time.
• Lower resource consumption in the Power BI Service.

Check our article: Configure an Incremental Refresh Policy in seven steps

– Group by and summarize combined with composite models: Depending on the level of granularity that you need to handle in your reports you can group your data by month, division, city for example.

In case you have to visualize a detail of the data, you can create a composite model and adding a drill through page that retrieves the detail data from a DirectQuery table.

For more detail check: Composite Models in Power BI Desktop

– Avoid many-to-many relationships: Most of the time, this can be avoided by creating a bridge table with unique values and one-to-many relationships from the bridge table to the fact tables, thus adopting star schema.

– Avoid both-directional relationships in your data model: You need to have a solid data modeling knowledge to measure the impact of this type of relationship in your data model, otherwise this could cause:

• Unexpected results
• Performance problems

Instead of creating a bi-directional relationship, try using a measure like the one below to get the value you need:

– Add a date table to your model and mark it as a date table.

– Disable auto date/time option: This avoids Power BI automatically create a lot of date dimension tables in the background.

– Disable the “Autodetect new relationships after data is loaded” option: Because you could mess up your data model if you do not have solid experience in data modeling.

– Remove columns with high cardinality: High cardinality columns can be particularly expensive. If possible:

• Remove columns such as GUID and timestamp from a SQL Server table from your model.
• Splitting Datetime columns in two columns, one for date and one for time.

– Select the correct data type: Don’t use decimal type if you only need the whole number or if your data granularity is at day level don’t use Date/Time data.

– Create measures instead of Calculated Columns: Using measures where applicable will help reduce data model size and increase the efficiency of the calculations.

 – Create custom columns in Power query instead of DAX calculated columns: The custom columns created in Power Query achieve a better and more efficient compression.

– Use tools as DAX Studio and Bravo you to analyze where your model consumes the most memory: and choose which columns and tables you should remove to optimize it.

– Use the tool Tabular model cleaner to analyze the impact in your reports of make changes or removing objects from your data models.

Click here for part 3 of this series.

Or after cleaning up your data model, do you find that connected reports break so you manually go through each one to find the cause?

In this post I will introduce you to the first version of a tool I created to help you in the process of clean your data models, which can become a nightmare if you don’t have an effective and fast way to do it.

Challenge

When developers remove unnecessary objects from large data models, connected reports break a and the validation process is manual, tedious and time-consuming.

Solution:

When I tried to clean my data models I couldn’t find any tool with all the functionalities I needed, so decided to create my own using Power BI, which allows:

– Obtain all the dependencies of the data model in an efficient and automated way.

– Identify the list of objects (measures, columns, calculated columns, tables) in use and those that are not, in the reports connected to the Power BI or SSAS Data model.

I hope it can help you in the process of keeping your data models optimized 😊.

How It Works

This tool contains two parts:

1. Get the dependencies between all objects in the data model: This tool identifies how all the objects in your data model are connected to each other. For example, if you want to remove a measure, you will be able to detect the impact on your reports.

To get the dependencies I used Dynamic Management Views (DMV) which are queries that return information about model objects, server operations and server status. If you want to know more that DMVs, check our article What are Dynamic Management Views (DMVs) and what are they for?.

The main DMV that I used is the DISCOVER_CALC_DEPENDENCY, which Returns information about the calculation dependency for an object that is specified in a Tabular database or in a DAX query that is executed against a Tabular database.

2. Identify the objects used in the data model: For that I used the tool “Report Analyzer” explained in detail in this article: Report Analyzer

Finally, I related the information provided for the DMVs and I merged it with the information generated for the Report Analyzer in a Power BI report.

How to Use the Tool

You can use this tool in any Power BI data model with the following simple steps:

1. Get the connection details of a Power BI Dataset

Local Power BI Dataset:

Open a Power BI Dataset that you want to document. Open DAX Studio and choose the “PBI / SSDT Model” data source option, then click on Connect:

In the lower right-hand corner of the screen is the name of the server, and the port number you have connected to. In this case it is localhost:52223

Run the following DMV query in a DAX query window. This will give you the GUID, the name of the only database in the Power BI data model:

SELECT [CATALOG_NAME] FROM $SYSTEM.DBSCHEMA_CATALOGS

Power BI Premium

– Server: Go to “Workspace settings” and copy the Workspace Connection.

– Database: Is the dataset name.

2. Update the Dataset parameters in the Data model cleaner tool:

3. Get all the objects used in the Power BI reports connected to the data model:

– Save all the PBI report files connected to the golden dataset in a folder.

– Open External tool: Report Analyzer:

– Click on “Select folder”:

– Select the respective folder, and select “Export Report Metadata”:

– You will see the following message:

The files in .txt format will be saved in the same folder where the reports are.

4. Update the “LocalSoucePBITemplates” parameter in Power BI:

Type the location of the folder with the .txt files generated by the Report Analyzer:

Information provided by the Data Model Cleaner tool

– Object dependencies: On this tab you can consult the list of calculated objects affected by the selected object:

– Calculated Objects Breakdown: This tab shows the breakdown of the queried calculated object.

– Objects Used / Unused List: On this tab you can easily consult the Used / Unused list of the objects that will help you make decisions about hiding or removing them from the model.

If you consult by table additionally you will see the “M” or DAX expression of the table.

To determine the used list of objects this tool consider:

• Visuals and custom visuals
• Report-Level measures
• Filters added to the visuals, pages, and reports
• Bookmarks

You can also drill through on any object used and consult the reports, pages and in which of the visualizations this object is being used:

– Used Objects in the Reports: List of objects used by report.

Download Tabular Model Cleaner tool here 👇:

Tabular Model Cleaner tool.zip