English
How to Automate your way into Microsoft Fabric
Questions
- How does AnalyticsCreator automate data engineering?
- Can AnalyticsCreator deploy to Microsoft Fabric?
- How are pipelines generated automatically?
- How does metadata-driven development work?
- How is schema evolution handled in data warehouses?
- Can AnalyticsCreator integrate with SSIS and ADF?
Key takeaways
- Data engineering represents ~80% of data warehouse effort.
- Automation reduces errors and improves delivery speed.
- Metadata-driven design enables easy platform switching (on-prem → cloud).
- Schema evolution is handled via metadata refresh and regeneration.
- Azure Data Factory pipelines can be generated automatically.
- Fabric integration supports modern analytics workflows.
- Data quality still depends on proper structuring before ingestion.
Transcript
Hello and welcome, everyone, to our webinar on how to automate your way into Microsoft Fabric.
We are very excited to have Andy Leonard with us. Andy knows his way around SSIS, Azure, and Fabric. He is a former Microsoft MVP and an outstanding data engineer.
We also have our CEO, Peter Smoly, and Gustavo Leo, who will demonstrate the solution.
AnalyticsCreator is a fully metadata-driven data warehouse automation technology.
You design your data objects graphically using a wizard, and from that metadata, AnalyticsCreator generates the source code.
You can develop on-premise and migrate to Azure without rewriting code. When Microsoft changes the technology stack, you do not need to change your development approach. New code is generated from the metadata.
Data engineering makes up about 80% of data warehouse projects.
It is complex and involves cleansing, mapping, staging, and ensuring high-quality data.
Automation removes errors such as typos and inconsistencies, improving both speed and reliability.
Tools like AnalyticsCreator automate predictable patterns, such as SSIS packages and Azure Data Factory pipelines.
AnalyticsCreator is a desktop application.
You create a new repository, which is a metadata repository containing all the structures needed for the warehouse.
You then connect a data source, in this case Northwind, and load the metadata instantly.
No production data is modified because AnalyticsCreator is a design-time tool.
The Data Warehouse Wizard loads metadata and allows you to select tables.
It automatically generates layers such as import, transformation, persistent staging, and data mart.
It also applies default transformations, such as trimming text and converting null values.
A calendar dimension is generated automatically.
The entire data warehouse is created from metadata without touching production systems.
All generated code is pure SQL and fully transparent.
Fact tables and dimensions can be added through wizard-driven processes.
Relationships and joins are created automatically.
A fact table is added using a wizard.
Relationships between dimensions are defined automatically.
Calculated columns, such as total amount equals unit price multiplied by quantity, are added using SQL expressions.
The system automatically integrates these into the model.
A deployment package is created targeting Fabric SQL.
Options include SSIS, Azure Data Factory, and Power BI project generation.
AnalyticsCreator generates DACPAC files and deploys them automatically to Fabric.
AnalyticsCreator automatically generates ARM templates with 47 resources.
These pipelines are imported into Azure Data Factory.
Linked services are configured for the source and destination systems.
Execution is managed through workflow pipelines.
Azure Data Factory pipelines load data into Fabric SQL.
Execution is monitored through workflow orchestration.
Data is validated directly in Fabric tables.
AnalyticsCreator generates a Power BI project, also known as PBIP.
Relationships and schemas are already defined.
Query folding ensures efficient SQL execution.
The model is ready for reporting without manual setup.
Schema evolution is handled through metadata refresh and merge.
Unstructured data must be structured before data quality rules can be applied.
AnalyticsCreator supports multiple connectors, including SAP and custom sources.
Support for the Fabric warehouse engine is in progress.
Automation reduces complexity and accelerates delivery.
Metadata-driven design ensures flexibility across platforms.
AnalyticsCreator enables rapid iteration and high-quality data engineering.
