Azure Data Factory: Our Favorite Features and When to Use It with Dynamics 365 and Power Apps

Azure Data Factory (ADF) is a cloud-based web integration application designed for transferring and editing data at scale.

Azure Data Factory is ideal for situations where you need to migrate or bulk update large quantities of data, especially if you’re already using other Azure services because it can handle larger volumes of data, has better performance, and can be less costly than building to Dynamics 365 or Power App APIs.

Handle Large Volumes of Data

While there are several applications you can use to connect, transfer, and bulk edit your data, not all of them have the capacity to manage huge volumes. These are the types of situations where ADF really excels. Azure Data Factory makes is possible to load and export millions of rows of data into Microsoft Dynamics 365 and Dataverse.

Take Advantage of Speed
ADF includes a Dynamics 365 connector that supports multiple connections and record batching capabilities, allowing you to upsert data at an impressively quick rate. We’ve seen it operate at speeds of up to 500K records per hour.

Save Time and Resources
While you can engage Altriva to build a custom application that will transfer or transform your data, Azure Data Factory is reasonably priced option and—as it’s already built, tried, tested and designed with pre-built connectors for dozens of databases—it can reduce development time and get you up and running with a shorter runway than a fully custom program.

We find ADF’s Copy Activity, Error Reviewing, and Mass Migration features particularly interesting and useful.

Copy Activity
Azure Data Factory includes a feature called "Copy Activity" which enables users to copy data from one database to another at scale. Copy Activity doesn’t just work for Microsoft products like Azure SQL server or Azure Storage; use to it to copy data from an on-premises Oracle database or flat files into Dynamics 365.

We also appreciate the Copy Activity status page, which shows the number of source rows that ADF retrieved so far, the number of rows upserted in Dynamics 365 / Dataverse and the total time the process has taken. This status page makes it easy to keep track of the migration process.

Error Reviewing
Another way that ADF keeps the user informed about the migration status is through its ability to write errors or warnings to Azure Storage. This allows the user to review row-level problems with the migration and take corrective action.

Long-Running Migration Capabilities
For major data migration projects, such as those involving millions of rows of data, it’s often important for the migration to run 24/7 and uninterrupted. This can be necessary, for example, when a business is transitioning from an old system on a Friday and wants employees to start using Dynamics 365 the following Monday. ADF can not only automatically retry failed upsert operations to help keep it running consistently, but also utilize advanced capabilities, such as looping controls, advanced error handling and alerts to help a data migration team to meet an important deadline.

Although Azure Data Factory can likely handle the majority of your data migration requirements, you should be aware that it may not have every capability you want for copying data into Dynamics 365 on its own.

For example, if your data migration involves copying support case records into Dynamics 365, it's likely that you’ll want the case (incident) records in Dynamics to be marked as closed or resolved. ADF doesn’t provide an option to close case records in Dynamics 365, but it does provide the ability to call Azure Functions to perform these types of operations on migrated data.

Interested in learning more about how Azure Data Factory could be used at your organization? Read the Microsoft documentation or get in touch with an Altriva consultant.

• The Latest Updates to Power Apps Portals
• How to Connect Excel to Dynamics 365 with Power Query
• Voice Channel for Dynamics 365 Customer Service

Altriva Solutions is an award-winning business technology consulting firm. Founded on CRM and Microsoft technologies, we help companies achieve their business objectives through smart, integrated, customer-centric solutions.