Overview: As we know, it's crucial that you run Database maintenance regularly in order to keep your database performance up with the latest statistics and healthy indexes, here we provide another way to schedule this job using Azure Data Factory; this can be done on a scheduled time interval weekly/bi-weekly/monthly.
Solution: This article will show steps to schedule Database maintenance using the Azure Data Factory(ADF). ADF can use as an SQL agent to schedule and automate Azure SQL database tasks. In addition, database maintenance plans not supported in Azure can also be replaced with Azure pipelines.
| Note: Using ADF will incur an additional cost. |
Use case:
- After large data deletion from database/tables or after data purge routine.
- Post ETL pipelines, as a final activity step to perform database update stats and rebuild indexes per business needs.
- After database migrations or database refresh/restore from production to Dev/Test environments.
- Use Azure Data Factory pipeline with store procedure activity replacement of Database maintenance plans, which are unsupported in Azure SQL DB.
This can be done via AzureSQLMaintenance store procedure (SP) to perform complete Database maintenance, Store procedure and blog written by Yochanan Rachamim are listed below.
Blogs:
How to maintain Azure SQL Indexes and Statistics - Microsoft Tech Community
Code:
Main steps:
- Use existing or create an Azure Data Factory
- Add SQL Authentication login on database
- Create "AzureSQLMaintenance" SP on the user databases
- Create pipeline
- Schedule pipeline
Step by Step walkthrough:
- Please create one using the Azure portal if you do not have a data factory.
https://docs.microsoft.com/en-us/azure/data-factory/quickstart-create-data-factory-portal#create-a-d... - Add New login to perform maintenance tasks, which will help identify when the maintenance tasks are running on the database. You can use the existing login and user.
-- Execute on Master
CREATE LOGIN DBAdmin WITH password='<your-password>';
-- Execute on Master and user database
CREATE USER DBAdmin FROM LOGIN DBAdmin;
-- permission
EXEC sp_addrolemember 'db_owner', 'DBAdmin';
3. Download and deploy AzureSQLMaintenance to a user database. Code: AzureSQLMaintenance
4. Open Azure data factory to create a pipeline and name it.
You can Import pipeline template using below zip file.
https://github.com/HitenBhavsarMSFT/DataFactoryTemplate/blob/main/AzureSQLMaintenance.zip
Add store procedure activity.
Press "+ New" and set up linked Serviced by adding a name, type, subscription, Server, database, authentication type, User, and password. Test connection for the connectivity check.
After selecting Linked Service, Select AzureSQLMaintenance store procedure
Import parameter, for operation parameter, provides the value "ALL".
Parameter value represents the database task it will perform. Blog
ALL = Run through all indexes and statistics and take a smart decision about steps taken for each object
STATISTICS = run smart maintenance only for statistics
INDEX = run smart maintenance only for indexes
All other parameters can be removed from the setting tab.
"Publish all " to publish and save the pipeline.
Note: Use trigger now to test the pipeline.
Use monitor pipeline run to check the Manual trigger.
5. Schedule pipeline using trigger using "New/Edit" button.
Enter trigger detail like name, Desc type. Then, publish the trigger to make it effective.
I hope you find this article helpful. If you have any feedback, please do not hesitate to provide it in the comment section below.
Hiten Bhavsar (Author)
Ahmed Mahmoud (Co-Author)
