Automating Databricks deployments via Octopus Deploy
If you are using Octopus Deploy to automate the deployment of solution(s) and part of your solution is using Databricks, then this post might interest you. We are using Databrick within my current project, and due to tight security controls, we cannot link our workspace to our git repository. We were manual importing our Databricks workbooks as our method of deployment. Until we automate the process using Octopus Deploy and the newly created community step template: Import Databricks Workbooks. Currently, the community step template only imports workbooks with the ‘.ipynb’or ‘.scala’ file extension and only communicates to the DataBricks instance using HTTPS. The task uses the DataBricks rest API v 2.0 to update the DataBricks instance, so there is no requirement to install any extra supporting software. The template only requires the parameters: DataBricks Workbook Package: The package of the Databricks workbooks for deployment. Databricks Instance Uri: The hostname of the Databrick