Tag: Databricks REST API 2.0

• Ingest Azure Event Hub Telemetry Data with Apache PySpark Structured Streaming on Databricks.

  Overview. Ingesting, storing and processing millions of telemetry data from a plethora of remote IoT devices and Sensors has become common place. One of the primary Cloud services used to process streaming telemetry events at scale is Azure Event Hub. Most documented implementations of Azure Databricks Ingestion from Azure Event Hub Data are based on…

  jbernec

  May 17, 2021

  Azure Databricks, Azure Event Hub

  Azure Data Factory, Azure Databricks, Azure Event Hub, Databricks, Databricks Notebooks, Databricks REST API 2.0, EntityPath, Event Hub Apache Connector, Event Hub Connection String, IoT, PySpark, Python, Telemetry

• Programmatically Provision an Azure Databricks Workspace and Cluster using Python Functions.

  Azure Databricks is a data analytics and machine learning platform based on Apache Spark. The first set of tasks to be performed before using Azure Databricks for any kind of Data exploration and machine learning execution is to create a Databricks workspace and Cluster. The following Python functions were developed to enable the automated provision…

  jbernec

  May 16, 2019

  Apache Spark, Azure Automation Account, Azure Databricks, Python

  ARM Templates, Automation, Azure Automation, Azure Databricks, Azure Databricks Cluster, Create Cluster API, Databricks REST API 2.0, Python3, yaml