Tag: Databricks Notebooks

• Ingest Azure Event Hub Telemetry Data with Apache PySpark Structured Streaming on Databricks.

  Overview. Ingesting, storing and processing millions of telemetry data from a plethora of remote IoT devices and Sensors has become common place. One of the primary Cloud services used to process streaming telemetry events at scale is Azure Event Hub. Most documented implementations of Azure Databricks Ingestion from Azure Event Hub Data are based on…

  jbernec

  May 17, 2021

  Azure Databricks, Azure Event Hub

  Azure Data Factory, Azure Databricks, Azure Event Hub, Databricks, Databricks Notebooks, Databricks REST API 2.0, EntityPath, Event Hub Apache Connector, Event Hub Connection String, IoT, PySpark, Python, Telemetry

• Configure a Databricks Cluster-scoped Init Script in Visual Studio Code.

  Databricks is a distributed data analytics and processing platform designed to run in the Cloud. This platform is built on Apache Spark which is currently at version 2.4.4. In this post, I will demonstrate the deployment and installation of custom R based machine learning packages into Azure Databricks Clusters using Cluster Init Scripts. So, what…

  jbernec

  March 2, 2020

  Apache Spark, Bash, Cluster Init Scripts, Databricks Notebooks, Install.packages(), Logs, R, Shell

  Apache Spark, Azure Databricks, Bash, Cluster Init Scripts, Databricks CLI, Databricks Notebooks, Install.packages(), Logs, R