Tag: Python Virtual Environmet

• Data Preparation of PySpark Dataframes in Azure Databricks Cluster using Databricks Connect.

  In my limited experience with processing big data workloads on the Azure Databricks platform powered by Apache Spark, it has become obvious that a significant part of the tasks are targeted towards Data Quality. Data quality in this context mostly refers to having data that is free of errors, inconsistencies, redundancies, poor formatting and other…

  jbernec

  March 1, 2020

  Azure Databricks

  Apache Spark, APIs, CSV, Databricks-Connect, DataCompy, Dataframes, Jupyter Notebook, PySpark, Python, Python Virtual Environmet, Venv

• Setting Up Jupyter Notebook to Run in a Python Virtual Environment.

  1) Install Jupyter on the local machine outside of any existing Python Virtual environment: pip install jupyter –no-cach-dir 2) Create a Python Virtual environment. mkdir virtualenv cd virtualenv python.exe -m venv dbconnect 3) Change directory into the virtual environment and activate .\scripts\activate 4) Install ipykernel package in the virtual environment pip install ipykernel 5) Use…

  jbernec

  May 20, 2019

  Python

  Jupyter Kernel, Jupyter Notebook, Python Virtual Environmet, Venv