PySpark/Spark SQL

Install dependencies for Ibis’s Spark dialect:

pip install ibis-framework[spark]

Create a client by passing in the spark session as a parameter to ibis.spark.connect():

con = ibis.spark.connect(spark_session)