Datafusion

Note

The Datafusion backend is experimental

Install

Install ibis along with its dependencies for the datafusion backend:

pip install 'ibis-framework[datafusion]'

or

conda install -c conda-forge ibis-datafusion

Connect

Create a client by passing a dictionary that maps table names to paths to ibis.datafusion.connect():

>>> import ibis
>>> data_sources = {"t": "path/to/file.parquet", "s": "path/to/file.csv"}
>>> client = ibis.datafusion.connect(data_sources)
>>> t = client.table("t")

API

The Datafusion client is accessible through the ibis.datafusion namespace.

Use ibis.datafusion.connect to create a Datafusion client.

Backend.connect(*args, **kwargs)

Return new client object with saved args/kwargs, having called .reconnect() on it.

Backend.database([name])

Return a Database object for the name database.

Backend.list_tables([like, database])

List the available tables.

Backend.table(name[, schema])

Get an ibis expression representing a DataFusion table.

Backend.register_csv(name, path[, schema])

Register a CSV file with with name located at path.

Backend.register_parquet(name, path[, schema])

Register a parquet file with with name located at path.