Dask

Experimental

The Dask backend is experimental and is subject to backwards incompatible changes.

Install

Install ibis and dependencies for the Dask backend:

pipcondamamba

pip install 'ibis-framework[dask]'

conda install -c conda-forge ibis-dask

mamba install -c conda-forge ibis-dask

Connect

API

Create a client by passing in a dictionary of paths to ibis.dask.connect.

See ibis.backends.dask.Backend.do_connect for connection parameter information.

ibis.dask.connect is a thin wrapper around ibis.backends.dask.Backend.do_connect.

Connection Parameters

do_connect(self, dictionary)

Construct a Dask backend client from a dictionary of data sources.

Parameters:

Name	Type	Description	Default
dictionary	MutableMapping[str, dd.DataFrame]	Mapping from str table names to Dask DataFrames.	required

Examples:

>>> import ibis
>>> data = {"t": "path/to/file.parquet", "s": "path/to/file.csv"}
>>> ibis.dask.connect(data)

Backend API

Backend (BasePandasBackend)

Attributes

current_database inherited property readonly

Return the name of the current database.

Backends that don't support different databases will return None.

Returns:

Type	Description
str | None	Name of the current database.

version property readonly

Return the version of the backend engine.

For database servers, return the server version.

For others such as SQLite and pandas return the version of the underlying library or application.

Returns:

Type	Description
str	The backend version

Classes

Options (BaseModel) inherited pydantic-model

Attributes

enable_trace: bool pydantic-field

Enable tracing for execution.

Methods

add_operation(self, operation) inherited

Add a translation function to the backend for a specific operation.

Operations are defined in ibis.expr.operations, and a translation function receives the translator object and an expression as parameters, and returns a value depending on the backend. For example, in SQL backends, a NullLiteral operation could be translated to the string "NULL".

Examples:

>>> @ibis.sqlite.add_operation(ibis.expr.operations.NullLiteral)
... def _null_literal(translator, expression):
...     return 'NULL'

compile(self, query, params=None, **kwargs)

Compile expr.

connect(self, *args, **kwargs) inherited

Connect to the database.

Parameters:

Name	Type	Description	Default
args	None	Connection parameters	()
kwargs	None	Additional connection parameters	{}

Returns:

Type	Description
BaseBackend	An instance of the backend

create_database(self, name, force=False) inherited

Create a new database.

Not all backends implement this method.

Parameters:

Name	Type	Description	Default
name	str	Name of the new database.	required
force	bool	If False, an exception is raised if the database already exists.	False

create_table(self, table_name, obj=None, schema=None)

Create a table.

create_view(self, name, expr, database=None) inherited

Create a view.

Parameters:

Name	Type	Description	Default
name	str	Name for the new view.	required
expr	ir.TableExpr	An Ibis table expression that will be used to extract the query of the view.	required
database	str | None	Name of the database where the view will be created, if not the default.	None

database(self, name=None) inherited

Return a Database object for the name database.

DEPRECATED: database is deprecated; use equivalent methods in the backend

Parameters:

Name	Type	Description	Default
name	None	Name of the database to return the object for.	None

Returns:

Type	Description
Database	A database object for the specified database.

drop_table(self, name, database=None, force=False) inherited

Drop a table.

Parameters:

Name	Type	Description	Default
name	str	Name of the table to drop.	required
database	str | None	Name of the database where the table exists, if not the default.	None
force	bool	If False, an exception is raised if the table does not exist.	False

drop_view(self, name, database=None, force=False) inherited

Drop a view.

Parameters:

Name	Type	Description	Default
name	str	Name of the view to drop.	required
database	str | None	Name of the database where the view exists, if not the default.	None
force	bool	If False, an exception is raised if the view does not exist.	False

execute(self, query, params=None, limit='default', **kwargs)

Execute an expression.

exists_database(self, name) inherited

Return whether a database name exists in the current connection.

DEPRECATED: exists_database is deprecated as of v2.0; use name in client.list_databases()

Parameters:

Name	Type	Description	Default
name	str	Database to check for existence	required

Returns:

Type	Description
bool	Whether name exists

exists_table(self, name, database=None) inherited

Return whether a table name exists in the database.

DEPRECATED: exists_table is deprecated as of v2.0; use name in client.list_tables()

Parameters:

Name	Type	Description	Default
name	str	Table name	required
database	str | None	Database to check if given	None

Returns:

Type	Description
bool	Whether name is a table

from_dataframe(self, df, name='df', client=None) inherited

Construct an ibis table from a pandas DataFrame.

Parameters:

Name	Type	Description	Default
df	pd.DataFrame	A pandas DataFrame	required
name	str	The name of the pandas DataFrame	'df'
client	BasePandasBackend | None	Client dictionary will be mutated with the name of the DataFrame, if not provided a new client is created	None

Returns:

Type	Description
ir.TableExpr	A table expression

list_databases(self, like=None) inherited

List existing databases in the current connection.

Parameters:

Name	Type	Description	Default
like	None	A pattern in Python's regex format to filter returned database names.	None

Returns:

Type	Description
list[str]	The database names that exist in the current connection, that match the like pattern if provided.

list_tables(self, like=None, database=None) inherited

Return the list of table names in the current database.

For some backends, the tables may be files in a directory, or other equivalent entities in a SQL database.

Parameters:

Name	Type	Description	Default
like	str	A pattern in Python's regex format.	None
database	str	The database to list tables of, if not the current one.	None

Returns:

Type	Description
list[str]	The list of the table names that match the pattern like.

table(self, name, schema=None) inherited

Return a table expression from the database.

DEPRECATED: table is deprecated as of v2.0; change the current database before calling .table()

verify(self, expr, params=None) inherited

Verify expr is an expression that can be compiled.

DEPRECATED: verify is deprecated as of v2.0; compile and capture TranslationError instead

---

Last update: March 1, 2022