Dask¶
Experimental
The Dask backend is experimental and is subject to backwards incompatible changes.
Install¶
Install ibis and dependencies for the Dask backend:
pip install 'ibis-framework[dask]'
conda install -c conda-forge ibis-dask
mamba install -c conda-forge ibis-dask
Connect¶
API¶
Create a client by passing in a dictionary of paths to ibis.dask.connect
.
See ibis.backends.dask.Backend.do_connect
for connection parameter information.
ibis.dask.connect
is a thin wrapper around ibis.backends.dask.Backend.do_connect
.
Connection Parameters¶
do_connect(self, dictionary)
¶
Construct a Dask backend client from a dictionary of data sources.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
dictionary |
MutableMapping[str, dd.DataFrame] |
Mapping from |
required |
Examples:
>>> import ibis
>>> data = {"t": "path/to/file.parquet", "s": "path/to/file.csv"}
>>> ibis.dask.connect(data)
Backend API¶
Backend (BasePandasBackend)
¶
Attributes¶
current_database
inherited
property
readonly
¶
Return the name of the current database.
Backends that don't support different databases will return None.
Returns:
Type | Description |
---|---|
str | None |
Name of the current database. |
version
property
readonly
¶
Return the version of the backend engine.
For database servers, return the server version.
For others such as SQLite and pandas return the version of the underlying library or application.
Returns:
Type | Description |
---|---|
str |
The backend version |
Classes¶
Options (BaseModel)
inherited
pydantic-model
¶
Methods¶
add_operation(self, operation)
inherited
¶
Add a translation function to the backend for a specific operation.
Operations are defined in ibis.expr.operations
, and a translation
function receives the translator object and an expression as
parameters, and returns a value depending on the backend. For example,
in SQL backends, a NullLiteral operation could be translated to the
string "NULL"
.
Examples:
>>> @ibis.sqlite.add_operation(ibis.expr.operations.NullLiteral)
... def _null_literal(translator, expression):
... return 'NULL'
compile(self, query, params=None, **kwargs)
¶
Compile expr
.
connect(self, *args, **kwargs)
inherited
¶
Connect to the database.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
args |
None |
Connection parameters |
() |
kwargs |
None |
Additional connection parameters |
{} |
Returns:
Type | Description |
---|---|
BaseBackend |
An instance of the backend |
create_database(self, name, force=False)
inherited
¶
Create a new database.
Not all backends implement this method.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
str |
Name of the new database. |
required |
force |
bool |
If |
False |
create_table(self, table_name, obj=None, schema=None)
¶
Create a table.
create_view(self, name, expr, database=None)
inherited
¶
Create a view.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
str |
Name for the new view. |
required |
expr |
ir.TableExpr |
An Ibis table expression that will be used to extract the query of the view. |
required |
database |
str | None |
Name of the database where the view will be created, if not the default. |
None |
database(self, name=None)
inherited
¶
Return a Database
object for the name
database.
DEPRECATED: database
is deprecated; use equivalent methods in the backend
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
None |
Name of the database to return the object for. |
None |
Returns:
Type | Description |
---|---|
Database |
A database object for the specified database. |
drop_table(self, name, database=None, force=False)
inherited
¶
Drop a table.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
str |
Name of the table to drop. |
required |
database |
str | None |
Name of the database where the table exists, if not the default. |
None |
force |
bool |
If |
False |
drop_view(self, name, database=None, force=False)
inherited
¶
Drop a view.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
str |
Name of the view to drop. |
required |
database |
str | None |
Name of the database where the view exists, if not the default. |
None |
force |
bool |
If |
False |
execute(self, query, params=None, limit='default', **kwargs)
¶
Execute an expression.
exists_database(self, name)
inherited
¶
Return whether a database name exists in the current connection.
DEPRECATED: exists_database
is deprecated as of v2.0; use name in client.list_databases()
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
str |
Database to check for existence |
required |
Returns:
Type | Description |
---|---|
bool |
Whether |
exists_table(self, name, database=None)
inherited
¶
Return whether a table name exists in the database.
DEPRECATED: exists_table
is deprecated as of v2.0; use name in client.list_tables()
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
str |
Table name |
required |
database |
str | None |
Database to check if given |
None |
Returns:
Type | Description |
---|---|
bool |
Whether |
from_dataframe(self, df, name='df', client=None)
inherited
¶
Construct an ibis table from a pandas DataFrame.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
df |
pd.DataFrame |
A pandas DataFrame |
required |
name |
str |
The name of the pandas DataFrame |
'df' |
client |
BasePandasBackend | None |
Client dictionary will be mutated with the name of the DataFrame, if not provided a new client is created |
None |
Returns:
Type | Description |
---|---|
ir.TableExpr |
A table expression |
list_databases(self, like=None)
inherited
¶
List existing databases in the current connection.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
like |
None |
A pattern in Python's regex format to filter returned database names. |
None |
Returns:
Type | Description |
---|---|
list[str] |
The database names that exist in the current connection, that match
the |
list_tables(self, like=None, database=None)
inherited
¶
Return the list of table names in the current database.
For some backends, the tables may be files in a directory, or other equivalent entities in a SQL database.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
like |
str |
A pattern in Python's regex format. |
None |
database |
str |
The database to list tables of, if not the current one. |
None |
Returns:
Type | Description |
---|---|
list[str] |
The list of the table names that match the pattern |
table(self, name, schema=None)
inherited
¶
Return a table expression from the database.
DEPRECATED: table
is deprecated as of v2.0; change the current database before calling .table()
verify(self, expr, params=None)
inherited
¶
Verify expr
is an expression that can be compiled.
DEPRECATED: verify
is deprecated as of v2.0; compile
and capture TranslationError
instead