Skip to content

Base Expression Types

These APIs are shared by both table and column expressions.

Expr

Bases: Immutable

Base expression class.

Functions

as_table()

Convert an expression to a table.

compile(limit=None, timecontext=None, params=None)

Compile to an execution target.

Parameters:

Name Type Description Default
limit int | None

An integer to effect a specific row limit. A value of None means "no limit". The default is in ibis/config.py.

None
timecontext TimeContext | None

Defines a time range of (begin, end). When defined, the execution will only compute result for data inside the time range. The time range is inclusive of both endpoints. This is conceptually same as a time filter. The time column must be named 'time' and should preserve across the expression. For example, if that column is dropped then execute will result in an error.

None
params Mapping[ir.Value, Any] | None

Mapping of scalar parameter expressions to value

None

equals(other)

Return whether this expression is structurally equivalent to other.

If you want to produce an equality expression, use == syntax.

Parameters:

Name Type Description Default
other

Another expression

required

Examples:

>>> import ibis
>>> t1 = ibis.table(dict(a="int"), name="t")
>>> t2 = ibis.table(dict(a="int"), name="t")
>>> t1.equals(t2)
True
>>> v = ibis.table(dict(a="string"), name="v")
>>> t1.equals(v)
False

execute(limit='default', timecontext=None, params=None, **kwargs)

Execute an expression against its backend if one exists.

Parameters:

Name Type Description Default
limit int | str | None

An integer to effect a specific row limit. A value of None means "no limit". The default is in ibis/config.py.

'default'
timecontext TimeContext | None

Defines a time range of (begin, end). When defined, the execution will only compute result for data inside the time range. The time range is inclusive of both endpoints. This is conceptually same as a time filter. The time column must be named 'time' and should preserve across the expression. For example, if that column is dropped then execute will result in an error.

None
params Mapping[ir.Value, Any] | None

Mapping of scalar parameter expressions to value

None
kwargs Any

Keyword arguments

{}

get_name()

Return the name of this expression.

has_name()

Check whether this expression has an explicit name.

pipe(f, *args, **kwargs)

Compose f with self.

Parameters:

Name Type Description Default
f

If the expression needs to be passed as anything other than the first argument to the function, pass a tuple with the argument name. For example, (f, 'data') if the function f expects a 'data' keyword

required
args Any

Positional arguments to f

()
kwargs Any

Keyword arguments to f

{}

Examples:

>>> import ibis
>>> t = ibis.table([('a', 'int64'), ('b', 'string')], name='t')
>>> f = lambda a: (a + 1).name('a')
>>> g = lambda a: (a * 2).name('a')
>>> result1 = t.a.pipe(f).pipe(g)
>>> result1
r0 := UnboundTable: t
  a int64
  b string
a: r0.a + 1 * 2
>>> result2 = g(f(t.a))  # equivalent to the above
>>> result1.equals(result2)
True

Returns:

Type Description
Expr

Result type of passed function

to_csv(path, *, params=None, **kwargs)

Write the results of executing the given expression to a CSV file

This method is eager and will execute the associated expression immediately.

Parameters:

Name Type Description Default
path str | Path

The data source. A string or Path to the CSV file.

required
params Mapping[ir.Scalar, Any] | None

Mapping of scalar parameter expressions to value.

None
**kwargs Any

Additional keyword arguments passed to pyarrow.csv.CSVWriter

{}

to_parquet(path, *, params=None, **kwargs)

Write the results of executing the given expression to a parquet file

This method is eager and will execute the associated expression immediately.

Parameters:

Name Type Description Default
path str | Path

The data source. A string or Path to the parquet file.

required
params Mapping[ir.Scalar, Any] | None

Mapping of scalar parameter expressions to value.

None
**kwargs Any

Additional keyword arguments passed to pyarrow.parquet.ParquetWriter

{}

Examples:

Write out an expression to a single parquet file.

>>> import ibis
>>> penguins = ibis.examples.penguins.fetch()
>>> penguins.to_parquet("penguins.parquet")

Write out an expression to a hive-partitioned parquet file.

>>> import ibis
>>> penguins = ibis.examples.penguins.fetch()
>>> # partition on single column
>>> penguins.to_parquet("penguins_hive_dir", partition_by="year")
>>> # partition on multiple columns
>>> penguins.to_parquet("penguins_hive_dir", partition_by=("year", "island"))

Hive-partitioned output is currently only supported when using DuckDB

to_pyarrow(*, params=None, limit=None, **kwargs)

Execute expression and return results in as a pyarrow table.

This method is eager and will execute the associated expression immediately.

Parameters:

Name Type Description Default
params Mapping[ir.Scalar, Any] | None

Mapping of scalar parameter expressions to value.

None
limit int | str | None

An integer to effect a specific row limit. A value of None means "no limit". The default is in ibis/config.py.

None
kwargs Any

Keyword arguments

{}

Returns:

Type Description
Table

A pyarrow table holding the results of the executed expression.

to_pyarrow_batches(*, limit=None, params=None, chunk_size=1000000, **kwargs)

Execute expression and return a RecordBatchReader.

This method is eager and will execute the associated expression immediately.

Parameters:

Name Type Description Default
limit int | str | None

An integer to effect a specific row limit. A value of None means "no limit". The default is in ibis/config.py.

None
params Mapping[ir.Value, Any] | None

Mapping of scalar parameter expressions to value.

None
chunk_size int

Maximum number of rows in each returned record batch.

1000000
kwargs Any

Keyword arguments

{}

Returns:

Type Description
results

RecordBatchReader

unbind()

Return an expression built on UnboundTable instead of backend-specific objects.

visualize(format='svg', *, label_edges=False, verbose=False)

Visualize an expression as a GraphViz graph in the browser.

Parameters:

Name Type Description Default
format str

Image output format. These are specified by the graphviz Python library.

'svg'
label_edges bool

Show operation input names as edge labels

False
verbose bool

Print the graphviz DOT code to stderr if True

False

Raises:

Type Description
ImportError

If graphviz is not installed.


Last update: February 3, 2022