Skip to content

Column Selectors

selectors

Convenient column selectors.

Rationale

Column selectors are convenience functions for selecting columns that share some property.

Discussion

For example, a common task is to be able to select all numeric columns for a subsequent computation.

Without selectors this becomes quite verbose and tedious to write:

>>> t.select([t[c] for c in t.columns if t[c].type().is_numeric()])  # doctest: +SKIP

Compare that to the numeric selector:

>>> t.select(s.numeric())  # doctest: +SKIP

When there are multiple properties to check it gets worse:

>>> t.select(  # doctest: +SKIP
...     [
...         t[c] for c in t.columns
...         if t[c].type().is_numeric()
...         if ("a" in c.get_name() or "cd" in c.get_name())
...     ]
... )

Using a composition of selectors this is much less tiresome:

>>> t.select(s.numeric() & s.contains(("a", "cd")))  # doctest: +SKIP

Classes

Predicate

Bases: Selector

Functions
__and__(other)

Compute the conjunction of two Selectors.

Parameters:

Name Type Description Default
other Selector

Another selector

required
__invert__()

Compute the logical negation of two Selectors.

__or__(other)

Compute the disjunction of two Selectors.

Parameters:

Name Type Description Default
other Selector

Another selector

required
expand(table)

Evaluate self.predicate on every column of table.

Parameters:

Name Type Description Default
table ir.Table

An ibis table expression

required

Functions

all()

Return every column from a table.

all_of(*predicates)

Include columns satisfying all of predicates.

any_of(*predicates)

Include columns satisfying any of predicates.

c(*names)

Select specific column names.

contains(needles, how=any)

Return columns whose name contains needles.

Parameters:

Name Type Description Default
needles str | tuple[str, ...]

One or more strings to search for in column names

required
how Callable[[Iterable[bool]], bool]

A boolean reduction to allow the configuration of how needles are summarized.

any

Examples:

Select columns that contain either "a" or "b"

>>> t.select(s.contains(("a", "b")))

Select columns that contain all of "a" and "b"

>>> t.select(s.contains(("a", "b"), how=all))
See Also

matches

endswith(suffixes)

Select columns whose name ends with one of suffixes.

Parameters:

Name Type Description Default
suffixes str | tuple[str, ...]

Suffixes to compare column names against

required
See Also

startswith

first()

Return the first column of a table.

last()

Return the last column of a table.

matches(regex)

Return columns whose name matches the regular expression regex.

Parameters:

Name Type Description Default
regex str | re.Pattern

A string or re.Pattern object

required

Examples:

>>> t.select(s.matches(r"ab+"))
See Also

contains

numeric()

Return numeric columns.

Examples:

>>> import ibis
>>> import ibis.expr.selectors as s
>>> t = ibis.table(dict(a="int", b="string", c="array<string>"), name="t")
>>> t
UnboundTable: t
  a int64
  b string
  c array<string>
>>> expr = t.select(s.numeric())  # `a` has integer type, so it's numeric
>>> expr.columns
['a']
See Also

of_type

of_type(dtype)

Select columns of type dtype.

Parameters:

Name Type Description Default
dtype dt.DataType | str | type[dt.DataType]

DataType instance, str or DataType class

required

Examples:

Select according to a specific DataType instance

>>> t.select(s.of_type(dt.Array(dt.string)))

Strings are also accepted

>>> t.select(s.of_type("map<string, float>"))

Select by category of DataType by passing the DataType class

>>> t.select(s.of_type(dt.Struct))
See Also

numeric

startswith(prefixes)

Select columns whose name starts with one of prefixes.

Parameters:

Name Type Description Default
prefixes str | tuple[str, ...]

Prefixes to compare column names against

required

Examples:

>>> import ibis
>>> import ibis.expr.selectors as s
>>> t = ibis.table(dict(apples="int", oranges="float", bananas="bool"), name="t")
>>> expr = t.select(s.startswith(("a", "b")))
>>> expr.columns
['apples', 'bananas']
See Also

endswith

where(predicate)

Return columns that satisfy predicate.

Use this selector when one of the other selectors does not meet your needs.

Parameters:

Name Type Description Default
predicate Callable[[ir.Value], bool]

A callable that accepts an ibis value expression and returns a bool

required

Examples:

>>> import ibis
>>> import ibis.expr.selectors as s
>>> t = ibis.table(dict(a="float32"), name="t")
>>> expr = t.select(s.where(lambda col: col.get_name() == "a"))
>>> expr.columns
['a']

Last update: January 24, 2023