Column Selectors

selectors

Convenient column selectors.

Rationale

Column selectors are convenience functions for selecting columns that share some property.

Discussion

For example, a common task is to be able to select all numeric columns for a subsequent computation.

Without selectors this becomes quite verbose and tedious to write:

>>> t.select([t[c] for c in t.columns if t[c].type().is_numeric()])  # doctest: +SKIP

Compare that to the numeric selector:

>>> t.select(s.numeric())  # doctest: +SKIP

When there are multiple properties to check it gets worse:

>>> t.select(  # doctest: +SKIP
...     [
...         t[c] for c in t.columns
...         if t[c].type().is_numeric()
...         if ("a" in c.get_name() or "cd" in c.get_name())
...     ]
... )

Using a composition of selectors this is much less tiresome:

>>> t.select(s.numeric() & s.contains(("a", "cd")))  # doctest: +SKIP

Classes

Predicate

Bases: Selector

Functions

__and__(other)

Compute the conjunction of two Selectors.

Parameters:

Name	Type	Description	Default
other	Selector	Another selector	required

__invert__()

Compute the logical negation of two Selectors.

__or__(other)

Compute the disjunction of two Selectors.

Parameters:

Name	Type	Description	Default
other	Selector	Another selector	required

expand(table)

Evaluate self.predicate on every column of table.

Parameters:

Name	Type	Description	Default
table	ir.Table	An ibis table expression	required

Functions

all()

Return every column from a table.

all_of(*predicates)

Include columns satisfying all of predicates.

any_of(*predicates)

Include columns satisfying any of predicates.

c(*names)

Select specific column names.

contains(needles, how=any)

Return columns whose name contains needles.

Parameters:

Name	Type	Description	Default
needles	str | tuple[str, ...]	One or more strings to search for in column names	required
how	Callable[[Iterable[bool]], bool]	A boolean reduction to allow the configuration of how needles are summarized.	any

Examples:

Select columns that contain either "a" or "b"

>>> t.select(s.contains(("a", "b")))

Select columns that contain all of "a" and "b"

>>> t.select(s.contains(("a", "b"), how=all))

See Also

matches

endswith(suffixes)

Select columns whose name ends with one of suffixes.

Parameters:

Name	Type	Description	Default
suffixes	str | tuple[str, ...]	Suffixes to compare column names against	required

See Also

startswith

first()

Return the first column of a table.

last()

Return the last column of a table.

matches(regex)

Return columns whose name matches the regular expression regex.

Parameters:

Name	Type	Description	Default
regex	str | re.Pattern	A string or re.Pattern object	required

Examples:

>>> t.select(s.matches(r"ab+"))

See Also

contains

numeric()

Return numeric columns.

Examples:

>>> import ibis
>>> import ibis.expr.selectors as s
>>> t = ibis.table(dict(a="int", b="string", c="array<string>"), name="t")
>>> t
UnboundTable: t
  a int64
  b string
  c array<string>
>>> expr = t.select(s.numeric())  # `a` has integer type, so it's numeric
>>> expr.columns
['a']

See Also

of_type

of_type(dtype)

Select columns of type dtype.

Parameters:

Name	Type	Description	Default
dtype	dt.DataType | str | type[dt.DataType]	DataType instance, str or DataType class	required

Examples:

Select according to a specific DataType instance

>>> t.select(s.of_type(dt.Array(dt.string)))

Strings are also accepted

>>> t.select(s.of_type("map<string, float>"))

Select by category of DataType by passing the DataType class

>>> t.select(s.of_type(dt.Struct))

See Also

numeric

startswith(prefixes)

Select columns whose name starts with one of prefixes.

Parameters:

Name	Type	Description	Default
prefixes	str | tuple[str, ...]	Prefixes to compare column names against	required

Examples:

>>> import ibis
>>> import ibis.expr.selectors as s
>>> t = ibis.table(dict(apples="int", oranges="float", bananas="bool"), name="t")
>>> expr = t.select(s.startswith(("a", "b")))
>>> expr.columns
['apples', 'bananas']

See Also

endswith

where(predicate)

Return columns that satisfy predicate.

Use this selector when one of the other selectors does not meet your needs.

Parameters:

Name	Type	Description	Default
predicate	Callable[[ir.Value], bool]	A callable that accepts an ibis value expression and returns a bool	required

Examples:

>>> import ibis
>>> import ibis.expr.selectors as s
>>> t = ibis.table(dict(a="float32"), name="t")
>>> expr = t.select(s.where(lambda col: col.get_name() == "a"))
>>> expr.columns
['a']

---

Last update: January 24, 2023