Operation support matrix
We provide Ibis’s operation support matrix as a Streamlit app that shows supported operations for each backend. Ibis defines a common API for analytics and data transformation code that is transpiled to native code for each backend. This code is often, but not always, SQL – see the backends concept page for details. Due to differences in SQL dialects and support for different operations in different backends, support for the full breadth of the Ibis API varies.
You can use this page to see which operations are supported on each backend.
Backends with low coverage are good places to start contributing!
Each backend implements operations differently, but this is usually very similar to other backends. If you want to start contributing to ibis, it’s a good idea to start by adding missing operations to backends that have low operation coverage.
This app is built using streamlit
.
You can develop the app locally by editing docs/backends/app/backend_info_app.py
and opening a PR with your changes.
Test your changes locally by running
$ streamlit run docs/backends/app/backend_info_app.py
The changes will show up in the dev docs when your PR is merged!
Raw Data
You can also download data from the above tables in CSV format.
The code used to generate the linked CSV file is below.
from __future__ import annotations
from pathlib import Path
import pandas as pd
import ibis
import ibis.expr.operations as ops
def get_backends(exclude=()):
= sorted(ep.name for ep in ibis.util.backend_entry_points())
entry_points return [
getattr(ibis, backend))
(backend, for backend in entry_points
if backend not in exclude
]
def get_leaf_classes(op):
for child_class in op.__subclasses__():
if not child_class.__subclasses__():
yield child_class
else:
yield from get_leaf_classes(child_class)
def main():
= {
internal_ops # Never translates into anything
ops.UnresolvedExistsSubquery,
ops.UnresolvedNotExistsSubquery,
ops.ScalarParameter,
}
= frozenset(get_leaf_classes(ops.Value)) - internal_ops
public_ops = {"operation": [f"{op.__module__}.{op.__name__}" for op in public_ops]}
support
support.update(list(map(backend.has_operation, public_ops)))
(name, # exclude flink until https://github.com/apache/flink/pull/23141 is
# merged and released we also need to roll it into poetry
for name, backend in get_backends(exclude=("flink",))
)
= pd.DataFrame(support).set_index("operation").sort_index()
df
with Path(ibis.__file__).parents[1].joinpath(
"docs", "backends", "raw_support_matrix.csv"
open(mode="w") as f:
).="FullOperation")
df.to_csv(f, index_label
if __name__ == "__main__":
main()