Complex Type Expressions¶

These APIs are available on arrays, maps and structs.

`ArrayValue` ¶

Bases: Value

Functions¶

`add(other)` ¶

Concatenate this array with another.

Parameters:

Name	Type	Description	Default
`other`	`ArrayValue`	Array to concat with `self`	required

Returns:

Type	Description
`ArrayValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ ∅                    │
└──────────────────────┘
>>> t.a + t.a
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayConcat(a, a)    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7, 7]               │
│ [3, 3]               │
│ ∅                    │
└──────────────────────┘
>>> t.a + [4]
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayConcat(a, (4,)) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7, 4]               │
│ [3, 4]               │
│ [4]                  │
└──────────────────────┘

`getitem(index)` ¶

Extract one or more elements of self.

Parameters:

Name	Type	Description	Default
`index`	`int \| ir.IntegerValue \| slice`	Index into `array`	required

Returns:

Type	Description
`Value`	If `index` is an `int` or `IntegerValue` then the return type is the element type of `self`. If `index` is a `slice` then the return type is the same type as the input.

Examples:

Extract a single element

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7, 42], [3], None]})
>>> t.a[0]
┏━━━━━━━━━━━━━━━━━━┓
┃ ArrayIndex(a, 0) ┃
┡━━━━━━━━━━━━━━━━━━┩
│ int8             │
├──────────────────┤
│                7 │
│                3 │
│                ∅ │
└──────────────────┘

Extract a range of elements

>>> t = ibis.memtable({"a": [[7, 42, 72], [3] * 5, None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7, 42, ... +1]      │
│ [3, 3, ... +3]       │
│ ∅                    │
└──────────────────────┘
>>> t.a[1:2]
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArraySlice(a, 1, 2)  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [42]                 │
│ [3]                  │
│ ∅                    │
└──────────────────────┘

`mul(n)` ¶

Repeat this array n times.

Parameters:

Name	Type	Description	Default
`n`	`int \| ir.IntegerValue`	Number of times to repeat `self`.	required

Returns:

Type	Description
`ArrayValue`	`self` repeated `n` times

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ ∅                    │
└──────────────────────┘
>>> t.a * 2
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayRepeat(a, 2)    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7, 7]               │
│ [3, 3]               │
│ []                   │
└──────────────────────┘

`radd(other)` ¶

Concatenate this array with another.

Parameters:

Name	Type	Description	Default
`other`	`ArrayValue`	Array to concat with `self`	required

Returns:

Type	Description
`ArrayValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ ∅                    │
└──────────────────────┘
>>> [4] + t.a
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayConcat((4,), a) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [4, 7]               │
│ [4, 3]               │
│ [4]                  │
└──────────────────────┘

`rmul(n)` ¶

Repeat this array n times.

Parameters:

Name	Type	Description	Default
`n`	`int \| ir.IntegerValue`	Number of times to repeat `self`.	required

Returns:

Type	Description
`ArrayValue`	`self` repeated `n` times

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ ∅                    │
└──────────────────────┘
>>> 2 * t.a
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayRepeat(a, 2)    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7, 7]               │
│ [3, 3]               │
│ []                   │
└──────────────────────┘

`filter(predicate)` ¶

Filter array elements using predicate.

Parameters:

Name	Type	Description	Default
`predicate`	`Callable[[ir.Value], ir.BooleanValue]`	Function to use to filter array elements	required

Returns:

Type	Description
`ArrayValue`	Array elements filtered using `predicate`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[1, None, 2], [4], []]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [1, None, ... +1]    │
│ [4]                  │
│ []                   │
└──────────────────────┘
>>> t.a.filter(lambda x: x > 1)
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayFilter(a)       ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [2]                  │
│ [4]                  │
│ []                   │
└──────────────────────┘

`join(sep)` ¶

Join the elements of this array expression with sep.

Parameters:

Name	Type	Description	Default
`sep`	`str \| ir.StringValue`	Separator to use for joining array elements	required

Returns:

Type	Description
`StringValue`	Elements of `self` joined with `sep`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr": [["a", "b", "c"], None, [], ["b", None]]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<string>        │
├──────────────────────┤
│ ['a', 'b', ... +1]   │
│ ∅                    │
│ []                   │
│ ['b', None]          │
└──────────────────────┘
>>> t.arr.join("|")
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayStringJoin('|', arr) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ string                    │
├───────────────────────────┤
│ a|b|c                     │
│ ∅                         │
│ ∅                         │
│ b                         │
└───────────────────────────┘

`length()` ¶

Compute the length of an array.

Returns:

Type	Description
`IntegerValue`	The integer length of each element of `self`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7, 42], [3], None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7, 42]              │
│ [3]                  │
│ ∅                    │
└──────────────────────┘
>>> t.a.length()
┏━━━━━━━━━━━━━━━━┓
┃ ArrayLength(a) ┃
┡━━━━━━━━━━━━━━━━┩
│ int64          │
├────────────────┤
│              2 │
│              1 │
│              ∅ │
└────────────────┘

`map(func)` ¶

Apply a callable func to each element of this array expression.

Parameters:

Name	Type	Description	Default
`func`	`Callable[[ir.Value], ir.Value]`	Function to apply to each element of this array	required

Returns:

Type	Description
`ArrayValue`	`func` applied to every element of this array expression.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[1, None, 2], [4], []]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [1, None, ... +1]    │
│ [4]                  │
│ []                   │
└──────────────────────┘
>>> t.a.map(lambda x: (x + 100).cast("float"))
┏━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayMap(a)           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━┩
│ array<float64>        │
├───────────────────────┤
│ [101.0, None, ... +1] │
│ [104.0]               │
│ []                    │
└───────────────────────┘

`unnest()` ¶

Flatten an array into a column.

This operation changes the cardinality of the result

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7, 42], [3, 3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int8>          │
├──────────────────────┤
│ [7, 42]              │
│ [3, 3]               │
│ ∅                    │
└──────────────────────┘
>>> t.a.unnest()
┏━━━━━━┓
┃ a    ┃
┡━━━━━━┩
│ int8 │
├──────┤
│    7 │
│   42 │
│    3 │
│    3 │
└──────┘

Returns:

Type	Description
`ir.Value`	Unnested array

`StructValue` ¶

Bases: Value

Attributes¶

`fields: Mapping[str, dt.DataType]` `property` ¶

Return a mapping from field name to field type of the struct.

`names: Sequence[str]` `property` ¶

Return the field names of the struct.

`types: Sequence[dt.DataType]` `property` ¶

Return the field types of the struct.

Functions¶

`getattr(name)` ¶

Extract the name field from this struct.

`getitem(name)` ¶

Extract the name field from this struct.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the field to access.	required

Returns:

Type	Description
`Value`	An expression with the type of the field being accessed.

Examples:

>>> import ibis
>>> s = ibis.struct(dict(fruit="pear", weight=0))
>>> s['fruit']
fruit: StructField(...)

`destructure()` ¶

Destructure a StructValue into the corresponding struct fields.

When assigned, a destruct value will be destructured and assigned to multiple columns.

Returns:

Type	Description
`list[AnyValue]`	Value expressions corresponding to the struct fields.

`lift()` ¶

Project the fields of self into a table.

This method is useful when analyzing data that has deeply nested structs or arrays of structs. lift can be chained to avoid repeating column names and table references.

Returns:

Type	Description
`Table`	A projection with this struct expression's fields.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> lines = '''
...     {"pos": {"lat": 10.1, "lon": 30.3}}
...     {"pos": {"lat": 10.2, "lon": 30.2}}
...     {"pos": {"lat": 10.3, "lon": 30.1}}
... '''
>>> with open("/tmp/lines.json", "w") as f:
...     _ = f.write(lines)
>>> t = ibis.read_json("/tmp/lines.json")
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ pos                                ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ struct<lat: float64, lon: float64> │
├────────────────────────────────────┤
│ {'lat': 10.1, 'lon': 30.3}         │
│ {'lat': 10.2, 'lon': 30.2}         │
│ {'lat': 10.3, 'lon': 30.1}         │
└────────────────────────────────────┘
>>> t.pos.lift()
┏━━━━━━━━━┳━━━━━━━━━┓
┃ lat     ┃ lon     ┃
┡━━━━━━━━━╇━━━━━━━━━┩
│ float64 │ float64 │
├─────────┼─────────┤
│    10.1 │    30.3 │
│    10.2 │    30.2 │
│    10.3 │    30.1 │
└─────────┴─────────┘

`MapValue` ¶

Bases: Value

Functions¶

`add(other)` ¶

Concatenate this map with another.

Parameters:

Name	Type	Description	Default
`other`	`MapValue`	Map to concatenate with `self`	required

Returns:

Type	Description
`MapValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> m1 = ibis.map({"a": 1, "b": 2})
>>> m2 = ibis.map({"c": 3, "d": 4})
>>> m1 + m2
MapMerge(...)

`getitem(key)` ¶

Get the value for a given map key.

This operation may have different semantics depending on the backend.

Some backends return NULL when a key is missing, others may fail the query.

Parameters:

Name	Type	Description	Default
`key`	`ir.Value`	A map key	required

Returns:

Type	Description
`Value`	An element with the value type of the map

Examples:

>>> import ibis
>>> m = ibis.map({"a": 1, "b": 2})
>>> m["a"]
MapGet(...)
>>> m["c"]  # note that this does not fail on construction
MapGet(...)

`radd(other)` ¶

Concatenate this map with another.

Parameters:

Name	Type	Description	Default
`other`	`MapValue`	Map to concatenate with `self`	required

Returns:

Type	Description
`MapValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> m1 = ibis.map({"a": 1, "b": 2})
>>> m2 = ibis.map({"c": 3, "d": 4})
>>> m1 + m2
MapMerge(...)

`contains(key)` ¶

Return whether the map contains key.

Parameters:

Name	Type	Description	Default
`key`	`int \| str \| ir.IntegerValue \| ir.StringValue`	Mapping key for which to check	required

Returns:

Type	Description
`BooleanValue`	Boolean indicating the presence of `key` in the map expression

`get(key, default=None)` ¶

Return the value for key from expr.

Return default if key is not in the map.

Parameters:

Name	Type	Description	Default
`key`	`ir.Value`	Expression to use for key	required
`default`	`ir.Value \| None`	Expression to return if `key` is not a key in `expr`	`None`

Returns:

Type	Description
`Value`	The element type of `self`

Examples:

>>> import ibis
>>> m = ibis.map({"a": 1, "b": 2})
>>> m.get("a")
MapGet(...)
>>> m.get("c", 3)
MapGet(...)
>>> m.get("d")
MapGet(...)

`keys()` ¶

Extract the keys of a map.

Returns:

Type	Description
`ArrayValue`	The keys of `self`

Examples:

>>> import ibis
>>> m = ibis.map({"a": 1, "b": 2})
>>> m.keys()
MapKeys(...)

`length()` ¶

Return the number of key-value pairs in the map.

Returns:

Type	Description
`IntegerValue`	The number of elements in `self`

Examples:

>>> import ibis
>>> m = ibis.map({"a": 1, "b": 2})
>>> m.length()
MapLength(...)

`values()` ¶

Extract the values of a map.

Returns:

Type	Description
`ArrayValue`	The values of `self`

Examples:

>>> import ibis
>>> m = ibis.map({"a": 1, "b": 2})
>>> m.values()
MapValues(...)

Last update: August 5, 2022

Complex Type Expressions¶

`ArrayValue` ¶

Functions¶

`add(other)` ¶

`getitem(index)` ¶

`mul(n)` ¶

`radd(other)` ¶

`rmul(n)` ¶

`filter(predicate)` ¶

`join(sep)` ¶

See Also¶

`length()` ¶

`map(func)` ¶

`unnest()` ¶

`StructValue` ¶

Attributes¶

`fields: Mapping[str, dt.DataType]` `property` ¶

`names: Sequence[str]` `property` ¶

`types: Sequence[dt.DataType]` `property` ¶

Functions¶

`getattr(name)` ¶

`getitem(name)` ¶

`destructure()` ¶

`lift()` ¶

See Also¶

`MapValue` ¶

Functions¶

`add(other)` ¶

`getitem(key)` ¶

`radd(other)` ¶

`contains(key)` ¶

`get(key, default=None)` ¶

`keys()` ¶

`length()` ¶

`values()` ¶

Complex Type Expressions¶

ArrayValue ¶

Functions¶

__add__(other) ¶

__getitem__(index) ¶

__mul__(n) ¶

__radd__(other) ¶

__rmul__(n) ¶

filter(predicate) ¶

join(sep) ¶

See Also¶

length() ¶

map(func) ¶

unnest() ¶

StructValue ¶

Attributes¶

fields: Mapping[str, dt.DataType] property ¶

names: Sequence[str] property ¶

types: Sequence[dt.DataType] property ¶

Functions¶

__getattr__(name) ¶

__getitem__(name) ¶

destructure() ¶

lift() ¶

See Also¶

MapValue ¶

Functions¶

__add__(other) ¶

__getitem__(key) ¶

__radd__(other) ¶

contains(key) ¶

get(key, default=None) ¶

keys() ¶

length() ¶

values() ¶

`ArrayValue` ¶

`add(other)` ¶

`getitem(index)` ¶

`mul(n)` ¶

`radd(other)` ¶

`rmul(n)` ¶

`filter(predicate)` ¶

`join(sep)` ¶

`length()` ¶

`map(func)` ¶

`unnest()` ¶

`StructValue` ¶

`fields: Mapping[str, dt.DataType]` `property` ¶

`names: Sequence[str]` `property` ¶

`types: Sequence[dt.DataType]` `property` ¶

`getattr(name)` ¶

`getitem(name)` ¶

`destructure()` ¶

`lift()` ¶

`MapValue` ¶

`add(other)` ¶

`getitem(key)` ¶

`radd(other)` ¶

`contains(key)` ¶

`get(key, default=None)` ¶

`keys()` ¶

`length()` ¶

`values()` ¶