Complex Type Expressions¶

These APIs are available on arrays, maps and structs.

`ArrayValue` ¶

Bases: Value

Functions¶

`add(other)` ¶

Concatenate this array with another.

Parameters:

Name	Type	Description	Default
`other`	`ArrayValue`	Array to concat with `self`	required

Returns:

Type	Description
`ArrayValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ NULL                 │
└──────────────────────┘
>>> t.a + t.a
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayConcat(a, a)    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7, 7]               │
│ [3, 3]               │
│ NULL                 │
└──────────────────────┘
>>> t.a + ibis.literal([4], type="array<int64>")
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayConcat(a, (4,)) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7, 4]               │
│ [3, 4]               │
│ [4]                  │
└──────────────────────┘

`getitem(index)` ¶

Extract one or more elements of self.

Parameters:

Name	Type	Description	Default
`index`	`int \| ir.IntegerValue \| slice`	Index into `array`	required

Returns:

Type	Description
`Value`	If `index` is an `int` or `IntegerValue` then the return type is the element type of `self`. If `index` is a `slice` then the return type is the same type as the input.

Examples:

Extract a single element

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7, 42], [3], None]})
>>> t.a[0]
┏━━━━━━━━━━━━━━━━━━┓
┃ ArrayIndex(a, 0) ┃
┡━━━━━━━━━━━━━━━━━━┩
│ int64            │
├──────────────────┤
│                7 │
│                3 │
│             NULL │
└──────────────────┘

Extract a range of elements

>>> t = ibis.memtable({"a": [[7, 42, 72], [3] * 5, None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7, 42, ... +1]      │
│ [3, 3, ... +3]       │
│ NULL                 │
└──────────────────────┘
>>> t.a[1:2]
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArraySlice(a, 1, 2)  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [42]                 │
│ [3]                  │
│ NULL                 │
└──────────────────────┘

`mul(n)` ¶

Repeat this array n times.

Parameters:

Name	Type	Description	Default
`n`	`int \| ir.IntegerValue`	Number of times to repeat `self`.	required

Returns:

Type	Description
`ArrayValue`	`self` repeated `n` times

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ NULL                 │
└──────────────────────┘
>>> t.a * 2
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayRepeat(a, 2)    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7, 7]               │
│ [3, 3]               │
│ []                   │
└──────────────────────┘

`radd(other)` ¶

Concatenate this array with another.

Parameters:

Name	Type	Description	Default
`other`	`ArrayValue`	Array to concat with `self`	required

Returns:

Type	Description
`ArrayValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ NULL                 │
└──────────────────────┘
>>> ibis.literal([4], type="array<int64>") + t.a
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayConcat((4,), a) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [4, 7]               │
│ [4, 3]               │
│ [4]                  │
└──────────────────────┘

`rmul(n)` ¶

Repeat this array n times.

Parameters:

Name	Type	Description	Default
`n`	`int \| ir.IntegerValue`	Number of times to repeat `self`.	required

Returns:

Type	Description
`ArrayValue`	`self` repeated `n` times

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7], [3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7]                  │
│ [3]                  │
│ NULL                 │
└──────────────────────┘
>>> 2 * t.a
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayRepeat(a, 2)    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7, 7]               │
│ [3, 3]               │
│ []                   │
└──────────────────────┘

`contains(other)` ¶

Return whether the array contains other.

Parameters:

Name	Type	Description	Default
`other`	`ir.Value`	Ibis expression to check for existence of in `self`	required

Returns:

Type	Description
`BooleanValue`	Whether `other` is contained in `self`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr": [[1], [], [42, 42], None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [1]                  │
│ []                   │
│ [42, 42]             │
│ NULL                 │
└──────────────────────┘
>>> t.arr.contains(42)
┏━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayContains(arr, 42) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━┩
│ boolean                │
├────────────────────────┤
│ False                  │
│ False                  │
│ True                   │
│ NULL                   │
└────────────────────────┘
>>> t.arr.contains(None)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayContains(arr, None) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ boolean                  │
├──────────────────────────┤
│ NULL                     │
│ NULL                     │
│ NULL                     │
│ NULL                     │
└──────────────────────────┘

`filter(predicate)` ¶

Filter array elements using predicate.

Parameters:

Name	Type	Description	Default
`predicate`	`Callable[[ir.Value], ir.BooleanValue]`	Function to use to filter array elements	required

Returns:

Type	Description
`ArrayValue`	Array elements filtered using `predicate`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[1, None, 2], [4], []]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [1, None, ... +1]    │
│ [4]                  │
│ []                   │
└──────────────────────┘
>>> t.a.filter(lambda x: x > 1)
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayFilter(a)       ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [2]                  │
│ [4]                  │
│ []                   │
└──────────────────────┘

`index(other)` ¶

Return the position of other in an array.

Parameters:

Name	Type	Description	Default
`other`	`ir.Value`	Ibis expression to existence of in `self`	required

Returns:

Type	Description
`BooleanValue`	The position of `other` in `self`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr": [[1], [], [42, 42], None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [1]                  │
│ []                   │
│ [42, 42]             │
│ NULL                 │
└──────────────────────┘
>>> t.arr.index(42)
┏━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayPosition(arr, 42) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━┩
│ int64                  │
├────────────────────────┤
│                     -1 │
│                     -1 │
│                      0 │
│                   NULL │
└────────────────────────┘
>>> t.arr.index(800)
┏━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayPosition(arr, 800) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ int64                   │
├─────────────────────────┤
│                      -1 │
│                      -1 │
│                      -1 │
│                    NULL │
└─────────────────────────┘
>>> t.arr.index(None)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayPosition(arr, None) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ int64                    │
├──────────────────────────┤
│                     NULL │
│                     NULL │
│                     NULL │
│                     NULL │
└──────────────────────────┘

`join(sep)` ¶

Join the elements of this array expression with sep.

Parameters:

Name	Type	Description	Default
`sep`	`str \| ir.StringValue`	Separator to use for joining array elements	required

Returns:

Type	Description
`StringValue`	Elements of `self` joined with `sep`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr": [["a", "b", "c"], None, [], ["b", None]]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<string>        │
├──────────────────────┤
│ ['a', 'b', ... +1]   │
│ NULL                 │
│ []                   │
│ ['b', None]          │
└──────────────────────┘
>>> t.arr.join("|")
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayStringJoin('|', arr) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ string                    │
├───────────────────────────┤
│ a|b|c                     │
│ NULL                      │
│ NULL                      │
│ b                         │
└───────────────────────────┘

`length()` ¶

Compute the length of an array.

Returns:

Type	Description
`IntegerValue`	The integer length of each element of `self`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7, 42], [3], None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7, 42]              │
│ [3]                  │
│ NULL                 │
└──────────────────────┘
>>> t.a.length()
┏━━━━━━━━━━━━━━━━┓
┃ ArrayLength(a) ┃
┡━━━━━━━━━━━━━━━━┩
│ int64          │
├────────────────┤
│              2 │
│              1 │
│           NULL │
└────────────────┘

`map(func)` ¶

Apply a callable func to each element of this array expression.

Parameters:

Name	Type	Description	Default
`func`	`Callable[[ir.Value], ir.Value]`	Function to apply to each element of this array	required

Returns:

Type	Description
`ArrayValue`	`func` applied to every element of this array expression.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[1, None, 2], [4], []]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [1, None, ... +1]    │
│ [4]                  │
│ []                   │
└──────────────────────┘
>>> t.a.map(lambda x: (x + 100).cast("float"))
┏━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayMap(a)           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━┩
│ array<float64>        │
├───────────────────────┤
│ [101.0, None, ... +1] │
│ [104.0]               │
│ []                    │
└───────────────────────┘

`remove(other)` ¶

Remove other from self.

Parameters:

Name	Type	Description	Default
`other`	`ir.Value`	Element to remove from `self`.	required

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr": [[3, 2], [], [42, 2], [2, 2], None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [3, 2]               │
│ []                   │
│ [42, 2]              │
│ [2, 2]               │
│ NULL                 │
└──────────────────────┘
>>> t.arr.remove(2)
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayRemove(arr, 2)  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [3]                  │
│ []                   │
│ [42]                 │
│ []                   │
│ NULL                 │
└──────────────────────┘

`sort()` ¶

Sort the elements in an array.

Returns:

Type	Description
`ArrayValue`	Sorted values in an array

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr": [[3, 2], [], [42, 42], None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [3, 2]               │
│ []                   │
│ [42, 42]             │
│ NULL                 │
└──────────────────────┘
>>> t.arr.sort()
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArraySort(arr)       ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [2, 3]               │
│ []                   │
│ [42, 42]             │
│ NULL                 │
└──────────────────────┘

`union(other)` ¶

Union two arrays.

Parameters:

Name	Type	Description	Default
`other`	`ir.ArrayValue`	Another array to union with `self`	required

Returns:

Type	Description
`ArrayValue`	Unioned arrays

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr1": [[3, 2], [], None], "arr2": [[1, 3], [None], [5]]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr1                 ┃ arr2                 ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │ array<int64>         │
├──────────────────────┼──────────────────────┤
│ [3, 2]               │ [1, 3]               │
│ []                   │ [None]               │
│ NULL                 │ [5]                  │
└──────────────────────┴──────────────────────┘
>>> t.arr1.union(t.arr2)
┏━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayUnion(arr1, arr2) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>           │
├────────────────────────┤
│ [1, 2, ... +1]         │
│ []                     │
│ [5]                    │
└────────────────────────┘
>>> t.arr1.union(t.arr2).contains(3)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayContains(ArrayUnion(arr1, arr2), 3) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ boolean                                  │
├──────────────────────────────────────────┤
│ True                                     │
│ False                                    │
│ False                                    │
└──────────────────────────────────────────┘

`unique()` ¶

Return the unique values in an array.

Element ordering in array may not be retained.

Returns:

Type	Description
`ArrayValue`	Unique values in an array

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"arr": [[1, 3, 3], [], [42, 42], None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ arr                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [1, 3, ... +1]       │
│ []                   │
│ [42, 42]             │
│ NULL                 │
└──────────────────────┘
>>> t.arr.unique()
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayDistinct(arr)   ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [3, 1]               │
│ []                   │
│ [42]                 │
│ NULL                 │
└──────────────────────┘

`unnest()` ¶

Flatten an array into a column.

This operation changes the cardinality of the result

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [[7, 42], [3, 3] , None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ a                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │
├──────────────────────┤
│ [7, 42]              │
│ [3, 3]               │
│ NULL                 │
└──────────────────────┘
>>> t.a.unnest()
┏━━━━━━━┓
┃ a     ┃
┡━━━━━━━┩
│ int64 │
├───────┤
│     7 │
│    42 │
│     3 │
│     3 │
└───────┘

Returns:

Type	Description
`ir.Value`	Unnested array

`zip(other, *others)` ¶

Zip two or more arrays together.

Parameters:

Name	Type	Description	Default
`other`	`ir.Array`	Another array to zip with `self`	required
`others`	`ir.Array`	Additional arrays to zip with `self`	`()`

Returns:

Type	Description
`Array`	Array of structs where each struct field is an element of each input array.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"numbers": [[3, 2], [], None], "strings": [["a", "c"], None, ["e"]]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ numbers              ┃ strings              ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ array<int64>         │ array<string>        │
├──────────────────────┼──────────────────────┤
│ [3, 2]               │ ['a', 'c']           │
│ []                   │ NULL                 │
│ NULL                 │ ['e']                │
└──────────────────────┴──────────────────────┘
>>> expr = t.numbers.zip(t.strings)
>>> expr
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayZip()                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ array<struct<f1: int64, f2: string>> │
├──────────────────────────────────────┤
│ [{...}, {...}]                       │
│ []                                   │
│ [{...}]                              │
└──────────────────────────────────────┘
>>> expr.unnest()
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ArrayZip()                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ struct<f1: int64, f2: string> │
├───────────────────────────────┤
│ {'f1': 3, 'f2': 'a'}          │
│ {'f1': 2, 'f2': 'c'}          │
│ {'f1': None, 'f2': 'e'}       │
└───────────────────────────────┘

`StructValue` ¶

Bases: Value

A struct literal or column.

Can be constructed with ibis.struct().

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({'s': [{'a': 1, 'b': 'foo'}, {'a': 3, 'b': None}, None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ s                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ struct<a: int64, b: string> │
├─────────────────────────────┤
│ {'a': 1, 'b': 'foo'}        │
│ {'a': 3, 'b': None}         │
│ NULL                        │
└─────────────────────────────┘

Can use either . or [] to access fields:

>>> t.s.a
┏━━━━━━━┓
┃ a     ┃
┡━━━━━━━┩
│ int64 │
├───────┤
│     1 │
│     3 │
│  NULL │
└───────┘
>>> t.s['a']
┏━━━━━━━┓
┃ a     ┃
┡━━━━━━━┩
│ int64 │
├───────┤
│     1 │
│     3 │
│  NULL │
└───────┘

Attributes¶

`fields: Mapping[str, dt.DataType]` `property` ¶

Return a mapping from field name to field type of the struct.

`names: Sequence[str]` `property` ¶

Return the field names of the struct.

`types: Sequence[dt.DataType]` `property` ¶

Return the field types of the struct.

Functions¶

`getattr(name)` ¶

Extract the name field from this struct.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the field to access.	required

Returns:

Type	Description
`Value`	An expression with the type of the field being accessed.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({'s': [{'a': 1, 'b': 'foo'}, {'a': 3, 'b': None}, None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ s                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ struct<a: int64, b: string> │
├─────────────────────────────┤
│ {'a': 1, 'b': 'foo'}        │
│ {'a': 3, 'b': None}         │
│ NULL                        │
└─────────────────────────────┘
>>> t.s.a
┏━━━━━━━┓
┃ a     ┃
┡━━━━━━━┩
│ int64 │
├───────┤
│     1 │
│     3 │
│  NULL │
└───────┘
>>> t.s.b
┏━━━━━━━━┓
┃ b      ┃
┡━━━━━━━━┩
│ string │
├────────┤
│ foo    │
│ NULL   │
│ NULL   │
└────────┘
>>> t.s.foo_bar
Traceback (most recent call last):
    ...
AttributeError: foo_bar

`getitem(name)` ¶

Extract the name field from this struct.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the field to access.	required

Returns:

Type	Description
`Value`	An expression with the type of the field being accessed.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({'s': [{'a': 1, 'b': 'foo'}, {'a': 3, 'b': None}, None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ s                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ struct<a: int64, b: string> │
├─────────────────────────────┤
│ {'a': 1, 'b': 'foo'}        │
│ {'a': 3, 'b': None}         │
│ NULL                        │
└─────────────────────────────┘
>>> t.s['a']
┏━━━━━━━┓
┃ a     ┃
┡━━━━━━━┩
│ int64 │
├───────┤
│     1 │
│     3 │
│  NULL │
└───────┘
>>> t.s['b']
┏━━━━━━━━┓
┃ b      ┃
┡━━━━━━━━┩
│ string │
├────────┤
│ foo    │
│ NULL   │
│ NULL   │
└────────┘
>>> t.s['foo_bar']
Traceback (most recent call last):
    ...
KeyError: 'foo_bar'

`destructure()` ¶

Destructure a StructValue into the corresponding struct fields.

When assigned, a destruct value will be destructured and assigned to multiple columns.

Returns:

Type	Description
`list[AnyValue]`	Value expressions corresponding to the struct fields.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({'s': [{'a': 1, 'b': 'foo'}, {'a': 3, 'b': None}, None]})
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ s                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ struct<a: int64, b: string> │
├─────────────────────────────┤
│ {'a': 1, 'b': 'foo'}        │
│ {'a': 3, 'b': None}         │
│ NULL                        │
└─────────────────────────────┘
>>> a, b = t.s.destructure()
>>> a
┏━━━━━━━┓
┃ a     ┃
┡━━━━━━━┩
│ int64 │
├───────┤
│     1 │
│     3 │
│  NULL │
└───────┘
>>> b
┏━━━━━━━━┓
┃ b      ┃
┡━━━━━━━━┩
│ string │
├────────┤
│ foo    │
│ NULL   │
│ NULL   │
└────────┘

`lift()` ¶

Project the fields of self into a table.

This method is useful when analyzing data that has deeply nested structs or arrays of structs. lift can be chained to avoid repeating column names and table references.

Returns:

Type	Description
`Table`	A projection with this struct expression's fields.

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable(
...     {
...         "pos": [
...             {"lat": 10.1, "lon": 30.3},
...             {"lat": 10.2, "lon": 30.2},
...             {"lat": 10.3, "lon": 30.1},
...         ]
...     }
... )
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ pos                                ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ struct<lat: float64, lon: float64> │
├────────────────────────────────────┤
│ {'lat': 10.1, 'lon': 30.3}         │
│ {'lat': 10.2, 'lon': 30.2}         │
│ {'lat': 10.3, 'lon': 30.1}         │
└────────────────────────────────────┘
>>> t.pos.lift()
┏━━━━━━━━━┳━━━━━━━━━┓
┃ lat     ┃ lon     ┃
┡━━━━━━━━━╇━━━━━━━━━┩
│ float64 │ float64 │
├─────────┼─────────┤
│    10.1 │    30.3 │
│    10.2 │    30.2 │
│    10.3 │    30.1 │
└─────────┴─────────┘

`MapValue` ¶

Bases: Value

A map literal or column expression.

Can be constructed with ibis.map().

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> import pyarrow as pa
>>> tab = pa.table({
...    "m": pa.array([{"a": 1, "b": 2}, {"a": 1}, None],
...                  type=pa.map_(pa.utf8(), pa.int64()))})
>>> t = ibis.memtable(tab)
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ m                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ map<!string, int64>  │
├──────────────────────┤
│ {'a': 1, 'b': 2}     │
│ {'a': 1}             │
│ NULL                 │
└──────────────────────┘

Can use [] to access values:

>>> t.m['a']
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ MapGet(m, 'a', None) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ int64                │
├──────────────────────┤
│                    1 │
│                    1 │
│                 NULL │
└──────────────────────┘

To provide default values, use get:

>>> t.m.get('b', 0)
┏━━━━━━━━━━━━━━━━━━━┓
┃ MapGet(m, 'b', 0) ┃
┡━━━━━━━━━━━━━━━━━━━┩
│ int64             │
├───────────────────┤
│                 2 │
│                 0 │
│                 0 │
└───────────────────┘

Functions¶

`add(other)` ¶

Concatenate this map with another.

Parameters:

Name	Type	Description	Default
`other`	`MapValue`	Map to concatenate with `self`	required

Returns:

Type	Description
`MapValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> m1 = ibis.map({"a": 1, "b": 2})
>>> m2 = ibis.map({"c": 3, "d": 4})
>>> m1 + m2
{'a': 1, 'b': 2, 'c': 3, 'd': 4}

`getitem(key)` ¶

Get the value for a given map key.

This operation may have different semantics depending on the backend.

Some backends return NULL when a key is missing, others may fail the query.

Parameters:

Name	Type	Description	Default
`key`	`ir.Value`	A map key	required

Returns:

Type	Description
`Value`	An element with the value type of the map

Examples:

>>> import ibis
>>> import pyarrow as pa
>>> ibis.options.interactive = True
>>> tab = pa.table({
...    "m": pa.array([{"a": 1, "b": 2}, {"a": 1}, None],
...                  type=pa.map_(pa.utf8(), pa.int64()))})
>>> t = ibis.memtable(tab)
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ m                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ map<!string, int64>  │
├──────────────────────┤
│ {'a': 1, 'b': 2}     │
│ {'a': 1}             │
│ NULL                 │
└──────────────────────┘
>>> t.m["a"]
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ MapGet(m, 'a', None) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ int64                │
├──────────────────────┤
│                    1 │
│                    1 │
│                 NULL │
└──────────────────────┘

`radd(other)` ¶

Concatenate this map with another.

Parameters:

Name	Type	Description	Default
`other`	`MapValue`	Map to concatenate with `self`	required

Returns:

Type	Description
`MapValue`	`self` concatenated with `other`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> m1 = ibis.map({"a": 1, "b": 2})
>>> m2 = ibis.map({"c": 3, "d": 4})
>>> m1 + m2
{'a': 1, 'b': 2, 'c': 3, 'd': 4}

`contains(key)` ¶

Return whether the map contains key.

Parameters:

Name	Type	Description	Default
`key`	`int \| str \| ir.IntegerValue \| ir.StringValue`	Mapping key for which to check	required

Returns:

Type	Description
`BooleanValue`	Boolean indicating the presence of `key` in the map expression

Examples:

>>> import ibis
>>> import pyarrow as pa
>>> ibis.options.interactive = True
>>> tab = pa.table({
...    "m": pa.array([{"a": 1, "b": 2}, {"a": 1}, None],
...                  type=pa.map_(pa.utf8(), pa.int64()))})
>>> t = ibis.memtable(tab)
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ m                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ map<!string, int64>   │
├──────────────────────┤
│ {'a': 1, 'b': 2}     │
│ {'a': 1}             │
│ NULL                 │
└──────────────────────┘
>>> t.m.contains("b")
┏━━━━━━━━━━━━━━━━━━━━━┓
┃ MapContains(m, 'b') ┃
┡━━━━━━━━━━━━━━━━━━━━━┩
│ boolean             │
├─────────────────────┤
│ True                │
│ False               │
│ False               │
└─────────────────────┘

`get(key, default=None)` ¶

Return the value for key from expr.

Return default if key is not in the map.

Parameters:

Name	Type	Description	Default
`key`	`ir.Value`	Expression to use for key	required
`default`	`ir.Value \| None`	Expression to return if `key` is not a key in `expr`	`None`

Returns:

Type	Description
`Value`	The element type of `self`

Examples:

>>> import ibis
>>> import pyarrow as pa
>>> ibis.options.interactive = True
>>> tab = pa.table({
...    "m": pa.array([{"a": 1, "b": 2}, {"a": 1}, None],
...                  type=pa.map_(pa.utf8(), pa.int64()))})
>>> t = ibis.memtable(tab)
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ m                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ map<!string, int64>  │
├──────────────────────┤
│ {'a': 1, 'b': 2}     │
│ {'a': 1}             │
│ NULL                 │
└──────────────────────┘
>>> t.m.get("a")
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ MapGet(m, 'a', None) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ int64                │
├──────────────────────┤
│                    1 │
│                    1 │
│                 NULL │
└──────────────────────┘
>>> t.m.get("b")
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ MapGet(m, 'b', None) ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ int64                │
├──────────────────────┤
│                    2 │
│                 NULL │
│                 NULL │
└──────────────────────┘
>>> t.m.get("b", 0)
┏━━━━━━━━━━━━━━━━━━━┓
┃ MapGet(m, 'b', 0) ┃
┡━━━━━━━━━━━━━━━━━━━┩
│ int64             │
├───────────────────┤
│                 2 │
│                 0 │
│                 0 │
└───────────────────┘

`keys()` ¶

Extract the keys of a map.

Returns:

Type	Description
`ArrayValue`	The keys of `self`

Examples:

>>> import ibis
>>> import pyarrow as pa
>>> ibis.options.interactive = True
>>> tab = pa.table({
...    "m": pa.array([{"a": 1, "b": 2}, {"a": 1}, None],
...                  type=pa.map_(pa.utf8(), pa.int64()))})
>>> t = ibis.memtable(tab)
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ m                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ map<!string, int64>  │
├──────────────────────┤
│ {'a': 1, 'b': 2}     │
│ {'a': 1}             │
│ NULL                 │
└──────────────────────┘
>>> t.m.keys()
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ MapKeys(m)           ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ array<!string>       │
├──────────────────────┤
│ ['a', 'b']           │
│ ['a']                │
│ NULL                 │
└──────────────────────┘

`length()` ¶

Return the number of key-value pairs in the map.

Returns:

Type	Description
`IntegerValue`	The number of elements in `self`

Examples:

>>> import ibis
>>> import pyarrow as pa
>>> ibis.options.interactive = True
>>> tab = pa.table({
...    "m": pa.array([{"a": 1, "b": 2}, {"a": 1}, None],
...                  type=pa.map_(pa.utf8(), pa.int64()))})
>>> t = ibis.memtable(tab)
>>> t
┏━━━━━━━━━━━━━━━━━━━━━━┓
┃ m                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━┩
│ map<!string, int64>  │
├──────────────────────┤
│ {'a': 1, 'b': 2}     │
│ {'a': 1}             │
│ NULL                 │
└──────────────────────┘
>>> t.m.length()
┏━━━━━━━━━━━━━━┓
┃ MapLength(m) ┃
┡━━━━━━━━━━━━━━┩
│ int64        │
├──────────────┤
│            2 │
│            1 │
│         NULL │
└──────────────┘

`values()` ¶

Extract the values of a map.

Returns:

Type	Description
`ArrayValue`	The values of `self`

Examples:

>>> import ibis
>>> ibis.options.interactive = True
>>> m = ibis.map({"a": 1, "b": 2})
>>> m.values()
[1, 2]

Last update: August 5, 2022

Complex Type Expressions¶

ArrayValue ¶

Functions¶

__add__(other) ¶

__getitem__(index) ¶

__mul__(n) ¶

__radd__(other) ¶

__rmul__(n) ¶

contains(other) ¶

filter(predicate) ¶

index(other) ¶

join(sep) ¶

See Also¶

length() ¶

map(func) ¶

remove(other) ¶

sort() ¶

union(other) ¶

unique() ¶

unnest() ¶

zip(other, *others) ¶

StructValue ¶

Attributes¶

fields: Mapping[str, dt.DataType] property ¶

names: Sequence[str] property ¶

types: Sequence[dt.DataType] property ¶

Functions¶

__getattr__(name) ¶

__getitem__(name) ¶

destructure() ¶

lift() ¶

See Also¶

MapValue ¶

Functions¶

__add__(other) ¶

__getitem__(key) ¶

__radd__(other) ¶

contains(key) ¶

get(key, default=None) ¶

keys() ¶

length() ¶

values() ¶

`ArrayValue` ¶

`add(other)` ¶

`getitem(index)` ¶

`mul(n)` ¶

`radd(other)` ¶

`rmul(n)` ¶

`contains(other)` ¶

`filter(predicate)` ¶

`index(other)` ¶

`join(sep)` ¶

`length()` ¶

`map(func)` ¶

`remove(other)` ¶

`sort()` ¶

`union(other)` ¶

`unique()` ¶

`unnest()` ¶

`zip(other, *others)` ¶

`StructValue` ¶

`fields: Mapping[str, dt.DataType]` `property` ¶

`names: Sequence[str]` `property` ¶

`types: Sequence[dt.DataType]` `property` ¶

`getattr(name)` ¶

`getitem(name)` ¶

`destructure()` ¶

`lift()` ¶

`MapValue` ¶

`add(other)` ¶

`getitem(key)` ¶

`radd(other)` ¶

`contains(key)` ¶

`get(key, default=None)` ¶

`keys()` ¶

`length()` ¶

`values()` ¶