Schemas

Table Schemas

schema

ibis.schema(pairs=None, /, *, names=None, types=None)

Validate and return a Schema object.

Parameters

Name Type Description Default
pairs SchemaLike | None List or dictionary of name, type pairs. Mutually exclusive with names and types arguments. None
names Iterable[str] | None Field names. Mutually exclusive with pairs. None
types Iterable[str | dt.DataType] | None Field types. Mutually exclusive with pairs. None

Returns

Name Type Description
Schema An ibis schema

Examples

>>> from ibis import schema, Schema
>>> sc = schema([("foo", "string"), ("bar", "int64"), ("baz", "boolean")])
>>> sc = schema(names=["foo", "bar", "baz"], types=["string", "int64", "boolean"])
>>> sc = schema({"nullable-str": "string", "non-nullable-str": "!string"})
>>> sc = schema(dict(foo="string"))
>>> sc = schema(Schema(dict(foo="string")))  # no-op

Schema

Schema(**kwargs)

An ordered mapping of str -> datatype, used to hold a Table’s schema.

Attributes

Name Description
fields A mapping of str to

Methods

Name Description
equals Return whether other is equal to self.
from_numpy Return the equivalent ibis schema.
from_pandas Return the equivalent ibis schema.
from_polars Return the equivalent ibis schema.
from_pyarrow Return the equivalent ibis schema.
from_sqlglot Construct an Ibis Schema from a SQLGlot Schema.
from_tuples Construct a Schema from an iterable of pairs.
name_at_position Return the name of a schema column at position i.
to_numpy Return the equivalent numpy dtypes.
to_pandas Return the equivalent pandas datatypes.
to_polars Return the equivalent polars schema.
to_pyarrow Return the equivalent pyarrow schema.
to_sqlglot DEPRECATED: use to_sqlglot_column_defs() instead.
to_sqlglot_column_defs Convert the schema to a list of SQL column definitions.

equals

equals(other)

Return whether other is equal to self.

The order of fields in the schema is taken into account when computing equality.

Parameters

Name Type Description Default
other Schema Schema to compare self to. required

Examples

>>> import ibis
>>> xy = ibis.schema({"x": int, "y": str})
>>> xy2 = ibis.schema({"x": int, "y": str})
>>> yx = ibis.schema({"y": str, "x": int})
>>> xy_float = ibis.schema({"x": float, "y": str})
>>> assert xy.equals(xy2)
>>> assert not xy.equals(yx)
>>> assert not xy.equals(xy_float)

from_numpy

from_numpy(numpy_schema)

Return the equivalent ibis schema.

from_pandas

from_pandas(pandas_schema)

Return the equivalent ibis schema.

from_polars

from_polars(polars_schema)

Return the equivalent ibis schema.

from_pyarrow

from_pyarrow(pyarrow_schema)

Return the equivalent ibis schema.

from_sqlglot

from_sqlglot(schema, dialect=None)

Construct an Ibis Schema from a SQLGlot Schema.

Parameters

Name Type Description Default
schema sge.Schema A SQLGlot Schema containing column definitions. required
dialect str | sg.Dialect | None Optional dialect to use for type conversion. None

Returns

Name Type Description
Schema An Ibis Schema.

Examples

>>> import ibis
>>> import sqlglot as sg
>>> import sqlglot.expressions as sge
>>> columns = [
...     sge.ColumnDef(
...         this=sg.to_identifier("a", quoted=True),
...         kind=sge.DataType(this=sge.DataType.Type.BIGINT),
...     ),
...     sge.ColumnDef(
...         this=sg.to_identifier("b", quoted=True),
...         kind=sge.DataType(this=sge.DataType.Type.VARCHAR),
...         constraints=[sge.ColumnConstraint(kind=sge.NotNullColumnConstraint())],
...     ),
... ]
>>> schema_expr = sge.Schema(expressions=columns)
>>> sch = ibis.Schema.from_sqlglot(schema_expr)
>>> sch
ibis.Schema {
  a  int64
  b  !string
}

Different source dialects are supported using the dialect keyword argument.

>>> columns = [
...     sge.ColumnDef(
...         this=sg.to_identifier("a", quoted=True),
...         kind=sge.DataType(
...             this=sge.DataType.Type.ARRAY,
...             expressions=[sge.DataType(this=sge.DataType.Type.BIGINT, nested=False)],
...             nested=True,
...         ),
...     )
... ]
>>> schema_expr = sge.Schema(expressions=columns)
>>> snowflake_schema = ibis.Schema.from_sqlglot(schema_expr, dialect="snowflake")
>>> snowflake_schema
ibis.Schema {
  a  array<json>
}
>>> bigquery_schema = ibis.Schema.from_sqlglot(schema_expr, dialect="bigquery")
>>> bigquery_schema
ibis.Schema {
  a  array<int64>
}

from_tuples

from_tuples(values)

Construct a Schema from an iterable of pairs.

Parameters

Name Type Description Default
values Iterable[tuple[str, str | dt.DataType]] An iterable of pairs of name and type. required

Returns

Name Type Description
Schema A new schema

Examples

>>> import ibis
>>> ibis.Schema.from_tuples([("a", "int"), ("b", "string")])
ibis.Schema {
  a  int64
  b  string
}

name_at_position

name_at_position(i)

Return the name of a schema column at position i.

Parameters

Name Type Description Default
i int The position of the column required

Returns

Name Type Description
str The name of the column in the schema at position i.

Examples

>>> import ibis
>>> sch = ibis.Schema({"a": "int", "b": "string"})
>>> sch.name_at_position(0)
'a'
>>> sch.name_at_position(1)
'b'

to_numpy

to_numpy()

Return the equivalent numpy dtypes.

to_pandas

to_pandas()

Return the equivalent pandas datatypes.

to_polars

to_polars()

Return the equivalent polars schema.

to_pyarrow

to_pyarrow()

Return the equivalent pyarrow schema.

to_sqlglot

to_sqlglot(dialect)

DEPRECATED: use to_sqlglot_column_defs() instead.

to_sqlglot_column_defs

to_sqlglot_column_defs(dialect)

Convert the schema to a list of SQL column definitions.

Parameters

Name Type Description Default
dialect str | sg.Dialect The SQL dialect to use. required

Returns

Name Type Description
list[sqlglot.expressions.ColumnDef] A list of SQL column definitions.

Examples

>>> import ibis
>>> sch = ibis.schema({"a": "int", "b": "!string"})
>>> sch
ibis.Schema {
  a  int64
  b  !string
}
>>> columns = sch.to_sqlglot_column_defs(dialect="duckdb")
>>> columns
[ColumnDef(
   this=Identifier(this='a', quoted=True),
   kind=DataType(this=Type.BIGINT)),
 ColumnDef(
   this=Identifier(this='b', quoted=True),
   kind=DataType(this=Type.VARCHAR),
   constraints=[
     ColumnConstraint(
       kind=NotNullColumnConstraint())])]

One use case for this method is to embed its output into a SQLGlot CREATE TABLE expression.

>>> import sqlglot as sg
>>> import sqlglot.expressions as sge
>>> table = sg.table("t", quoted=True)
>>> ct = sge.Create(
...     kind="TABLE",
...     this=sge.Schema(
...         this=table,
...         expressions=columns,
...     ),
... )
>>> ct.sql(dialect="duckdb")
'CREATE TABLE "t" ("a" BIGINT, "b" TEXT NOT NULL)'
Back to top