Polars DataFrame validation with pandera, collecting all errors: working versions (tested 2026-10-15)
Validate Polars DataFrames with a pandera.polars DataFrameModel, collect every failing check with lazy validation, and test the schema offline with in-memory frames.
passed5 passed · 0 failed2026-10-15 13:32 UTC- Status
- passed
- Last passed
- 2026-10-15 13:32 UTC
- Tests
- 5 / 5
- Language
- Python
- Runtime
- Python 3.13.16
uv venv -q .venv && uv pip install -q --python .venv/bin/python -r requirements.txt .venv/bin/python -m pytest -q
Python 3.13.16 linux 6.17.0-1022-azure x64 runner: github-actions 38742087302
The code
Exactly the files that passed. Your agent can pull them with get_recipe("polars-pandera-validation") (install), or see them on GitHub.
polars==2.0.0 pandera==0.34.1 pytest==9.1.1
import polars as pl
import pandera.polars as pa
from pandera.typing.polars import Series
class Orders(pa.DataFrameModel):
order_id: Series[int] = pa.Field(ge=1)
amount: Series[float] = pa.Field(ge=0)
currency: Series[str] = pa.Field(isin=["USD", "EUR"])
class StrictOrders(Orders):
class Config:
strict = True # reject columns that are not declared in the schema
def validate_orders(df: pl.DataFrame) -> pl.DataFrame:
"""Return df if valid; raise SchemaErrors listing every failing check."""
return Orders.validate(df, lazy=True)
import polars as pl
import pytest
from pandera.errors import SchemaError, SchemaErrors
from schemas import Orders, StrictOrders, validate_orders
def good_frame() -> pl.DataFrame:
return pl.DataFrame({"order_id": [1, 2], "amount": [9.5, 0.0], "currency": ["USD", "EUR"]})
def bad_frame() -> pl.DataFrame:
# three independent failures: order_id < 1, amount < 0, currency not allowed
return pl.DataFrame({"order_id": [0, 2], "amount": [-1.0, 3.0], "currency": ["GBP", "EUR"]})
def test_valid_frame_passes():
assert validate_orders(good_frame()).height == 2
def test_lazy_collects_all_errors():
with pytest.raises(SchemaErrors) as exc:
validate_orders(bad_frame())
columns = set(exc.value.failure_cases["column"].to_list())
assert columns == {"order_id", "amount", "currency"}
def test_eager_stops_at_first_error():
# default lazy=False raises on the first failing check only
with pytest.raises(SchemaError):
Orders.validate(bad_frame())
def test_nulls_fail_by_default():
df = pl.DataFrame({"order_id": [1, None], "amount": [1.0, 2.0], "currency": ["USD", "EUR"]})
with pytest.raises(SchemaErrors):
validate_orders(df)
def test_extra_columns_need_strict():
df = good_frame().with_columns(pl.lit("x").alias("note"))
Orders.validate(df) # extra columns allowed by default
with pytest.raises(SchemaErrors):
StrictOrders.validate(df, lazy=True)
[pytest] pythonpath = .
Connect to ProvenPin
Add the free MCP server so your agent can pull this recipe and others like it. Same setup as the docs.
claude mcp add --transport http provenpin https://mcp.provenpin.com/mcp
{ "mcpServers": { "provenpin": { "url": "https://mcp.provenpin.com/mcp" } } }{ "servers": { "provenpin": { "type": "http", "url": "https://mcp.provenpin.com/mcp" } } }{ "mcpServers": { "provenpin": { "serverUrl": "https://mcp.provenpin.com/mcp" } } }{ "mcpServers": { "provenpin": { "type": "streamableHttp", "url": "https://mcp.provenpin.com/mcp" } } }Any other MCP client: Streamable HTTP at https://mcp.provenpin.com/mcp. No API key, no signup.
Related stacks
DuckDB over Polars DataFrames: replacement scans, parquet roundtrip, group-by parity
Query Polars DataFrames directly with DuckDB, return .pl() results, round-trip through parquet in tmp_path, and check group-by aggregations against Polars native output.
pandas + Pandera DataFrame schema validation
Validate pandas DataFrames with Pandera DataFrameModel schemas: dtypes, coercion, value checks, custom dataframe checks and lazy error reports.
Validating polars DataFrame rows with pydantic v2 TypeAdapter and collecting errors
Validate each polars row against a strict pydantic v2 model with a shared TypeAdapter, collect per-row errors instead of failing the batch, and catch polars and pydantic gotchas such as nulls, NaN, numeric strings and extra columns.