Data Validation With Pydantic

Introduction

#

Pydantic is a Python package that provides data validation and settings management functionality. It is built on top of Python’s typing module, which allows you to specify the types of data that you expect. Pydantic is designed to be lightweight and extensible, making it a popular choice for building APIs and microservices.

Validating primitive data types

#

Pydantic can be used to validate primitive data types such as strings, integers and floats. In the example that follows, we define a Pydantic model that represents a person’s name, age and height:

In [1]: from pydantic import BaseModel

In [2]: class Person(BaseModel):
   ...:     name: str
   ...:     age: int
   ...:     height: float
   ...:

In [3]: person = Person(name="Jack Smith", age=32, height=1.82)

In [4]: person
Out[4]: Person(name='Jack Smith', age=32, height=1.82)

We can then create instances of the Person model and validate the data that we pass to it. If we try to pass in data that does not conform to the expected types, Pydantic will raise a validation error:

In [5]: Person(name='Jack Smith', age='thirty two', height=1.82)
ValidationError: 1 validation error for Person
age
  Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='thirty two', input_type=str]

Validating nested data structures

#

Pydantic can also be used to validate more complex data structures, such as nested dictionaries. In the following example, we define a Pydantic model that represents a customer’s contact information:

In [7]: from typing import Dict

In [9]: class Contact(BaseModel):
   ...:     name: str
   ...:     phone: str
   ...:

In [10]: class Customer(BaseModel):
   ...:     name: str
   ...:     contact: Contact
   ...:

In [15]: customer_data = {
   ...:     'name': 'Jack Smith',
   ...:     'contact': {
   ...:         'email': 'jack.smith@company.com',
   ...:         'phone': '321-654-9876'
   ...:         }
   ...:     }

We can then create instances of the Customer model and validate the data that we pass in:

In [16]: customer = Customer(**customer_data)

In [17]: customer
Out[17]: Customer(name='Jack Smith', contact=Contact(email='jack.smith@company.com', phone='321-654-9876'))

If we try to pass in data that does not conform to the expected structure, Pydantic will raise a validation error:

In [8]: customer_data = {
   ...:     'name': 'Jack Smith',
   ...:     'contact': {
   ...:         'email': 'jack.smith@company.com',
   ...:         'mobile': '321-654-9876'
   ...:     }
   ...: }
   ...:

In [9]: customer = Customer(**customer_data)
ValidationError: 1 validation error for Customer
contact.phone
  Field required [type=missing, input_value={'email': 'jack.smith@com...mobile': '321-654-9876'}, input_type=dict]

Validating data from external sources

#

Pydantic can be used to validate data that comes from external sources such as JSON files. In the following example, we define a Pydantic model that represents a product’s name, price, and quantity:

In [10]: from pathlib import Path
In [11]: from pydantic import BaseModel, PositiveFloat, PositiveInt, TypeAdapter
In [12]: class Product(BaseModel):
    ...:     name: str
    ...:     price: PositiveFloat
    ...:     quantity: PositiveInt
    ...:

TypeAdapter is used to validate a list of Product objects. We can now load data from a JSON file and validate it using Pydantic:

[
  {
    "name": "Apple",
    "price": 0.5,
    "quantity": 100
  },
  {
    "name": "Banana",
    "price": 0.25,
    "quantity": 200
  },
  {
    "name": "Orange",
    "price": 0.75,
    "quantity": 50
  }
]

In [13]: json_string = Path('products.json').read_text()
In [14]: product_list_adapter = TypeAdapter(list[Product])
In [15]: products = product_list_adapter.validate_json(json_string)
In [16]: print(repr(products))
[Product(name='Apple', price=0.5, quantity=100), Product(name='Banana', price=0.25, quantity=200), Product(name='Orange', price=0.75, quantity=50)]

If the data in the file does not come from to the expected structure, Pydantic will raise a validation error. Let’s add the following data to the JSON file:

  {
      "name": "Pineapple",
      "price": 1.5,
      "quantity": -5.0
  }

We’ll now receive the following error message:

In [23]: products = product_list_adapter.validate_json(json_string)

ValidationError: 1 validation error for list[Product]
3.quantity
  Input should be greater than 0 [type=greater_than, input_value=-5.0, input_type=float]

Constraining fields with Field()

#

Beyond basic type checks, Pydantic’s Field() function lets you attach extra constraints directly to a model’s attributes, such as string lengths, numeric ranges, and regular expression patterns. This is often a simpler alternative to writing a custom validator for straightforward rules. According to the Pydantic documentation on fields, these constraints are also translated into the corresponding JSON Schema keywords (for example, min_length becomes minLength and ge/le become minimum/maximum), which is useful if the model doubles as part of an API contract.

In [1]: from pydantic import BaseModel, Field

In [2]: class Employee(BaseModel):
   ...:     name: str = Field(min_length=1, max_length=50)
   ...:     age: int = Field(ge=18, le=65)
   ...:     employee_id: str = Field(pattern=r'^[A-Z]{2}\d{4}$')
   ...:

In [3]: Employee(name="Jane Doe", age=29, employee_id="HR1234")
Out[3]: Employee(name='Jane Doe', age=29, employee_id='HR1234')

Passing an age below the minimum, or an ID that doesn’t match the pattern, produces the same kind of ValidationError we’ve already seen:

In [4]: Employee(name="Jane Doe", age=16, employee_id="HR1234")
ValidationError: 1 validation error for Employee
age
  Input should be greater than or equal to 18 [type=greater_than_equal, input_value=16, input_type=int]

Some of the most commonly used constraint arguments are summarised below:

Using custom validators

#

Field() constraints cover many common cases, but sometimes you need to validate a field against custom logic, or validate the relationship between several fields together. Pydantic provides two decorators for this: field_validator, for checking or transforming a single field, and model_validator, for checks that depend on more than one field at once. Both are described in detail in Pydantic’s validators documentation.

Field validators

#

A field validator is attached to a single named field using the @field_validator decorator. In the example below, we define a Pydantic model that represents a user’s password and add a validator that rejects passwords under eight characters:

In [1]: from pydantic import BaseModel, field_validator

In [2]: class UserModel(BaseModel):
   ...:     username: str
   ...:     password: str
   ...:
   ...:     @field_validator('password')
   ...:     @classmethod
   ...:     def password_must_be_long_enough(cls, value: str) -> str:
   ...:         if len(value) < 8:
   ...:             raise ValueError('password must be at least 8 characters long')
   ...:         return value
   ...:

In [3]: UserModel(username='jsmith', password='short')
ValidationError: 1 validation error for UserModel
password
  Value error, password must be at least 8 characters long [type=value_error, input_value='short', input_type=str]

By default, a field validator runs in mode='after', meaning it receives the value once Pydantic has already coerced it to the expected type. Setting mode='before' instead lets the validator run on the raw input, before Pydantic’s own type coercion takes place, which is useful for cleaning up messy input (for example, stripping whitespace from a string before it’s checked).

Model validators

#

When a check depends on more than one field, such as confirming that two password fields match, use @model_validator instead, which runs against the whole model:

In [1]: from pydantic import BaseModel, model_validator

In [2]: class RegistrationForm(BaseModel):
   ...:     password: str
   ...:     password_confirmation: str
   ...:
   ...:     @model_validator(mode='after')
   ...:     def passwords_match(self) -> 'RegistrationForm':
   ...:         if self.password != self.password_confirmation:
   ...:             raise ValueError('passwords do not match')
   ...:         return self
   ...:

In [3]: RegistrationForm(password='hunter2', password_confirmation='hunter3')
ValidationError: 1 validation error for RegistrationForm
  Value error, passwords do not match [type=value_error, input_value={'password': 'hunter2', 'password_confirmation': 'hunter3'}, input_type=dict]

As with field validators, model validators also support a mode='before' option, which runs on the raw input dictionary before any of the individual fields have been validated. This is useful for reshaping incoming data before Pydantic attempts to build the model.

Settings management with pydantic-settings

#

One of Pydantic’s most popular uses, beyond validating request or file data, is managing application configuration. Since Pydantic version 2, this functionality lives in a companion package, pydantic-settings, which is installed separately with pip install pydantic-settings.

A settings class inherits from BaseSettings rather than BaseModel. When it’s instantiated, Pydantic automatically looks for a matching environment variable for any field that isn’t passed in explicitly, falling back to a default value where one is defined:

In [1]: from pydantic_settings import BaseSettings

In [2]: class AppSettings(BaseSettings):
   ...:     app_name: str = "My App"
   ...:     debug: bool = False
   ...:     database_url: str
   ...:

In [3]: # Reads DATABASE_URL (and, if present, APP_NAME / DEBUG) from the environment
In [4]: settings = AppSettings()

Environment variable names are matched case-insensitively by default, and a shared prefix can be applied to every field using SettingsConfigDict(env_prefix=...), so that, for example, database_url maps to MYAPP_DATABASE_URL. Settings can also be loaded from a .env file by pointing model_config at it:

In [1]: from pydantic_settings import BaseSettings, SettingsConfigDict

In [2]: class AppSettings(BaseSettings):
   ...:     model_config = SettingsConfigDict(env_file='.env', env_prefix='myapp_')
   ...:
   ...:     app_name: str = "My App"
   ...:     debug: bool = False
   ...:     database_url: str
   ...:

This pattern gives you a single, type-checked, validated source of truth for configuration, rather than reading raw strings out of os.environ throughout a codebase.

Conclusion

#

Pydantic is a powerful Python package that allows you to easily define and validate data models. With Pydantic, you can write concise and readable code that is also robust and maintainable. By using Pydantic, you can reduce the time and effort required to validate and sanitize user input, and make your code more resilient to errors and bugs. Whether you’re working on a web application, a data processing pipeline, or any other type of software, Pydantic is a valuable tool to have in your toolkit.

Further reading

#

• Pydantic documentation: get started
• Pydantic documentation: fields and Field() constraints
• Pydantic documentation: validators
• Pydantic documentation: settings management
• pydantic-settings on GitHub