Data Validation With Pydantic

Introduction

#

Pydantic is a Python package that provides data validation and settings management functionality. It is built on top of Python’s typing module, which allows you to specify the types of data that you expect. Pydantic is designed to be lightweight and extensible, making it a popular choice for building APIs and microservices.

Validating primitive data types

#

Pydantic can be used to validate primitive data types such as strings, integers and floats. In the example that follows, we define a Pydantic model that represents a person’s name, age and height:

In [1]: from pydantic import BaseModel

In [2]: class Person(BaseModel):
   ...:     name: str
   ...:     age: int
   ...:     height: float
   ...:

In [3]: person = Person(name="Jack Smith", age=32, height=1.82)

In [4]: person
Out[4]: Person(name='Jack Smith', age=32, height=1.82)

We can then create instances of the Person model and validate the data that we pass to it. If we try to pass in data that does not conform to the expected types, Pydantic will raise a validation error:

In [5]: Person(name='Jack Smith', age='thirty two', height=1.82)
ValidationError: 1 validation error for Person
age
  Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='thirty two', input_type=str]

Validating Nested Data Structures

#

Pydantic can also be used to validate more complex data structures, such as nested dictionaries. In the following example, we define a Pydantic model that represents a customer’s contact information:

In [7]: from typing import Dict

In [9]: class Contact(BaseModel):
   ...:     name: str
   ...:     phone: str
   ...:

In [10]: class Customer(BaseModel):
    ...:     name: str
    ...:     contact: Contact
    ...:

In [15]: customer_data = {
    ...:     'name': 'Jack Smith',
    ...:     'contact': {
    ...:         'email': 'jack.smith@company.com',
    ...:         'phone': '321-654-9876'
    ...:         }
    ...:     }

We can then create instances of the Customer model and validate the data that we pass in:

In [16]: customer = Customer(**customer_data)

In [17]: customer
Out[17]: Customer(name='Jack Smith', contact=Contact(email='jack.smith@company.com', phone='321-654-9876'))

If we try to pass in data that does not conform to the expected structure, Pydantic will raise a validation error:

In [8]: customer_data = {
   ...:     'name': 'Jack Smith',
   ...:     'contact': {
   ...:         'email': 'jack.smith@company.com',
   ...:         'mobile': '321-654-9876'
   ...:     }
   ...: }
   ...:

In [9]: customer = Customer(**customer_data)
ValidationError: 1 validation error for Customer
contact.phone
  Field required [type=missing, input_value={'email': 'jack.smith@com...mobile': '321-654-9876'}, input_type=dict]

Validating Data from External Sources

#

Pydantic can be used to validate data that comes from external sources such as JSON files. In the following example, we define a Pydantic model that represents a product’s name, price, and quantity:

In [10]: from pathlib import Path
In [11]: from pydantic import BaseModel, PositiveFloat, PositiveInt, TypeAdapter
In [12]: class Product(BaseModel):
    ...:     name: str
    ...:     price: PositiveFloat
    ...:     quantity: PositiveInt
    ...:

TypeAdapter is used to validate a list of Product objects. We can now load data from a JSON file and validate it using Pydantic:

[
  {
    "name": "Apple",
    "price": 0.5,
    "quantity": 100
  },
  {
    "name": "Banana", 
    "price": 0.25,
    "quantity": 200
  },
  {
    "name": "Orange",
    "price": 0.75,
    "quantity": 50
  }
]

In [13]: json_string = Path('products.json').read_text()
In [14]: product_list_adapter = TypeAdapter(list[Product])
In [15]: products = product_list_adapter.validate_json(json_string)
In [16]: print(repr(products))
[Product(name='Apple', price=0.5, quantity=100), Product(name='Banana', price=0.25, quantity=200), Product(name='Orange', price=0.75, quantity=50)]

If the data in the file does not come from to the expected structure, Pydantic will raise a validation error. Let’s add the following data to the JSON file:

  {
      "name": "Pineapple",
      "price": 1.5,
      "quantity": -5.0
  }

We’ll now receive the following error message:

In [23]: products = product_list_adapter.validate_json(json_string)

ValidationError: 1 validation error for list[Product]
3.quantity
  Input should be greater than 0 [type=greater_than, input_value=-5.0, input_type=float]

Using Custom Validators

#

Pydantic provides a way to define validators that can be used to check data after it has been validated. In the following example, we define a Pydantic model that represents a user’s password:

Conclusion

#

Pydantic is a powerful Python package that allows you to easily define and validate data models. With Pydantic, you can write concise and readable code that is also robust and maintainable. By using Pydantic, you can reduce the time and effort required to validate and sanitize user input, and make your code more resilient to errors and bugs. Whether you’re working on a web application, a data processing pipeline, or any other type of software, Pydantic is a valuable tool to have in your toolkit.