To attain knowledge, add things every day. To attain wisdom, remove things every day -Lao Tzu
NoSQL Databases has been growing in popularity since 2014 Show Figure 1, this is due to different uses cases that relational database can't match. One of the most popular uses cases is big data, since relational databases have grown at an accelerated rate, companies now require flexible technology which can be found in Databases such as MongoDB, Redis and Elasticsearch.
Currently, MongoDB is the most popular NoSQL database, companies like Forbes, Toyota, and eBay are using it. In python there exists pymongo which is a tool for interacting with MongoDB using python code. However, as applications grow, they will need to support complex use cases, complex queries and complex interactions, all of which can be time-consuming when using only pymongo.
ODM
ODM stands from object document mapping, its propose is similar to the ORM, but using documents database, there are some popular libraries in python like MongoEngine, odmantic, and umongo which are pretty good, each one have fetures that I really like. For instance MongoEngine has this Django ORM mood, odmantic is built using pydantic too (which is great) and umongo is pretty simple to use.
Having said that, I recently found a library that contains good features and pretty good performance, which is also built using Pydantic v2 adding an incredible performance.
Beanie
Beanie is an asynchronous Python object-document mapper (ODM) for MongoDB. Data models are based in Pydantic which makes the FastAPI integration very friendly. It is built using two main libraries motor and Pydantic Show Figure 3. There's nothing more to say; let's write some code.
Install beanie using poetry
poetry add beanieDefine a module call models.py and add the following code
from typing import Optional
from enum import Enum
from beanie import Document, PydanticObjectId, Link
from pydantic import Field, BaseModel
class Profession(str, Enum):
engineering = "ENGINEERING"
accounter = "ACCOUNTER"
class Pet(Document):
name: str = None
class Person(Document):
name: str
email: str
age: int
profession: Profession
pet: Optional[Link[Pet]]
country: str = NoneDefine a module called main_local.py and add the following code
import asyncio
from beanie import WriteRules, init_beanie
from motor.motor_asyncio import AsyncIOMotorClient
from src.models import Person, Pet, Profession
from beanie.operators import In
async def insert_person():
client = AsyncIOMotorClient("mongodb://localhost:27017/myproject")
await init_beanie(database=client["myproject"], document_models=[Person, Pet])
my_pet = Pet(name="My pet")
person = Person(
age=27,
name="Jairo",
email="me@jairoandres.com",
pet=my_pet,
profession=Profession.engineering,
country="Colombia",
)
await person.insert(link_rule=WriteRules.WRITE)
persons = await Person.find(In(Person.name, ["Jairo", "Andres"])).to_list()
print(persons)
avg_age = await Person.find(
Person.country == "Colombia", Person.profession == Profession.engineering
).avg(Person.age)
print(avg_age)
if __name__ == "__main__":
asyncio.run(insert_person())
run the code
petry run python src/main_local.pyGo to MongoDB, there will appear the collection called Person and Pet
myproject> db.Person.find({})
[
{
_id: ObjectId("64f7de8e9b60fedaeb139212"),
name: 'Jairo',
email: 'me@jairoandres.com',
age: 26,
profession: 'ENGINEERING',
pet: DBRef("Pet", ObjectId("64f7de8e9b60fedaeb139211"))
}
]
Explanation
models.py: The file called models.py will contain collections that will be reflected in MongoDB (as long as the class inherit from Document), there are two models Person and Pet, the behavior of them is exactly as Pydantic models, the different are in the methods available to interact with MongoDB. If you have worked with Pydantic before, the code should look familiar, the only concern would be in the Link class.
How I mentioned before Beanie is an extension from Pydantic making easier communicate with MongoDB , Link comes from the DBRef concept, this is used to reference another documents, it is kind of relationship, which is not recommended in many cases, but you still have it if you need it
main_local.py: The file called main_local.py contains configurations and initialization instructions. The line 9 and 10 are pretty standard, line 9 is creating a new connection with MongoDB and line 10 is initiating, which means creating a collection if it does not exist.
Line 11 to 18 are creating two objects from Pet and Person (look that the pet object is used to pass as a parameter to Person). The line 19 is inserting the data into MongoDB, as you can see, there is one parameter called:
link_rule=WriteRules.WRITE)The last instruction is because of the link reference, so this parameter is indicating to beanie that every Link reference should be also written into the MongoDB before to create the relationship.
The line 20 is calling another method called find that return some Person objects, the filter is "pythonic" what means you can use attributes and special operator to query data.
Line 23 is performing an aggregation query to calculate the avg age using the country as a filter, in this case the country is Colombia
Using FastAPI
Using beanie is so intuitive; there are a bunch of things that you can do, and you can find the in the docs. Another interesting thing about beanie is how well it integrates with FastAPI, let' dot some code.
The first thing I would recommend is to use schemas for serialization, building them using Pydantic, the reason is that you can play with them depending on what are your needs. Therefore, the first file will be called schemas.py and will contain the following code
from typing import Optional
from beanie import PydanticObjectId
from pydantic import BaseModel, Field
from src.models import Profession, Pet
class Pet(BaseModel):
name: Optional[str] = None
class BasePerson(BaseModel):
name: Optional[str] = None
email: str
age: int
profession: Optional[Profession] = None
pet: Optional[Pet] = None
country: Optional[str] = None
class PersonCreate(BasePerson):
pass
class PersonRead(BasePerson):
id: PydanticObjectId = Field()
The second file is the file that contains the FastAPI endpoints and the logic related with beanie, it is called main.py
from motor.motor_asyncio import AsyncIOMotorClient
from beanie import WriteRules, init_beanie
from src.models import Pet, Person
from fastapi import FastAPI
from contextlib import asynccontextmanager
from src.schema import (
PersonCreate,
PersonRead,
Pet as PetSchema,
)
from beanie.operators import In
async def init():
client = AsyncIOMotorClient("mongodb://localhost:27017/myproject")
await init_beanie(database=client["myproject"], document_models=[Person, Pet])
@asynccontextmanager
async def lifespan(app: FastAPI):
await init()
yield
app = FastAPI(lifespan=lifespan)
@app.post("/person", response_model=PersonRead)
async def create_person(person_create: PersonCreate):
person = Person(**person_create.model_dump(mode="json"))
await person.insert(link_rule=WriteRules.WRITE)
return person
@app.get("/person", response_model=list[PersonRead])
async def get_person(name: str = None):
if name:
return await Person.find(Person.name == name).to_list()
return await Person.find_all().sort(+Person.name).to_list()
@app.get("/pet", response_model=list[PetSchema])
async def get_pet(name: str = None):
if name:
return await Pet.find(Pet.name == name).to_list()
return await Pet.find_all().sort(+Pet.name).to_list()
@app.post("/pet", response_model=PetSchema)
async def create_pet(pet_create: PetSchema):
pet = Pet(**pet_create.model_dump(mode="json"))
await pet.insert()
return pet
Run the app
poetry run uvicorn src.main:app --reloadTesting using swagger docs
The main.py file contains Beanie and FastAPI logic, most of the Beanie logic was already explained, there are some code lines to point out.
Line14 to 16 represent a Beanie initialization method, it registers the models and management the connection to MongoDB
Beanie initialization have to be management in efficient way, FastAPI has these events called lifespan, where you can define and execute a piece of code in different moments, line 19 to 22 define that the initialization will be executed when the app start
The rest of the code are methods connected to endpoints, they receive variables and return objects related with the schemas defined before. For instance, the 'create_person' method receives an instance from 'PersonCreate,' and this object is used to create a 'Person' instance (converting the object to dict and sending the fields as parameters).
Conclusion
Beanie is a simple, fast, and easy-to-integrate ODM with the FastAPI ecosystem. Its popularity has been growing in the last month. In addition, Beanie is one of the ODMs that support both versions of Pydantic (although support for V1 will likely be removed), and it is constantly adding new interesting features. Without a doubt, it is a library to consider for projects that require interactions with MongoDB and APIs.