Data Management in Microservices: The Database-per-Service Pattern

Quick Summary (TL;DR)

The database-per-service pattern …a core principle of microservices architecture. It dictates that each microservice should own and manage its own private database. Other services are not allowed to access this database directly; they must interact with the data only through the owning service’s public API. This pattern ensures loose coupling between services, allowing them to be developed, deployed, and scaled independently. However, it introduces the significant challenge of how to query data that is spread across multiple databases.

Key Takeaways

• Encapsulation and Loose Coupling: By making each service’s database private, you enforce encapsulation. This means a team can change their service’s database schema without coordinating with or breaking any other teams, as long as the service’s API contract remains the same.
• Polyglot Persistence: This pattern allows each service to choose the database technology that is best suited for its specific needs. The user service might use a relational database like PostgreSQL, while the product catalog service might use a document database like MongoDB. This is known as polyglot persistence.
• The Challenge is Querying Across Services: The biggest drawback of this pattern is that it makes it difficult to run queries that join data from multiple services. Since you cannot directly query another service’s database, you must use other patterns to solve this problem.

The Solution: From Shared Database to Database-per-Service

In a monolithic application, it’s common… for all code to share a single, large database. This creates tight coupling; a change to the database schema for one feature can have unintended consequences for another. The database-per-service pattern solves this by decentralizing data ownership. Each microservice becomes the single source of truth for a specific business domain. This aligns with the core microservice principle of autonomy and enables true independent deployment.

Strategies for Querying Data Across Services

Since you can’t use distributed transactions or direct joins, you need to use other patterns to combine data from multiple services.

1. API Composition

• How it Works: A higher-level service …(like an API Gateway or a dedicated aggregator service)… is responsible for fetching data from multiple downstream services and combining it before returning it to the client. For example, to display an order details page, the aggregator would fetch the order data from the Order Service and the customer data from the User Service and then join them in memory.
• Best For: Simple queries and use cases where the number of downstream services to call is small.

2. CQRS (Command Query Responsibility Segregation)

• How it Works: CQRS is an advanced pattern where you separate the model for writing data (the Command model) from the model for reading data (the Query model). You can create a dedicated, de-normalized “read database” that is specifically designed to support a particular query. This read database is kept up-to-date by subscribing to events published by the services that own the data.
• Best For: Complex querying scenarios where performance is critical and eventual consistency is acceptable.

Data Consistency and the Saga Pattern

Without a shared database, you can’t use traditional ACID transactions to maintain consistency across services. The solution is the Saga pattern. A saga is a sequence of local transactions. Each local transaction updates the database within a single service and publishes an event that triggers the next local transaction in the next service. If a local transaction fails, the saga executes a series of compensating transactions to undo the preceding changes.

Common Questions

Q: Does every service literally need its own database server? No. The pattern is about logical separation, not necessarily physical separation. In the early stages, multiple services might use separate schemas within the same physical database server. The critical rule is that they are not allowed to access each other’s schemas directly.

Q: How do I handle a foreign key relationship? Instead of a database foreign key constraint, you simply store the ID of the entity from the other service. For example, the Order Service would store the customer_id, but it would not have a foreign key constraint to the customers table in the User Service’s database.

Tools & Resources

• The Saga Pattern: A detailed article from Microsoft explaining the Saga pattern for managing distributed transactions.
• API Composition Pattern: An explanation of how to implement queries by composing data from multiple services.
• CQRS (Command Query Responsibility Segregation): Martin Fowler’s in-depth article on the CQRS pattern.

Related Topics

Microservices Architecture & Communication

• An Introduction to Microservices Architecture
• Decomposing a Monolith into Microservices
• Microservices Communication Patterns

Distributed Data Management

• Distributed Database Consistency Patterns
• CQRS (Command Query Responsibility Segregation)
• Event Sourcing Database Patterns
• API Composition Patterns

Database Architecture & Selection

• Database Scaling Patterns: Read Replicas, Connection Pooling, and Caching
• NoSQL vs SQL: Database Selection Strategy
• Polyglot Persistence Patterns

Need Help With Implementation?

Decentralized data management is one of the most complex aspects of a microservices architecture. Built By Dakic provides expert consulting on microservices and distributed systems, helping you design a data management strategy that ensures both service autonomy and data consistency. Get in touch for a free consultation.