Some time ago, SoundCloud operated through a single monolithic system that managed all interactions across its web client and mobile apps.
They handled many requests, and when a new feature was added, their services became very complex.
SoundCloud started with a shared API for all platforms like iOS, Android, and Web, but soon, this approach reached its limitations because of the company’s growth.
So they embarked on a journey to transition from a monolith to a microservice architecture.
Sketched using Multiplayer
Transition To Microservices And Backend-For-Frontend (BFF)
Because of the need for scalability both from an organizational and operational point, SoundClound decided to transition to a microservice architecture.
They started breaking down the monolith into smaller, more manageable services, each responsible for a specific feature.
SoundCloud also introduced a BFF architecture. This allowed each team to develop APIs tailored to the needs of different frontends - iOS, Android, Web, etc.
Using BFF helped the teams to be more autonomous and effective.
Sketched using Multiplayer. You can create a 1-month free trial here.
Challenges With BFF And Introduction of VAS
Despite the initial success with the upgraded architecture, SoundCloud faced challenges like duplication of business and authorization logic across multiple BFFs.
This duplication and complexity led to maintenance challenges and synchronization issues.
SoundCloud addressed this challenges by introducing Value-Added-Services (VAS).
VAS operates on a few core principles from Domain-Driven Design (DDD) like domains, entities, value objects, and aggregators.
You can think of a VAS like a separated service tailored to a specific domain like “Notifications”, “Songs”, “Tracks”, “Preferences”, etc.
Here’s is a high-level diagram outlining the overall architecture after this change:
The different service layers at SoundCloud that make up this new architecture. Source: https://developers.soundcloud.com/blog/.
With the introduction of Value-Added-Services (VAS), SoundCloud managed to centralize the logic related to entity handling.
This reduced the duplication and improved the system coherence.
VAS managed entities like Tracks, Users, Comments, etc., and their life cycles.
Each VAS returned an aggregate that included all the necessary metadata.
This new improvement in the design significantly streamlined the operations.
Sketched using Multiplayer. You can create a 1-month free trial here.
And here is a diagram of an example of the Playlists VAS and its usage:
Sketched using Multiplayer. You can create a 1-month free trial here.
The Impact Of VAS On System Design
The implementation of VAS changed how SoundCloud handled data.
This change offered a more centralized approach where entities could be managed through commands and queries, following the Command Query Responsibility Segregation (CQRS) pattern.
The separation allowed different parts of the system to evolve independently, improving maintainability and scalability.
For example, operations that altered the entity’s state were handled separately from those that retrieved data. This ensured efficient processing and consistency.
Here is an example of Tracks VAS and querying different parts of the Aggregate from several services:
Sketched using Multiplayer. You can create a 1-month free trial here.
Here is an example of liking and disliking a Track through Tracks VAS using commands:
Sketched using Multiplayer. You can create a 1-month free trial here.
Evolution To Domain Gateways
As with the SoundCloud business growth and new business domains, a new need has emerged - handle the same entities in different business contexts.
To handle this, SoundCloud introduced a Domain Gateway - a tailored VAT to serve the specific business needs and context.
Each gateway was managed by a different team, providing customized views and operations on entities like Tracks, which could be used differently by Consumers and Creators.
This new approach minimized the cross-team dependencies and maximized the domain-specific enhancements.
Sketched using Multiplayer. You can create a 1-month free trial here.
📋 Recap
- SoundCloud transitioned from monolithic to microservice architecture to improve scalability and maintainability.
- They implemented BFF pattern to optimize APIs for different clients, enhancing team autonomy and operational scalability.
- SoundCloud introduced VAS (DDD based services) to centralize core functionalities and reduce duplication and complexity.
- They developed Domain Gateways to manage and centralize services for specific business domains, further improving scalability and system coherence.