Airbnb recently described how he built Himeji, a scalable centralized authorization system. Himeji stores authorization data and performs authorization checks as a central source of truth. It uses a fragmented and replicated in-memory cache to improve performance and reduce latency, and served as controls in production for about a year. Its throughput increased from 0 in March 2020 to 850,000 entities / s in March 2021, while maintaining 99.999% availability and 12 millisecond latency at the 99th percentile.
The image below illustrates the three-layered architecture of Himeji.
First, the orchestration layer receives requests from clients and is responsible for retrieving data from the cache. The caching layer, which is fragmented and replicated, is responsible for in-memory filtering and loading from the database if the cache fails. Airbnb is aiming for a cache hit rate of around 98%. Finally, the data layer uses Amazonian Aurora for durable database storage. Airbnb SpinalTap detects data mutations and sends notifications on Apache Kafka to invalidate the cache.
Airbnb Engineer Alain Yao describes the reasons that led to this architecture:
Over the past two years, Airbnb’s engineering has shifted from a monolithic Ruby on Rails architecture to a service-oriented architecture. In our Rails architecture, we had one API per resource to access the underlying data. These APIs had authorization controls to protect sensitive data. Since there was only one way to access a resource’s data, managing these checks was straightforward. In the transition to SOA, we moved to a layered architecture where data services wrap around databases and presentation services hydrate from multiple data services.
According to Yao, Airbnb initially moved authorization controls to presentation services, as shown in the image below.
This choice posed several problems. First, authorization controls were now duplicated and difficult to manage. Second, each authorization check had to deploy across multiple services to perform the required logic, which drastically degraded performance and reliability. The solution was to move authorization controls to data services instead of presentation services and create Himeji, allowing authorization data to be stored centrally and in a scalable manner. The figure below illustrates Himeji and its use in the Airbnb system.
Himeji’s Verification API allows data services to perform authorization verifications. Data services can ask Himeji if a particular principal (for example, a user) has a relationship (for example, a privilege or action) on a specific entity. For example, a data service might ask, “Can user 123 write in the description for ad 10?” This structure is called a tuple. It is inspired by Google Zanzibar, which is Google’s global authorization system.
Authorization rules in Himeji can be stored in the database or derived from configuration. For example, the following configured rule allows a principal to read the location of a list if the principal is the owner of the lists (an owner is a list permission stored in the database). Alternatively, it allows customers of a booking linked to the ad to also read the location.
LISTING:
LOCATION:
'#READ':
union:
- #OWNER
- LISTING : $id # RESERVATION @
Reference(RESERVATION : $reservationId # GUEST)
Therefore, if a guest tries to read the location of the announcement, the data service will check if that user’s principal is authorized to do so. Based on the above rule, Himeji will automatically ask if the principal is a guest on the ad reservation automatically, and it will return the appropriate result.
To reduce onboarding times and drive developer adoption, Airbnb has created tools. These include tools for porting pre-existing authorization data with Apache airflow and Apache Spark and scripts to automatically generate Java and Scala code.
Comments are closed.