Search Federation Architecture at LinkedIn

Co-authors: Yi Shen, Claire Liu, and Ali Mohamed

Introduction: A brief history of Search Federation at LinkedIn

Almost every part of LinkedIn contains data that needs to be discoverable by our members or customers. Use cases range from a member looking up a news article posted by someone in their network to a recruiter looking for candidates on the platform. One of the primary mechanisms for discovering this content is search.

Accordingly, at LinkedIn, we have many different search engines for a variety of use cases, such as People Search (members) and Job Search (posted job opportunities). The Search Federation team’s mission is to help our members and customers find resources by searching across these engines. We call these use case-focused engines “search backends.”

We also provide a family of mid-tier services designed to answer search and search typeahead queries. These services typically follow the federation pattern, where the user’s query is first expanded through query understanding, spell checking, and other techniques. This expansion is used to write queries specific for each search backend. The documents returned from the backends are then combined to be most useful to the user through a process called “blending.”

Collectively, this process makes up Search Federation at LinkedIn. Search Federation provides us with a way to personalize ambiguous queries. For instance, when a user searches for “machine learning” on LinkedIn, they could mean to search for people with machine learning skills, jobs requiring machine learning skills, or content about the topic.

Search Federation mid-tier building blocks
Historically, our Search Federation architecture has consisted of three mid-tier services.

The federated-search-rest service was created in 2012 to provide a API for the Search endpoint and served as a Search Federation mid-tier for the blended search experience. Additionally, federated-search-rest calls the search-decoration service to decorate the result with unindexed information.

The typeahead-rest service was created in Q1 2014 to provide a API for the typeahead endpoint and to serve as a typeahead federation mid-tier for the blended typeahead experience.

The seas-federated-search service was created in Q2 2014 to be the new federator used by all external services to access search. During the Galene migrations, all calls from federated-search-rest to the legacy backends were moved over to seas-federated-search. In this blog post, we use “new federation mid-tier” to refer to seas-federated-search service.

Like with any service, over the years product requirements have changed, the number of callers has grown, and an increase in use cases has resulted in a good amount of legacy code. At the same time, there has been a tremendous increase in demand for ease-of-use and rapid iteration from developers modifying Search Federation architecture (110+ code contributors and 800+ commits, for instance, in 2017) for different verticals.

In early 2015, the team identified three main challenges in our legacy federation architecture that were impeding our ability to iterate quickly and threatened search stability as our scale increased:

  • Leverage between search and typeahead – Similar federation logic in these closely-related mid-tiers was written with different design patterns. This meant adding a new feature would always involve changing in stacks of multiple services and making separate deployments.

  • Code isolation – The legacy design was based on inheritance so that verticals serving different use cases were inevitably coupled. On top of that, it wasn’t easy to extend for new vertical use cases in a clean way, and complicated if-else clauses were added in many shared components.

  • Test and deployment – Low test coverage in 2015 and no Search Federation services supporting daily deployment made it difficult to reliably deploy changes without regressions.

Roadmap: Building a bridge to the future

To address the above problems, the Search Federation team began implementing a redesign of the federation service with the following strategies: services consolidation, code isolation, and improving our overall test coverage.

Services consolidation:

  • We merged the legacy search federation mid-tier and legacy typeahead federation mid-tier into a new federation mid-tier to ease leveraging between search and typeahead. 
  • Access to all backends now goes through this new federation mid-tier, the same place where query understanding, spell checking, and data fetching are happening.

Code isolation:

  • We used a workflow framework to compose a use case specific workflow, which provides code isolation among verticals. No if-else clauses are needed, and new use cases can be onboarded easily by reusing common workflows. 
  • Modularize code per vertical use case.

Overall test coverage improvement:

  • Add more unit tests to increase code coverage. 
  • Add more integration tests to enhance PCx test. 
  • Add dark canary hosts.

Additionally, this had to be accomplished while accommodating the search needs of a rapidly-growing global membership, which was dramatically increasing the overall amount and number of types of content that needed to be searchable on LinkedIn. These issues needed to be addressed without disrupting the the 10,000+ searches that LinkedIn handles every single day.

Progress so far

The Search Federation team kicked off the Search Federation Re-architecture Project by building the SeaS workflow framework, then we rolled-out the migration in stages: individual workflow migration, typeahead federation consolidation, and Search Federation consolidation.

Workflow migration (2016)
Starting in Q4 2015 and working through Q2 2016, we successfully migrated all query processing to new federation mid-tier and all 40+ search use cases to SeaS workflow. Access to all Galene backends was routed through this new federation mid-tier.

Source link