Four methods used to measure network impact.
Results: This story has a happy ending
It turns out that the new feed model exceeded our expectations. We worried that we might see some declines in feed engagement that we would have to weigh against the benefits we were bringing to creators, but actually, this feature turned out to be win-win for both creators and feed viewers. Members like seeing more content from people they know!
Of course, prioritizing posts from close connections means we have less space in the feed to show posts from top creators. The overall effect of the model was to take about 8% of all feedback away from the top 0.1% of creators and redistribute it to the bottom 98%. Since rolling out the model, we’ve seen a reduction in out-of-network highly-viral posts shown in top slots of the feed, indicating that, in aggregate, this change is making more room for posts from close connections at the top of the feed.
As a result, we saw big wins for creators, especially the members with smaller networks. The fraction of creators receiving feedback on their posts increased, and as a result we saw a 5% increase in creators returning to post again.
If this all sounds like our top creators are now going to be hurting because we’ve taken feedback away from them, recall that we’re in an environment of 50% YoY growth in overall responses to posts—taking 8% of the likes away from the top 0.1% still leaves them better off than they were a year ago. These changes just help ensure that the rising tide is lifting all the boats in a fair and equitable fashion.
Of course, we have more plans for optimizing and extending this model, as well as teaching the feed relevance model lots of new tricks involving content understanding, transfer learning, and more. We want to optimize feed as an ecosystem; after all, LinkedIn is a social network where your actions not only affect your own experience, but also impact many other members. We also want to understand members’ long-term behavior, such as interactions with the feed over time, and incorporate that into the relevance model. Other signals that we want to explore further include dwell time, freshness, affinity, and others. Beyond the feed, we plan to also incorporate creator-side optimization into other areas of LinkedIn content recommendations, such as notifications.
We’re not done exploring new methods of experimentation and analysis either. We’re working on new and better ways of measuring the impact of our work on complex, interconnected systems. We know we’ll win some and lose some, but we always want to try lots of new things, take intelligent risks, learn, and iterate, so that ultimately we can enable members to advance their careers through their LinkedIn community.
This story spans the work of several teams, including feed AI (Wei Xue, Ying Xuan, Souvik Ghosh, Tim Jurka) and the greater flagship relevance org (Shaunak Chatterjee, Kinjal Basu), analytics (Guillaume Saint-Jacques, Jia Ding, Ya Xu), and product (Pete Davies).