Pinterest’s Visual Lens: How computer vision explores your taste while building virality into the product

Pinterest’s Visual Lens: How computer vision explores your taste while building virality into the product

Pinterest, the popular visual discovery engine, has been utilizing machine learning techniques to solve various challenging problems. From recommending interests to generating engaging home-feeds, machine learning has played a crucial role in enhancing the user experience on Pinterest.

In January 2015, Pinterest made a significant move by acquiring Kosei, a machine learning startup specializing in recommender systems. This acquisition marked a critical moment for Pinterest, as it allowed them to delve deeper into the world of computer vision and explore the possibilities it offers.

One of the key applications of computer vision on Pinterest is identifying visual similarities. Machine learning algorithms not only determine the subject of an image but can also identify visual patterns and match them to other photos. This capability enables Pinterest to offer personalized recommendations based on the visual preferences of its users.

Categorizing and curating is another area where computer vision has made a significant impact. For example, if a user pins a mid-century dining-room table, Pinterest can now suggest other objects from the same era, expanding the user's interests and providing a more engaging experience.

Predicting engagement is crucial for any platform, and Pinterest has leveraged machine learning to achieve this. By analyzing an individual's tastes and habits, such as what they've pinned and when they've pinned it, Pinterest can surface more personalized recommendations, increasing user engagement.

Pinterest has also prioritized local taste by training its recommendation engine to suggest popular content from users' local regions in their native languages. This localization strategy helps Pinterest cater to the unique preferences and interests of its diverse user base.

In addition to images, Pinterest also looks at captions from previously pinned content and the items that are pinned to the same virtual boards. This holistic approach allows Pinterest to link related items, even if they may not look alike, based on the context in which they are pinned together.

One of Pinterest's major endeavors in the field of computer vision was its focus on visual search. In 2014, Pinterest acquired VisualGraph, an image-recognition startup, and established its computer vision team. This move marked the beginning of Pinterest's exploration of visual search and its potential applications.

Instant Ideas, a feature introduced by Pinterest, enables users to transform their home feed with similar ideas in just a tap. This feature showcases Pinterest's ability to combine its understanding of images and objects with its discovery technologies to offer a diverse set of results to its users.

To achieve this, Pinterest's Lens architecture is divided into two logical components. The first component focuses on understanding the query image by computing visual features such as object detection, salient colors, and image quality conditions. The second component, known as the blender, combines results from multiple sources, including visual search, object search, and image search, to provide a comprehensive set of recommendations.

Pinterest conducted extensive experiments and A/B testing to refine its computer vision capabilities. These experiments allowed Pinterest to show visually similar pin recommendations based on specific objects in an image. By simplifying the discovery experience, Pinterest made it easier for users to explore related items and expand their interests.

Object detection in visual search plays a crucial role in Pinterest's computer vision capabilities. To train its object detection algorithms, Pinterest collected labeled bounding boxes for regions of interest in images. This data aggregation process helped Pinterest understand which objects users are interested in and improve its object detection capabilities.

Real-time object detection is essential for Pinterest's visual search engine, as it allows users to search using any image, including unseen content from the web or their own camera. Pinterest's network uses a fully convolutional network to identify regions of an image that likely contain objects of interest. The network also outputs adjustments to these regions to better frame the objects.

Object search, another aspect of Pinterest's computer vision, treats objects as the unit for search. Given an input image, Pinterest finds the most visually similar objects in billions of images and returns scenes containing these objects. This capability enhances the user experience by providing relevant and visually appealing results.

According to Pinterest's CEO, Ben Silbermann, computer vision serves three primary purposes for the company. Firstly, it helps understand the aesthetic qualities of a product or service, enabling better recommendations. Secondly, it allows Pinterest to zoom in on specific objects within an image and provide similar recommendations. Lastly, Pinterest aims to develop a camera tool that can query the world around the user, leveraging the power of computer vision.

Building virality into a product is crucial for its success, and Pinterest has implemented several strategies to achieve this. One effective approach is creating incentives for users to invite their friends and for their friends to accept the invitation. By rewarding both sides, Pinterest ensures that the incentive is compelling enough to catch on.

LinkedIn, in its early days, utilized a similar strategy by showcasing the number of connections on a user's profile. This feature incentivized users to invite more people to connect with them, expanding the network and increasing user engagement.

Collaborative or communication-focused apps inherently have viral potential. By enabling multiple people to collaborate or communicate within the app, the value of the product increases, attracting more users through word-of-mouth.

Another way to build virality is by allowing others to embed the product into their websites. This not only creates exposure for the product but also familiarizes potential users with its content and behaviors.

Social sharing is a powerful tool for spreading knowledge of a product. If a product generates uniquely interesting content, users can be encouraged to share it on their social networks. Auto-shared content onto other networks has proven to be highly successful in attracting new users.

Creating artifacts or URLs that are expected to be shared via messaging is another effective strategy for virality. Additionally, providing a rich experience around the shared information increases the likelihood of users sharing it with their network.

Viral loops, which involve users spreading awareness and consideration of a product, are essential. However, conversion and loyalty depend on the quality of the product itself. Virality should be built into the core mechanics of the product from the start, rather than being added as an afterthought.

In conclusion, Pinterest's exploration of computer vision has revolutionized the visual discovery experience for its users. By leveraging machine learning techniques and incorporating computer vision into its core architecture, Pinterest has been able to offer personalized recommendations, enhance user engagement, and provide a seamless visual search experience. Additionally, Pinterest's focus on building virality into its product has been instrumental in its growth and success. By implementing strategies such as incentives, collaboration features, social sharing, and rich experiences, Pinterest has effectively harnessed the power of virality to expand its user base and increase user engagement.

3 Actionable Advice:

1. Incorporate machine learning and computer vision techniques into your product to enhance the user experience and provide personalized recommendations.
2. Build virality into your product from the start by creating incentives for users to invite their friends and embedding the product into other websites.
3. Utilize social sharing and messaging features to encourage users to share your product with their network, increasing awareness and user engagement.