Retrofitting | Stanford CS224U Natural Language Understanding | Spring 2021

TL;DR

Retrofitting allows for the combination of powerful distributional representations with rich knowledge graphs, enabling diverse semantic distinctions.

Transcript

hello everyone welcome to part six in our series on distributed word representations this can be considered an optional part but it's on the irresistibly cool idea of retrofitting vectors to knowledge graphs here are central goals on the one hand as we've seen distributional representations are powerful and also easy to obtain but they tend to refl... Read More

Key Insights

• 📈 Retrofitting combines distributional representations and knowledge graphs to capture both primitive semantic notions and diverse semantic distinctions.
• 📈 Balancing faithfulness to original vectors and resemblance to neighbors in the knowledge graph is crucial in the retrofitting model.
• 🤑 The retrofitting model can be applied to various types of knowledge graphs, including those with rich edge relations.
• 📈 Extensions like functional retrofitting and graph embedding methods enhance the flexibility and applicability of retrofitting.

Questions & Answers

Q: What is the goal of retrofitting vectors to knowledge graphs?

The goal is to combine the power of distributional representations with the rich semantic distinctions offered by knowledge graphs.

Q: How does the retrofitting model balance the opposing forces?

The model balances the pressure to remain faithful to original vectors with the pressure to resemble neighbors in the knowledge graph using alpha and beta parameters.

Q: What happens to nodes in a simple knowledge graph when the retrofitting model is applied?

Nodes in the knowledge graph are pulled closer together, becoming equidistant and more similar to each other compared to the original embedding.

Q: What is the limitation of the retrofitting model?

The model assumes that an edge between nodes indicates similarity, which may not always be the case. Extensions like functional retrofitting can address this limitation and allow for learning different retrofitting modes for different edge semantics.

Summary & Key Takeaways

• Retrofitting is a method that combines existing embedding spaces with knowledge graphs to enhance semantic understanding.

• The retrofitting model consists of two opposing forces: remaining faithful to original vectors and making representations that resemble neighbors in the knowledge graph.

• By balancing these forces, retrofitting pulls related nodes closer together in the embedding space.