Homework 2: Sentiment Analysis | Stanford CS224U Natural Language Understanding | Spring 2021

TL;DR

An overview of Homework 2, focusing on cross-domain sentiment analysis and the tasks and datasets involved.

Transcript

hello everyone this video is an overview of homework 2 which is on supervised sentiment analysis and i would actually think of it as an experiment in cross domain sentiment analysis let's just walk through this notebook and i'll try to give you a feel for the problem and our thinking behind it so the plot is the usual one we're going to introduce a... Read More

Key Insights

• 😵 Homework 2 focuses on cross-domain sentiment analysis using the SST and a new restaurant review dataset.
• 🧑‍🎓 Students are encouraged to explore additional training data, such as the DynaSent dataset, to improve system performance.
• 🖐️ Feature representation plays a crucial role, with suggestions to experiment with vector averaging and Burston coding methods.
• 😒 Students have the flexibility to use various model architectures, including logistic regression and shallow and deep neural classifiers.
• ❓ Error analysis is essential for identifying system weaknesses and improving performance.
• 😒 The use of the sst.experiment framework is recommended for efficient experimentation.
• 😫 The SST test set is off-limits for development to ensure the scientific integrity of the evaluation process.

Questions & Answers

Q: What is the purpose of Homework 2?

The purpose of Homework 2 is to familiarize students with supervised sentiment analysis and the development lifecycle of sentiment analysis systems. It involves tasks such as feature function writing, model architecture exploration, hyperparameter tuning, and error analysis.

Q: What datasets are used in Homework 2?

Homework 2 utilizes two datasets: the Stanford Sentiment Tree Bank (SST) and a new assessment dataset consisting of sentences from restaurant reviews. Students train their models on the SST train set and evaluate them on SST dev set and the restaurant review dev set for cross-domain analysis.

Q: Can additional training data be used in Homework 2?

Yes, students have the freedom to bring in additional training datasets. It is suggested to consider using the DynaSent dataset, which shares the same labeling protocols as the restaurant review dev set, to address label shift and improve performance on the new data.

Q: How should students approach system development in Homework 2?

Students should focus on feature function creation, model architecture exploration, and potentially using deep learning approaches. They have the freedom to experiment with different models and bring in new data for training. The final system performance is evaluated based on the macro average F1 scores on both the SST dev set and the restaurant review dev set.

Summary & Key Takeaways

• The video provides an overview of Homework 2 on supervised sentiment analysis, emphasizing cross-domain analysis using two datasets: the Stanford Sentiment Tree Bank and a new assessment dataset from restaurant reviews.

• The assignment aims to reinforce core concepts and techniques in sentiment analysis, including model development, feature functions, model architectures, hyperparameter tuning, and error analysis.

• Students are encouraged to explore the data, set up additional baselines, and potentially bring in new training data to improve system performance.