How Do ChatGPT Plugins, Browsing, and Code Work?

TL;DR

ChatGPT’s alpha tools add three practical capabilities: plugins connect conversations to services, web browsing retrieves information from websites, and Code Interpreter analyzes uploaded files by writing Python. Demonstrations show hotel and flight searches through Expedia, grocery-list creation through Instacart, research-paper summarization with GPT-4 browsing, and rapid charting and regression analysis of a LEGO dataset.

Transcript

I got access to multiple alpha models from OpenAI, and we're gonna review them today. Let's get into it. So I got access to plugins, which is already a few weeks old, but I haven't done a review on it. You've probably already seen it out there, but I want to do my own review. I also got GPT 3.5 with browsing, and that actually means you can have GP... Read More

Key Insights

• ChatGPT plugins connect conversational requests to external services that can supply actionable results. In the demonstration, Expedia returns hotel and flight information, Instacart builds a purchasable grocery list, and plugin-generated links transfer the user to the relevant booking or ordering page.
• The plugin store is presented as an app marketplace for ChatGPT. Available examples include Kayak, OpenTable, Wolfram Alpha, Instacart, Expedia, and Zapier, while the reviewed interface permits only three plugins to be enabled at the same time.
• Expedia integration supports a conversational travel-planning sequence. ChatGPT first suggests European destinations, asks for travel dates, retrieves Amsterdam lodging options with ratings and nightly prices, filters for luxury five-star hotels, and provides a link to continue the booking process.
• Plugin calls can fail during ordinary use. The Expedia flight search produces errors more than once, but regenerating the response allows the conversation to continue and eventually returns nonstop options from San Francisco, including departure time, duration, and ticket price.
• Instacart converts a generated meal plan into a practical shopping workflow. ChatGPT proposes breakfasts, lunches, and dinners for two days, compiles their ingredients, sends a titled request to Instacart, and returns a link containing the groceries needed for the plan.
• GPT-4 web browsing retrieves information directly from a provided website instead of requiring the user to paste long source material into the prompt. Its visible process includes opening links, reading content, scrolling through pages, and following references found within the source.
• Web browsing remains useful despite visible retrieval errors. When asked about the Generative Agents research paper, the model records several failed reading attempts but still summarizes the paper and correctly states that twenty-five agents lived in the simulated world.
• Code Interpreter analyzes uploaded files by generating and executing Python. With a LEGO CSV, it uses pandas to read columns, creates plots, and applies linear regression, estimating that the average number of parts per set increased by 2.86 for every advancing year.

Questions & Answers

Q: What can ChatGPT plugins do?

ChatGPT plugins connect a conversation to external services that provide specialized information or actions. The demonstrations show Expedia searching for hotels and flights, then linking to a booking page, while Instacart converts ingredients from a two-day meal plan into an online shopping list. The reviewed interface allows three plugins to be enabled at one time.

Q: How does the Expedia plugin help plan a trip?

The Expedia plugin can enter a conversation after the user chooses a destination and provides travel dates. It retrieves lodging options with hotel ratings, guest ratings, and average nightly prices, then can filter those choices for luxury five-star properties. It also searches for flights after asking for a departure city and provides links that continue the booking process.

Q: How reliable were ChatGPT plugins during the demonstration?

The plugin experience included repeated errors, particularly during the Expedia flight search. Regenerating the response allowed the request to proceed, and the system eventually returned nonstop flight options from San Francisco. The demonstration therefore shows that plugins can complete useful searches and produce actionable links, but users may need to retry a response when a plugin call fails.

Q: How can Instacart turn a meal plan into a grocery order?

ChatGPT first creates a two-day healthy meal plan containing breakfast, lunch, and dinner, including meals such as Greek yogurt parfait, grilled chicken, and baked salmon. It then compiles the required ingredients and sends a titled shopping-list request to Instacart. The resulting link opens a list containing the groceries needed for purchase through Instacart.

Q: How does GPT-4 browsing summarize a research paper?

GPT-4 browsing accepts the paper’s URL, opens the link, reads the available content, scrolls through the page, and may follow connected links. It then produces a summary with references to specific parts of the paper. This process avoids manually copying and pasting a long paper into multiple prompts solely to fit its contents into the conversation.

Q: Can GPT-4 browsing answer follow-up questions about a source?

GPT-4 browsing can answer follow-up questions using information retrieved from a source, although the demonstration shows imperfect retrieval. After summarizing the Generative Agents paper, it is asked how many agents lived in the simulated world. Despite several visible reading failures, it correctly responds that the study’s simulated world was populated by twenty-five generative agents.

Q: What does ChatGPT Code Interpreter do with a CSV file?

Code Interpreter loads an uploaded CSV, inspects its structure, and writes Python to answer questions about the data. For the LEGO dataset, it uses pandas to read columns for set number, name, year, theme ID, and number of parts. It then creates requested visualizations and performs calculations directly from those uploaded records.

Q: How did Code Interpreter analyze changes in LEGO sets?

Code Interpreter creates a scatter plot with release year on the horizontal axis and parts per set on the vertical axis. To estimate the rate of change, it performs linear regression on the dataset. The resulting calculation reports that each advancing year corresponds to an average increase of 2.86 parts per set, and it also creates a line graph of yearly set releases.

Summary & Key Takeaways

• ChatGPT plugins function like apps within a broader ecosystem. The demonstration uses Expedia to recommend Amsterdam hotels and search for first-class nonstop flights, then provides links for booking. Instacart turns a generated two-day healthy meal plan into an online shopping list containing the required ingredients for immediate purchase.

• GPT-4 with web browsing can open a supplied research-paper URL, read its contents, follow links, and produce a referenced summary. The browsing interface reveals actions such as clicking, reading, scrolling, and thinking. Although several reading attempts fail, the model correctly answers that the simulated world contained twenty-five generative agents.

• Code Interpreter accepts an uploaded CSV and writes Python to inspect, analyze, and visualize its contents. Using a LEGO dataset, it identifies the columns, creates a scatter plot of parts per set over time, performs linear regression, calculates an annual increase of 2.86 parts per set, and generates yearly-release charts.