How to Use Hugging Face Models with LangChain

TL;DR

Use Hugging Face models with LangChain by installing langchain-huggingface and Transformers, then creating a HuggingFacePipeline with a model ID, task, and pipeline parameters. The walkthrough loads Microsoft Phi-3 Mini 4K Instruct, a 3.8-billion-parameter model, in Google Colab and configures text generation with 100 maximum new tokens and top_k of 50. Read on for the exact setup, token configuration, invocation steps, and endpoint alternative.

Transcript

hello all my name is krishak and welcome to my YouTube channel so guys uh here you can actually see an amazing announcement from both hugging face and langin now they are actually creating a new partner package uh specifically in langen to help langen API specifically call amazing LL models that are available in hugging face right uh so in this vid... Read More

Key Insights

• The langchain-huggingface package is a partner integration jointly maintained by Hugging Face and LangChain. Its purpose is to connect LangChain applications with Hugging Face models while keeping the integration aligned with current Hugging Face developments.
• HuggingFacePipeline is the main class used in the local demonstration. It is imported from langchain_huggingface and instantiated through its from_model_id method, which accepts a Hugging Face model identifier, a task, and pipeline keyword arguments.
• Microsoft Phi-3 Mini 4K Instruct is the model selected for the walkthrough. The transcript identifies it as a 3.8 billion parameter model and explains that it can be loaded directly within the demonstrated Google Colab environment.
• The task parameter determines what the selected Hugging Face model should perform. The walkthrough uses text generation, while also identifying summarization, text-to-text processing, translation, visual question answering, and text-to-image generation as possible model-dependent tasks.
• Generation behavior is configured through pipeline keyword arguments. The example uses max_new_tokens set to 100, top_k set to 50, and a temperature parameter, while noting that padding, maximum length, and truncation settings can also be supplied.
• A Hugging Face access token is required for the demonstrated setup. The token is generated from the access-token area of Hugging Face settings and stored as a private key named HF_TOKEN in the notebook environment.
• Some Hugging Face repositories require custom code to run correctly. The notebook displays a confirmation prompt and a warning to check for malicious code before execution, after which the presenter approves the repository code and downloads the model tensors.
• HuggingFaceEndpoint is introduced as another way to connect a model through LangChain. The shown preparation sets the Hugging Face Hub API token and imports the endpoint integration, presenting it as a simpler alternative to loading the complete model locally.

Questions & Answers

Q: How do you use a Hugging Face model with LangChain?

Install langchain-huggingface and Transformers, then import HuggingFacePipeline from langchain_huggingface. Create the pipeline with HuggingFacePipeline.from_model_id, providing the model ID, task, and pipeline keyword arguments. After the model loads, pass a prompt to llm.invoke to generate a response.

Q: What is the Hugging Face and LangChain partner package?

The langchain-huggingface package is a partner integration maintained by Hugging Face and LangChain. It connects LangChain applications with supported Hugging Face models and is intended to bring current Hugging Face developments into LangChain.

Q: How do you create a HuggingFacePipeline in LangChain?

Import HuggingFacePipeline from langchain_huggingface and call HuggingFacePipeline.from_model_id. Supply the Hugging Face repository model ID, choose a task such as text generation, and provide generation settings through pipeline keyword arguments.

Q: Which model does the LangChain and Hugging Face walkthrough use?

The walkthrough uses Microsoft Phi-3 Mini 4K Instruct. It is described as a 3.8-billion-parameter model and selected because it can be loaded in the demonstrated Google Colab environment.

Q: Which generation parameters are configured for Microsoft Phi-3 Mini 4K Instruct?

The example sets max_new_tokens to 100 and top_k to 50, along with a temperature parameter. It uses max_new_tokens instead of max_length for the Phi-3 setup. Padding, maximum-length, and truncation options are also mentioned as available settings.

Q: Which tasks can HuggingFacePipeline perform in this walkthrough?

The demonstrated task is text generation. The walkthrough also identifies summarization, text-to-text processing, translation, visual question answering, and text-to-image generation as possibilities, depending on the selected Hugging Face model.

Q: How is the Hugging Face access token configured in Google Colab?

Create a token from the access-token area in Hugging Face settings. Store it privately in the Google Colab environment under the name HF_TOKEN so the notebook can access it without displaying the complete value. The token is also used when preparing HuggingFaceEndpoint access.

Q: How do HuggingFacePipeline and HuggingFaceEndpoint differ in the walkthrough?

HuggingFacePipeline downloads and loads Microsoft Phi-3 Mini 4K Instruct in Google Colab before prompts are submitted through the LangChain object. HuggingFaceEndpoint is introduced as a simpler second method that starts by configuring the Hugging Face Hub API token. The walkthrough shows its preparation but not a complete endpoint execution sequence.

Summary & Key Takeaways

• The langchain-huggingface partner package is jointly maintained by Hugging Face and LangChain. It is intended to bring current Hugging Face developments into LangChain, allowing developers to call supported open-source models and incorporate them into applications through dedicated LangChain integrations.

• The local pipeline demonstration installs langchain-huggingface and Transformers, imports HuggingFacePipeline, and creates a pipeline with the from_model_id method. Microsoft Phi-3 Mini 4K Instruct is selected because its 3.8 billion parameters can be loaded in the demonstrated Google Colab environment.

• The pipeline is configured for text generation with max_new_tokens set to 100, top_k set to 50, and a temperature parameter. After the model downloads and any required custom-code confirmation is accepted, the presenter calls llm.invoke with a prompt and reviews the generated continuation.