How to Use Hugging Face Models with LangChain

TL;DR

Install langchain-huggingface and Transformers, then create a HuggingFacePipeline with a model ID, task, and generation parameters before calling the model with invoke. The demonstrated setup uses Microsoft Phi-3 Mini 4K Instruct, a Hugging Face access token, and Google Colab, while HuggingFaceEndpoint is introduced as a second way to connect LangChain to hosted models.

Transcript

hello all my name is krishak and welcome to my YouTube channel so guys uh here you can actually see an amazing announcement from both hugging face and langin now they are actually creating a new partner package uh specifically in langen to help langen API specifically call amazing LL models that are available in hugging face right uh so in this vid... Read More

Key Insights

• The langchain-huggingface package is a partner integration jointly maintained by Hugging Face and LangChain. Its purpose is to connect LangChain applications with Hugging Face models while keeping the integration aligned with current Hugging Face developments.
• HuggingFacePipeline is the main class used in the local demonstration. It is imported from langchain_huggingface and instantiated through its from_model_id method, which accepts a Hugging Face model identifier, a task, and pipeline keyword arguments.
• Microsoft Phi-3 Mini 4K Instruct is the model selected for the walkthrough. The transcript identifies it as a 3.8 billion parameter model and explains that it can be loaded directly within the demonstrated Google Colab environment.
• The task parameter determines what the selected Hugging Face model should perform. The walkthrough uses text generation, while also identifying summarization, text-to-text processing, translation, visual question answering, and text-to-image generation as possible model-dependent tasks.
• Generation behavior is configured through pipeline keyword arguments. The example uses max_new_tokens set to 100, top_k set to 50, and a temperature parameter, while noting that padding, maximum length, and truncation settings can also be supplied.
• A Hugging Face access token is required for the demonstrated setup. The token is generated from the access-token area of Hugging Face settings and stored as a private key named HF_TOKEN in the notebook environment.
• Some Hugging Face repositories require custom code to run correctly. The notebook displays a confirmation prompt and a warning to check for malicious code before execution, after which the presenter approves the repository code and downloads the model tensors.
• HuggingFaceEndpoint is introduced as another way to connect a model through LangChain. The shown preparation sets the Hugging Face Hub API token and imports the endpoint integration, presenting it as a simpler alternative to loading the complete model locally.

Questions & Answers

Q: How do you use a Hugging Face model with LangChain?

Install the langchain-huggingface and Transformers packages, then import HuggingFacePipeline from langchain_huggingface. Create the model through HuggingFacePipeline.from_model_id, supplying a Hugging Face model ID, the required task, and any pipeline keyword arguments. Once the model finishes downloading and loading, send a prompt through llm.invoke to generate the result.

Q: What is the langchain-huggingface partner package?

The langchain-huggingface package is a Python partner package jointly maintained by Hugging Face and LangChain. It is designed to bring recent Hugging Face developments into LangChain and keep the integration current. The package lets LangChain applications call supported Hugging Face models through components such as HuggingFacePipeline and HuggingFaceEndpoint.

Q: How do you create a HuggingFacePipeline in LangChain?

Import HuggingFacePipeline from langchain_huggingface and call HuggingFacePipeline.from_model_id. Provide the repository model ID, select a task such as text generation, and pass model-specific generation settings through pipeline keyword arguments. The walkthrough initially produces a validation error because from_model_id was omitted, then corrects the construction by using that method.

Q: Which model is used in the LangChain Hugging Face example?

The example uses Microsoft Phi-3 Mini 4K Instruct from Hugging Face. The transcript describes it as having 3.8 billion parameters and selects it because it can be loaded in the demonstrated Google Colab environment. Its Hugging Face repository name is copied into the model_id field used by HuggingFacePipeline.from_model_id.

Q: Which generation parameters are configured for Phi-3?

The pipeline configuration uses max_new_tokens set to 100, top_k set to 50, and a temperature parameter. The presenter specifically uses max_new_tokens instead of max_length for the Phi-3 example. Padding, maximum-length, and truncation settings are mentioned as additional possibilities, but they are not added to the demonstrated configuration.

Q: How do you configure a Hugging Face access token?

Open the settings area on the Hugging Face website, navigate to the access-token page, and obtain a token. In the demonstrated Google Colab setup, the token is stored privately under the name HF_TOKEN so the notebook can use it without displaying the full value. The same token is also assigned for Hugging Face Hub endpoint access.

Q: How do you run prompts after loading the Hugging Face model?

After HuggingFacePipeline finishes downloading and loading the selected model, call llm.invoke with a text prompt. The demonstration supplies short text beginnings, including a prompt about LangChain, and lets the text-generation model continue them. Warnings about sampling settings or GPU utilization may appear before the generated output is returned.

Q: What is the difference between HuggingFacePipeline and HuggingFaceEndpoint in the walkthrough?

HuggingFacePipeline is demonstrated by downloading and loading the selected model in Google Colab, then invoking it through the LangChain object. HuggingFaceEndpoint is introduced as a second and simpler calling method that begins by configuring the Hugging Face Hub API token. The transcript shows the endpoint setup but does not provide its complete execution sequence.

Summary & Key Takeaways

• The langchain-huggingface partner package is jointly maintained by Hugging Face and LangChain. It is intended to bring current Hugging Face developments into LangChain, allowing developers to call supported open-source models and incorporate them into applications through dedicated LangChain integrations.

• The local pipeline demonstration installs langchain-huggingface and Transformers, imports HuggingFacePipeline, and creates a pipeline with the from_model_id method. Microsoft Phi-3 Mini 4K Instruct is selected because its 3.8 billion parameters can be loaded in the demonstrated Google Colab environment.

• The pipeline is configured for text generation with max_new_tokens set to 100, top_k set to 50, and a temperature parameter. After the model downloads and any required custom-code confirmation is accepted, the presenter calls llm.invoke with a prompt and reviews the generated continuation.