Using OpenAI's Speech to Text API with the Glitch Code Editor for Building an Eleventy Blog

Using OpenAI's Speech to Text API with the Glitch Code Editor for Building an Eleventy Blog

Introduction:
The Speech to Text API offered by OpenAI provides users with the ability to convert audio files into text using their state-of-the-art open-source large-v2 Whisper model. This API offers two endpoints: transcriptions and translations. Transcriptions allow users to transcribe audio into any language, while translations transcribe and translate the audio into English. This article will explore how to use the Speech to Text API in conjunction with the Glitch Code Editor to build an Eleventy blog.

Transcriptions Endpoint:
The transcriptions API endpoint is used to transcribe audio files into text. Users can provide the audio file they wish to transcribe, along with the desired output file format for the transcription. OpenAI's Speech to Text API supports multiple input and output file formats. To transcribe audio using Python, users need to have OpenAI Python v0.27.0 installed. The following code snippet demonstrates how to transcribe audio using Python:

import openai  
  
audio_file = open("/path/to/file/audio.mp3", "rb")  
transcript = openai.Audio.transcribe("whisper-1", audio_file)  

Alternatively, users can use cURL to make a POST request to the transcriptions endpoint. The following cURL command demonstrates how to transcribe audio:

curl --request POST \  
  --url https://api.openai.com/v1/audio/transcriptions \  
  --header 'Authorization: Bearer TOKEN' \  
  --header 'Content-Type: multipart/form-data' \  
  --form file=@/path/to/file/openai.mp3 \  
  --form model=whisper-1  

The response from the API will be in JSON format, with the raw transcribed text included.

Translations Endpoint:
The translations API endpoint is used to transcribe audio files in any supported language and translate them into English. This endpoint differs from the transcriptions endpoint, as the output text is not in the original input language but is instead translated into English. Similar to the transcriptions endpoint, users can provide the audio file they wish to translate, along with the desired output file format. The following code snippet demonstrates how to translate audio using Python:

import openai  
  
audio_file = open("/path/to/file/german.mp3", "rb")  
transcript = openai.Audio.translate("whisper-1", audio_file)  

Users can also use cURL to make a POST request to the translations endpoint. The following cURL command demonstrates how to translate audio:

curl --request POST \  
  --url https://api.openai.com/v1/audio/translations \  
  --header 'Authorization: Bearer TOKEN' \  
  --header 'Content-Type: multipart/form-data' \  
  --form file=@/path/to/file/german.mp3 \  
  --form model=whisper-1  

Supported Languages:
OpenAI's Speech to Text API currently supports a wide range of languages for both the transcriptions and translations endpoints. These languages include Afrikaans, Arabic, Armenian, Azerbaijani, Belarusian, Bosnian, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Marathi, Maori, Nepali, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tagalog, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Welsh. It's important to note that while the Whisper model was trained on 98 languages, only languages with a word error rate (WER) below 50% are listed. The model can still process languages not listed, but the quality may be lower.

Handling Longer Inputs:
By default, the Whisper API has a limit of 25 MB for input audio files. If users have an audio file that exceeds this limit, they will need to break it up into smaller chunks of 25 MB or less. Alternatively, users can use compressed audio formats to stay within the size limit. It's recommended to avoid splitting audio files mid-sentence to ensure the context is not lost. The PyDub open-source Python package can be used to split audio files into smaller chunks. The following code snippet demonstrates how to split an audio file using PyDub:

from pydub import AudioSegment  
  
song = AudioSegment.from_mp3("good_morning.mp3")  
ten_minutes = 10 * 60 * 1000  
first_10_minutes = song[:ten_minutes]  
first_10_minutes.export("good_morning_10.mp3", format="mp3")  

Prompting for Improved Transcriptions:
Users can use prompts to enhance the quality of the transcriptions generated by the Whisper API. The model will attempt to match the style of the prompt, resulting in more accurate capitalization and punctuation usage. Prompting is particularly helpful for correcting specific words or acronyms that the model frequently misrecognizes in the audio. To preserve context in files split into segments, users can prompt the model with the transcript of the preceding segment. This will improve the accuracy of the transcript by incorporating relevant information from the previous audio. It's important to note that the model only considers the final 224 tokens of the prompt and ignores anything earlier. Additionally, prompts can help address issues such as the model skipping punctuation or leaving out common filler words in the transcript. Users can create prompts that include punctuation or contain filler words to ensure the desired transcription format.

Using the Glitch Code Editor for Building an Eleventy Blog:
The Glitch Code Editor is a powerful tool for building web apps and websites. When combined with the Speech to Text API, it provides an efficient way to transcribe audio files and integrate them into an Eleventy blog. Eleventy is a lightweight static site generator that builds plain HTML files for fast loading by visitors. The Glitch Code Editor allows for real-time previewing of changes while coding, making the development process seamless. The Eleventy blog project on Glitch includes default posts and layouts that serve as a foundation for customization. Users can leverage their HTML and JavaScript knowledge to modify the site's build process in the code.

Actionable Advice:

1. Utilize prompts effectively: Experiment with different prompts to improve the accuracy and formatting of transcriptions. Prompting can be especially useful for correcting specific words or acronyms that the model may misrecognize.
2. Split longer audio files: If you encounter audio files that exceed the 25 MB limit, consider using the PyDub package to split them into smaller chunks. This will ensure compatibility with the Whisper API and maintain context within the transcripts.
3. Leverage the Glitch Code Editor: Take advantage of the real-time preview capabilities of the Glitch Code Editor to visualize changes to your Eleventy blog while coding. This feature enables a smoother development experience and faster iteration.

In conclusion, the combination of OpenAI's Speech to Text API and the Glitch Code Editor offers developers a powerful solution for transcribing audio files and building an Eleventy blog. By following the guidelines provided and implementing the actionable advice, users can harness the full potential of these tools to create engaging and dynamic content.