[Notes] OpenAi Whisper for Video Subtitles
This is just my notes that I’m taking for my own use while I was trying to figure out OpenAi Whisper to:
- Automatically transcribe videos that have no subtitles
- Works well, even if the speakers have accents or speak fast or hard to hear
- Translate videos from other languages to English
See this post for a less-cluttered list of useful tools
OpenAi Whisper via Google Drive > Colaboratory
April 5th, 2023 – I just found this tool from a YouTube video and I just followed his instructions and immediately transcribed a podcast while I was watching (confirming his instructions are accurate).
Easily transcribe videos (particularly from sites that have no subtitle files: Telegram/Rumble/Brighteon/BitChute and other censored platforms).
It can also help us automatically translate and transcribe all those awesome non-English truth videos, which opens up the world.
This guy explains exactly step-by-step how to use Google Drive (Colaboratory) to install and use Whisper AI right from your browser without anything to download.
Keep in mind:
- ) Download the text straight away (it times out and deletes your files after a certain timeframe).
- So don’t load up and leave the house, just download the text/subtitles straight away.
- If you time-out, you have to reload the !Whisper code (30seconds) and re-upload your file.
- ) Google can decide at any moment that it isn’t going to allow you to use GPU (without payment); thereby, alternatives to Google Colab will be listed as I come across them below.
Alternatives: Other Online versions (alternatives to Google Colab version)
- Replicate: https://replicate.com/openai/whisper (uses Large and Large-v2)
Alternative: for YouTube or really crappy Subtitles:
I’ve been using chatGPT to clean-up crappy YouTube subtitles and make them more accurate:
When using chatGPT to clean up YouTube’s god-awful auto-generated subtitles, my “go-to” commands are generally one of the following:
- Please correct punctuation and spelling of the following subtitles: “your crappy subtitle text here “
- Please summarize the following text: ” “
- Please provide key highlights of the following subtitles, in bulletpoint form: ” “
- Please correct grammar for following subtitles: ” “
Alternative: (for Very Large files) VB-CABLE Virtual Audio & Dictation Pro
An alternative to Whisper AI for larger videos (for example if you don’t want to upload a 3-hour video file, the same guy (Kevin Stratvert) has another video on how to use Window’s own built-in Dictator Pro software with Virtual Audio Cable (both free), which you have to run on your own computer but might be a life-saver for someone. However it looks to me that it transcribes in real time i.e. a 3 hour video will take 3 hours playtime. All the other apps transcribe the video in minutes. but their downside is that you have to sometimes make the file smaller, and it might just be easier to use this method in some circumstances.
Alternative: Computer Apps using Whisper AI to get subtitles
- Buzz: Instructions (YouTube)
- Using: https://github.com/chidiwilliams/buzz (no internet connection needed)
- Subtitle Edit: Instructions (YouTube)
- Using: https://github.com/SubtitleEdit/subtitleedit/releases/tag/3.6.10
- (tested just now using 37min video, it takes 16 minutes using base model – downside is it keeps the timestamps)
- Speech Translate: Extremely Boring Instructions (YouTube)
- Using: https://github.com/Dadangdut33/Speech-Translate/releases/tag/1.1.0
- Finally got this working by being patient. It looks like there is an error, but eventually it spits out a subtitle file.
Python & WhisperAI
From what I have learnt, I’ll use these commands the most using Python:
To transcribe a video and output just a text file:
whisper "test file.mp4" --model small.en --output_format txt --task transcribe --length_penalty 0 --device cuda
To transcribe & translate a Spanish mp4 into mp3 to translate into English subtitles & text:
ffmpeg -i "deadwhistleblower.mp4" -q:a 0 -map a deadwhistleblower.mp3 whisper "deadwhistleblower.mp3" --model tiny --language Spanish --task translate --patience 2 --device cuda
To transcribe a Spanish video into Spanish text: (to use a different program to translate if the whisper translation is rubbish):
whisper "deadwhistleblower.mp4" --model tiny --language Spanish --output_format txt --task transcribe --patience 2 --device cuda
Convert existing mp4 into mp3 in seconds
- Browse to folder with mp4 file
- basic use:
- ffmpeg -i example.mp4 example.mp3
For my use:
ffmpeg -i "testvideo.mp4" -q:a 0 -map a testvideo.mp3
Split Videos in seconds (to less than 25mb for use with whisper)
Might as well learn how to do this too (as a faster option to my video editing software)
Instructions: timestamp 10:20 from first video (YouTube)
- Using https://github.com/mifi/lossless-cut/releases
- For me: https://github.com/mifi/lossless-cut/releases/download/v3.54.0/LosslessCut-win-x64.7z
- Open LosslessCut exe
- Drag video into app
- Change timestamp
- Export the cut
- (WOW, that was so fast… normally this takes a long time with video editing software, it split the video in seconds!)
chatGPT wrote a whisper script for me but it isn’t supported. but I’m keeping it on here as a must-have wish-list:
--punctuate --filter_outputs "uh, um, like, and and, I I, you know, like, basically, actually, sort of, kind of, hmm, mm, mhm, mmm, oh"
Download Videos in seconds
Instructions timestamp 07:48 on this video (YouTube) or Basic instructions 01:35 (YouTube) and GUI instructions (YouTube) and you’ll need FFMPEG as well (listed further up the post in the !whisper instructions)
- Also uses: yt-dlp: https://github.com/yt-dlp/yt-dlp
- for my pc: https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.exe
- OR for a GUI version yt-dlp (above)
- Or Alternative GUI: https://github.com/ErrorFlynn/ytdlp-interface
- Or Alternative GUI: https://github.com/BKSalman/ytdlp-gui
To use GUI version
- Move all 3 to same folder
- Launch yt-dlp-gui
- Enter url > analyze
- Select which video resolution you want
- Select which audio quality you want
- Select or uncheck thumbnail
- Select output directory
To use CMD version
- Move all 3 to same folder
- CMD from folder
- yt-dlp https://youtu.be/7QPbfKDOkO4
- Downloads the video in seconds
Filtering/adjusting CMD version – go to this post: [CMD] Download videos & snippets in seconds
Must-use resources for researchers are on these posts:
- Please SAVE & SHARE With Other RESEARCHERS (https://pennybutler.com/truthseeker-tools/)
- Useful Tools for TruthSeekers (https://pennybutler.com/sleuth-tools/)
- Free Online Video Downloader (https://pennybutler.com/video-dl/)
- [CMD] Download videos & snippets in seconds (https://pennybutler.com/cmd-download-videos/)
Telegram Channels: Get Post Updates | Post Comments | Videos
Get post updates from either Gettr or my Telegram channel: @JourneyToABetterLife. To comment on a post, the easiest way is to post in Telegram. (Note: I stopped logging into Facebook in January 2023 and haven't looked back!). Best way to reach me (although I don't login everyday) is putting a message in Telegram, Gettr, or SMS.