AI-video-lifeplan-demo/backend/SUBTITLE_INVESTIGATION_README.md

Subtitle Extraction Investigation

Overview

This script compares different methods and granularity levels for subtitle extraction from video files.

Installation Requirements

FFmpeg (Required for Audio Extraction)

Windows:

1. Download from: https://ffmpeg.org/download.html
2. Or use winget: winget install ffmpeg
3. Or use chocolatey: choco install ffmpeg

After installation, restart your terminal and verify:

ffmpeg -version

Features

Granularity Levels

1. Movie-level: Entire video summary + full transcript
2. Second-level: Time-stamped subtitles by phrase/sentence (MM:SS.mmm)
3. Word-level: Individual word timestamps

Comparison Methods

• Method A: Upload video directly to Gemini
• Method B: Extract audio with ffmpeg, then upload audio only

Results Analysis

Token Usage (from first run - video only):

• Movie-level: 15,618 tokens | 32s processing
• Second-level: 19,456 tokens | 96s processing
• Word-level: 40,279 tokens | 226s processing

Key Findings:

1. Higher granularity = more tokens + longer processing
2. Word-level is ~2.5x more expensive than movie-level
3. Second-level provides good balance for subtitle generation

Usage

# Activate virtual environment
.\venv\Scripts\Activate.ps1

# Run investigation
python backend/investigate_subtitle_extraction.py

Output Files

• assets/1_processed/subtitle_investigation/*.json - Individual comparison results
• complete_comparison.json - Full comparison data with metadata

Next Steps

• Install ffmpeg to test Method B (audio extraction)
• Compare audio vs video method performance
• Determine optimal approach for production pipeline