61 lines
1.7 KiB
Markdown
61 lines
1.7 KiB
Markdown
# Subtitle Extraction Investigation
|
|
|
|
## Overview
|
|
This script compares different methods and granularity levels for subtitle extraction from video files.
|
|
|
|
## Installation Requirements
|
|
|
|
### FFmpeg (Required for Audio Extraction)
|
|
|
|
**Windows:**
|
|
1. Download from: https://ffmpeg.org/download.html
|
|
2. Or use winget: `winget install ffmpeg`
|
|
3. Or use chocolatey: `choco install ffmpeg`
|
|
|
|
After installation, restart your terminal and verify:
|
|
```powershell
|
|
ffmpeg -version
|
|
```
|
|
|
|
## Features
|
|
|
|
### Granularity Levels
|
|
1. **Movie-level**: Entire video summary + full transcript
|
|
2. **Second-level**: Time-stamped subtitles by phrase/sentence (MM:SS.mmm)
|
|
3. **Word-level**: Individual word timestamps
|
|
|
|
### Comparison Methods
|
|
- **Method A**: Upload video directly to Gemini
|
|
- **Method B**: Extract audio with ffmpeg, then upload audio only
|
|
|
|
## Results Analysis
|
|
|
|
### Token Usage (from first run - video only):
|
|
- **Movie-level**: 15,618 tokens | 32s processing
|
|
- **Second-level**: 19,456 tokens | 96s processing
|
|
- **Word-level**: 40,279 tokens | 226s processing
|
|
|
|
### Key Findings:
|
|
1. Higher granularity = more tokens + longer processing
|
|
2. Word-level is ~2.5x more expensive than movie-level
|
|
3. Second-level provides good balance for subtitle generation
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
# Activate virtual environment
|
|
.\venv\Scripts\Activate.ps1
|
|
|
|
# Run investigation
|
|
python backend/investigate_subtitle_extraction.py
|
|
```
|
|
|
|
## Output Files
|
|
- `assets/1_processed/subtitle_investigation/*.json` - Individual comparison results
|
|
- `complete_comparison.json` - Full comparison data with metadata
|
|
|
|
## Next Steps
|
|
- Install ffmpeg to test Method B (audio extraction)
|
|
- Compare audio vs video method performance
|
|
- Determine optimal approach for production pipeline
|