5.9 KiB

Raw Permalink Blame History

Auto Life Plan Video Generator

A hybrid Python/Node.js system that automatically generates financial advice videos from PDF reports and interview footage.

🎯 Project Overview

This system transforms static Life Plan reports (PDF) and raw interview videos into dynamic, personalized 3-5 minute explanatory videos combining:

Animated Charts (Asset depletion, cash flow analysis)
AI-Generated Narration (Natural Japanese scripts)
B-Roll Footage (Contextual stock footage)
Interview Clips (Synchronized with content)

🏗️ Architecture

┌─────────────┐
│   Python    │  ← Director (Orchestration, AI, Data Processing)
│   Backend   │
└──────┬──────┘
       │ generates
       ↓
┌─────────────────┐
│render_props.json│  ← Data Contract
└──────┬──────────┘
       │ consumed by
       ↓
┌─────────────┐
│  Remotion   │  ← Engine (Video Rendering)
│  Frontend   │
└─────────────┘

Tech Stack

Backend (Python 3.10+)

Google Gemini 1.5 Pro (PDF parsing, script generation)
OpenAI Whisper (Audio transcription)
FFmpeg (Video processing)
Pexels API (B-Roll footage)

Frontend (Node.js/TypeScript)

Remotion (Programmatic video rendering)
React 19 + TypeScript
Recharts (Dynamic charts)
Zod (Schema validation)

📁 Directory Structure

auto-life-plan-video/
├── main.py                    # CLI entry point
├── requirements.txt           # Python dependencies
├── package.json               # Node.js dependencies
├── remotion.config.ts         # Remotion configuration
├── .env                       # API keys (create from .env.example)
├── assets/                    # Shared storage
│   ├── 0_input/               # Raw inputs (PDF, MP4)
│   ├── 1_processed/           # Intermediate files
│   └── 2_output/              # Final rendered videos
├── backend/                   # Python modules
│   ├── __init__.py
│   ├── extractor.py           # PDF/Video parsing
│   ├── scripter.py            # Script generation
│   ├── media_fetcher.py       # B-Roll fetching
│   └── data_builder.py        # Chart data simulation
└── frontend/                  # Remotion source
    └── src/
        ├── index.ts           # Remotion entry
        ├── Root.tsx           # Composition registry
        ├── render-props-types.ts  # TypeScript schema
        ├── types.ts           # Component types
        └── components/        # React components
            ├── LifePlanChart.tsx
            ├── CashFlowChart.tsx
            └── ...

🚀 Quick Start

1. Install Dependencies

Python:

pip install -r requirements.txt

Node.js:

npm install

2. Configure API Keys

Copy .env.example to .env and add your API keys:

cp .env.example .env
# Edit .env with your actual keys

Required keys:

GEMINI_API_KEY - Google Gemini API
OPENAI_API_KEY - OpenAI (for Whisper)
PEXELS_API_KEY - Pexels (for B-Roll)

3. Run the Pipeline

Basic usage:

python main.py --pdf assets/0_input/life_plan_report.pdf

With interview video:

python main.py --pdf report.pdf --video interview.mp4

Dry run (generate data only, skip rendering):

python main.py --pdf report.pdf --dry-run

📊 Data Flow

Input → PDF + optional video
Extract → Gemini parses PDF structure
Transcribe → Whisper extracts audio/timestamps (if video provided)
Generate → Gemini creates narration script
Fetch → Pexels downloads B-Roll based on keywords
Simulate → Python calculates financial projections
Build → Create render_props.json
Render → Remotion generates final MP4

🔧 Development

Run Remotion Studio (Frontend Development)

npm run dev

Test Python Backend

python -m pytest

Type Check

npm run type-check

Lint

npm run lint

📝 Data Contract

The render_props.json file bridges Python and Remotion:

{
  "meta": {
    "title": "山田太郎様の人生設計",
    "currencyUnit": "万円",
    "duration": 180
  },
  "charts": {
    "assetDepletion": {
      "timeline": [
        /* YearlySnapshot[] */
      ],
      "meta": { "currencyUnit": "万円", "startYear": 2025 }
    },
    "cashFlow": {
      "timeline": [
        /* CashFlowData[] */
      ],
      "meta": { "currencyUnit": "万円" }
    }
  },
  "script": [
    {
      "id": 1,
      "text": "これからあなたの将来設計について...",
      "duration": 3.5,
      "type": "narration"
    }
  ],
  "assets": {
    "broll": { "retirement": "path/to/video.mp4" },
    "videoClips": []
  }
}

See frontend/src/render-props-types.ts for full TypeScript schema.

🎨 Adding New Components

Create component in frontend/src/components/
Import in frontend/src/Root.tsx
Register as <Composition> with props from render_props.json

🐛 Troubleshooting

"API key not found"

Ensure .env file exists and contains valid keys

"Remotion render failed"

Check that npm install completed successfully
Verify frontend/src/ structure is correct

"PDF extraction failed"

Ensure PDF is readable and not password-protected
Check Gemini API quota/limits

📚 Documentation

🤝 Contributing

See project structure guidelines in .github/copilot-instructions.md

📄 License

UNLICENSED - Private project

5.9 KiB Raw Permalink Blame History