AI-video-lifeplan-demo/PROJECT_README.md

# Auto Life Plan Video Generator

A hybrid Python/Node.js system that automatically generates financial advice videos from PDF reports and interview footage.

## 🎯 Project Overview

This system transforms static Life Plan reports (PDF) and raw interview videos into dynamic, personalized 3-5 minute explanatory videos combining:

- **Animated Charts** (Asset depletion, cash flow analysis)
- **AI-Generated Narration** (Natural Japanese scripts)
- **B-Roll Footage** (Contextual stock footage)
- **Interview Clips** (Synchronized with content)

## 🏗️ Architecture

```
┌─────────────┐
│   Python    │  ← Director (Orchestration, AI, Data Processing)
│   Backend   │
└──────┬──────┘
       │ generates
       ↓
┌─────────────────┐
│render_props.json│  ← Data Contract
└──────┬──────────┘
       │ consumed by
       ↓
┌─────────────┐
│  Remotion   │  ← Engine (Video Rendering)
│  Frontend   │
└─────────────┘
```

### Tech Stack

**Backend (Python 3.10+)**

- Google Gemini 1.5 Pro (PDF parsing, script generation)
- OpenAI Whisper (Audio transcription)
- FFmpeg (Video processing)
- Pexels API (B-Roll footage)

**Frontend (Node.js/TypeScript)**

- Remotion (Programmatic video rendering)
- React 19 + TypeScript
- Recharts (Dynamic charts)
- Zod (Schema validation)

## 📁 Directory Structure

```
auto-life-plan-video/
├── main.py                    # CLI entry point
├── requirements.txt           # Python dependencies
├── package.json               # Node.js dependencies
├── remotion.config.ts         # Remotion configuration
├── .env                       # API keys (create from .env.example)
├── assets/                    # Shared storage
│   ├── 0_input/               # Raw inputs (PDF, MP4)
│   ├── 1_processed/           # Intermediate files
│   └── 2_output/              # Final rendered videos
├── backend/                   # Python modules
│   ├── __init__.py
│   ├── extractor.py           # PDF/Video parsing
│   ├── scripter.py            # Script generation
│   ├── media_fetcher.py       # B-Roll fetching
│   └── data_builder.py        # Chart data simulation
└── frontend/                  # Remotion source
    └── src/
        ├── index.ts           # Remotion entry
        ├── Root.tsx           # Composition registry
        ├── render-props-types.ts  # TypeScript schema
        ├── types.ts           # Component types
        └── components/        # React components
            ├── LifePlanChart.tsx
            ├── CashFlowChart.tsx
            └── ...
```

## 🚀 Quick Start

### 1. Install Dependencies

**Python:**

```bash
pip install -r requirements.txt
```

**Node.js:**

```bash
npm install
```

### 2. Configure API Keys

Copy `.env.example` to `.env` and add your API keys:

```bash
cp .env.example .env
# Edit .env with your actual keys
```

Required keys:

- `GEMINI_API_KEY` - Google Gemini API
- `OPENAI_API_KEY` - OpenAI (for Whisper)
- `PEXELS_API_KEY` - Pexels (for B-Roll)

### 3. Run the Pipeline

**Basic usage:**

```bash
python main.py --pdf assets/0_input/life_plan_report.pdf
```

**With interview video:**

```bash
python main.py --pdf report.pdf --video interview.mp4
```

**Dry run (generate data only, skip rendering):**

```bash
python main.py --pdf report.pdf --dry-run
```

## 📊 Data Flow

1. **Input** → PDF + optional video
2. **Extract** → Gemini parses PDF structure
3. **Transcribe** → Whisper extracts audio/timestamps (if video provided)
4. **Generate** → Gemini creates narration script
5. **Fetch** → Pexels downloads B-Roll based on keywords
6. **Simulate** → Python calculates financial projections
7. **Build** → Create `render_props.json`
8. **Render** → Remotion generates final MP4

## 🔧 Development

### Run Remotion Studio (Frontend Development)

```bash
npm run dev
```

### Test Python Backend

```bash
python -m pytest
```

### Type Check

```bash
npm run type-check
```

### Lint

```bash
npm run lint
```

## 📝 Data Contract

The `render_props.json` file bridges Python and Remotion:

```json
{
  "meta": {
    "title": "山田太郎様の人生設計",
    "currencyUnit": "万円",
    "duration": 180
  },
  "charts": {
    "assetDepletion": {
      "timeline": [
        /* YearlySnapshot[] */
      ],
      "meta": { "currencyUnit": "万円", "startYear": 2025 }
    },
    "cashFlow": {
      "timeline": [
        /* CashFlowData[] */
      ],
      "meta": { "currencyUnit": "万円" }
    }
  },
  "script": [
    {
      "id": 1,
      "text": "これからあなたの将来設計について...",
      "duration": 3.5,
      "type": "narration"
    }
  ],
  "assets": {
    "broll": { "retirement": "path/to/video.mp4" },
    "videoClips": []
  }
}
```

See `frontend/src/render-props-types.ts` for full TypeScript schema.

## 🎨 Adding New Components

1. Create component in `frontend/src/components/`
2. Import in `frontend/src/Root.tsx`
3. Register as `<Composition>` with props from `render_props.json`

## 🐛 Troubleshooting

**"API key not found"**

- Ensure `.env` file exists and contains valid keys

**"Remotion render failed"**

- Check that `npm install` completed successfully
- Verify `frontend/src/` structure is correct

**"PDF extraction failed"**

- Ensure PDF is readable and not password-protected
- Check Gemini API quota/limits

## 📚 Documentation

- [Remotion Documentation](https://www.remotion.dev/)
- [Google Gemini API](https://ai.google.dev/)
- [Copilot Instructions](.github/copilot-instructions.md)

## 🤝 Contributing

See project structure guidelines in `.github/copilot-instructions.md`

## 📄 License

UNLICENSED - Private project