Initial commit: Life Plan Video Generator PoC

- Remotion-based video rendering system for financial life plans
- Python backend for PDF parsing, video extraction, and B-roll fetching
- Animated charts: Life Plan, Cash Flow, and Dual Chart views
- Integration with Google Gemini, OpenAI, and Pexels APIs
- Complete testing framework and documentation
This commit is contained in:
LIAOWEIRAN 2026-01-14 13:55:38 +09:00
parent f64e798574
commit 47d1f01448
57 changed files with 20278 additions and 72 deletions

22
.env.example Normal file
View File

@ -0,0 +1,22 @@
# =============================================================================
# Auto Life Plan Video Generator - Environment Variables
# =============================================================================
# Copy this file to .env and fill in your actual API keys
# DO NOT commit .env to version control!
# Google Gemini API (for PDF parsing and script generation)
GEMINI_API_KEY=your_gemini_api_key_here
# OpenAI API (for Whisper transcription)
OPENAI_API_KEY=your_openai_api_key_here
# Pexels API (for B-Roll footage)
PEXELS_API_KEY=your_pexels_api_key_here
# Google Cloud TTS (optional, if using TTS)
# GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json
# Configuration
DEFAULT_INFLATION_RATE=0.02
DEFAULT_DEFICIT_INTEREST_RATE=0.15
DEFAULT_INVESTMENT_RETURN=0.03

275
.github/copilot-instructions.md vendored Normal file
View File

@ -0,0 +1,275 @@
# Project Overview: Life Plan Video Generator (PoC)
## 1\. Ultimate Goal
To build an automated pipeline that transforms a static **Life Plan Report (PDF)** and raw **Interview Footage** into a **3-5 minute dynamic explanatory video (MP4)**. The system is designed to help insurance consultants visualize complex financial data (e.g., asset depletion, inflation risks) for clients, creating an emotional and logical narrative to support decision-making.
## 2\. Core Architecture
The system follows a "Hybrid Local/Cloud" architecture to handle sensitive data and heavy video processing efficiently.
### **Backend (Python / Logic Layer)**
- **Role:** The "Brain". Handles data extraction, logic calculation, and asset preparation.
- **Tech Stack:** Python 3.10+, LangChain, Google Gemini 1.5 Pro (via API).
- **Key Responsibilities:**
1. **Ingestion:** Parse complex Japanese PDFs (Sony Life Reports) using Gemini 1.5 Pro.
2. **Structured Output:** Convert unstructured text into strict JSON (Zod Schema compliant).
3. **Simulation Engine:** Calculate future asset projections (inflation, compound interest on deficits) to fill missing data points (Linear Interpolation).
4. **Asset Prep:** Use `FFmpeg` (stream copy) to slice raw interview videos based on timestamps.
### **Frontend / Rendering (Remotion / Visual Layer)**
- **Role:** The "Renderer". Converts JSON data into video frames.
- **Tech Stack:** TypeScript, React, Remotion, Recharts, Zod.
- **Key Responsibilities:**
1. **Programmatic Video:** Render UI components as video frames.
2. **Dynamic Charts:** Visualize "Asset Depletion" and "Sufficiency Rates" using `Recharts`.
3. **Frame Synchronization:** Drive animations strictly via `useCurrentFrame()` (no CSS transitions).
---
## 3\. Remotion Specifics (Copilot Instructions)
### **What does Remotion do in this project?**
Remotion is **NOT** just a video editor; it is a **code-based rendering engine**. It treats the video timeline as a function of state `f(frame) = UI`.
### **Critical Implementation Rules**
1. **No Internal Animations:**
- When using `Recharts`, always set `isAnimationActive={false}`.
- **Reason:** Remotion renders frame-by-frame. Internal library animations rely on real-time timers, which break during rendering, causing flickering or static frames.
- **Solution:** Drive data slicing using `interpolate(frame, [start, end], [0, dataLength])`.
2. **Asset Handling:**
- Do **NOT** import huge video files directly into React. Use URL references or local paths managed by the Python backend.
- Use `<Video startFrom={x} durationInFrames={y} />` for virtual cutting.
3. **The "3-Layer X-Axis" Component:**
- The Life Plan chart requires a custom X-Axis showing: `Year`, `Representative Age`, and `Spouse Age` vertically stacked.
- Use Recharts' `<XAxis tick={<CustomTick />} />` pattern.
---
## 4\. Data Structures (The "Contract")
The bridge between Python and Remotion is `src/mock-data.json`.
### **Core Schema (`YearlySnapshot`)**
TypeScript
interface YearlySnapshot {
year: number; // 西暦 (e.g., 2027)
ages: {
rep: number; // 世帯主年齢
spouse?: number; // 配偶者年齢
};
financials: {
income: number; // 年間収入 (HUD Display)
expenditure: number; // 年間支出 (HUD Display)
assetBalance: number; // 資産残高 (Y-Axis, Red if < 0)
};
events?: string[]; // Life Events (e.g., "Retirement", "Buy House")
isCritical?: boolean; // Trigger for "Red Alert" visual effect
}
---
## 5\. Business Logic & Domain Knowledge
### **The "400 Million JPY" Logic**
- **Context:** The report often predicts a massive deficit (e.g., -400M JPY) at age 100, even if the deficit at age 70 is small (-1.7M JPY).
- **Why:** The simulation engine applies a **high borrowing interest rate** (e.g., 15%) once assets dip below zero. This creates a "compound interest debt trap."
- **Visual Requirement:** The chart must turn **RED** immediately when `assetBalance < 0` and grow exponentially downwards to visualize this risk.
### **The "Sufficiency Rate" (充足率)**
- **Formula:** `(Current Assets + Insurance) / Future Needs * 100%`.
- **Visual:** Displayed as a Donut Chart.
- < 100%: **Danger** (Red/Orange).
- > \= 100%: **Safe** (Green/Blue).
### **Hidden Costs**
- **Housing:** Even after loan repayment, maintenance costs & taxes often rise to ~1.5M JPY/year due to inflation (2% YoY).
- **Life Expectancy:** Simulations default to **Age 100** to account for longevity risk.
---
## 6\. Directory Structure
**This Repository (Remotion Frontend)**
/src
index.ts # Entry point - registers RemotionRoot
Root.tsx # Composition registry - defines video specs
Composition.tsx # Main composition component (currently placeholder)
LifePlanChart.tsx # Core animated chart component
mock-data.json # Sample data matching YearlySnapshot schema
types.ts # (MISSING) TypeScript interfaces for data contract
/public # Static assets (if needed)
remotion.config.ts # Global Remotion settings (frame format, overwrite)
package.json # Dependencies: Remotion 4.0.400, Recharts, React 19
**Full System Context** (Python backend not in this repo):
/0_input_raw # User uploads (PDF, MP4) - handled by Python
/1_processed_data # JSONs extracted by Gemini - consumed by this repo
/scripts # Python orchestration (extraction, simulation)
---
## 7\. Essential Development Workflows
### **Local Development**
```bash
npm run dev # Opens Remotion Studio (interactive preview)
npm run lint # ESLint + TypeScript type check
npx remotion render # Render final MP4
```
### **Project Standards**
- **Frame Rate**: All compositions use **60 fps** for smooth animations
- **Duration**: Typically 15 seconds (900 frames at 60fps)
- **Resolution**: 1920x1080 (Full HD)
### **Debugging Animation Issues**
1. Use Remotion Studio's timeline scrubber to inspect specific frames
2. Check `useCurrentFrame()` values with console.log (visible in Studio)
3. Verify `interpolate()` input/output ranges match data array bounds
4. If chart appears static: confirm `isAnimationActive={false}` on Recharts components
### **Data Integration from Python Backend**
- Python scripts generate JSON files → Place in `src/mock-data.json` or import dynamically
- Validate against `YearlySnapshot` schema before rendering
- **Critical**: Asset balance can go deeply negative (-400M JPY) due to compound interest simulation
---
## 8\. Remotion-Specific Patterns in This Project
### **Frame-Driven Animation Pattern**
```typescript
const frame = useCurrentFrame();
const { durationInFrames } = useVideoConfig();
// Map frame to data index (with end buffer to prevent abrupt cutoff)
const dataEndIndex = Math.floor(
interpolate(frame, [0, durationInFrames - 30], [0, timeline.length - 1], {
extrapolateRight: "clamp", // Prevent out-of-bounds access
}),
);
const visibleData = timeline.slice(0, dataEndIndex + 1);
```
### **Custom Recharts Components**
- **3-Row X-Axis**: Implemented via `<XAxis tick={<CustomXAxisTick />} />`
- Row 1: Year (西暦)
- Row 2: Representative age (bold)
- Row 3: Spouse age (parentheses, or "-")
- **Bi-Directional Gradient**: Uses `linearGradient` with dynamic offset calculation
```typescript
const gradientOffset = () => {
const dataMax = Math.max(...timeline.map((i) => i.financials.assetBalance));
const dataMin = Math.min(...timeline.map((i) => i.financials.assetBalance));
if (dataMax <= 0) return 0;
if (dataMin >= 0) return 1;
return dataMax / (dataMax - dataMin);
};
```
### **Visual Convention Rules**
- **Colors**: Green income (#38A169), Red expenses/deficit (#E53E3E), Blue positive balance (#4299E1)
- **Currency**: Large values formatted as `億` (÷10,000), e.g., `-400億` for -4,000,000 万円
- **HUD Panel**: Shows "current frame" snapshot (not entire dataset aggregation)
---
## 9\. Common Pitfalls & Solutions
| Issue | Why It Happens | Solution |
| --------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------- |
| Chart animation flickers/jumps | Recharts internal animations conflict with frame rendering | Set `isAnimationActive={false}` on all Recharts components |
| Data extends beyond visible range | `interpolate()` extrapolation defaults allow overflow | Use `extrapolateRight: "clamp"` option |
| Type errors on `YearlySnapshot` | `types.ts` file doesn't exist yet | Create `src/types.ts` with interfaces matching mock-data.json |
| Root.tsx renders blank video | Uses placeholder `MyComposition` instead of `LifePlanChart` | Update `<Composition component={LifePlanChart} />` |
| Video cuts off abruptly | No buffer at end of timeline | Use `[0, durationInFrames - 30]` instead of full duration |
---
## 10\. Repeated Mistakes (AI Assistant Learning)
### 🚨 CRITICAL: Gradient Offset Calculation
**MISTAKE (Repeated Multiple Times):**
```typescript
// ❌ WRONG - Uses full timeline data
const gradientOffset = React.useMemo(() => {
const dataMax = Math.max(...timeline.map((i) => i.financials.assetBalance));
const dataMin = Math.min(...timeline.map((i) => i.financials.assetBalance));
return dataMax / (dataMax - dataMin);
}, [timeline]);
```
**PROBLEM:**
- When timeline contains future negative values (e.g., -400M at age 100)
- But current animation only shows positive years (ages 30-50)
- The gradient calculates offset based on full range [-400M to +500M]
- Result: Zero-line appears **above** the visible positive data
- Visual impact: Blue area appears in wrong location
**CORRECT SOLUTION:**
```typescript
// ✅ CORRECT - Uses only visible data
const gradientOffset = React.useMemo(() => {
const dataMax = Math.max(
...visibleData.map((i) => i.financials.assetBalance),
);
const dataMin = Math.min(
...visibleData.map((i) => i.financials.assetBalance),
);
if (dataMin >= 0) return 1; // All positive: fully blue
if (dataMax <= 0) return 0; // All negative: fully red
return dataMax / (dataMax - dataMin); // Mixed: calculate zero crossing
}, [visibleData]); // ← Depends on visibleData, NOT timeline
```
**WHY THIS MATTERS:**
- The gradient must match what's **currently visible** on screen
- Animation progressively reveals data over time
- Early frames show only positive balances → should be fully blue
- Later frames reveal deficit → gradient transitions to red
- Using `timeline` breaks this progressive revelation
**RULE FOR AI:**
> When calculating visual properties (gradients, colors, domains) for animated charts, ALWAYS use `visibleData` (the currently rendered slice), NOT the full `timeline` dataset, unless you explicitly need stable axis ranges for Y-axis domains.
---
## 11\. Next Steps / Known TODOs
- [ ] Create `src/types.ts` with `LifePlanChartProps`, `YearlySnapshot`, `LifePlanMeta` interfaces
- [ ] Wire `LifePlanChart` into `Root.tsx` composition
- [ ] Add Zod schema validation for runtime data checking
- [ ] Implement "Sufficiency Rate" donut chart component
- [ ] Add video interview footage integration using `<Video>` components
- [ ] Create transition sequences between chart and interview clips

33
.github/workflows/lint.yml vendored Normal file
View File

@ -0,0 +1,33 @@
name: Lint & Type Check
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
cache: "npm"
- name: Install dependencies
run: npm ci
- name: Run ESLint
run: npm run lint
- name: Run Prettier check
run: npm run format:check
- name: Type check
run: npm run type-check

117
.gitignore vendored
View File

@ -1,7 +1,112 @@
node_modules # ============================================
dist # Node.js / JavaScript / TypeScript
.DS_Store # ============================================
.env node_modules/
dist/
build/
*.tsbuildinfo
# Ignore the output video from Git but not videos you import into src/. # ============================================
out # Python
# ============================================
# Virtual Environment
venv/
env/
ENV/
.venv/
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
*.so
# Distribution / packaging
*.egg-info/
dist-py/
build-py/
.eggs/
# Testing
.pytest_cache/
.coverage
htmlcov/
# ============================================
# Asset Directories (Input/Output/Processed)
# ============================================
# Keep directory structure but ignore large files
assets/0_input_raw/*
!assets/0_input_raw/.gitkeep
# Input videos and renders
assets/0_input/out/*.mp4
assets/0_input/**/*.mp4
assets/0_input/**/*.mov
# Processed data - ignore videos but keep JSON metadata
assets/1_processed/broll/*.mp4
assets/1_processed/broll/*.mov
assets/1_processed/**/*.mp4
assets/1_processed/**/*.mov
# Keep JSON files for metadata
!assets/1_processed/**/*.json
# Output directory
assets/2_output/*
!assets/2_output/.gitkeep
# Test fixtures (large video files)
tests/fixtures/*.mp4
tests/fixtures/*.mp3
tests/fixtures/*.wav
!tests/fixtures/.gitkeep
# ============================================
# Remotion / Video Output
# ============================================
# Ignore the output video from Git but not videos you import into src/
out/
.remotion/
# ============================================
# Environment Variables & Secrets
# ============================================
.env
.env.local
.env.*.local
*.key
*.pem
# ============================================
# Logs & Temporary Files
# ============================================
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
*.tmp
*.temp
logs/
# ============================================
# OS & IDE Files
# ============================================
# macOS
.DS_Store
.AppleDouble
.LSOverride
# Windows
Thumbs.db
Desktop.ini
$RECYCLE.BIN/
# Vim
*.swp
*.swo
*~
# VSCode (optional - uncomment if you don't want to share settings)
# .vscode/

10
.prettierignore Normal file
View File

@ -0,0 +1,10 @@
node_modules
dist
build
out
.remotion
coverage
*.min.js
package-lock.json
yarn.lock
pnpm-lock.yaml

View File

@ -1,5 +1,12 @@
{ {
"semi": true,
"trailingComma": "all",
"singleQuote": false,
"printWidth": 80,
"tabWidth": 2,
"useTabs": false, "useTabs": false,
"arrowParens": "always",
"endOfLine": "lf",
"bracketSpacing": true, "bracketSpacing": true,
"tabWidth": 2 "jsxSingleQuote": false
} }

245
PROJECT_README.md Normal file
View File

@ -0,0 +1,245 @@
# Auto Life Plan Video Generator
A hybrid Python/Node.js system that automatically generates financial advice videos from PDF reports and interview footage.
## 🎯 Project Overview
This system transforms static Life Plan reports (PDF) and raw interview videos into dynamic, personalized 3-5 minute explanatory videos combining:
- **Animated Charts** (Asset depletion, cash flow analysis)
- **AI-Generated Narration** (Natural Japanese scripts)
- **B-Roll Footage** (Contextual stock footage)
- **Interview Clips** (Synchronized with content)
## 🏗️ Architecture
```
┌─────────────┐
│ Python │ ← Director (Orchestration, AI, Data Processing)
│ Backend │
└──────┬──────┘
│ generates
↓
┌─────────────────┐
│render_props.json│ ← Data Contract
└──────┬──────────┘
│ consumed by
↓
┌─────────────┐
│ Remotion │ ← Engine (Video Rendering)
│ Frontend │
└─────────────┘
```
### Tech Stack
**Backend (Python 3.10+)**
- Google Gemini 1.5 Pro (PDF parsing, script generation)
- OpenAI Whisper (Audio transcription)
- FFmpeg (Video processing)
- Pexels API (B-Roll footage)
**Frontend (Node.js/TypeScript)**
- Remotion (Programmatic video rendering)
- React 19 + TypeScript
- Recharts (Dynamic charts)
- Zod (Schema validation)
## 📁 Directory Structure
```
auto-life-plan-video/
├── main.py # CLI entry point
├── requirements.txt # Python dependencies
├── package.json # Node.js dependencies
├── remotion.config.ts # Remotion configuration
├── .env # API keys (create from .env.example)
├── assets/ # Shared storage
│ ├── 0_input/ # Raw inputs (PDF, MP4)
│ ├── 1_processed/ # Intermediate files
│ └── 2_output/ # Final rendered videos
├── backend/ # Python modules
│ ├── __init__.py
│ ├── extractor.py # PDF/Video parsing
│ ├── scripter.py # Script generation
│ ├── media_fetcher.py # B-Roll fetching
│ └── data_builder.py # Chart data simulation
└── frontend/ # Remotion source
└── src/
├── index.ts # Remotion entry
├── Root.tsx # Composition registry
├── render-props-types.ts # TypeScript schema
├── types.ts # Component types
└── components/ # React components
├── LifePlanChart.tsx
├── CashFlowChart.tsx
└── ...
```
## 🚀 Quick Start
### 1. Install Dependencies
**Python:**
```bash
pip install -r requirements.txt
```
**Node.js:**
```bash
npm install
```
### 2. Configure API Keys
Copy `.env.example` to `.env` and add your API keys:
```bash
cp .env.example .env
# Edit .env with your actual keys
```
Required keys:
- `GEMINI_API_KEY` - Google Gemini API
- `OPENAI_API_KEY` - OpenAI (for Whisper)
- `PEXELS_API_KEY` - Pexels (for B-Roll)
### 3. Run the Pipeline
**Basic usage:**
```bash
python main.py --pdf assets/0_input/life_plan_report.pdf
```
**With interview video:**
```bash
python main.py --pdf report.pdf --video interview.mp4
```
**Dry run (generate data only, skip rendering):**
```bash
python main.py --pdf report.pdf --dry-run
```
## 📊 Data Flow
1. **Input** → PDF + optional video
2. **Extract** → Gemini parses PDF structure
3. **Transcribe** → Whisper extracts audio/timestamps (if video provided)
4. **Generate** → Gemini creates narration script
5. **Fetch** → Pexels downloads B-Roll based on keywords
6. **Simulate** → Python calculates financial projections
7. **Build** → Create `render_props.json`
8. **Render** → Remotion generates final MP4
## 🔧 Development
### Run Remotion Studio (Frontend Development)
```bash
npm run dev
```
### Test Python Backend
```bash
python -m pytest
```
### Type Check
```bash
npm run type-check
```
### Lint
```bash
npm run lint
```
## 📝 Data Contract
The `render_props.json` file bridges Python and Remotion:
```json
{
"meta": {
"title": "山田太郎様の人生設計",
"currencyUnit": "万円",
"duration": 180
},
"charts": {
"assetDepletion": {
"timeline": [
/* YearlySnapshot[] */
],
"meta": { "currencyUnit": "万円", "startYear": 2025 }
},
"cashFlow": {
"timeline": [
/* CashFlowData[] */
],
"meta": { "currencyUnit": "万円" }
}
},
"script": [
{
"id": 1,
"text": "これからあなたの将来設計について...",
"duration": 3.5,
"type": "narration"
}
],
"assets": {
"broll": { "retirement": "path/to/video.mp4" },
"videoClips": []
}
}
```
See `frontend/src/render-props-types.ts` for full TypeScript schema.
## 🎨 Adding New Components
1. Create component in `frontend/src/components/`
2. Import in `frontend/src/Root.tsx`
3. Register as `<Composition>` with props from `render_props.json`
## 🐛 Troubleshooting
**"API key not found"**
- Ensure `.env` file exists and contains valid keys
**"Remotion render failed"**
- Check that `npm install` completed successfully
- Verify `frontend/src/` structure is correct
**"PDF extraction failed"**
- Ensure PDF is readable and not password-protected
- Check Gemini API quota/limits
## 📚 Documentation
- [Remotion Documentation](https://www.remotion.dev/)
- [Google Gemini API](https://ai.google.dev/)
- [Copilot Instructions](.github/copilot-instructions.md)
## 🤝 Contributing
See project structure guidelines in `.github/copilot-instructions.md`
## 📄 License
UNLICENSED - Private project

View File

@ -1,54 +0,0 @@
# Remotion video
<p align="center">
<a href="https://github.com/remotion-dev/logo">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/remotion-dev/logo/raw/main/animated-logo-banner-dark.apng">
<img alt="Animated Remotion Logo" src="https://github.com/remotion-dev/logo/raw/main/animated-logo-banner-light.gif">
</picture>
</a>
</p>
Welcome to your Remotion project!
## Commands
**Install Dependencies**
```console
npm i
```
**Start Preview**
```console
npm run dev
```
**Render video**
```console
npx remotion render
```
**Upgrade Remotion**
```console
npx remotion upgrade
```
## Docs
Get started with Remotion by reading the [fundamentals page](https://www.remotion.dev/docs/the-fundamentals).
## Help
We provide help on our [Discord server](https://discord.gg/6VzzNDwUwV).
## Issues
Found an issue with Remotion? [File an issue here](https://github.com/remotion-dev/remotion/issues/new).
## License
Note that for some entities a company license is needed. [Read the terms here](https://github.com/remotion-dev/remotion/blob/main/LICENSE.md).

1
assets/0_input/.gitkeep Normal file
View File

@ -0,0 +1 @@
# Place raw input files here (PDF reports, interview videos)

View File

@ -0,0 +1 @@
# Intermediate processed files (JSON, TTS audio, video clips, B-Roll)

View File

@ -0,0 +1,77 @@
{
"家族": {
"english": "Family",
"videos": [
{
"filepath": "assets\\1_processed\\broll\\Family_3198161.mp4",
"duration": 19,
"photographer": "cottonbro studio"
},
{
"filepath": "assets\\1_processed\\broll\\Family_3192261.mp4",
"duration": 21,
"photographer": "Pressmaster"
}
]
},
"老後": {
"english": "Retirement",
"videos": [
{
"filepath": "assets\\1_processed\\broll\\Retirement_10367208.mp4",
"duration": 43,
"photographer": "Ron Lach"
},
{
"filepath": "assets\\1_processed\\broll\\Retirement_8860237.mp4",
"duration": 30,
"photographer": "cottonbro studio"
}
]
},
"貯蓄": {
"english": "Savings",
"videos": [
{
"filepath": "assets\\1_processed\\broll\\Savings_1793337.mp4",
"duration": 26,
"photographer": "Miguel Á. Padriñán"
},
{
"filepath": "assets\\1_processed\\broll\\Savings_3752548.mp4",
"duration": 25,
"photographer": "Ricky Esquivel"
}
]
},
"住宅": {
"english": "Housing",
"videos": [
{
"filepath": "assets\\1_processed\\broll\\Housing_35628107.mp4",
"duration": 7,
"photographer": "Altaf Shah"
},
{
"filepath": "assets\\1_processed\\broll\\Housing_35628066.mp4",
"duration": 9,
"photographer": "Altaf Shah"
}
]
},
"教育": {
"english": "Education",
"videos": [
{
"filepath": "assets\\1_processed\\broll\\Education_3209663.mp4",
"duration": 25,
"photographer": "Pressmaster"
},
{
"filepath": "assets\\1_processed\\broll\\Education_3209298.mp4",
"duration": 17,
"photographer": "Pressmaster"
}
]
}
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,24 @@
{
"method_a_video": {
"summary": "ライフプランの作成のために、インタビュアーが顧客である男性(テスト・タロウ)に基本情報をヒアリングしている場面です。男性は自身の生年月日、職業(IT系)、年収(700万円)に加え、妻(ハナコ)と子供(一人っ子)の年齢や職業、就学状況などの家族構成について詳細に回答しています。",
"full_transcript": "インタビュアー: はい、すいません。えー、えーと、お客様の、えーと、今後の、えーと、ライフプランを提供するために、えーと、お客様の基本情報をお願いいたします。\nインタビュアー: え、すいませんが、お客様のお名前からお願いいたします。\n男性: あ、はい。えーと、私がそのテストタロウとして、はい。テストタロウですね。はい、まあ、はい。\n男性: えーと、家族情報としては…あ、はい。\nインタビュアー: で、あの、生年月日の方は?\n男性: はい、生年月日の方は、えーと、1000、1980、80年のその1月の1日です。\nインタビュアー: はい。え、ということは47歳ということでよろしいでしょうか。\n男性: あ、そうですね。はい。\nインタビュアー: 現在のお勤め先はどこになりますか?\n男性: そうですね、IT系ですね。はい。\nインタビュアー: あ、ITですね。はい。じゃあ、あの、すいませんが、あの、現在の、まあ、ざっくりでよろしいので、年収はどのぐらいになりますでしょうか?\n男性: ああ、年収が今700万ぐらいですね。はい。\nインタビュアー: では、あの、家族構成について、えー、お客様、ご結婚はなされておりますか?\n男性: あ、そうですね。はい。\nインタビュアー: あ、で、あの、下の名前だけでいいので、あの、奥様のお名前をお願いいたします。\n男性: あ、ハナコです。はい。\nインタビュアー: 奥様の生年月日を教えていただけますか?\n男性: あ、1000、1990年の1日の、え、あ、1月のその10ですけど。はい。\nインタビュアー: ということは、えーと、現在37歳ということでよろしいでしょうか。\n男性: あ、そうですね。はい。\nインタビュアー: え、奥様、あの、ご勤めになられてますでしょうか?\n男性: そうですね、普通のサービス業ですけど、はい。\nインタビュアー: まあ、ざっくりでよろしいんで、奥様の年収とか…\n男性: 300、ぐらいですね。はい。\nインタビュアー: あ、300万ぐらいですね。はい。じゃあ、続きまして、お子様の方は、え、いらっしゃいますか?\n男性: ああ、はい、います。はい。\nインタビュアー: え、何人いらっしゃいますか?\n男性: あ、一人っ子だけですね。はい。\nインタビュアー: あ、お一人だけですね。あ、生年月日とかは…\n男性: あ、2018年の、はい、の3月の3ですけど。はい。\nインタビュアー: あ、そうですか。えーと、現在、えーと、まあ7歳ということで、えーと、多分小学校だと思いますけども、小学校の方は、あの、公立でしょうか?\n男性: あ、公立ですね。はい。\nインタビュアー: あ、公立ですね。あ、ありがとうございます。",
"speakers": [
"インタビュアー",
"男性(テスト・タロウ)"
],
"duration": "02:22",
"_metadata": {
"level": "Movie-level",
"token_usage": {
"input_tokens": 12776,
"output_tokens": 845,
"total_tokens": 15618
},
"processing_time_seconds": 32.01176190376282,
"model": "gemini-3-pro-preview"
}
},
"method_b_audio": {
"error": "Audio extraction failed"
}
}

View File

@ -0,0 +1,300 @@
{
"method_a_video": {
"subtitles": [
{
"index": 1,
"start_time": "00:00.000",
"end_time": "00:03.500",
"speaker": "インタビュアー",
"text": "はい、すいません。えっと、あ、"
},
{
"index": 2,
"start_time": "00:03.500",
"end_time": "00:13.000",
"speaker": "インタビュアー",
"text": "えっと、お客様の、えっと、今後、えっと、ライフプランを提供する為に、えっと、お客様の基本情報をお願い致します。"
},
{
"index": 3,
"start_time": "00:13.000",
"end_time": "00:18.000",
"speaker": "インタビュアー",
"text": "え、すいませんが、お客様のお名前からお願い致します。"
},
{
"index": 4,
"start_time": "00:18.000",
"end_time": "00:26.000",
"speaker": "男性",
"text": "あ、はい。えっと、私がそのテスト太郎として、はい。テスト太郎ですね。はい、まあ、はい。"
},
{
"index": 5,
"start_time": "00:26.000",
"end_time": "00:30.000",
"speaker": "男性",
"text": "えっと、家族情報としては、あ、"
},
{
"index": 6,
"start_time": "00:30.000",
"end_time": "00:32.000",
"speaker": "インタビュアー",
"text": "で、あの、生年月日の方は"
},
{
"index": 7,
"start_time": "00:32.000",
"end_time": "00:41.000",
"speaker": "男性",
"text": "はい。生年月日の方は、えっと、1980年のその1月の1日です。"
},
{
"index": 8,
"start_time": "00:41.000",
"end_time": "00:44.000",
"speaker": "インタビュアー",
"text": "はい、ということは47歳ということでよろしいでしょうか。"
},
{
"index": 9,
"start_time": "00:44.000",
"end_time": "00:45.000",
"speaker": "男性",
"text": "あ、そうですね、はい。"
},
{
"index": 10,
"start_time": "00:45.000",
"end_time": "00:48.000",
"speaker": "インタビュアー",
"text": "現在の、お勤め先はどこになりますか?"
},
{
"index": 11,
"start_time": "00:48.000",
"end_time": "00:51.000",
"speaker": "男性",
"text": "そうですね、IT系ですね、はい。"
},
{
"index": 12,
"start_time": "00:51.000",
"end_time": "00:52.000",
"speaker": "インタビュアー",
"text": "あ、ITですね。"
},
{
"index": 13,
"start_time": "00:52.000",
"end_time": "01:00.000",
"speaker": "インタビュアー",
"text": "はい、じゃああの、すいませんが、あの、現在の、まあざっくりでよろしいので、年収はどのぐらいになってますでしょうか。"
},
{
"index": 14,
"start_time": "01:00.000",
"end_time": "01:03.000",
"speaker": "男性",
"text": "あ、年収が今700万ぐらいですね、はい。"
},
{
"index": 15,
"start_time": "01:03.000",
"end_time": "01:09.000",
"speaker": "インタビュアー",
"text": "では、あの、家族構成について、えー、お客様ご結婚はなされておりますか?"
},
{
"index": 16,
"start_time": "01:09.000",
"end_time": "01:11.000",
"speaker": "男性",
"text": "あ、そうですね、はい。"
},
{
"index": 17,
"start_time": "01:11.000",
"end_time": "01:16.000",
"speaker": "インタビュアー",
"text": "あ、で、あの、下の名前だけでいいので、あの、奥様のお名前をお願い致します。"
},
{
"index": 18,
"start_time": "01:16.000",
"end_time": "01:17.000",
"speaker": "男性",
"text": "あ、花子です。はい。"
},
{
"index": 19,
"start_time": "01:17.000",
"end_time": "01:21.000",
"speaker": "インタビュアー",
"text": "で、奥様の生年月日を教えていただけますか。"
},
{
"index": 20,
"start_time": "01:21.000",
"end_time": "01:28.000",
"speaker": "男性",
"text": "1990年の、ついたちの、え、あ、1月のその10ですけど、はい。"
},
{
"index": 21,
"start_time": "01:28.000",
"end_time": "01:32.000",
"speaker": "インタビュアー",
"text": "ということは、えっと、現在37歳ということでよろしいでしょうか。"
},
{
"index": 22,
"start_time": "01:32.000",
"end_time": "01:33.000",
"speaker": "男性",
"text": "あ、そうですね、はい。"
},
{
"index": 23,
"start_time": "01:33.000",
"end_time": "01:37.000",
"speaker": "インタビュアー",
"text": "え、奥様、あの、お勤めになられてますでしょうか。"
},
{
"index": 24,
"start_time": "01:37.000",
"end_time": "01:40.000",
"speaker": "男性",
"text": "そうですね、普通のサービス業ですけど、はい。"
},
{
"index": 25,
"start_time": "01:40.000",
"end_time": "01:44.000",
"speaker": "インタビュアー",
"text": "まあ、ざっくりでよろしいので、奥様の年収とかは…"
},
{
"index": 26,
"start_time": "01:44.000",
"end_time": "01:46.000",
"speaker": "男性",
"text": "300万ぐらいですね、はい。"
},
{
"index": 27,
"start_time": "01:46.000",
"end_time": "01:47.000",
"speaker": "インタビュアー",
"text": "あ、300万ぐらいですね。はい。"
},
{
"index": 28,
"start_time": "01:47.000",
"end_time": "01:50.000",
"speaker": "インタビュアー",
"text": "じゃあ続きまして、お子様の方は、えー、いらっしゃいますか?"
},
{
"index": 29,
"start_time": "01:50.000",
"end_time": "01:52.000",
"speaker": "男性",
"text": "あ、はい、います。はい。"
},
{
"index": 30,
"start_time": "01:52.000",
"end_time": "01:54.000",
"speaker": "インタビュアー",
"text": "えー、何人いらっしゃいますか。"
},
{
"index": 31,
"start_time": "01:54.000",
"end_time": "01:56.000",
"speaker": "男性",
"text": "あ、一人っ子だけです。はい。"
},
{
"index": 32,
"start_time": "01:56.000",
"end_time": "01:57.000",
"speaker": "インタビュアー",
"text": "あ、一人だけですね。"
},
{
"index": 33,
"start_time": "01:57.000",
"end_time": "02:00.000",
"speaker": "インタビュアー",
"text": "あ、生年月日とかは…"
},
{
"index": 34,
"start_time": "02:00.000",
"end_time": "02:04.000",
"speaker": "男性",
"text": "あ、2018年の、はい、の3月の3ですけど、はい。"
},
{
"index": 35,
"start_time": "02:04.000",
"end_time": "02:05.000",
"speaker": "インタビュアー",
"text": "あ、そうですか。"
},
{
"index": 36,
"start_time": "02:05.000",
"end_time": "02:14.000",
"speaker": "インタビュアー",
"text": "えっと、現在、えっと、まあ7歳ということで、えっと、多分小学校だと思いますけども、小学校の方は、あの…"
},
{
"index": 37,
"start_time": "02:14.000",
"end_time": "02:15.000",
"speaker": "男性",
"text": "あ、公立です。はい。"
},
{
"index": 38,
"start_time": "02:15.000",
"end_time": "02:16.000",
"speaker": "インタビュアー",
"text": "あ、公立ですね。"
},
{
"index": 39,
"start_time": "02:16.000",
"end_time": "02:18.000",
"speaker": "男性",
"text": "あ、公立ですね、はい。"
},
{
"index": 40,
"start_time": "02:18.000",
"end_time": "02:22.000",
"speaker": "インタビュアー",
"text": "あ、ありがとうございます。"
}
],
"total_segments": 40,
"_metadata": {
"level": "Second-level",
"token_usage": {
"input_tokens": 12809,
"output_tokens": 3211,
"total_tokens": 19456
},
"processing_time_seconds": 96.17553997039795,
"model": "gemini-3-pro-preview"
}
},
"method_b_audio": {
"error": "Audio extraction failed"
}
}

View File

@ -0,0 +1,12 @@
{
"家族": "Family",
"老後": "Retirement",
"貯蓄": "Savings",
"住宅": "Housing",
"教育": "Education",
"保険": "Insurance",
"将来": "Future",
"不安": "Anxiety",
"安心": "Peace of mind",
"計画": "Planning"
}

View File

@ -0,0 +1,210 @@
{
"№1_基本情報.mp4": {
"summary": "保険コンサルタントが顧客に対して、ライフプラン提案のための基本情報をヒアリングしているインタビュー映像です。顧客の氏名、生年月日、職業、年収に加え、配偶者や子供の有無、家族の生年月日や職業、学校区分(公立)などの詳細な家族構成について確認しています。",
"speakers": [
"保険コンサルタント(声のみ)",
"顧客(テスト・タロウ様)"
],
"duration": "02:22",
"segments": [
{
"start_time": "00:00",
"end_time": "00:26",
"speaker": "コンサルタント",
"transcript": "お客様の今後のライフプランを提供するためにお客様の基本情報をお願いいたします。お名前からお願いします。",
"topic": "導入・氏名確認",
"emotion": "丁寧・真剣",
"importance": 5,
"keywords": [
"ライフプラン",
"基本情報",
"氏名"
]
},
{
"start_time": "00:27",
"end_time": "00:44",
"speaker": "顧客",
"transcript": "生年月日は1980年の1月1日です。(47歳ですね、と確認される)",
"topic": "生年月日・年齢",
"emotion": "淡々",
"importance": 4,
"keywords": [
"生年月日",
"年齢",
"1980年"
]
},
{
"start_time": "00:45",
"end_time": "01:03",
"speaker": "顧客",
"transcript": "今の勤め先はIT系です。年収は今700万ぐらいですね。",
"topic": "職業・年収",
"emotion": "落ち着いた",
"importance": 5,
"keywords": [
"職業",
"IT系",
"年収",
"700万"
]
},
{
"start_time": "01:04",
"end_time": "01:46",
"speaker": "顧客",
"transcript": "結婚しています。妻の名前はハナコ、生年月日は1990年1月10日(37歳)。サービス業で年収は300万くらい。",
"topic": "配偶者情報",
"emotion": "協力的",
"importance": 4,
"keywords": [
"配偶者",
"既婚",
"共働き",
"世帯年収"
]
},
{
"start_time": "01:47",
"end_time": "02:22",
"speaker": "顧客",
"transcript": "子供は一人います。2018年3月3日生まれ。小学校は公立です。",
"topic": "子供・教育環境",
"emotion": "穏やか",
"importance": 4,
"keywords": [
"子供",
"一人っ子",
"公立小学校",
"教育費"
]
}
],
"broll_keywords": [
"ライフプランニング",
"保険相談",
"ビジネスマン",
"インタビュー",
"ITエンジニア",
"家族構成",
"家計診断",
"カウンセリング",
"夫婦",
"教育資金"
],
"_metadata": {
"token_usage": {
"input_tokens": 13041,
"output_tokens": 881,
"total_tokens": 15774
},
"model": "gemini-3-pro-preview",
"video_name": "№1_基本情報.mp4"
}
},
"№2_ライフイベント.mp4": {
"summary": "保険コンサルタントが顧客に対し、今後のライフプランに関する詳細なヒアリングを行っている映像です。子供の教育進路、車の買い替え、住宅のリフォーム計画、記念日の海外旅行、そして定年後の働き方や年金受給について話し合われています。",
"speakers": [
"コンサルタント",
"顧客"
],
"duration": "04:48",
"segments": [
{
"start_time": "00:00",
"end_time": "01:05",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 今後のライフプランについてお伺いします。お子様はお一人と言いましたが、今後中学高校、大学などはどのようにお考えですか?\n顧客: 今の想定については、中学から私立が参加して、高校も同じ私立。できる限り大学は国公立の理系に行ければ一番いいかなという想定です。",
"topic": "子供の教育プラン",
"emotion": "真剣・希望",
"importance": 5,
"keywords": [
"教育費",
"私立中学",
"国公立大学",
"理系"
]
},
{
"start_time": "01:06",
"end_time": "01:38",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 現在お車はお持ちでしょうか?\n顧客: 持っています。3、4年くらい乗っています。\nコンサルタント: 今後買い替える予定はありますか?\n顧客: そうですね、もう少し大きいの車に買い替えしたいかな。もう2、3年したら。",
"topic": "車の所有と買い替え",
"emotion": "前向き",
"importance": 3,
"keywords": [
"マイカー",
"買い替え",
"車種変更"
]
},
{
"start_time": "01:39",
"end_time": "02:35",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: ご自宅の方は持ち家でしょうか?今後リフォームなどは考えていますか?\n顧客: 持ち家です。住んで10年くらい。新築ではないので、あと10年後くらいに一回リフォームかなと。\nコンサルタント: 住宅ローンはどのくらい組んでおられますか?\n顧客: 最初は35年なので、あと25年くらい残っています。固定金利です。",
"topic": "住宅事情とローン",
"emotion": "冷静・計画的",
"importance": 4,
"keywords": [
"持ち家",
"リフォーム",
"住宅ローン",
"固定金利"
]
},
{
"start_time": "02:36",
"end_time": "03:07",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 結婚されてからどのくらいになりますか?今後、結婚記念日とかでお祝いや海外旅行などは考えていますか?\n顧客: 今10年ですね。モルディブとか、またはアメリカとかに一回行こうかなと考えています。",
"topic": "記念日と旅行計画",
"emotion": "楽しい・夢",
"importance": 3,
"keywords": [
"結婚記念日",
"海外旅行",
"レジャー資金"
]
},
{
"start_time": "03:08",
"end_time": "04:48",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 今後定年を迎えると思いますが、定年後のお仕事はどうなされる計画などありますか?\n顧客: できれば定年後も仕事を継続したいかなと。65歳で年金をもらうまでは今の会社、もしくは別のところで働くイメージです。\nコンサルタント: 65歳から年金をもらう形で考えていますか?\n顧客: そうですね、65歳から年金をもらうという形です。",
"topic": "老後の働き方と年金",
"emotion": "堅実・少し不安",
"importance": 5,
"keywords": [
"定年退職",
"再雇用",
"年金受給",
"老後資金"
]
}
],
"broll_keywords": [
"学校",
"勉強する子供",
"新車ディーラー",
"住宅リフォーム",
"住宅ローン計算",
"海外リゾート",
"飛行機",
"オフィスで働く高齢者",
"年金手帳",
"ライフプランニング"
],
"_metadata": {
"token_usage": {
"input_tokens": 26044,
"output_tokens": 1177,
"total_tokens": 28611
},
"model": "gemini-3-pro-preview",
"video_name": "№2_ライフイベント.mp4"
}
}
}

View File

@ -0,0 +1,104 @@
{
"summary": "保険コンサルタントが顧客に対して、ライフプラン提案のための基本情報をヒアリングしているインタビュー映像です。顧客の氏名、生年月日、職業、年収に加え、配偶者や子供の有無、家族の生年月日や職業、学校区分(公立)などの詳細な家族構成について確認しています。",
"speakers": [
"保険コンサルタント(声のみ)",
"顧客(テスト・タロウ様)"
],
"duration": "02:22",
"segments": [
{
"start_time": "00:00",
"end_time": "00:26",
"speaker": "コンサルタント",
"transcript": "お客様の今後のライフプランを提供するためにお客様の基本情報をお願いいたします。お名前からお願いします。",
"topic": "導入・氏名確認",
"emotion": "丁寧・真剣",
"importance": 5,
"keywords": [
"ライフプラン",
"基本情報",
"氏名"
]
},
{
"start_time": "00:27",
"end_time": "00:44",
"speaker": "顧客",
"transcript": "生年月日は1980年の1月1日です。(47歳ですね、と確認される)",
"topic": "生年月日・年齢",
"emotion": "淡々",
"importance": 4,
"keywords": [
"生年月日",
"年齢",
"1980年"
]
},
{
"start_time": "00:45",
"end_time": "01:03",
"speaker": "顧客",
"transcript": "今の勤め先はIT系です。年収は今700万ぐらいですね。",
"topic": "職業・年収",
"emotion": "落ち着いた",
"importance": 5,
"keywords": [
"職業",
"IT系",
"年収",
"700万"
]
},
{
"start_time": "01:04",
"end_time": "01:46",
"speaker": "顧客",
"transcript": "結婚しています。妻の名前はハナコ、生年月日は1990年1月10日(37歳)。サービス業で年収は300万くらい。",
"topic": "配偶者情報",
"emotion": "協力的",
"importance": 4,
"keywords": [
"配偶者",
"既婚",
"共働き",
"世帯年収"
]
},
{
"start_time": "01:47",
"end_time": "02:22",
"speaker": "顧客",
"transcript": "子供は一人います。2018年3月3日生まれ。小学校は公立です。",
"topic": "子供・教育環境",
"emotion": "穏やか",
"importance": 4,
"keywords": [
"子供",
"一人っ子",
"公立小学校",
"教育費"
]
}
],
"broll_keywords": [
"ライフプランニング",
"保険相談",
"ビジネスマン",
"インタビュー",
"ITエンジニア",
"家族構成",
"家計診断",
"カウンセリング",
"夫婦",
"教育資金"
],
"_metadata": {
"token_usage": {
"input_tokens": 13041,
"output_tokens": 881,
"total_tokens": 15774
},
"model": "gemini-3-pro-preview",
"video_name": "№1_基本情報.mp4"
}
}

View File

@ -0,0 +1,104 @@
{
"summary": "保険コンサルタントが顧客に対し、今後のライフプランに関する詳細なヒアリングを行っている映像です。子供の教育進路、車の買い替え、住宅のリフォーム計画、記念日の海外旅行、そして定年後の働き方や年金受給について話し合われています。",
"speakers": [
"コンサルタント",
"顧客"
],
"duration": "04:48",
"segments": [
{
"start_time": "00:00",
"end_time": "01:05",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 今後のライフプランについてお伺いします。お子様はお一人と言いましたが、今後中学高校、大学などはどのようにお考えですか?\n顧客: 今の想定については、中学から私立が参加して、高校も同じ私立。できる限り大学は国公立の理系に行ければ一番いいかなという想定です。",
"topic": "子供の教育プラン",
"emotion": "真剣・希望",
"importance": 5,
"keywords": [
"教育費",
"私立中学",
"国公立大学",
"理系"
]
},
{
"start_time": "01:06",
"end_time": "01:38",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 現在お車はお持ちでしょうか?\n顧客: 持っています。3、4年くらい乗っています。\nコンサルタント: 今後買い替える予定はありますか?\n顧客: そうですね、もう少し大きいの車に買い替えしたいかな。もう2、3年したら。",
"topic": "車の所有と買い替え",
"emotion": "前向き",
"importance": 3,
"keywords": [
"マイカー",
"買い替え",
"車種変更"
]
},
{
"start_time": "01:39",
"end_time": "02:35",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: ご自宅の方は持ち家でしょうか?今後リフォームなどは考えていますか?\n顧客: 持ち家です。住んで10年くらい。新築ではないので、あと10年後くらいに一回リフォームかなと。\nコンサルタント: 住宅ローンはどのくらい組んでおられますか?\n顧客: 最初は35年なので、あと25年くらい残っています。固定金利です。",
"topic": "住宅事情とローン",
"emotion": "冷静・計画的",
"importance": 4,
"keywords": [
"持ち家",
"リフォーム",
"住宅ローン",
"固定金利"
]
},
{
"start_time": "02:36",
"end_time": "03:07",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 結婚されてからどのくらいになりますか?今後、結婚記念日とかでお祝いや海外旅行などは考えていますか?\n顧客: 今10年ですね。モルディブとか、またはアメリカとかに一回行こうかなと考えています。",
"topic": "記念日と旅行計画",
"emotion": "楽しい・夢",
"importance": 3,
"keywords": [
"結婚記念日",
"海外旅行",
"レジャー資金"
]
},
{
"start_time": "03:08",
"end_time": "04:48",
"speaker": "コンサルタント・顧客",
"transcript": "コンサルタント: 今後定年を迎えると思いますが、定年後のお仕事はどうなされる計画などありますか?\n顧客: できれば定年後も仕事を継続したいかなと。65歳で年金をもらうまでは今の会社、もしくは別のところで働くイメージです。\nコンサルタント: 65歳から年金をもらう形で考えていますか?\n顧客: そうですね、65歳から年金をもらうという形です。",
"topic": "老後の働き方と年金",
"emotion": "堅実・少し不安",
"importance": 5,
"keywords": [
"定年退職",
"再雇用",
"年金受給",
"老後資金"
]
}
],
"broll_keywords": [
"学校",
"勉強する子供",
"新車ディーラー",
"住宅リフォーム",
"住宅ローン計算",
"海外リゾート",
"飛行機",
"オフィスで働く高齢者",
"年金手帳",
"ライフプランニング"
],
"_metadata": {
"token_usage": {
"input_tokens": 26044,
"output_tokens": 1177,
"total_tokens": 28611
},
"model": "gemini-3-pro-preview",
"video_name": "№2_ライフイベント.mp4"
}
}

1
assets/2_output/.gitkeep Normal file
View File

@ -0,0 +1 @@
# Final rendered video outputs

View File

@ -0,0 +1,60 @@
# Subtitle Extraction Investigation
## Overview
This script compares different methods and granularity levels for subtitle extraction from video files.
## Installation Requirements
### FFmpeg (Required for Audio Extraction)
**Windows:**
1. Download from: https://ffmpeg.org/download.html
2. Or use winget: `winget install ffmpeg`
3. Or use chocolatey: `choco install ffmpeg`
After installation, restart your terminal and verify:
```powershell
ffmpeg -version
```
## Features
### Granularity Levels
1. **Movie-level**: Entire video summary + full transcript
2. **Second-level**: Time-stamped subtitles by phrase/sentence (MM:SS.mmm)
3. **Word-level**: Individual word timestamps
### Comparison Methods
- **Method A**: Upload video directly to Gemini
- **Method B**: Extract audio with ffmpeg, then upload audio only
## Results Analysis
### Token Usage (from first run - video only):
- **Movie-level**: 15,618 tokens | 32s processing
- **Second-level**: 19,456 tokens | 96s processing
- **Word-level**: 40,279 tokens | 226s processing
### Key Findings:
1. Higher granularity = more tokens + longer processing
2. Word-level is ~2.5x more expensive than movie-level
3. Second-level provides good balance for subtitle generation
## Usage
```bash
# Activate virtual environment
.\venv\Scripts\Activate.ps1
# Run investigation
python backend/investigate_subtitle_extraction.py
```
## Output Files
- `assets/1_processed/subtitle_investigation/*.json` - Individual comparison results
- `complete_comparison.json` - Full comparison data with metadata
## Next Steps
- Install ffmpeg to test Method B (audio extraction)
- Compare audio vs video method performance
- Determine optimal approach for production pipeline

3
backend/__init__.py Normal file
View File

@ -0,0 +1,3 @@
"""Backend package for Auto Life Plan Video Generator"""
__version__ = "0.1.0"

199
backend/data_builder.py Normal file
View File

@ -0,0 +1,199 @@
"""
Chart data builder and financial simulation engine
This module:
- Simulates future asset projections
- Handles inflation calculations
- Calculates compound interest on deficits
- Generates data in format compatible with Remotion charts
"""
from typing import Dict, Any, List
import math
class FinancialSimulator:
"""Simulate financial projections for Life Plan analysis"""
def __init__(self, config: Optional[Dict[str, Any]] = None):
self.config = config or {}
self.inflation_rate = self.config.get("inflation_rate", 0.02) # 2% default
self.deficit_interest_rate = self.config.get("deficit_interest_rate", 0.15) # 15% default
self.investment_return = self.config.get("investment_return", 0.03) # 3% default
def simulate_timeline(
self,
current_age: int,
retirement_age: int,
life_expectancy: int,
current_assets: float,
annual_income: float,
annual_expenses: float,
life_events: Optional[List[Dict]] = None,
) -> List[Dict[str, Any]]:
"""
Simulate year-by-year financial timeline
Args:
current_age: Current age of representative
retirement_age: Planned retirement age
life_expectancy: Expected life expectancy
current_assets: Current asset balance (万円)
annual_income: Annual income (万円)
annual_expenses: Annual expenses (万円)
life_events: List of major life events {year, type, amount}
Returns:
List of yearly snapshots compatible with YearlySnapshot interface
"""
timeline = []
current_year = 2025 # TODO: Make this dynamic
asset_balance = current_assets
life_events = life_events or []
for age in range(current_age, life_expectancy + 1):
year = current_year + (age - current_age)
# Adjust income (stops at retirement)
income = annual_income if age < retirement_age else 0
# Apply inflation to expenses
years_from_now = age - current_age
inflated_expenses = annual_expenses * math.pow(1 + self.inflation_rate, years_from_now)
# Check for life events
event_cost = self._get_event_cost(year, life_events)
total_expenses = inflated_expenses + event_cost
# Calculate net cash flow
net_cash_flow = income - total_expenses
# Update asset balance
if asset_balance >= 0:
# Positive balance: apply investment return
asset_balance = asset_balance * (1 + self.investment_return) + net_cash_flow
else:
# Negative balance: apply deficit interest (compound debt)
asset_balance = asset_balance * (1 + self.deficit_interest_rate) + net_cash_flow
# Create snapshot
snapshot = {
"year": year,
"ages": {
"rep": age,
"spouse": None, # TODO: Add spouse calculation
},
"financials": {
"income": round(income, 0),
"expenditure": round(total_expenses, 0),
"assetBalance": round(asset_balance, 0),
},
"events": self._get_event_labels(year, life_events),
"isCritical": asset_balance < 0,
}
timeline.append(snapshot)
return timeline
def _get_event_cost(self, year: int, life_events: List[Dict]) -> float:
"""Get total event cost for a specific year"""
total = 0
for event in life_events:
if event.get("year") == year:
total += event.get("amount", 0)
return total
def _get_event_labels(self, year: int, life_events: List[Dict]) -> List[str]:
"""Get event labels for a specific year"""
labels = []
for event in life_events:
if event.get("year") == year:
labels.append(event.get("label", "Event"))
return labels
def build_cashflow_data(self, timeline: List[Dict]) -> List[Dict]:
"""
Convert timeline to CashFlow chart format
Args:
timeline: YearlySnapshot format timeline
Returns:
List compatible with CashFlowData interface
"""
cashflow = []
for snapshot in timeline:
cashflow.append({
"year": snapshot["year"],
"age": snapshot["ages"]["rep"],
"income": {
"total": snapshot["financials"]["income"],
},
"expenses": {
"living": snapshot["financials"]["expenditure"] * 0.4, # Mock breakdown
"housing": snapshot["financials"]["expenditure"] * 0.3,
"education": snapshot["financials"]["expenditure"] * 0.15,
"tax": snapshot["financials"]["expenditure"] * 0.1,
"event": snapshot["financials"]["expenditure"] * 0.05,
},
"totalExpense": snapshot["financials"]["expenditure"],
"netCashFlow": snapshot["financials"]["income"] - snapshot["financials"]["expenditure"],
"assetBalance": snapshot["financials"]["assetBalance"],
})
return cashflow
def build_chart_data(pdf_data: Dict[str, Any]) -> Dict[str, Any]:
"""
Build chart data from PDF extraction results
Args:
pdf_data: Structured data from PDF extraction
Returns:
Dictionary with 'assetDepletion' and 'cashFlow' chart data
"""
simulator = FinancialSimulator()
# Extract parameters from PDF data
current_age = pdf_data.get("current_age", 35)
retirement_age = pdf_data.get("retirement_age", 65)
life_expectancy = pdf_data.get("life_expectancy", 100)
current_assets = pdf_data.get("current_assets", 5000)
annual_income = pdf_data.get("annual_income", 600)
annual_expenses = pdf_data.get("annual_expenses", 450)
life_events = pdf_data.get("life_events", [])
# Simulate timeline
timeline = simulator.simulate_timeline(
current_age=current_age,
retirement_age=retirement_age,
life_expectancy=life_expectancy,
current_assets=current_assets,
annual_income=annual_income,
annual_expenses=annual_expenses,
life_events=life_events,
)
# Build cashflow data
cashflow = simulator.build_cashflow_data(timeline)
return {
"assetDepletion": {
"timeline": timeline,
"meta": {
"currencyUnit": "万円",
"startYear": timeline[0]["year"] if timeline else 2025,
},
},
"cashFlow": {
"timeline": cashflow,
"meta": {
"currencyUnit": "万円",
},
},
}

83
backend/extractor.py Normal file
View File

@ -0,0 +1,83 @@
"""
Data extraction module using Google Gemini Pro and OpenAI Whisper
This module handles:
- PDF parsing (Life Plan reports in Japanese)
- Video transcription and timestamp extraction
- Structured data extraction using LLMs
"""
import os
from typing import Dict, Any, List, Optional
from pathlib import Path
class PDFExtractor:
"""Extract structured data from Life Plan PDF reports using Gemini Pro"""
def __init__(self, api_key: Optional[str] = None):
self.api_key = api_key or os.getenv("GEMINI_API_KEY")
if not self.api_key:
raise ValueError("GEMINI_API_KEY not found in environment")
def extract(self, pdf_path: str) -> Dict[str, Any]:
"""
Extract structured data from PDF
Args:
pdf_path: Path to the PDF file
Returns:
Dictionary containing:
- client_info: Name, age, family structure
- financial_data: Income, expenses, assets
- life_events: Major events (retirement, education, housing)
- insurance_data: Current policies and recommendations
"""
# TODO: Implement Gemini Pro PDF parsing
# 1. Upload PDF to Gemini API
# 2. Use prompt engineering to extract structured JSON
# 3. Validate against schema
raise NotImplementedError("PDF extraction not yet implemented")
class VideoExtractor:
"""Extract transcription and timestamps from interview videos"""
def __init__(self, api_key: Optional[str] = None):
self.api_key = api_key or os.getenv("OPENAI_API_KEY")
if not self.api_key:
raise ValueError("OPENAI_API_KEY not found in environment")
def extract(self, video_path: str) -> Dict[str, Any]:
"""
Extract audio transcription and meaningful timestamps
Args:
video_path: Path to the video file
Returns:
Dictionary containing:
- transcription: Full text transcription
- timestamps: List of {time, text, keywords} objects
- segments: Recommended video segments for B-Roll
"""
# TODO: Implement Whisper transcription
# 1. Extract audio from video using FFmpeg
# 2. Run Whisper API
# 3. Parse timestamps and keywords
raise NotImplementedError("Video extraction not yet implemented")
def extract_pdf_data(pdf_path: str) -> Dict[str, Any]:
"""Convenience function for PDF extraction"""
extractor = PDFExtractor()
return extractor.extract(pdf_path)
def extract_video_data(video_path: str) -> Dict[str, Any]:
"""Convenience function for video extraction"""
extractor = VideoExtractor()
return extractor.extract(video_path)

View File

@ -0,0 +1,358 @@
"""
Investigation Script 2: Keyword Translation and Pexels Media Fetcher
This script demonstrates:
1. Translating Japanese keywords to English using Gemini
2. Searching Pexels API for relevant B-Roll footage
3. Downloading videos to local assets folder
4. Caching translations to avoid repeated API calls
Target: Find stock footage for insurance/financial video production
"""
import os
import json
import time
from pathlib import Path
from typing import List, Dict, Any, Optional
from dotenv import load_dotenv
from google import genai
import requests
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn, DownloadColumn, TransferSpeedColumn
from rich.panel import Panel
from rich.table import Table
# Load environment variables
load_dotenv()
# Initialize Rich console
console = Console()
# Configure APIs
GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
PEXELS_API_KEY = os.getenv("PEXELS_API_KEY")
if not GEMINI_API_KEY or not PEXELS_API_KEY:
console.print("[red]❌ API keys not found in .env file[/red]")
console.print("Required: GEMINI_API_KEY, PEXELS_API_KEY")
exit(1)
# Initialize Gemini client
client = genai.Client(api_key=GEMINI_API_KEY)
class KeywordTranslator:
"""Translate Japanese keywords to English using Gemini"""
def __init__(self, cache_file: str = "assets/1_processed/translation_cache.json"):
self.model = 'gemini-3-pro-preview' # Latest model for translation
self.cache_file = Path(cache_file)
self.cache = self._load_cache()
console.print("[green]✓[/green] Initialized Gemini translator with cache")
def _load_cache(self) -> Dict[str, str]:
"""Load translation cache from file"""
if self.cache_file.exists():
with open(self.cache_file, 'r', encoding='utf-8') as f:
return json.load(f)
return {}
def _save_cache(self):
"""Save translation cache to file"""
self.cache_file.parent.mkdir(parents=True, exist_ok=True)
with open(self.cache_file, 'w', encoding='utf-8') as f:
json.dump(self.cache, f, ensure_ascii=False, indent=2)
def translate(self, japanese_keywords: List[str]) -> Dict[str, str]:
"""
Translate Japanese keywords to English
Args:
japanese_keywords: List of Japanese keywords
Returns:
Dictionary mapping Japanese to English keywords
"""
translations = {}
keywords_to_translate = []
# Check cache first
for keyword in japanese_keywords:
if keyword in self.cache:
translations[keyword] = self.cache[keyword]
else:
keywords_to_translate.append(keyword)
if keywords_to_translate:
console.print(f"[cyan]Translating {len(keywords_to_translate)} new keywords...[/cyan]")
# Batch translation for efficiency
prompt = f"""
Translate the following Japanese keywords to English for searching stock footage/videos.
Provide natural, searchable English terms that would work well for video search.
Japanese keywords: {', '.join(keywords_to_translate)}
Return ONLY a JSON object mapping Japanese to English, like this:
{{
"日本語キーワード1": "english keyword 1",
"日本語キーワード2": "english keyword 2"
}}
"""
response = client.models.generate_content(
model=self.model,
contents=prompt
)
# Display token usage
if hasattr(response, 'usage_metadata'):
token_info = {
"input_tokens": response.usage_metadata.prompt_token_count,
"output_tokens": response.usage_metadata.candidates_token_count,
"total_tokens": response.usage_metadata.total_token_count
}
console.print(f"[cyan]📊 Translation Tokens:[/cyan] {token_info['total_tokens']:,} (in: {token_info['input_tokens']:,}, out: {token_info['output_tokens']:,})")
try:
# Parse response
response_text = response.text
if "```json" in response_text:
json_start = response_text.find("```json") + 7
json_end = response_text.find("```", json_start)
response_text = response_text[json_start:json_end].strip()
elif "```" in response_text:
json_start = response_text.find("```") + 3
json_end = response_text.find("```", json_start)
response_text = response_text[json_start:json_end].strip()
new_translations = json.loads(response_text)
# Update cache and results
self.cache.update(new_translations)
translations.update(new_translations)
self._save_cache()
console.print(f"[green]✓[/green] Translated {len(new_translations)} keywords")
except json.JSONDecodeError as e:
console.print(f"[yellow]⚠[/yellow] Translation parsing failed: {e}")
# Fallback: use romanized version
for keyword in keywords_to_translate:
translations[keyword] = keyword
return translations
class PexelsFetcher:
"""Fetch B-Roll videos from Pexels API"""
def __init__(self, api_key: str, output_dir: str = "assets/1_processed/broll"):
self.api_key = api_key
self.output_dir = Path(output_dir)
self.output_dir.mkdir(parents=True, exist_ok=True)
self.base_url = "https://api.pexels.com/videos/search"
console.print("[green]✓[/green] Initialized Pexels API client")
def search_videos(self, query: str, max_results: int = 3, min_duration: int = 5) -> List[Dict[str, Any]]:
"""
Search for videos on Pexels
Args:
query: Search query (in English)
max_results: Maximum number of videos to return
min_duration: Minimum video duration in seconds
Returns:
List of video metadata dictionaries
"""
headers = {
"Authorization": self.api_key
}
params = {
"query": query,
"per_page": max_results,
"orientation": "landscape", # Better for editing
"size": "medium" # Balance quality and file size
}
try:
response = requests.get(self.base_url, headers=headers, params=params)
response.raise_for_status()
data = response.json()
videos = []
for video in data.get("videos", []):
# Find best quality HD file
video_files = video.get("video_files", [])
hd_file = None
for vf in video_files:
if vf.get("quality") == "hd" and vf.get("width") == 1920:
hd_file = vf
break
# Fallback to first available file
if not hd_file and video_files:
hd_file = video_files[0]
if hd_file and video.get("duration", 0) >= min_duration:
videos.append({
"id": video["id"],
"url": video["url"],
"duration": video["duration"],
"width": video["width"],
"height": video["height"],
"download_url": hd_file["link"],
"file_size": hd_file.get("size", 0),
"photographer": video["user"]["name"]
})
return videos[:max_results]
except requests.exceptions.RequestException as e:
console.print(f"[red]❌ Pexels API error:[/red] {e}")
return []
def download_video(self, video: Dict[str, Any], keyword: str) -> Optional[str]:
"""
Download a video file
Args:
video: Video metadata dictionary
keyword: Keyword used for search (for filename)
Returns:
Path to downloaded file, or None if failed
"""
# Create safe filename
safe_keyword = "".join(c for c in keyword if c.isalnum() or c in (' ', '-', '_')).strip()
safe_keyword = safe_keyword.replace(' ', '_')
filename = f"{safe_keyword}_{video['id']}.mp4"
filepath = self.output_dir / filename
# Skip if already downloaded
if filepath.exists():
console.print(f"[yellow]⊘[/yellow] Already exists: {filename}")
return str(filepath)
try:
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
DownloadColumn(),
TransferSpeedColumn(),
console=console
) as progress:
task = progress.add_task(f"Downloading {filename[:40]}...", total=video.get('file_size'))
response = requests.get(video['download_url'], stream=True)
response.raise_for_status()
with open(filepath, 'wb') as f:
for chunk in response.iter_content(chunk_size=8192):
f.write(chunk)
progress.update(task, advance=len(chunk))
console.print(f"[green]✓[/green] Downloaded: {filename}")
return str(filepath)
except Exception as e:
console.print(f"[red]❌ Download failed:[/red] {e}")
if filepath.exists():
filepath.unlink() # Remove partial file
return None
def main():
"""Main investigation workflow"""
console.print(Panel.fit(
"[bold cyan]Pexels Media Fetch Investigation Script[/bold cyan]\n"
"Testing keyword translation and B-Roll video download",
border_style="cyan"
))
# Initialize components
translator = KeywordTranslator()
fetcher = PexelsFetcher(PEXELS_API_KEY)
# Test keywords (typical for insurance/life planning videos)
test_keywords_ja = [
"家族",
"老後",
"貯蓄",
"住宅",
"教育",
"保険",
"将来",
"不安",
"安心",
"計画"
]
console.print("\n[bold]Test Keywords (Japanese):[/bold]")
console.print(", ".join(test_keywords_ja))
# Translate keywords
console.print("\n" + "="*60)
translations = translator.translate(test_keywords_ja)
# Display translations
table = Table(title="Translation Results", show_header=True, header_style="bold cyan")
table.add_column("Japanese", style="yellow")
table.add_column("English", style="green")
for ja, en in translations.items():
table.add_row(ja, en)
console.print(table)
# Search and download videos
console.print("\n" + "="*60)
console.print("[bold]Searching Pexels for B-Roll footage...[/bold]\n")
results = {}
for ja_keyword, en_keyword in list(translations.items())[:5]: # Limit to 5 for testing
console.print(f"\n[cyan]Searching:[/cyan] '{en_keyword}' (from '{ja_keyword}')")
videos = fetcher.search_videos(en_keyword, max_results=2)
if videos:
console.print(f" Found {len(videos)} video(s)")
downloaded = []
for video in videos:
filepath = fetcher.download_video(video, en_keyword)
if filepath:
downloaded.append({
"filepath": filepath,
"duration": video["duration"],
"photographer": video["photographer"]
})
results[ja_keyword] = {
"english": en_keyword,
"videos": downloaded
}
else:
console.print(f" [yellow]No videos found[/yellow]")
# Save results summary
output_file = Path("assets/1_processed/broll_fetch_results.json")
output_file.parent.mkdir(parents=True, exist_ok=True)
with open(output_file, 'w', encoding='utf-8') as f:
json.dump(results, f, ensure_ascii=False, indent=2)
console.print("\n" + "="*60)
console.print(f"[bold green]✓ Investigation Complete![/bold green]")
console.print(f"Downloaded videos saved to: {fetcher.output_dir}")
console.print(f"Results summary saved to: {output_file}")
if __name__ == "__main__":
main()

View File

@ -0,0 +1,412 @@
"""
Investigation Script 3: Subtitle/Transcript Extraction - Granularity Comparison
This script tests different approaches for subtitle extraction:
1. Movie-level (entire video summary)
2. Second-level (time-stamped subtitles per second/phrase)
3. Word-level (individual word timestamps)
Compares two methods:
- Method A: Upload video directly to Gemini
- Method B: Extract audio with ffmpeg, upload audio only
Tests which is faster, more accurate, and uses fewer tokens.
"""
import os
import json
import time
import subprocess
from pathlib import Path
from typing import Dict, Any, List, Optional, Literal
from dotenv import load_dotenv
from google import genai
from google.genai import types
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn
from rich.panel import Panel
from rich.table import Table
from rich.json import JSON
# Load environment variables
load_dotenv()
# Initialize Rich console
console = Console()
# Configure Gemini API
GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
if not GEMINI_API_KEY:
console.print("[red]❌ GEMINI_API_KEY not found in .env file[/red]")
exit(1)
# Initialize Gemini client
client = genai.Client(api_key=GEMINI_API_KEY)
class AudioExtractor:
"""Extract audio from video using ffmpeg"""
@staticmethod
def extract_audio(video_path: str, output_path: str) -> Optional[str]:
"""
Extract audio from video file using ffmpeg
Args:
video_path: Path to video file
output_path: Path for output audio file
Returns:
Path to extracted audio file, or None if failed
"""
try:
console.print(f"[cyan]Extracting audio from video...[/cyan]")
# Use ffmpeg to extract audio as MP3
cmd = [
'ffmpeg',
'-i', video_path,
'-vn', # No video
'-acodec', 'libmp3lame', # MP3 codec
'-q:a', '2', # High quality
'-y', # Overwrite output
output_path
]
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=True
)
console.print(f"[green]✓[/green] Audio extracted: {Path(output_path).name}")
return output_path
except subprocess.CalledProcessError as e:
console.print(f"[red]❌ FFmpeg error:[/red] {e.stderr}")
return None
except FileNotFoundError:
console.print("[red]❌ FFmpeg not found. Please install ffmpeg.[/red]")
return None
class SubtitleExtractor:
"""Extract subtitles at different granularity levels"""
def __init__(self):
self.model = 'gemini-3-pro-preview'
console.print("[green]✓[/green] Initialized Subtitle Extractor")
def upload_media(self, file_path: str, mime_type: str) -> Any:
"""Upload video or audio file to Gemini"""
console.print(f"[cyan]Uploading {mime_type.split('/')[0]}:[/cyan] {Path(file_path).name}")
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task("Uploading to Gemini API...", total=None)
with open(file_path, 'rb') as f:
upload_config = types.UploadFileConfig(mime_type=mime_type)
media_file = client.files.upload(file=f, config=upload_config)
# Wait for processing
while media_file.state == "PROCESSING":
time.sleep(2)
media_file = client.files.get(name=media_file.name)
progress.update(task, completed=True)
if media_file.state == "FAILED":
raise ValueError(f"Media processing failed: {media_file.state}")
console.print(f"[green]✓[/green] Media uploaded successfully")
return media_file
def extract_movie_level(self, media_file: Any) -> Dict[str, Any]:
"""
Extract movie-level subtitle (entire video summary)
Returns:
Dictionary with summary and full transcript
"""
prompt = """
このビデオ/音声の内容を全体として書き起こしてください。
以下の情報を抽出してください:
1. **全体の要約**: ビデオ全体の内容を2-3文で要約
2. **完全な書き起こし**: すべての発言を時系列で書き起こし
3. **話者**: 誰が話しているか識別
**出力形式**: 必ずJSON形式で返してください:
{
"summary": "全体要約",
"full_transcript": "完全な書き起こしテキスト",
"speakers": ["話者1", "話者2"],
"duration": "MM:SS"
}
"""
return self._generate_content(media_file, prompt, "Movie-level")
def extract_second_level(self, media_file: Any) -> Dict[str, Any]:
"""
Extract second-level subtitles (time-stamped by phrase/sentence)
Returns:
Dictionary with time-stamped subtitle segments
"""
prompt = """
この音声/ビデオの内容を時間軸に沿って正確に書き起こし、字幕形式で出力してください。
**要件**:
- 各発言を開始時刻と終了時刻付きでセグメント化
- 話者を識別
- 時刻はMM:SS.mmm形式(秒単位で正確に)
**出力形式**: 必ずJSON形式で返してください:
{
"subtitles": [
{
"index": 1,
"start_time": "00:00.000",
"end_time": "00:03.500",
"speaker": "話者名",
"text": "発言内容"
}
],
"total_segments": 0
}
"""
return self._generate_content(media_file, prompt, "Second-level")
def extract_word_level(self, media_file: Any) -> Dict[str, Any]:
"""
Extract word-level subtitles (timestamp for each word)
Returns:
Dictionary with word-by-word timestamps
"""
prompt = """
この音声/ビデオの内容を単語レベルで書き起こし、各単語のタイムスタンプを付けてください。
**要件**:
- 各単語に開始時刻を付与
- 話者を識別
- 時刻はMM:SS.mmm形式
**出力形式**: 必ずJSON形式で返してください:
{
"word_timestamps": [
{
"word": "単語",
"start_time": "00:00.000",
"speaker": "話者名"
}
],
"total_words": 0
}
"""
return self._generate_content(media_file, prompt, "Word-level")
def _generate_content(self, media_file: Any, prompt: str, level: str) -> Dict[str, Any]:
"""Generate content with token tracking"""
console.print(f"[cyan]Extracting {level} subtitles...[/cyan]")
start_time = time.time()
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task(f"Generating {level} content...", total=None)
response = client.models.generate_content(
model=self.model,
contents=[media_file, prompt]
)
progress.update(task, completed=True)
elapsed_time = time.time() - start_time
# Extract token usage
token_info = {
"input_tokens": response.usage_metadata.prompt_token_count if hasattr(response, 'usage_metadata') else 0,
"output_tokens": response.usage_metadata.candidates_token_count if hasattr(response, 'usage_metadata') else 0,
"total_tokens": response.usage_metadata.total_token_count if hasattr(response, 'usage_metadata') else 0
}
console.print(f"[cyan]📊 Tokens:[/cyan] {token_info['total_tokens']:,} | [cyan]⏱ Time:[/cyan] {elapsed_time:.2f}s")
# Parse response
try:
response_text = response.text
if "```json" in response_text:
json_start = response_text.find("```json") + 7
json_end = response_text.find("```", json_start)
response_text = response_text[json_start:json_end].strip()
elif "```" in response_text:
json_start = response_text.find("```") + 3
json_end = response_text.find("```", json_start)
response_text = response_text[json_start:json_end].strip()
result = json.loads(response_text)
result["_metadata"] = {
"level": level,
"token_usage": token_info,
"processing_time_seconds": elapsed_time,
"model": self.model
}
console.print(f"[green]✓[/green] {level} extraction complete")
return result
except json.JSONDecodeError as e:
console.print(f"[yellow]⚠[/yellow] JSON parsing failed, returning raw response")
return {
"raw_response": response.text,
"error": str(e),
"_metadata": {
"level": level,
"token_usage": token_info,
"processing_time_seconds": elapsed_time
}
}
def compare_methods(video_path: str, output_dir: Path, granularity: Literal["movie", "second", "word"]):
"""
Compare video vs audio extraction for a specific granularity level
Args:
video_path: Path to video file
output_dir: Output directory for results
granularity: Level of granularity to test
"""
extractor = SubtitleExtractor()
audio_extractor = AudioExtractor()
results = {}
console.print(f"\n[bold cyan]Testing {granularity.upper()}-level extraction[/bold cyan]")
console.print("="*60)
# Method A: Direct video upload
console.print("\n[bold]Method A: Video Upload[/bold]")
try:
video_file = extractor.upload_media(video_path, 'video/mp4')
if granularity == "movie":
result_a = extractor.extract_movie_level(video_file)
elif granularity == "second":
result_a = extractor.extract_second_level(video_file)
else: # word
result_a = extractor.extract_word_level(video_file)
results["method_a_video"] = result_a
except Exception as e:
console.print(f"[red]❌ Method A failed:[/red] {e}")
results["method_a_video"] = {"error": str(e)}
# Method B: Audio extraction + upload
console.print("\n[bold]Method B: Audio Extraction + Upload[/bold]")
try:
# Extract audio
audio_path = str(output_dir / f"{Path(video_path).stem}_audio.mp3")
extracted_audio = audio_extractor.extract_audio(video_path, audio_path)
if extracted_audio:
audio_file = extractor.upload_media(extracted_audio, 'audio/mpeg')
if granularity == "movie":
result_b = extractor.extract_movie_level(audio_file)
elif granularity == "second":
result_b = extractor.extract_second_level(audio_file)
else: # word
result_b = extractor.extract_word_level(audio_file)
results["method_b_audio"] = result_b
else:
results["method_b_audio"] = {"error": "Audio extraction failed"}
except Exception as e:
console.print(f"[red]❌ Method B failed:[/red] {e}")
results["method_b_audio"] = {"error": str(e)}
return results
def main():
"""Main investigation workflow"""
console.print(Panel.fit(
"[bold cyan]Subtitle Extraction Investigation[/bold cyan]\n"
"Comparing Video vs Audio | Movie/Second/Word Level Granularity",
border_style="cyan"
))
# Setup paths
input_dir = Path("assets/0_input")
output_dir = Path("assets/1_processed/subtitle_investigation")
output_dir.mkdir(parents=True, exist_ok=True)
# Get first video for testing
video_files = list(input_dir.glob("*.mp4"))
if not video_files:
console.print("[red]❌ No video files found[/red]")
return
test_video = video_files[0] # Use first video for comparison
console.print(f"\n[cyan]Test Video:[/cyan] {test_video.name}")
# Test all granularity levels
all_results = {}
for granularity in ["movie", "second", "word"]:
results = compare_methods(str(test_video), output_dir, granularity)
all_results[granularity] = results
# Save individual results
result_file = output_dir / f"{test_video.stem}_{granularity}_comparison.json"
with open(result_file, 'w', encoding='utf-8') as f:
json.dump(results, f, ensure_ascii=False, indent=2)
console.print(f"[green]✓[/green] Results saved: {result_file.name}")
# Generate comparison summary
console.print("\n" + "="*60)
console.print("[bold green]Comparison Summary[/bold green]\n")
table = Table(title="Performance Comparison", show_header=True)
table.add_column("Granularity", style="cyan")
table.add_column("Method", style="yellow")
table.add_column("Tokens", justify="right", style="magenta")
table.add_column("Time (s)", justify="right", style="blue")
for granularity, results in all_results.items():
for method_name, data in results.items():
if "_metadata" in data:
meta = data["_metadata"]
table.add_row(
granularity.upper(),
method_name.replace("_", " ").title(),
f"{meta['token_usage']['total_tokens']:,}",
f"{meta['processing_time_seconds']:.2f}"
)
console.print(table)
# Save complete comparison
summary_file = output_dir / "complete_comparison.json"
with open(summary_file, 'w', encoding='utf-8') as f:
json.dump(all_results, f, ensure_ascii=False, indent=2)
console.print(f"\n[green]✓[/green] Complete comparison saved: {summary_file}")
if __name__ == "__main__":
main()

View File

@ -0,0 +1,290 @@
"""
Investigation Script 1: Video Upload and Timeline/Dialogue Extraction using Gemini
This script demonstrates:
1. Uploading video files to Google Gemini API
2. Extracting dialogue/transcription from video
3. Identifying timestamps for key moments
4. Structuring the output for video editing pipeline
Target: Process interview videos (№1_基本情報.mp4, №2_ライフイベント.mp4)
"""
import os
import json
import time
from pathlib import Path
from typing import Dict, Any, List
from dotenv import load_dotenv
from google import genai
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn
from rich.panel import Panel
from rich.json import JSON
# Load environment variables
load_dotenv()
# Initialize Rich console for beautiful output
console = Console()
# Configure Gemini API
GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
if not GEMINI_API_KEY:
console.print("[red]❌ GEMINI_API_KEY not found in .env file[/red]")
exit(1)
# Initialize Gemini client
client = genai.Client(api_key=GEMINI_API_KEY)
class VideoExtractor:
"""Extract timeline and dialogue from video using Gemini"""
def __init__(self):
# Use Gemini 3 Pro for video understanding
self.model = 'gemini-3-pro-preview' # Latest model supporting video
console.print("[green]✓[/green] Initialized Gemini 3 Pro model")
def upload_video(self, video_path: str) -> Any:
"""
Upload video file to Gemini API
Args:
video_path: Path to video file
Returns:
Uploaded file object
"""
console.print(f"[cyan]Uploading video:[/cyan] {Path(video_path).name}")
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task("Uploading to Gemini API...", total=None)
# Upload the video file with config
from google.genai import types
with open(video_path, 'rb') as f:
upload_config = types.UploadFileConfig(
mime_type='video/mp4'
)
video_file = client.files.upload(
file=f,
config=upload_config
)
# Wait for processing to complete
while video_file.state == "PROCESSING":
time.sleep(2)
video_file = client.files.get(name=video_file.name)
progress.update(task, completed=True)
if video_file.state == "FAILED":
raise ValueError(f"Video processing failed: {video_file.state}")
console.print(f"[green]✓[/green] Video uploaded successfully: {video_file.name}")
return video_file
def extract_timeline_and_dialogue(self, video_file: Any, video_name: str) -> Dict[str, Any]:
"""
Extract structured timeline and dialogue from video
Args:
video_file: Uploaded video file object
video_name: Name of the video for context
Returns:
Dictionary containing timeline segments with timestamps and transcription
"""
console.print("\n[cyan]Analyzing video content...[/cyan]")
# Prompt for structured extraction (in Japanese for better understanding)
prompt = f"""
このビデオは保険コンサルタントと顧客のインタビュー映像です。
ビデオのタイトル: {video_name}
以下の情報を抽出してください:
1. **全体の要約**: ビデオの内容を2-3文で要約
2. **話者情報**: 誰が話しているか(例:コンサルタント、顧客)
3. **タイムライン**: 重要な場面ごとに以下の情報を含むセグメントに分割
- 開始時刻(MM:SS形式)
- 終了時刻(MM:SS形式)
- 発言内容(日本語の書き起こし)
- トピック/キーワード(例:「基本情報」「家族構成」「将来の不安」)
- 感情/トーン(例:「心配」「前向き」「真剣」)
- 映像使用の推奨度(1-5、5が最も重要)
4. **キーワード**: B-Roll素材検索用のキーワードを5-10個抽出(日本語)
**出力形式**: 必ずJSON形式で返してください。以下の構造に従ってください:
{{
"summary": "ビデオ全体の要約",
"speakers": ["話者1", "話者2"],
"duration": "MM:SS",
"segments": [
{{
"start_time": "00:00",
"end_time": "00:30",
"speaker": "コンサルタント",
"transcript": "発言内容の書き起こし",
"topic": "トピック名",
"emotion": "感情/トーン",
"importance": 4,
"keywords": ["キーワード1", "キーワード2"]
}}
],
"broll_keywords": ["素材検索用キーワード1", "キーワード2", "..."]
}}
"""
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
console=console
) as progress:
task = progress.add_task("Extracting content with Gemini...", total=None)
# Generate content from video - use file object directly with prompt
response = client.models.generate_content(
model=self.model,
contents=[
video_file, # Pass the file object directly
prompt
]
)
progress.update(task, completed=True)
# Parse the response
try:
# Extract token usage information
token_info = {
"input_tokens": response.usage_metadata.prompt_token_count if hasattr(response, 'usage_metadata') else 0,
"output_tokens": response.usage_metadata.candidates_token_count if hasattr(response, 'usage_metadata') else 0,
"total_tokens": response.usage_metadata.total_token_count if hasattr(response, 'usage_metadata') else 0
}
# Display token usage
console.print(f"[cyan]📊 Token Usage:[/cyan] Input: {token_info['input_tokens']:,} | Output: {token_info['output_tokens']:,} | Total: {token_info['total_tokens']:,}")
# Extract JSON from response (handle markdown code blocks if present)
response_text = response.text
if "```json" in response_text:
json_start = response_text.find("```json") + 7
json_end = response_text.find("```", json_start)
response_text = response_text[json_start:json_end].strip()
elif "```" in response_text:
json_start = response_text.find("```") + 3
json_end = response_text.find("```", json_start)
response_text = response_text[json_start:json_end].strip()
result = json.loads(response_text)
result["_metadata"] = {
"token_usage": token_info,
"model": self.model,
"video_name": video_name
}
console.print("[green]✓[/green] Successfully extracted timeline and dialogue")
return result
except json.JSONDecodeError as e:
console.print(f"[yellow]⚠[/yellow] Failed to parse JSON, returning raw response")
return {
"raw_response": response.text,
"error": str(e),
"_metadata": {
"token_usage": token_info if 'token_info' in locals() else {},
"model": self.model
}
}
def save_results(self, results: Dict[str, Any], output_path: str):
"""Save extraction results to JSON file"""
output_file = Path(output_path)
output_file.parent.mkdir(parents=True, exist_ok=True)
with open(output_file, 'w', encoding='utf-8') as f:
json.dump(results, f, ensure_ascii=False, indent=2)
console.print(f"[green]✓[/green] Results saved to: {output_file}")
def main():
"""Main investigation workflow"""
console.print(Panel.fit(
"[bold cyan]Video Extraction Investigation Script[/bold cyan]\n"
"Testing Gemini 1.5 Pro for video timeline and dialogue extraction",
border_style="cyan"
))
# Initialize extractor
extractor = VideoExtractor()
# Define input and output paths
input_dir = Path("assets/0_input")
output_dir = Path("assets/1_processed/video_extractions")
# Get all MP4 files
video_files = list(input_dir.glob("*.mp4"))
if not video_files:
console.print("[red]❌ No video files found in assets/0_input/[/red]")
return
console.print(f"\n[cyan]Found {len(video_files)} video(s) to process:[/cyan]")
for video in video_files:
console.print(f" • {video.name}")
# Process each video
all_results = {}
total_tokens_used = 0
for video_path in video_files:
console.print(f"\n{'='*60}")
console.print(f"[bold]Processing: {video_path.name}[/bold]")
console.print(f"{'='*60}\n")
try:
# Upload video
video_file = extractor.upload_video(str(video_path))
# Extract timeline and dialogue
results = extractor.extract_timeline_and_dialogue(video_file, video_path.name)
# Display results
console.print("\n[bold green]Extraction Results:[/bold green]")
console.print(JSON(json.dumps(results, ensure_ascii=False)))
# Save results
output_file = output_dir / f"{video_path.stem}_extraction.json"
extractor.save_results(results, str(output_file))
all_results[video_path.name] = results
# Track total tokens
if "_metadata" in results and "token_usage" in results["_metadata"]:
total_tokens_used += results["_metadata"]["token_usage"].get("total_tokens", 0)
except Exception as e:
console.print(f"[red]❌ Error processing {video_path.name}:[/red] {e}")
continue
# Save combined results
if all_results:
combined_output = output_dir / "all_extractions.json"
extractor.save_results(all_results, str(combined_output))
console.print("\n" + "="*60)
console.print("[bold green]✓ Investigation Complete![/bold green]")
console.print(f"Processed {len(all_results)} video(s)")
console.print(f"Results saved to: {output_dir}")
console.print(f"\n[cyan]💰 Total Tokens Consumed: {total_tokens_used:,}[/cyan]")
if __name__ == "__main__":
main()

71
backend/media_fetcher.py Normal file
View File

@ -0,0 +1,71 @@
"""
Media asset fetcher using Pexels API
This module fetches B-Roll footage based on keywords extracted from:
- PDF content
- Video transcription
- Generated script
"""
import os
import requests
from typing import Dict, Any, List, Optional
class MediaFetcher:
"""Fetch B-Roll videos from Pexels API"""
def __init__(self, api_key: Optional[str] = None):
self.api_key = api_key or os.getenv("PEXELS_API_KEY")
if not self.api_key:
raise ValueError("PEXELS_API_KEY not found in environment")
self.base_url = "https://api.pexels.com/videos/search"
def fetch_broll(self, keywords: List[str], max_per_keyword: int = 3) -> Dict[str, List[str]]:
"""
Fetch B-Roll videos based on keywords
Args:
keywords: List of Japanese keywords (will be translated to English)
max_per_keyword: Maximum videos to fetch per keyword
Returns:
Dictionary mapping keywords to video URLs
"""
# TODO: Implement Pexels API integration
# 1. Translate Japanese keywords to English (using Gemini)
# 2. Search Pexels for each keyword
# 3. Download videos to assets/1_processed/broll/
# 4. Return local file paths
raise NotImplementedError("B-Roll fetching not yet implemented")
def _translate_keyword(self, japanese_keyword: str) -> str:
"""Translate Japanese keyword to English using Gemini"""
# TODO: Implement translation
# Use Gemini Pro for accurate translation
# Cache translations to avoid repeated API calls
return japanese_keyword
def _download_video(self, url: str, output_path: str) -> str:
"""Download video from URL"""
# TODO: Implement video download
# Use requests to download
# Show progress bar
# Validate video format
return output_path
def fetch_broll(script: Dict[str, Any]) -> Dict[str, Any]:
"""Convenience function for fetching B-Roll assets"""
# Extract keywords from script
keywords = []
for segment in script.get("segments", []):
# TODO: Extract keywords from segment text
pass
fetcher = MediaFetcher()
return fetcher.fetch_broll(keywords)

68
backend/scripter.py Normal file
View File

@ -0,0 +1,68 @@
"""
Script generation module using Google Gemini Pro
This module generates natural Japanese narration scripts based on:
- Extracted PDF data
- Video transcription
- Template constraints
"""
import os
from typing import Dict, Any, List, Optional
class ScriptGenerator:
"""Generate narration scripts using Gemini Pro"""
def __init__(self, api_key: Optional[str] = None):
self.api_key = api_key or os.getenv("GEMINI_API_KEY")
if not self.api_key:
raise ValueError("GEMINI_API_KEY not found in environment")
def generate(
self,
pdf_data: Dict[str, Any],
video_data: Optional[Dict[str, Any]] = None,
style: str = "friendly"
) -> Dict[str, Any]:
"""
Generate narration script
Args:
pdf_data: Structured data from PDF extraction
video_data: Optional video transcription data
style: Narration style ('friendly', 'professional', 'casual')
Returns:
Dictionary containing:
- segments: List of narration segments
- total_duration: Estimated total duration in seconds
- sync_points: Recommended chart/video sync points
"""
# TODO: Implement script generation
# 1. Build prompt with PDF data and constraints
# 2. Call Gemini Pro API
# 3. Parse and structure the response
# 4. Estimate timing for each segment
raise NotImplementedError("Script generation not yet implemented")
def _build_prompt(self, pdf_data: Dict, style: str) -> str:
"""Build the prompt for Gemini Pro"""
# TODO: Implement prompt engineering
# Should include:
# - PDF data summary
# - Style guidelines (natural Japanese, friendly tone)
# - Output format requirements (JSON with segments)
# - Length constraints (target 3-5 minutes)
return ""
def generate_script(
pdf_data: Dict[str, Any],
video_data: Optional[Dict[str, Any]] = None
) -> Dict[str, Any]:
"""Convenience function for script generation"""
generator = ScriptGenerator()
return generator.generate(pdf_data, video_data)

View File

@ -0,0 +1,59 @@
"""
Video Processing Module
Provides two strategies for analyzing video content:
1. Direct Analysis: Send video directly to Gemini 1.5 Pro
2. Audio-First: Extract audio → Whisper transcription → Text analysis
Usage:
from backend.video_processor import analyze_video
result = analyze_video(
video_path="input.mp4",
strategy="audio", # or "direct"
config={"language": "ja"}
)
"""
from .schemas import VideoAnalysisResult, VideoSegment, KeyMoment
from .utils import extract_audio, get_video_duration
__all__ = [
"analyze_video",
"VideoAnalysisResult",
"VideoSegment",
"KeyMoment",
"extract_audio",
"get_video_duration",
]
def analyze_video(
video_path: str,
strategy: str = "audio",
config: dict | None = None
) -> VideoAnalysisResult:
"""
Main entry point for video analysis.
Args:
video_path: Path to video file
strategy: "audio" or "direct"
config: Optional configuration dict
Returns:
VideoAnalysisResult object with segments and key moments
"""
if config is None:
config = {}
if strategy == "audio":
# Lazy import to avoid dependency issues when just checking deps
from .strategy_audio import AudioFirstStrategy
processor = AudioFirstStrategy(config)
return processor.analyze(video_path)
elif strategy == "direct":
# TODO: Implement in next sprint
raise NotImplementedError("Direct video analysis coming in next sprint")
else:
raise ValueError(f"Unknown strategy: {strategy}")

View File

@ -0,0 +1,90 @@
"""
Data schemas for video processing results.
Defines the standard output format that all strategies must produce.
This ensures consistency when integrating with the main pipeline.
"""
from dataclasses import dataclass, field
from typing import Literal
@dataclass
class VideoSegment:
"""Represents a meaningful segment in the video."""
start_time: float # seconds
end_time: float # seconds
description: str # What happens in this segment
transcript: str | None = None # Spoken text (if available)
importance: Literal["high", "medium", "low"] = "medium"
tags: list[str] = field(default_factory=list)
speaker: str | None = None # For multi-speaker videos
@property
def duration(self) -> float:
"""Duration of this segment in seconds."""
return self.end_time - self.start_time
def to_dict(self) -> dict:
"""Convert to JSON-serializable dict."""
return {
"startTime": self.start_time,
"endTime": self.end_time,
"description": self.description,
"transcript": self.transcript,
"importance": self.importance,
"tags": self.tags,
"speaker": self.speaker,
"duration": self.duration,
}
@dataclass
class KeyMoment:
"""Represents a single important moment (quote, visual, emotion)."""
timestamp: float # seconds
type: Literal["quote", "chart", "emotion", "action"]
content: str # Description or transcript
context: str | None = None # Additional context
def to_dict(self) -> dict:
"""Convert to JSON-serializable dict."""
return {
"timestamp": self.timestamp,
"type": self.type,
"content": self.content,
"context": self.context,
}
@dataclass
class VideoAnalysisResult:
"""Complete analysis result for a video."""
source_video: str # Original file path
duration: float # Total video duration in seconds
strategy: Literal["direct", "audio-first"]
segments: list[VideoSegment] = field(default_factory=list)
key_moments: list[KeyMoment] = field(default_factory=list)
metadata: dict = field(default_factory=dict) # Extra info (language, etc.)
def to_dict(self) -> dict:
"""Convert to JSON-serializable dict."""
return {
"sourceVideo": self.source_video,
"duration": self.duration,
"strategy": self.strategy,
"segments": [seg.to_dict() for seg in self.segments],
"keyMoments": [moment.to_dict() for moment in self.key_moments],
"metadata": self.metadata,
}
def get_high_importance_segments(self) -> list[VideoSegment]:
"""Filter segments marked as high importance."""
return [seg for seg in self.segments if seg.importance == "high"]
def get_quotes(self) -> list[KeyMoment]:
"""Get all key moments that are quotes."""
return [moment for moment in self.key_moments if moment.type == "quote"]

View File

@ -0,0 +1,186 @@
"""
Audio-First Strategy for Video Analysis
Workflow:
1. Extract audio from video (FFmpeg)
2. Transcribe audio with timestamps (OpenAI Whisper)
3. Analyze transcript to identify important segments (Gemini)
4. Generate VideoAnalysisResult with precise timestamps
Pros:
- Lower token cost (text vs video frames)
- Precise timestamps from Whisper
- Fast processing
Cons:
- Misses visual information (charts, expressions)
- Only suitable for dialogue-heavy videos
"""
import os
from pathlib import Path
from typing import Any
from openai import OpenAI
from .schemas import VideoAnalysisResult, VideoSegment, KeyMoment
from .utils import extract_audio, get_video_duration
class AudioFirstStrategy:
"""Implements audio-first video analysis strategy."""
def __init__(self, config: dict[str, Any] | None = None):
"""
Initialize strategy with configuration.
Args:
config: Optional configuration dict
- language: Language code (default: "ja")
- model: Whisper model (default: "whisper-1")
- prompt: Optional prompt to guide transcription
"""
self.config = config or {}
self.language = self.config.get("language", "ja")
self.model = self.config.get("model", "whisper-1")
self.prompt = self.config.get("prompt", None)
# Initialize OpenAI client
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
raise ValueError("OPENAI_API_KEY not found in environment")
self.client = OpenAI(api_key=api_key)
def analyze(self, video_path: str) -> VideoAnalysisResult:
"""
Analyze video using audio-first strategy.
Args:
video_path: Path to video file
Returns:
VideoAnalysisResult with segments and key moments
"""
# Step 1: Extract audio
print(f"📹 Extracting audio from {video_path}...")
audio_path = extract_audio(video_path)
# Step 2: Transcribe with Whisper
print("🎤 Transcribing audio with Whisper...")
transcript_data = self._transcribe_audio(audio_path)
# Step 3: Convert to segments
print("📝 Processing transcript segments...")
segments = self._create_segments(transcript_data)
# Step 4: Identify key moments
print("⭐ Identifying key moments...")
key_moments = self._identify_key_moments(transcript_data)
# Step 5: Get video duration
duration = get_video_duration(video_path)
# Clean up temporary audio file
if os.path.exists(audio_path):
os.remove(audio_path)
return VideoAnalysisResult(
source_video=video_path,
duration=duration,
strategy="audio-first",
segments=segments,
key_moments=key_moments,
metadata={
"language": self.language,
"model": self.model,
"total_segments": len(segments),
"total_key_moments": len(key_moments),
}
)
def _transcribe_audio(self, audio_path: str) -> dict:
"""
Transcribe audio using OpenAI Whisper API.
Args:
audio_path: Path to audio file
Returns:
Whisper API response with segments and timestamps
"""
with open(audio_path, "rb") as audio_file:
# Use verbose_json response format to get word-level timestamps
transcript = self.client.audio.transcriptions.create(
model=self.model,
file=audio_file,
language=self.language,
response_format="verbose_json",
timestamp_granularities=["segment"]
)
# Convert to dict for easier processing
return transcript.model_dump()
def _create_segments(self, transcript_data: dict) -> list[VideoSegment]:
"""
Convert Whisper segments to VideoSegment objects.
Args:
transcript_data: Whisper API response
Returns:
List of VideoSegment objects
"""
segments = []
for seg in transcript_data.get("segments", []):
segment = VideoSegment(
start_time=seg["start"],
end_time=seg["end"],
description=f"Dialogue segment {seg['id'] + 1}",
transcript=seg["text"].strip(),
importance="medium", # Will be updated by analysis
tags=["dialogue"],
)
segments.append(segment)
return segments
def _identify_key_moments(self, transcript_data: dict) -> list[KeyMoment]:
"""
Identify key moments from transcript.
For now, this is a simple heuristic. In the future, we can use
Gemini to analyze the full transcript and identify:
- Important quotes
- Topic transitions
- Emotional moments
Args:
transcript_data: Whisper API response
Returns:
List of KeyMoment objects
"""
key_moments = []
# Simple heuristic: Mark first segment as key moment
segments = transcript_data.get("segments", [])
if segments:
first_seg = segments[0]
key_moments.append(
KeyMoment(
timestamp=first_seg["start"],
type="quote",
content=first_seg["text"].strip(),
context="Opening statement"
)
)
# TODO: Add Gemini-based analysis here
# - Analyze full transcript
# - Identify topic changes
# - Extract important quotes
# - Mark emotional moments
return key_moments

View File

@ -0,0 +1,57 @@
"""
Direct Video Analysis Strategy (Placeholder)
Workflow:
1. Split long video into 5-minute segments (FFmpeg)
2. Upload each segment to Gemini 1.5 Pro
3. Analyze with vision + audio understanding
4. Merge results and normalize timestamps
Pros:
- Captures visual information (charts, expressions, gestures)
- Holistic understanding (audio + visual)
Cons:
- Higher token cost (video frames)
- Slower processing
- API upload limits
Status: TODO - Implement in next sprint
"""
from typing import Any
from .schemas import VideoAnalysisResult
class DirectVideoStrategy:
"""Implements direct video analysis strategy using Gemini 1.5 Pro."""
def __init__(self, config: dict[str, Any] | None = None):
"""
Initialize strategy with configuration.
Args:
config: Optional configuration dict
- max_segment_duration: Max seconds per segment (default: 300)
- language: Language code (default: "ja")
"""
self.config = config or {}
self.max_segment_duration = self.config.get("max_segment_duration", 300)
self.language = self.config.get("language", "ja")
def analyze(self, video_path: str) -> VideoAnalysisResult:
"""
Analyze video using direct upload strategy.
Args:
video_path: Path to video file
Returns:
VideoAnalysisResult with segments and key moments
"""
# TODO: Implement in next sprint
raise NotImplementedError(
"Direct video analysis will be implemented in next sprint. "
"Use strategy='audio' for now."
)

View File

@ -0,0 +1,163 @@
"""
Utility functions for video processing.
Provides FFmpeg wrappers and common operations used by all strategies.
"""
import json
import os
import subprocess
from pathlib import Path
def get_video_duration(video_path: str) -> float:
"""
Get video duration in seconds using ffprobe.
Args:
video_path: Path to video file
Returns:
Duration in seconds
Raises:
RuntimeError: If ffprobe fails or video is invalid
"""
cmd = [
"ffprobe",
"-v", "error",
"-show_entries", "format=duration",
"-of", "json",
video_path
]
try:
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=True
)
data = json.loads(result.stdout)
return float(data["format"]["duration"])
except subprocess.CalledProcessError as e:
raise RuntimeError(f"Failed to get video duration: {e.stderr}")
except (KeyError, ValueError) as e:
raise RuntimeError(f"Invalid ffprobe output: {e}")
def extract_audio(
video_path: str,
output_path: str | None = None,
sample_rate: int = 16000,
codec: str = "pcm_s16le"
) -> str:
"""
Extract audio from video file using FFmpeg.
Args:
video_path: Path to input video
output_path: Path for output audio (default: same name with .wav)
sample_rate: Audio sample rate in Hz (16000 recommended for Whisper)
codec: Audio codec (pcm_s16le for WAV)
Returns:
Path to extracted audio file
Raises:
RuntimeError: If FFmpeg fails
"""
if output_path is None:
video_name = Path(video_path).stem
output_path = str(Path(video_path).parent / f"{video_name}_audio.wav")
cmd = [
"ffmpeg",
"-i", video_path,
"-vn", # No video
"-acodec", codec,
"-ar", str(sample_rate),
"-ac", "1", # Mono
"-y", # Overwrite output file
output_path
]
try:
subprocess.run(
cmd,
capture_output=True,
text=True,
check=True
)
return output_path
except subprocess.CalledProcessError as e:
raise RuntimeError(f"Failed to extract audio: {e.stderr}")
def slice_video(
video_path: str,
start_time: float,
end_time: float,
output_path: str | None = None
) -> str:
"""
Slice a video segment using FFmpeg stream copy (fast, no re-encoding).
Args:
video_path: Path to input video
start_time: Start time in seconds
end_time: End time in seconds
output_path: Path for output segment
Returns:
Path to output segment
Raises:
RuntimeError: If FFmpeg fails
"""
if output_path is None:
video_name = Path(video_path).stem
output_path = str(
Path(video_path).parent / f"{video_name}_{start_time:.1f}-{end_time:.1f}.mp4"
)
duration = end_time - start_time
cmd = [
"ffmpeg",
"-ss", str(start_time),
"-i", video_path,
"-t", str(duration),
"-c", "copy", # Stream copy (no re-encoding)
"-y",
output_path
]
try:
subprocess.run(
cmd,
capture_output=True,
text=True,
check=True
)
return output_path
except subprocess.CalledProcessError as e:
raise RuntimeError(f"Failed to slice video: {e.stderr}")
def check_ffmpeg_installed() -> bool:
"""Check if FFmpeg and FFprobe are available in PATH."""
try:
subprocess.run(
["ffmpeg", "-version"],
capture_output=True,
check=True
)
subprocess.run(
["ffprobe", "-version"],
capture_output=True,
check=True
)
return True
except (subprocess.CalledProcessError, FileNotFoundError):
return False

View File

@ -1,3 +1,78 @@
import { config } from "@remotion/eslint-config-flat"; import { config } from "@remotion/eslint-config-flat";
import importPlugin from "eslint-plugin-import";
import jsxA11y from "eslint-plugin-jsx-a11y";
export default config; export default [
...config,
{
files: ["**/*.{ts,tsx}"],
plugins: {
import: importPlugin,
"jsx-a11y": jsxA11y,
},
settings: {
"import/resolver": {
typescript: true,
node: true,
},
},
rules: {
// Enhanced TypeScript rules
"@typescript-eslint/no-unused-vars": [
"warn",
{
argsIgnorePattern: "^_",
varsIgnorePattern: "^_",
},
],
"@typescript-eslint/no-explicit-any": "warn",
"@typescript-eslint/consistent-type-imports": [
"error",
{
prefer: "type-imports",
},
],
// Import organization for better code structure
"import/order": [
"error",
{
groups: [
"builtin",
"external",
"internal",
"parent",
"sibling",
"index",
],
"newlines-between": "always",
alphabetize: {
order: "asc",
caseInsensitive: true,
},
},
],
"import/no-duplicates": "error",
// Accessibility rules
"jsx-a11y/alt-text": "warn",
"jsx-a11y/aria-props": "error",
"jsx-a11y/aria-role": "error",
// Code quality rules
"no-console": ["warn", { allow: ["warn", "error"] }],
"prefer-const": "error",
"no-var": "error",
eqeqeq: ["error", "always"],
},
},
{
ignores: [
"node_modules/**",
"dist/**",
"build/**",
"out/**",
".remotion/**",
],
},
];

View File

@ -0,0 +1,250 @@
/**
* Type definitions for render_props.json
*
* This file defines the TypeScript interfaces that match the Python backend's
* render_props.json structure. It serves as the data contract between:
* - Python backend (data generator)
* - Remotion frontend (video renderer)
*
* CRITICAL: Keep this file in sync with backend/data_builder.py
*/
import { z } from "zod";
// ============================================================================
// Core Data Types (matches existing types.ts)
// ============================================================================
export interface YearlySnapshot {
year: number;
ages: {
rep: number;
spouse?: number;
};
financials: {
income: number;
expenditure: number;
assetBalance: number;
};
events?: string[];
isCritical?: boolean;
}
export interface CashFlowData {
year: number;
age: number;
income: {
total: number;
};
expenses: {
living: number;
housing: number;
education: number;
tax: number;
event: number;
};
totalExpense?: number;
netCashFlow?: number;
assetBalance?: number;
}
// ============================================================================
// Chart Data Structures
// ============================================================================
export interface AssetDepletionChartData {
timeline: YearlySnapshot[];
meta: {
currencyUnit: string;
startYear: number;
};
}
export interface CashFlowChartData {
timeline: CashFlowData[];
meta: {
currencyUnit: string;
};
}
// ============================================================================
// Script & Narration Types
// ============================================================================
export type ScriptSegmentType =
| "narration" // Pure narration with B-Roll
| "chart_intro" // Introduction to chart
| "chart_highlight" // Highlighting specific data point
| "video_clip" // Original interview footage
| "transition"; // Transition between scenes
export interface ScriptSegment {
id: number;
text: string; // Japanese narration text
duration: number; // Duration in seconds
type: ScriptSegmentType;
assetUrl?: string; // URL or path to B-Roll/video clip
chartSync?: {
// Optional chart synchronization
chartId: "assetDepletion" | "cashFlow";
highlightYear?: number; // Year to highlight
animationStart?: number; // Start frame for animation
};
}
// ============================================================================
// Asset References
// ============================================================================
export interface VideoClipReference {
id: string;
sourcePath: string; // Path to source video
startTime: number; // Start timestamp in source (seconds)
duration: number; // Duration of clip (seconds)
outputPath?: string; // Path to extracted clip
}
export interface AssetCollection {
broll: Record<string, string>; // keyword -> video URL/path mapping
videoClips: VideoClipReference[];
ttsAudio?: string; // Path to generated TTS audio (if used)
}
// ============================================================================
// Main Render Props Structure
// ============================================================================
export interface RenderProps {
meta: {
title: string; // Video title (e.g., "山田太郎様の人生設計")
currencyUnit: string; // Currency unit for all financial data
duration: number; // Total video duration in seconds
clientName?: string; // Client name for personalization
generatedAt?: string; // ISO timestamp of generation
};
charts: {
assetDepletion: AssetDepletionChartData;
cashFlow: CashFlowChartData;
};
script: ScriptSegment[];
assets: AssetCollection;
}
// ============================================================================
// Zod Schemas for Runtime Validation
// ============================================================================
export const YearlySnapshotSchema = z.object({
year: z.number(),
ages: z.object({
rep: z.number(),
spouse: z.number().optional(),
}),
financials: z.object({
income: z.number(),
expenditure: z.number(),
assetBalance: z.number(),
}),
events: z.array(z.string()).optional(),
isCritical: z.boolean().optional(),
});
export const CashFlowDataSchema = z.object({
year: z.number(),
age: z.number(),
income: z.object({
total: z.number(),
}),
expenses: z.object({
living: z.number(),
housing: z.number(),
education: z.number(),
tax: z.number(),
event: z.number(),
}),
totalExpense: z.number().optional(),
netCashFlow: z.number().optional(),
assetBalance: z.number().optional(),
});
export const ScriptSegmentSchema = z.object({
id: z.number(),
text: z.string(),
duration: z.number(),
type: z.enum([
"narration",
"chart_intro",
"chart_highlight",
"video_clip",
"transition",
]),
assetUrl: z.string().optional(),
chartSync: z
.object({
chartId: z.enum(["assetDepletion", "cashFlow"]),
highlightYear: z.number().optional(),
animationStart: z.number().optional(),
})
.optional(),
});
export const RenderPropsSchema = z.object({
meta: z.object({
title: z.string(),
currencyUnit: z.string(),
duration: z.number(),
clientName: z.string().optional(),
generatedAt: z.string().optional(),
}),
charts: z.object({
assetDepletion: z.object({
timeline: z.array(YearlySnapshotSchema),
meta: z.object({
currencyUnit: z.string(),
startYear: z.number(),
}),
}),
cashFlow: z.object({
timeline: z.array(CashFlowDataSchema),
meta: z.object({
currencyUnit: z.string(),
}),
}),
}),
script: z.array(ScriptSegmentSchema),
assets: z.object({
broll: z.record(z.string()),
videoClips: z.array(
z.object({
id: z.string(),
sourcePath: z.string(),
startTime: z.number(),
duration: z.number(),
outputPath: z.string().optional(),
}),
),
ttsAudio: z.string().optional(),
}),
});
// ============================================================================
// Helper Functions
// ============================================================================
/**
* Validate render props at runtime
* Throws if validation fails
*/
export function validateRenderProps(props: unknown): RenderProps {
return RenderPropsSchema.parse(props);
}
/**
* Type guard for RenderProps
*/
export function isRenderProps(props: unknown): props is RenderProps {
return RenderPropsSchema.safeParse(props).success;
}

304
main.py Normal file
View File

@ -0,0 +1,304 @@
#!/usr/bin/env python3
"""
Auto Life Plan Video Generator - Main Entry Point
This script orchestrates the entire video generation pipeline:
1. Parse PDF/Video inputs (Gemini Pro / Whisper)
2. Generate script (Gemini Pro)
3. Fetch B-Roll assets (Pexels API)
4. Build chart data (Simulation engine)
5. Generate render_props.json
6. Trigger Remotion render
Usage:
python main.py --pdf path/to/report.pdf
python main.py --pdf report.pdf --video interview.mp4
python main.py --config config.json
"""
import argparse
import json
import subprocess
import sys
from pathlib import Path
from typing import Dict, Any
# Import backend modules (to be implemented)
# from backend.extractor import extract_pdf_data, extract_video_data
# from backend.scripter import generate_script
# from backend.media_fetcher import fetch_broll
# from backend.data_builder import build_chart_data
class VideoGeneratorPipeline:
"""Main orchestrator for video generation"""
def __init__(self, config: Dict[str, Any]):
self.config = config
self.project_root = Path(__file__).parent
self.assets_dir = self.project_root / "assets"
self.input_dir = self.assets_dir / "0_input"
self.processed_dir = self.assets_dir / "1_processed"
self.output_dir = self.assets_dir / "2_output"
# Ensure directories exist
for dir_path in [self.input_dir, self.processed_dir, self.output_dir]:
dir_path.mkdir(parents=True, exist_ok=True)
def run(self) -> str:
"""Execute the full pipeline and return output video path"""
print("🚀 Starting Auto Life Plan Video Generation Pipeline...")
# Step 1: Extract data from PDF
print("\n📄 Step 1/6: Extracting data from PDF...")
pdf_data = self.extract_pdf()
# Step 2: Extract data from video (if provided)
print("\n🎥 Step 2/6: Processing video footage...")
video_data = self.extract_video()
# Step 3: Generate narration script
print("\n✍️ Step 3/6: Generating narration script...")
script = self.generate_script(pdf_data, video_data)
# Step 4: Fetch B-Roll assets
print("\n🎬 Step 4/6: Fetching B-Roll assets...")
broll_assets = self.fetch_broll(script)
# Step 5: Build render props
print("\n📊 Step 5/6: Building render props...")
render_props = self.build_render_props(pdf_data, script, broll_assets)
# Save render_props.json
props_file = self.processed_dir / "render_props.json"
with open(props_file, "w", encoding="utf-8") as f:
json.dump(render_props, f, ensure_ascii=False, indent=2)
print(f" ✓ Saved render props to {props_file}")
# Step 6: Render video with Remotion
print("\n🎞️ Step 6/6: Rendering video with Remotion...")
output_path = self.render_video(props_file)
print(f"\n✅ Video generation complete! Output: {output_path}")
return str(output_path)
def extract_pdf(self) -> Dict[str, Any]:
"""Extract data from PDF using Gemini Pro"""
# TODO: Implement backend.extractor.extract_pdf_data
print(" [MOCK] Extracting PDF data...")
return {
"client_name": "山田太郎",
"current_age": 35,
"retirement_age": 65,
"life_expectancy": 100,
"current_assets": 5000,
"annual_income": 600,
"annual_expenses": 450,
}
def extract_video(self) -> Dict[str, Any]:
"""Extract data from video using Whisper (if video provided)"""
if not self.config.get("video_path"):
print(" [SKIP] No video file provided")
return {}
# TODO: Implement backend.extractor.extract_video_data
print(" [MOCK] Extracting video timestamps and transcription...")
return {
"transcription": "こんにちは...",
"timestamps": [
{"time": 0.5, "text": "Introduction", "keywords": ["保険", "将来"]},
{"time": 15.2, "text": "Main topic", "keywords": ["資産", "計画"]},
],
}
def generate_script(self, pdf_data: Dict, video_data: Dict) -> Dict[str, Any]:
"""Generate narration script using Gemini Pro"""
# TODO: Implement backend.scripter.generate_script
print(" [MOCK] Generating script with Gemini Pro...")
return {
"segments": [
{
"id": 1,
"text": "これからあなたの将来設計について説明します。",
"duration": 3.5,
"type": "narration",
},
{
"id": 2,
"text": "現在の資産状況を見てみましょう。",
"duration": 2.8,
"type": "chart_intro",
},
]
}
def fetch_broll(self, script: Dict) -> Dict[str, Any]:
"""Fetch B-Roll footage from Pexels API"""
# TODO: Implement backend.media_fetcher.fetch_broll
print(" [MOCK] Fetching B-Roll from Pexels...")
return {
"retirement": "https://example.com/retirement.mp4",
"family": "https://example.com/family.mp4",
}
def build_render_props(
self, pdf_data: Dict, script: Dict, broll: Dict
) -> Dict[str, Any]:
"""Build the complete render_props.json structure"""
# TODO: Implement backend.data_builder.build_chart_data
print(" [MOCK] Building chart data and render props...")
return {
"meta": {
"title": f"{pdf_data.get('client_name', 'お客様')}様の人生設計",
"currencyUnit": "万円",
"duration": 180, # 3 minutes
},
"charts": {
"assetDepletion": {
"timeline": [
{
"year": 2025,
"ages": {"rep": 35, "spouse": 33},
"financials": {
"income": 600,
"expenditure": 450,
"assetBalance": 5000,
},
},
# ... more years
],
"meta": {"currencyUnit": "万円", "startYear": 2025},
},
"cashFlow": {
"timeline": [
{
"year": 2025,
"age": 35,
"income": {"total": 600},
"expenses": {
"living": 200,
"housing": 150,
"education": 50,
"tax": 50,
"event": 0,
},
},
# ... more years
],
"meta": {"currencyUnit": "万円"},
},
},
"script": script["segments"],
"assets": {
"broll": broll,
"videoClips": [],
},
}
def render_video(self, props_file: Path) -> Path:
"""Render video using Remotion"""
output_file = self.output_dir / "life_plan_video.mp4"
# Remotion render command
cmd = [
"npx",
"remotion",
"render",
"frontend/src/index.ts",
"MainVideo",
str(output_file),
f"--props={props_file}",
]
print(f" Running: {' '.join(cmd)}")
# TODO: Uncomment when frontend is ready
# try:
# subprocess.run(cmd, check=True, cwd=self.project_root)
# except subprocess.CalledProcessError as e:
# print(f" ❌ Remotion render failed: {e}")
# sys.exit(1)
print(" [MOCK] Remotion render would execute here")
return output_file
def main():
"""CLI entry point"""
parser = argparse.ArgumentParser(
description="Auto Life Plan Video Generator",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python main.py --pdf assets/0_input/report.pdf
python main.py --pdf report.pdf --video interview.mp4
python main.py --config config.json --output custom_output.mp4
""",
)
parser.add_argument(
"--pdf",
type=str,
help="Path to the Life Plan PDF report",
)
parser.add_argument(
"--video",
type=str,
help="Path to the interview video (optional)",
)
parser.add_argument(
"--config",
type=str,
help="Path to configuration JSON file",
)
parser.add_argument(
"--output",
type=str,
help="Custom output filename (default: life_plan_video.mp4)",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Generate render_props.json without rendering",
)
args = parser.parse_args()
# Build configuration
config = {}
if args.config:
with open(args.config, "r", encoding="utf-8") as f:
config = json.load(f)
else:
if not args.pdf:
parser.error("--pdf is required when --config is not provided")
config["pdf_path"] = args.pdf
config["video_path"] = args.video
if args.output:
config["output_filename"] = args.output
config["dry_run"] = args.dry_run
# Run pipeline
pipeline = VideoGeneratorPipeline(config)
try:
output_path = pipeline.run()
print(f"\n🎉 Success! Video saved to: {output_path}")
except Exception as e:
print(f"\n❌ Pipeline failed: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()

8270
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@ -9,13 +9,23 @@
"@remotion/cli": "4.0.400", "@remotion/cli": "4.0.400",
"react": "19.2.3", "react": "19.2.3",
"react-dom": "19.2.3", "react-dom": "19.2.3",
"recharts": "^3.6.0",
"remotion": "4.0.400" "remotion": "4.0.400"
}, },
"devDependencies": { "devDependencies": {
"@remotion/eslint-config-flat": "4.0.400", "@remotion/eslint-config-flat": "4.0.400",
"@types/react": "19.2.7", "@types/react": "19.2.7",
"@types/web": "0.0.166", "@types/web": "0.0.166",
"@typescript-eslint/eslint-plugin": "^8.52.0",
"@typescript-eslint/parser": "^8.52.0",
"eslint": "9.19.0", "eslint": "9.19.0",
"eslint-config-prettier": "^10.1.8",
"eslint-import-resolver-typescript": "^4.4.4",
"eslint-plugin-import": "^2.32.0",
"eslint-plugin-jsx-a11y": "^6.10.2",
"eslint-plugin-prettier": "^5.5.4",
"eslint-plugin-react": "^7.37.5",
"eslint-plugin-react-hooks": "^7.0.1",
"prettier": "3.6.0", "prettier": "3.6.0",
"typescript": "5.9.3" "typescript": "5.9.3"
}, },
@ -23,6 +33,10 @@
"dev": "remotion studio", "dev": "remotion studio",
"build": "remotion bundle", "build": "remotion bundle",
"upgrade": "remotion upgrade", "upgrade": "remotion upgrade",
"lint": "eslint src && tsc" "lint": "eslint src && tsc",
"lint:fix": "eslint src --fix && prettier --write \"src/**/*.{ts,tsx,json}\"",
"format": "prettier --write \"src/**/*.{ts,tsx,json}\"",
"format:check": "prettier --check \"src/**/*.{ts,tsx,json}\"",
"type-check": "tsc --noEmit"
} }
} }

33
requirements.txt Normal file
View File

@ -0,0 +1,33 @@
# Auto Life Plan Video Generator - Python Dependencies
# Core
python-dotenv>=1.0.0
# AI/LLM - Install these separately when needed
# Check official docs for correct versions:
# - google-generativeai: https://pypi.org/project/google-generativeai/
# - openai: https://pypi.org/project/openai/
#
# To install manually:
# pip install google-generativeai (installs latest stable)
# pip install openai
# Media Processing
ffmpeg-python>=0.2.0
requests>=2.31.0
# Data Processing (optional - only needed for advanced simulations)
# numpy>=1.24.0
# pandas>=2.0.0
# PDF Processing (optional - Gemini API can handle PDFs directly)
# PyPDF2>=3.0.0
# pdfplumber>=0.10.0
# Progress bars and CLI
tqdm>=4.66.0
rich>=13.0.0
# Testing (optional)
# pytest>=7.4.0
# pytest-cov>=4.1.0

440
src/CashFlowChart.tsx Normal file
View File

@ -0,0 +1,440 @@
import React from "react";
import {
ComposedChart,
Bar,
Line,
XAxis,
YAxis,
CartesianGrid,
ResponsiveContainer,
Legend,
Tooltip,
} from "recharts";
import { useCurrentFrame, useVideoConfig, interpolate } from "remotion";
// --- Types (You can move this to types.ts later) ---
export interface CashFlowData {
year: number;
age: number;
income: {
total: number;
};
expenses: {
living: number;
housing: number;
education: number;
tax: number;
event: number;
};
totalExpense?: number;
netCashFlow?: number;
assetBalance?: number;
}
interface CashFlowChartProps {
timeline?: CashFlowData[];
meta?: {
currencyUnit: string;
};
hideHeader?: boolean;
externalDataIndex?: number;
externalYearProgress?: number;
}
// --- Custom Axis Tick (Year + Age) ---
const CustomXAxisTick = ({ x, y, payload, timelineData }: any) => {
const index = payload.index;
const dataPoint = timelineData[index];
if (!dataPoint) return null;
return (
<g transform={`translate(${x},${y})`}>
<text
x={0}
y={0}
dy={16}
textAnchor="middle"
fill="#666"
fontSize={12}
fontWeight="bold"
>
{dataPoint.year}
</text>
<text x={0} y={0} dy={32} textAnchor="middle" fill="#999" fontSize={10}>
{dataPoint.age}歳
</text>
</g>
);
};
// Stable configuration objects to prevent re-renders
// CRITICAL: Must match LifePlanChart margins for proper alignment in DualChartView
const CHART_MARGIN = { top: 10, right: 30, left: 0, bottom: 50 };
export const CashFlowChart: React.FC<CashFlowChartProps> = React.memo(
({
timeline = [],
meta = { currencyUnit: "万円" },
hideHeader = false,
externalDataIndex,
externalYearProgress,
}) => {
const frame = useCurrentFrame();
const { durationInFrames } = useVideoConfig();
// --- Animation Logic ---
const exactProgress = interpolate(
frame,
[0, durationInFrames - 30],
[0, timeline.length - 1],
{ extrapolateRight: "clamp" },
);
const dataEndIndex =
externalDataIndex !== undefined
? externalDataIndex
: Math.floor(exactProgress);
const yearProgress =
externalYearProgress !== undefined
? externalYearProgress
: exactProgress - dataEndIndex;
// Sequential growth animation for stacked bars
// Each category grows one after another
const getMultiplier = (segment: number) => {
const segmentSize = 1 / 6; // 6 segments: 5 expenses + 1 income line
const segmentStart = segment * segmentSize;
const segmentEnd = (segment + 1) * segmentSize;
if (yearProgress < segmentStart) return 0;
if (yearProgress > segmentEnd) return 1;
return (yearProgress - segmentStart) / segmentSize;
};
// Flatten data for Recharts (Recharts needs flat keys for Bars)
const flattenedData = timeline.map((item) => ({
...item,
incomeTotal: item.income.total as number | null,
expLiving: item.expenses.living,
expHousing: item.expenses.housing,
expEducation: item.expenses.education,
expTax: item.expenses.tax,
expEvent: item.expenses.event,
// Calculate total expense for HUD
totalExpense:
item.expenses.living +
item.expenses.housing +
item.expenses.education +
item.expenses.tax +
item.expenses.event,
}));
let visibleData = flattenedData
.slice(0, dataEndIndex + 1)
.map((item, idx) => {
const isCurrentBar = idx === dataEndIndex;
if (!isCurrentBar) {
// Previous bars are fully visible
return item;
}
// Current bar: apply sequential growth animation
// Income line stays at full value (not animated) to show target
return {
...item,
expLiving: item.expLiving * getMultiplier(0),
expHousing: item.expHousing * getMultiplier(1),
expEducation: item.expEducation * getMultiplier(2),
expEvent: item.expEvent * getMultiplier(3),
expTax: item.expTax * getMultiplier(4),
// incomeTotal keeps original value - no animation
};
});
// Add interpolated point for smooth line growth
// 在当前年动画期间,向下一年方向延伸红线
if (dataEndIndex >= 0 && dataEndIndex < flattenedData.length - 1) {
const current = flattenedData[dataEndIndex];
// 根据年内进度计算延伸位置(X轴)
const extensionYear = current.year + yearProgress;
const interpolatedPoint = {
year: extensionYear,
age: current.age,
income: current.income,
expenses: current.expenses,
incomeTotal: current.incomeTotal, // Y值保持当前年不变
expLiving: 0,
expHousing: 0,
expEducation: 0,
expTax: 0,
expEvent: 0,
totalExpense: current.totalExpense,
};
visibleData = [...visibleData, interpolatedPoint];
}
// When in fixed domain mode (< 10 points), pad with empty years to prevent offset
if (visibleData.length < 10 && visibleData.length > 0) {
const startYear = timeline[0].year;
const endYear = startYear + 9;
const paddedData = [];
for (let year = startYear; year <= endYear; year++) {
const existingData = visibleData.find((d) => d.year === year);
if (existingData) {
paddedData.push(existingData);
} else {
// Add placeholder for missing years
paddedData.push({
year,
age: 0,
income: { total: 0 },
expenses: { living: 0, housing: 0, education: 0, tax: 0, event: 0 },
incomeTotal: null, // null to break line (not render)
expLiving: 0,
expHousing: 0,
expEducation: 0,
expTax: 0,
expEvent: 0,
totalExpense: 0,
isEmpty: true,
});
}
}
visibleData = paddedData;
}
// For HUD: use original data (not animated values)
const currentData = flattenedData[dataEndIndex] || flattenedData[0];
// Memoize xAxis domain to prevent re-creating array every frame
const xAxisDomain = React.useMemo(
() =>
visibleData.length < 10
? [timeline[0].year, timeline[0].year + 9]
: undefined,
[visibleData.length, timeline],
);
const xAxisType = visibleData.length < 10 ? "number" : "category";
// Calculate Y-axis domain based on full timeline (not visibleData)
// This prevents flickering when data range changes dramatically (e.g., retirement)
const yAxisDomain = React.useMemo(() => {
const allIncomes = timeline.map((d) => d.income.total);
const allExpenses = timeline.map((d) => {
const item = flattenedData.find((f) => f.year === d.year);
return item?.totalExpense || 0;
});
const maxValue = Math.max(...allIncomes, ...allExpenses);
return [0, Math.ceil(maxValue / 100) * 100]; // Round up to nearest 100
}, [timeline, flattenedData]);
return (
<div
style={{
width: "100%",
height: "100%",
backgroundColor: "white",
padding: hideHeader ? "10px" : "40px",
fontFamily: "Noto Sans JP",
display: "flex",
flexDirection: "column",
}}
>
{/* --- HUD (Dashboard) --- */}
{!hideHeader && (
<div
style={{
display: "flex",
justifyContent: "space-between",
alignItems: "center",
padding: "20px 30px",
background: "#F7FAFC",
borderRadius: "12px",
marginBottom: "20px",
}}
>
<div>
<div style={{ fontSize: "14px", color: "#718096" }}>
CURRENT YEAR
</div>
<div
style={{
fontSize: "36px",
fontWeight: "800",
color: "#2D3748",
}}
>
{currentData.year}{" "}
<span style={{ fontSize: "20px", fontWeight: "normal" }}>
({currentData.age}歳)
</span>
</div>
</div>
<div style={{ display: "flex", gap: "40px" }}>
<div style={{ textAlign: "right" }}>
<div
style={{
fontSize: "12px",
color: "#718096",
fontWeight: "bold",
}}
>
TOTAL INCOME
</div>
<div
style={{
fontSize: "28px",
fontWeight: "bold",
color: "#E53E3E",
}}
>
{currentData.incomeTotal}{" "}
<span style={{ fontSize: "16px" }}>{meta.currencyUnit}</span>
</div>
</div>
<div style={{ textAlign: "right" }}>
<div
style={{
fontSize: "12px",
color: "#718096",
fontWeight: "bold",
}}
>
TOTAL EXPENSE
</div>
<div
style={{
fontSize: "28px",
fontWeight: "bold",
color: "#4A5568",
}}
>
{currentData.totalExpense}{" "}
<span style={{ fontSize: "16px" }}>{meta.currencyUnit}</span>
</div>
</div>
<div style={{ textAlign: "right" }}>
<div
style={{
fontSize: "12px",
color: "#718096",
fontWeight: "bold",
}}
>
BALANCE
</div>
<div
style={{
fontSize: "28px",
fontWeight: "bold",
color:
currentData.incomeTotal !== null &&
currentData.incomeTotal - currentData.totalExpense >= 0
? "#38A169"
: "#E53E3E",
}}
>
{currentData.incomeTotal !== null &&
currentData.incomeTotal - currentData.totalExpense > 0
? "+"
: ""}
{currentData.incomeTotal !== null
? currentData.incomeTotal - currentData.totalExpense
: "-"}
</div>
</div>
</div>
</div>
)}
{/* --- Chart --- */}
<div style={{ flex: 1, minHeight: 0, width: "100%" }}>
<ResponsiveContainer width="100%" height="100%">
<ComposedChart
data={visibleData}
margin={CHART_MARGIN}
barCategoryGap={0}
>
<CartesianGrid strokeDasharray="3 3" vertical={false} />
<XAxis
dataKey="year"
tick={<CustomXAxisTick timelineData={timeline} />}
interval={4}
tickLine={false}
height={60}
domain={xAxisDomain}
type={xAxisType}
/>
<YAxis unit={meta.currencyUnit} width={80} domain={yAxisDomain} />
<Tooltip />
<Legend verticalAlign="top" height={36} />
{/* --- Stacked Bars (Expenses) --- */}
{/* Note: stackId="a" makes them stack on top of each other */}
<Bar
name="Living"
dataKey="expLiving"
stackId="a"
fill="#ED8936"
isAnimationActive={false}
/>
<Bar
name="Housing"
dataKey="expHousing"
stackId="a"
fill="#4299E1"
isAnimationActive={false}
/>
<Bar
name="Education"
dataKey="expEducation"
stackId="a"
fill="#48BB78"
isAnimationActive={false}
/>
<Bar
name="Event"
dataKey="expEvent"
stackId="a"
fill="#ED64A6"
isAnimationActive={false}
/>
<Bar
name="Tax/Social"
dataKey="expTax"
stackId="a"
fill="#A0AEC0"
isAnimationActive={false}
/>
{/* --- Line (Income) --- */}
<Line
type="monotone"
name="Income"
dataKey="incomeTotal"
stroke="#E53E3E"
strokeWidth={4}
dot={false}
connectNulls={false}
isAnimationActive={false}
/>
</ComposedChart>
</ResponsiveContainer>
</div>
</div>
);
},
);
CashFlowChart.displayName = "CashFlowChart";

175
src/DualChartView.tsx Normal file
View File

@ -0,0 +1,175 @@
import React, { useMemo } from "react";
import { useCurrentFrame, useVideoConfig, interpolate } from "remotion";
import type { CashFlowData } from "./CashFlowChart";
import { CashFlowChart } from "./CashFlowChart";
import { LifePlanChart } from "./LifePlanChart";
import type { YearlySnapshot } from "./types";
import { UnifiedHUD } from "./UnifiedHUD";
interface DualChartViewProps {
cashflowData?: {
timeline: Array<
CashFlowData & {
totalExpense: number;
netCashFlow: number;
assetBalance: number;
}
>;
meta: {
currencyUnit: string;
};
};
}
export const DualChartView: React.FC<DualChartViewProps> = ({
cashflowData = { timeline: [], meta: { currencyUnit: "万円" } },
}) => {
const frame = useCurrentFrame();
const { durationInFrames } = useVideoConfig();
// --- Unified Animation Logic (Both charts sync) ---
// Animation completes at 1800 frames (30s), then holds for remaining duration
const animationEndFrame = 1800;
const exactProgress = interpolate(
frame,
[0, animationEndFrame - 30],
[0, cashflowData.timeline.length - 1],
{ extrapolateRight: "clamp" },
);
const dataEndIndex = Math.floor(exactProgress);
const yearProgress = exactProgress - dataEndIndex;
// Current data for HUD
const currentData =
cashflowData.timeline[dataEndIndex] || cashflowData.timeline[0];
// --- Convert cashflow data to LifePlanChart format ---
// useMemo prevents re-creating this array every frame, which causes chart flickering
const lifePlanTimeline: YearlySnapshot[] = useMemo(
() =>
cashflowData.timeline.map((item) => ({
year: item.year,
ages: {
rep: item.age,
spouse: undefined, // No spouse data in cashflow
},
financials: {
income: item.income.total,
expenditure: item.totalExpense,
assetBalance: item.assetBalance,
},
events: [],
isCritical: item.assetBalance < 0,
})),
[cashflowData.timeline],
);
// Memoize meta object to prevent re-creating it every frame
const lifePlanMeta = useMemo(
() => ({
currencyUnit: cashflowData.meta.currencyUnit,
startYear: cashflowData.timeline[0].year,
}),
[cashflowData.meta.currencyUnit, cashflowData.timeline],
);
return (
<div
style={{
width: "100%",
height: "100%",
backgroundColor: "white",
display: "flex",
flexDirection: "column",
gap: "20px",
padding: "20px",
}}
>
{/* --- Upper Chart: LifePlan (Asset Balance Trend) --- */}
<div
style={{
flex: 1,
backgroundColor: "white",
border: "2px solid #e2e8f0",
borderRadius: "8px",
overflow: "hidden",
display: "flex",
flexDirection: "column",
}}
>
<div
style={{
padding: "10px 20px",
borderBottom: "2px solid #e2e8f0",
background: "#4299E1",
}}
>
<h3
style={{
margin: 0,
fontSize: "16px",
fontWeight: "700",
color: "white",
}}
>
資産残高推移
</h3>
</div>
<div style={{ flex: 1, minHeight: 0 }}>
<LifePlanChart
key="lifeplan-chart"
timeline={lifePlanTimeline}
meta={lifePlanMeta}
hideHeader={true}
externalDataIndex={dataEndIndex}
externalFraction={yearProgress}
/>
</div>
</div>
{/* --- Lower Chart: CashFlow (Income vs Expenses) --- */}
<div
style={{
flex: 1,
backgroundColor: "white",
border: "2px solid #e2e8f0",
borderRadius: "8px",
overflow: "hidden",
display: "flex",
flexDirection: "column",
}}
>
<div
style={{
padding: "10px 20px",
borderBottom: "2px solid #e2e8f0",
background: "#ED8936",
}}
>
<h3
style={{
margin: 0,
fontSize: "16px",
fontWeight: "700",
color: "white",
}}
>
収支明細
</h3>
</div>
<div style={{ flex: 1, minHeight: 0 }}>
<CashFlowChart
key="cashflow-chart"
timeline={cashflowData.timeline}
meta={cashflowData.meta}
hideHeader={true}
externalDataIndex={dataEndIndex}
externalYearProgress={yearProgress}
/>
</div>
</div>
</div>
);
};

257
src/LifePlanBarChart.tsx Normal file
View File

@ -0,0 +1,257 @@
import React from "react";
import {
BarChart,
Bar,
XAxis,
YAxis,
CartesianGrid,
ReferenceLine,
ResponsiveContainer,
Cell,
} from "recharts";
import { useCurrentFrame, useVideoConfig, interpolate } from "remotion";
import type { LifePlanChartProps, YearlySnapshot } from "./types";
// --- Helper: The 3-Row X-Axis Tick ---
const CustomXAxisTick = ({ x, y, payload, timelineData }: any) => {
const index = payload.index;
const dataPoint = timelineData[index] as YearlySnapshot;
if (!dataPoint) return null;
return (
<g transform={`translate(${x},${y})`}>
{/* Row 1: Year */}
<text
x={0}
y={0}
dy={18}
textAnchor="middle"
fill="#666"
fontSize={22}
fontFamily="Roboto"
>
{dataPoint.year}
</text>
{/* Row 2: Rep Age (Bold) */}
<text
x={0}
y={0}
dy={44}
textAnchor="middle"
fill="#333"
fontWeight="bold"
fontSize={24}
fontFamily="Roboto"
>
{dataPoint.ages.rep}
</text>
{/* Row 3: Spouse Age */}
<text
x={0}
y={0}
dy={70}
textAnchor="middle"
fill="#999"
fontSize={20}
fontFamily="Roboto"
>
{dataPoint.ages.spouse ? `(${dataPoint.ages.spouse})` : "-"}
</text>
</g>
);
};
export const LifePlanBarChart: React.FC<LifePlanChartProps> = ({
timeline = [],
meta = { currencyUnit: "万円", startYear: 2025 },
}) => {
const frame = useCurrentFrame();
const { durationInFrames } = useVideoConfig();
// Calculate which year we're currently revealing
const framesPerYear = 15; // Slower: each bar takes 15 frames to grow
const currentYearIndex = Math.floor(
interpolate(frame, [0, durationInFrames - 30], [0, timeline.length - 1], {
extrapolateRight: "clamp",
}),
);
// Calculate growth progress for the current bar (0 to 1)
const exactProgress = interpolate(
frame,
[0, durationInFrames - 30],
[0, timeline.length - 1],
{ extrapolateRight: "clamp" },
);
const yearProgress = exactProgress - currentYearIndex;
// Simple linear growth (no bounce)
const growthMultiplier = yearProgress;
// Prepare data with animation
let visibleData = timeline.slice(0, currentYearIndex + 1).map((item, idx) => {
// Only the last (current) bar grows
const isCurrentBar = idx === currentYearIndex;
const multiplier = isCurrentBar ? growthMultiplier : 1;
return {
...item,
animatedBalance: item.financials.assetBalance * multiplier,
};
});
// When in fixed domain mode (< 10 points), pad with empty years to prevent offset
if (visibleData.length < 10 && visibleData.length > 0) {
const startYear = timeline[0].year;
const endYear = startYear + 9;
const paddedData = [];
for (let year = startYear; year <= endYear; year++) {
const existingData = visibleData.find((d) => d.year === year);
if (existingData) {
paddedData.push(existingData);
} else {
// Add placeholder for missing years
paddedData.push({
year,
ages: { rep: 0, spouse: 0 },
financials: { income: 0, expenditure: 0, assetBalance: 0 },
animatedBalance: 0,
isEmpty: true, // Flag to identify empty bars
});
}
}
visibleData = paddedData;
}
const currentData = timeline[currentYearIndex] || timeline[0];
return (
<div
style={{
flex: 1,
backgroundColor: "#ffffff",
padding: "40px",
fontFamily: "sans-serif",
display: "flex",
flexDirection: "column",
}}
>
{/* --- HUD (Heads-Up Display) --- */}
<div
style={{
display: "flex",
justifyContent: "space-between",
marginBottom: "20px",
padding: "20px",
background: "#f7fafc",
borderRadius: "12px",
boxShadow: "0 4px 6px rgba(0,0,0,0.05)",
}}
>
<div>
<div
style={{ fontSize: "14px", color: "#718096", marginBottom: "4px" }}
>
CURRENT YEAR
</div>
<div
style={{ fontSize: "32px", fontWeight: "800", color: "#2D3748" }}
>
{currentData.year}
</div>
</div>
<div style={{ textAlign: "right" }}>
<div
style={{ fontSize: "14px", color: "#718096", marginBottom: "4px" }}
>
ASSET BALANCE
</div>
<div
style={{
fontSize: "40px",
fontWeight: "900",
color:
currentData.financials.assetBalance < 0 ? "#E53E3E" : "#3182CE",
}}
>
{currentData.financials.assetBalance.toLocaleString()}{" "}
{meta.currencyUnit}
</div>
</div>
<div style={{ display: "flex", gap: "20px" }}>
<div>
<div style={{ fontSize: "12px", color: "#718096" }}>INCOME</div>
<div
style={{ fontSize: "20px", color: "#38A169", fontWeight: "600" }}
>
+{currentData.financials.income}
</div>
</div>
<div>
<div style={{ fontSize: "12px", color: "#718096" }}>EXPENSE</div>
<div
style={{ fontSize: "20px", color: "#E53E3E", fontWeight: "600" }}
>
-{currentData.financials.expenditure}
</div>
</div>
</div>
</div>
{/* --- Chart Area --- */}
<div style={{ flex: 1, position: "relative" }}>
<ResponsiveContainer width="100%" height="100%">
<BarChart
data={visibleData}
margin={{ top: 10, right: 30, left: 0, bottom: 95 }}
barCategoryGap={0}
>
<CartesianGrid strokeDasharray="3 3" vertical={false} />
<XAxis
dataKey="year"
interval={4}
tick={<CustomXAxisTick timelineData={timeline} />}
height={95}
tickLine={false}
domain={
visibleData.length < 10
? [timeline[0].year, timeline[0].year + 9]
: undefined
}
type={visibleData.length < 10 ? "number" : "category"}
/>
<YAxis
unit={meta.currencyUnit}
width={80}
tickFormatter={(val) =>
val >= 10000 || val <= -10000 ? `${val / 10000}億` : val
}
/>
<ReferenceLine y={0} stroke="#000" strokeWidth={2} />
<Bar dataKey="animatedBalance" isAnimationActive={false}>
{visibleData.map((entry: any, index) => (
<Cell
key={`cell-${index}`}
fill={
entry.isEmpty
? "transparent"
: entry.financials.assetBalance >= 0
? "#4299E1"
: "#E53E3E"
}
/>
))}
</Bar>
</BarChart>
</ResponsiveContainer>
</div>
</div>
);
};

348
src/LifePlanChart.tsx Normal file
View File

@ -0,0 +1,348 @@
import React from "react";
import {
AreaChart,
Area,
XAxis,
YAxis,
CartesianGrid,
ReferenceLine,
ResponsiveContainer,
} from "recharts";
import { useCurrentFrame, useVideoConfig, interpolate } from "remotion";
// Ensure this import points to your new types file
import type { LifePlanChartProps, YearlySnapshot } from "./types";
// --- Helper: The 3-Row X-Axis Tick ---
const CustomXAxisTick = ({ x, y, payload, timelineData }: any) => {
// Map index to data to ensure safety
const index = payload.index;
const dataPoint = timelineData[index] as YearlySnapshot;
if (!dataPoint) return null;
return (
<g transform={`translate(${x},${y})`}>
{/* Row 1: Year */}
<text
x={0}
y={0}
dy={18}
textAnchor="middle"
fill="#666"
fontSize={22}
fontFamily="Roboto"
>
{dataPoint.year}
</text>
{/* Row 2: Rep Age (Bold) */}
<text
x={0}
y={0}
dy={44}
textAnchor="middle"
fill="#333"
fontWeight="bold"
fontSize={24}
fontFamily="Roboto"
>
{dataPoint.ages.rep}
</text>
{/* Row 3: Spouse Age */}
<text
x={0}
y={0}
dy={70}
textAnchor="middle"
fill="#999"
fontSize={20}
fontFamily="Roboto"
>
{dataPoint.ages.spouse ? `(${dataPoint.ages.spouse})` : "-"}
</text>
</g>
);
};
// Stable configuration objects to prevent re-renders
const CHART_MARGIN = { top: 10, right: 30, left: 0, bottom: 50 };
export const LifePlanChart: React.FC<LifePlanChartProps> = React.memo(
({
timeline = [],
meta = { currencyUnit: "万円", startYear: 2025 },
hideHeader = false,
externalDataIndex,
externalFraction,
}) => {
const frame = useCurrentFrame();
const { durationInFrames } = useVideoConfig();
// --- 1. Animation Logic: Data Slicing with Smooth Interpolation ---
// Get fractional index (e.g., 5.7 means 70% between index 5 and 6)
const exactIndex = interpolate(
frame,
[0, durationInFrames - 30],
[0, timeline.length - 1],
{ extrapolateRight: "clamp" },
);
const dataEndIndex =
externalDataIndex !== undefined
? externalDataIndex
: Math.floor(exactIndex);
const fraction =
externalFraction !== undefined
? externalFraction
: exactIndex - dataEndIndex;
// HUD Data: The "Latest" available data point based on animation
const currentData = timeline[dataEndIndex] || timeline[0];
// Chart Data: The slice of history to render
let visibleData = timeline.slice(0, dataEndIndex + 1);
// Add interpolated point for smooth animation
if (fraction > 0 && dataEndIndex < timeline.length - 1) {
const current = timeline[dataEndIndex];
const next = timeline[dataEndIndex + 1];
const interpolatedPoint: YearlySnapshot = {
year: current.year + fraction,
ages: {
rep: Math.round(
current.ages.rep + fraction * (next.ages.rep - current.ages.rep),
),
spouse:
current.ages.spouse && next.ages.spouse
? Math.round(
current.ages.spouse +
fraction * (next.ages.spouse - current.ages.spouse),
)
: current.ages.spouse,
},
financials: {
income:
current.financials.income +
fraction * (next.financials.income - current.financials.income),
expenditure:
current.financials.expenditure +
fraction *
(next.financials.expenditure - current.financials.expenditure),
assetBalance:
current.financials.assetBalance +
fraction *
(next.financials.assetBalance - current.financials.assetBalance),
},
};
visibleData = [...visibleData, interpolatedPoint];
}
// --- 2. Gradient Logic (Blue above 0, Red below 0) ---
// CRITICAL: Calculate gradient based on VISIBLE data, not full timeline
// This ensures the gradient matches what's currently shown on screen
const gradientOffset = React.useMemo(() => {
if (!visibleData || visibleData.length === 0) return 0;
const dataMax = Math.max(
...visibleData.map((i: YearlySnapshot) => i.financials.assetBalance),
);
const dataMin = Math.min(
...visibleData.map((i: YearlySnapshot) => i.financials.assetBalance),
);
// All positive: gradient fully blue
if (dataMin >= 0) return 1;
// All negative: gradient fully red
if (dataMax <= 0) return 0;
// Mixed: calculate zero-line position
return dataMax / (dataMax - dataMin);
}, [visibleData]);
const off = gradientOffset;
// Memoize xAxis domain to prevent re-creating array every frame
const xAxisDomain = React.useMemo(
() =>
visibleData.length < 10
? [timeline[0].year, timeline[0].year + 9]
: undefined,
[visibleData.length, timeline],
);
const xAxisType = visibleData.length < 10 ? "number" : "category";
// Calculate Y-axis domain based on full timeline (not visibleData)
// This prevents Y-axis from jumping when asset balance changes dramatically
const yAxisDomain = React.useMemo(() => {
const allBalances = timeline.map((d) => d.financials.assetBalance);
const minValue = Math.min(...allBalances);
const maxValue = Math.max(...allBalances);
// Add 10% padding for visual comfort
const padding = (maxValue - minValue) * 0.1;
return [
Math.floor((minValue - padding) / 100) * 100,
Math.ceil((maxValue + padding) / 100) * 100,
];
}, [timeline]);
return (
<div
style={{
width: "100%",
height: "100%",
backgroundColor: "#ffffff",
padding: hideHeader ? "10px" : "40px",
fontFamily: "sans-serif",
display: "flex",
flexDirection: "column",
}}
>
{/* --- HUD (Heads-Up Display) --- */}
{!hideHeader && (
<div
style={{
display: "flex",
justifyContent: "space-between",
marginBottom: "20px",
padding: "20px",
background: "#f7fafc",
borderRadius: "12px",
boxShadow: "0 4px 6px rgba(0,0,0,0.05)",
}}
>
<div>
<div
style={{
fontSize: "14px",
color: "#718096",
marginBottom: "4px",
}}
>
CURRENT YEAR
</div>
<div
style={{
fontSize: "32px",
fontWeight: "800",
color: "#2D3748",
}}
>
{currentData.year}
</div>
</div>
<div style={{ textAlign: "right" }}>
<div
style={{
fontSize: "14px",
color: "#718096",
marginBottom: "4px",
}}
>
ASSET BALANCE
</div>
<div
style={{
fontSize: "40px",
fontWeight: "900",
// Turn Red if negative
color:
currentData.financials.assetBalance < 0
? "#E53E3E"
: "#3182CE",
}}
>
{currentData.financials.assetBalance.toLocaleString()}{" "}
{meta.currencyUnit}
</div>
</div>
<div style={{ display: "flex", gap: "20px" }}>
<div>
<div style={{ fontSize: "12px", color: "#718096" }}>INCOME</div>
<div
style={{
fontSize: "20px",
color: "#38A169",
fontWeight: "600",
}}
>
+{currentData.financials.income}
</div>
</div>
<div>
<div style={{ fontSize: "12px", color: "#718096" }}>
EXPENSE
</div>
<div
style={{
fontSize: "20px",
color: "#E53E3E",
fontWeight: "600",
}}
>
-{currentData.financials.expenditure}
</div>
</div>
</div>
</div>
)}
{/* --- Chart Area --- */}
<div style={{ flex: 1, minHeight: 0, width: "100%" }}>
<ResponsiveContainer width="100%" height="100%">
<AreaChart data={visibleData} margin={CHART_MARGIN}>
<defs>
<linearGradient
id="lifePlanGradient"
x1="0"
y1="0"
x2="0"
y2="1"
>
<stop offset={off} stopColor="#4299E1" stopOpacity={0.8} />
<stop offset={off} stopColor="#E53E3E" stopOpacity={0.8} />
</linearGradient>
</defs>
<CartesianGrid strokeDasharray="3 3" vertical={false} />
<XAxis
dataKey="year"
interval={4}
tick={<CustomXAxisTick timelineData={timeline} />}
height={95}
tickLine={false}
domain={xAxisDomain}
type={xAxisType}
/>
<YAxis
unit={meta.currencyUnit}
width={80}
domain={yAxisDomain}
tickFormatter={(val) =>
val >= 10000 || val <= -10000 ? `${val / 10000}億` : val
}
/>
<ReferenceLine y={0} stroke="#000" strokeWidth={2} />
<Area
type="monotone"
dataKey="financials.assetBalance"
stroke="#2D3748"
strokeWidth={2}
fill="url(#lifePlanGradient)"
isAnimationActive={false} // CRITICAL: Disable internal animation
/>
</AreaChart>
</ResponsiveContainer>
</div>
</div>
);
},
);
LifePlanChart.displayName = "LifePlanChart";

View File

@ -1,16 +1,66 @@
import { Composition } from "remotion"; import { Composition } from "remotion";
import { MyComposition } from "./Composition";
import { CashFlowChart } from "./CashFlowChart";
import { DualChartView } from "./DualChartView";
import { LifePlanBarChart } from "./LifePlanBarChart";
import { LifePlanChart } from "./LifePlanChart";
// Import the data directly (Make sure generate_mock_data.py was run!)
import mockCashFlow from "./mock-cashflow.json";
import mockData from "./mock-data.json";
// Import the NEW component
// Import the NEW data
// Import DualChartView
export const RemotionRoot: React.FC = () => { export const RemotionRoot: React.FC = () => {
return ( return (
<> <>
<Composition <Composition
id="MyComp" id="LifePlanSimulation"
component={MyComposition} component={LifePlanChart}
durationInFrames={60} durationInFrames={900} // 15 seconds video (60fps * 15)
fps={30} fps={60}
width={1280} width={1920}
height={720} height={1080}
// CRITICAL FIX: Pass the data here!
defaultProps={{
timeline: mockData.timeline,
meta: mockData.meta,
}}
/>
<Composition
id="LifePlanBarChart"
component={LifePlanBarChart}
durationInFrames={900} // 15 seconds video (60fps * 15)
fps={60}
width={1920}
height={1080}
defaultProps={{
timeline: mockData.timeline,
meta: mockData.meta,
}}
/>
<Composition
id="CashFlowChart"
component={CashFlowChart}
durationInFrames={900} // 15 seconds video (60fps * 15)
fps={60}
width={1920}
height={1080}
defaultProps={{
timeline: mockCashFlow.timeline,
meta: mockCashFlow.meta,
}}
/>
<Composition
id="DualChartView"
component={DualChartView}
durationInFrames={2100} // 35 seconds: 30s animation + 5s hold (60fps * 35)
fps={60}
width={1920}
height={1080}
defaultProps={{
cashflowData: mockCashFlow,
}}
/> />
</> </>
); );

163
src/UnifiedHUD.tsx Normal file
View File

@ -0,0 +1,163 @@
import React from "react";
interface UnifiedHUDProps {
year: number;
age: number;
income: number;
expense: number;
netCashFlow: number;
assetBalance: number;
currencyUnit: string;
}
export const UnifiedHUD: React.FC<UnifiedHUDProps> = ({
year,
age,
income,
expense,
netCashFlow,
assetBalance,
currencyUnit,
}) => {
return (
<div
style={{
display: "flex",
justifyContent: "space-between",
alignItems: "center",
padding: "20px 30px",
background: "linear-gradient(135deg, #667eea 0%, #764ba2 100%)",
borderRadius: "12px",
marginBottom: "15px",
boxShadow: "0 4px 6px rgba(0,0,0,0.1)",
}}
>
{/* Year & Age */}
<div>
<div
style={{
fontSize: "14px",
color: "rgba(255,255,255,0.8)",
fontWeight: "600",
letterSpacing: "0.5px",
}}
>
CURRENT YEAR
</div>
<div
style={{
fontSize: "36px",
fontWeight: "800",
color: "#ffffff",
marginTop: "4px",
}}
>
{year}{" "}
<span
style={{
fontSize: "20px",
fontWeight: "normal",
opacity: 0.9,
}}
>
({age}歳)
</span>
</div>
</div>
{/* Metrics */}
<div style={{ display: "flex", gap: "30px" }}>
{/* Income */}
<MetricCard
label="INCOME"
value={income}
unit={currencyUnit}
color="#FF6B6B"
icon="↑"
/>
{/* Expense */}
<MetricCard
label="EXPENSE"
value={expense}
unit={currencyUnit}
color="#FFA500"
icon="↓"
/>
{/* Net Cash Flow */}
<MetricCard
label="NET FLOW"
value={netCashFlow}
unit={currencyUnit}
color={netCashFlow >= 0 ? "#4ECDC4" : "#FF6B6B"}
icon={netCashFlow >= 0 ? "+" : "-"}
/>
{/* Asset Balance */}
<MetricCard
label="ASSET"
value={assetBalance}
unit={currencyUnit}
color={assetBalance >= 0 ? "#95E1D3" : "#F38181"}
icon="💰"
isLarge
/>
</div>
</div>
);
};
interface MetricCardProps {
label: string;
value: number;
unit: string;
color: string;
icon: string;
isLarge?: boolean;
}
const MetricCard: React.FC<MetricCardProps> = ({
label,
value,
unit,
color,
icon,
isLarge = false,
}) => {
return (
<div style={{ textAlign: "right" }}>
<div
style={{
fontSize: "11px",
color: "rgba(255,255,255,0.8)",
fontWeight: "700",
letterSpacing: "0.8px",
marginBottom: "4px",
}}
>
{label}
</div>
<div
style={{
fontSize: isLarge ? "32px" : "24px",
fontWeight: "bold",
color: "#ffffff",
display: "flex",
alignItems: "center",
justifyContent: "flex-end",
gap: "6px",
}}
>
<span style={{ fontSize: "16px", opacity: 0.9 }}>{icon}</span>
<span>
{value > 0 && label === "NET FLOW" ? "+" : ""}
{value.toLocaleString()}
</span>
<span style={{ fontSize: isLarge ? "16px" : "14px", opacity: 0.8 }}>
{unit}
</span>
</div>
</div>
);
};

View File

@ -1,4 +1,5 @@
import { registerRoot } from "remotion"; import { registerRoot } from "remotion";
import { RemotionRoot } from "./Root"; import { RemotionRoot } from "./Root";
registerRoot(RemotionRoot); registerRoot(RemotionRoot);

View File

@ -0,0 +1,908 @@
{
"meta": {
"currencyUnit": "万円"
},
"timeline": [
{
"year": 2027,
"age": 48,
"income": {
"total": 1100
},
"expenses": {
"living": 240,
"housing": 180,
"education": 80,
"tax": 165,
"event": 0
},
"totalExpense": 665,
"netCashFlow": 435,
"assetBalance": 935
},
{
"year": 2028,
"age": 49,
"income": {
"total": 1110
},
"expenses": {
"living": 242,
"housing": 180,
"education": 80,
"tax": 166,
"event": 0
},
"totalExpense": 668,
"netCashFlow": 442,
"assetBalance": 1377
},
{
"year": 2029,
"age": 50,
"income": {
"total": 1120
},
"expenses": {
"living": 244,
"housing": 181,
"education": 80,
"tax": 168,
"event": 100
},
"totalExpense": 773,
"netCashFlow": 347,
"assetBalance": 1724
},
{
"year": 2030,
"age": 51,
"income": {
"total": 1130
},
"expenses": {
"living": 247,
"housing": 181,
"education": 250,
"tax": 169,
"event": 0
},
"totalExpense": 847,
"netCashFlow": 283,
"assetBalance": 2007
},
{
"year": 2031,
"age": 52,
"income": {
"total": 1140
},
"expenses": {
"living": 249,
"housing": 182,
"education": 250,
"tax": 171,
"event": 0
},
"totalExpense": 852,
"netCashFlow": 288,
"assetBalance": 2295
},
{
"year": 2032,
"age": 53,
"income": {
"total": 1150
},
"expenses": {
"living": 252,
"housing": 182,
"education": 250,
"tax": 172,
"event": 0
},
"totalExpense": 856,
"netCashFlow": 294,
"assetBalance": 2589
},
{
"year": 2033,
"age": 54,
"income": {
"total": 1160
},
"expenses": {
"living": 254,
"housing": 183,
"education": 250,
"tax": 174,
"event": 0
},
"totalExpense": 861,
"netCashFlow": 299,
"assetBalance": 2888
},
{
"year": 2034,
"age": 55,
"income": {
"total": 1170
},
"expenses": {
"living": 257,
"housing": 183,
"education": 0,
"tax": 175,
"event": 100
},
"totalExpense": 715,
"netCashFlow": 455,
"assetBalance": 3343
},
{
"year": 2035,
"age": 56,
"income": {
"total": 1180
},
"expenses": {
"living": 259,
"housing": 184,
"education": 0,
"tax": 177,
"event": 0
},
"totalExpense": 620,
"netCashFlow": 560,
"assetBalance": 3903
},
{
"year": 2036,
"age": 57,
"income": {
"total": 1190
},
"expenses": {
"living": 262,
"housing": 184,
"education": 0,
"tax": 178,
"event": 0
},
"totalExpense": 624,
"netCashFlow": 566,
"assetBalance": 4469
},
{
"year": 2037,
"age": 58,
"income": {
"total": 1200
},
"expenses": {
"living": 265,
"housing": 185,
"education": 0,
"tax": 180,
"event": 0
},
"totalExpense": 630,
"netCashFlow": 570,
"assetBalance": 5039
},
{
"year": 2038,
"age": 59,
"income": {
"total": 1210
},
"expenses": {
"living": 267,
"housing": 185,
"education": 0,
"tax": 181,
"event": 0
},
"totalExpense": 633,
"netCashFlow": 577,
"assetBalance": 5616
},
{
"year": 2039,
"age": 60,
"income": {
"total": 2500
},
"expenses": {
"living": 270,
"housing": 186,
"education": 0,
"tax": 375,
"event": 500
},
"totalExpense": 1331,
"netCashFlow": 1169,
"assetBalance": 6785
},
{
"year": 2040,
"age": 61,
"income": {
"total": 500
},
"expenses": {
"living": 273,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 534,
"netCashFlow": -34,
"assetBalance": 6751
},
{
"year": 2041,
"age": 62,
"income": {
"total": 500
},
"expenses": {
"living": 276,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 537,
"netCashFlow": -37,
"assetBalance": 6714
},
{
"year": 2042,
"age": 63,
"income": {
"total": 500
},
"expenses": {
"living": 279,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 540,
"netCashFlow": -40,
"assetBalance": 6674
},
{
"year": 2043,
"age": 64,
"income": {
"total": 500
},
"expenses": {
"living": 282,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 543,
"netCashFlow": -43,
"assetBalance": 6631
},
{
"year": 2044,
"age": 65,
"income": {
"total": 280
},
"expenses": {
"living": 285,
"housing": 61,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 488,
"netCashFlow": -208,
"assetBalance": 6423
},
{
"year": 2045,
"age": 66,
"income": {
"total": 280
},
"expenses": {
"living": 288,
"housing": 62,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 392,
"netCashFlow": -112,
"assetBalance": 6311
},
{
"year": 2046,
"age": 67,
"income": {
"total": 280
},
"expenses": {
"living": 291,
"housing": 63,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 396,
"netCashFlow": -116,
"assetBalance": 6195
},
{
"year": 2047,
"age": 68,
"income": {
"total": 280
},
"expenses": {
"living": 294,
"housing": 63,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 399,
"netCashFlow": -119,
"assetBalance": 6076
},
{
"year": 2048,
"age": 69,
"income": {
"total": 280
},
"expenses": {
"living": 297,
"housing": 64,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 403,
"netCashFlow": -123,
"assetBalance": 5953
},
{
"year": 2049,
"age": 70,
"income": {
"total": 280
},
"expenses": {
"living": 300,
"housing": 64,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 506,
"netCashFlow": -226,
"assetBalance": 5727
},
{
"year": 2050,
"age": 71,
"income": {
"total": 280
},
"expenses": {
"living": 303,
"housing": 65,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 410,
"netCashFlow": -130,
"assetBalance": 5597
},
{
"year": 2051,
"age": 72,
"income": {
"total": 280
},
"expenses": {
"living": 306,
"housing": 66,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 414,
"netCashFlow": -134,
"assetBalance": 5463
},
{
"year": 2052,
"age": 73,
"income": {
"total": 280
},
"expenses": {
"living": 309,
"housing": 66,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 417,
"netCashFlow": -137,
"assetBalance": 5326
},
{
"year": 2053,
"age": 74,
"income": {
"total": 280
},
"expenses": {
"living": 312,
"housing": 67,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 421,
"netCashFlow": -141,
"assetBalance": 5185
},
{
"year": 2054,
"age": 75,
"income": {
"total": 280
},
"expenses": {
"living": 315,
"housing": 67,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 524,
"netCashFlow": -244,
"assetBalance": 4941
},
{
"year": 2055,
"age": 76,
"income": {
"total": 280
},
"expenses": {
"living": 318,
"housing": 68,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 428,
"netCashFlow": -148,
"assetBalance": 4793
},
{
"year": 2056,
"age": 77,
"income": {
"total": 280
},
"expenses": {
"living": 321,
"housing": 69,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 432,
"netCashFlow": -152,
"assetBalance": 4641
},
{
"year": 2057,
"age": 78,
"income": {
"total": 280
},
"expenses": {
"living": 324,
"housing": 69,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 435,
"netCashFlow": -155,
"assetBalance": 4486
},
{
"year": 2058,
"age": 79,
"income": {
"total": 280
},
"expenses": {
"living": 327,
"housing": 70,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 439,
"netCashFlow": -159,
"assetBalance": 4327
},
{
"year": 2059,
"age": 80,
"income": {
"total": 280
},
"expenses": {
"living": 330,
"housing": 70,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 542,
"netCashFlow": -262,
"assetBalance": 4065
},
{
"year": 2060,
"age": 81,
"income": {
"total": 280
},
"expenses": {
"living": 333,
"housing": 71,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 446,
"netCashFlow": -166,
"assetBalance": 3899
},
{
"year": 2061,
"age": 82,
"income": {
"total": 280
},
"expenses": {
"living": 336,
"housing": 72,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 450,
"netCashFlow": -170,
"assetBalance": 3729
},
{
"year": 2062,
"age": 83,
"income": {
"total": 280
},
"expenses": {
"living": 339,
"housing": 72,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 453,
"netCashFlow": -173,
"assetBalance": 3556
},
{
"year": 2063,
"age": 84,
"income": {
"total": 280
},
"expenses": {
"living": 342,
"housing": 73,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 457,
"netCashFlow": -177,
"assetBalance": 3379
},
{
"year": 2064,
"age": 85,
"income": {
"total": 280
},
"expenses": {
"living": 345,
"housing": 73,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 560,
"netCashFlow": -280,
"assetBalance": 3099
},
{
"year": 2065,
"age": 86,
"income": {
"total": 280
},
"expenses": {
"living": 348,
"housing": 74,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 464,
"netCashFlow": -184,
"assetBalance": 2915
},
{
"year": 2066,
"age": 87,
"income": {
"total": 280
},
"expenses": {
"living": 351,
"housing": 75,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 468,
"netCashFlow": -188,
"assetBalance": 2727
},
{
"year": 2067,
"age": 88,
"income": {
"total": 280
},
"expenses": {
"living": 354,
"housing": 75,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 471,
"netCashFlow": -191,
"assetBalance": 2536
},
{
"year": 2068,
"age": 89,
"income": {
"total": 280
},
"expenses": {
"living": 357,
"housing": 76,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 475,
"netCashFlow": -195,
"assetBalance": 2341
},
{
"year": 2069,
"age": 90,
"income": {
"total": 280
},
"expenses": {
"living": 360,
"housing": 76,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 578,
"netCashFlow": -298,
"assetBalance": 2043
},
{
"year": 2070,
"age": 91,
"income": {
"total": 280
},
"expenses": {
"living": 363,
"housing": 77,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 482,
"netCashFlow": -202,
"assetBalance": 1841
},
{
"year": 2071,
"age": 92,
"income": {
"total": 280
},
"expenses": {
"living": 366,
"housing": 78,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 486,
"netCashFlow": -206,
"assetBalance": 1635
},
{
"year": 2072,
"age": 93,
"income": {
"total": 280
},
"expenses": {
"living": 369,
"housing": 78,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 489,
"netCashFlow": -209,
"assetBalance": 1426
},
{
"year": 2073,
"age": 94,
"income": {
"total": 280
},
"expenses": {
"living": 372,
"housing": 79,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 493,
"netCashFlow": -213,
"assetBalance": 1213
},
{
"year": 2074,
"age": 95,
"income": {
"total": 280
},
"expenses": {
"living": 375,
"housing": 79,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 596,
"netCashFlow": -316,
"assetBalance": 897
},
{
"year": 2075,
"age": 96,
"income": {
"total": 280
},
"expenses": {
"living": 378,
"housing": 80,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 500,
"netCashFlow": -220,
"assetBalance": 677
},
{
"year": 2076,
"age": 97,
"income": {
"total": 280
},
"expenses": {
"living": 381,
"housing": 81,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 504,
"netCashFlow": -224,
"assetBalance": 453
},
{
"year": 2077,
"age": 98,
"income": {
"total": 280
},
"expenses": {
"living": 384,
"housing": 81,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 507,
"netCashFlow": -227,
"assetBalance": 226
},
{
"year": 2078,
"age": 99,
"income": {
"total": 280
},
"expenses": {
"living": 387,
"housing": 82,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 511,
"netCashFlow": -231,
"assetBalance": -5
},
{
"year": 2079,
"age": 100,
"income": {
"total": 280
},
"expenses": {
"living": 390,
"housing": 82,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 614,
"netCashFlow": -334,
"assetBalance": -339
}
]
}

908
src/mock-cashflow.json Normal file
View File

@ -0,0 +1,908 @@
{
"meta": {
"currencyUnit": "万円"
},
"timeline": [
{
"year": 2027,
"age": 48,
"income": {
"total": 950
},
"expenses": {
"living": 240,
"housing": 180,
"education": 80,
"tax": 165,
"event": 0
},
"totalExpense": 665,
"netCashFlow": 285,
"assetBalance": 785
},
{
"year": 2028,
"age": 49,
"income": {
"total": 960
},
"expenses": {
"living": 242,
"housing": 180,
"education": 80,
"tax": 166,
"event": 0
},
"totalExpense": 668,
"netCashFlow": 292,
"assetBalance": 1077
},
{
"year": 2029,
"age": 50,
"income": {
"total": 970
},
"expenses": {
"living": 244,
"housing": 181,
"education": 80,
"tax": 168,
"event": 100
},
"totalExpense": 773,
"netCashFlow": 197,
"assetBalance": 1274
},
{
"year": 2030,
"age": 51,
"income": {
"total": 980
},
"expenses": {
"living": 247,
"housing": 181,
"education": 250,
"tax": 169,
"event": 0
},
"totalExpense": 847,
"netCashFlow": 133,
"assetBalance": 1407
},
{
"year": 2031,
"age": 52,
"income": {
"total": 990
},
"expenses": {
"living": 249,
"housing": 182,
"education": 250,
"tax": 171,
"event": 0
},
"totalExpense": 852,
"netCashFlow": 138,
"assetBalance": 1545
},
{
"year": 2032,
"age": 53,
"income": {
"total": 1000
},
"expenses": {
"living": 252,
"housing": 182,
"education": 250,
"tax": 172,
"event": 0
},
"totalExpense": 856,
"netCashFlow": 144,
"assetBalance": 1689
},
{
"year": 2033,
"age": 54,
"income": {
"total": 1010
},
"expenses": {
"living": 254,
"housing": 183,
"education": 250,
"tax": 174,
"event": 0
},
"totalExpense": 861,
"netCashFlow": 149,
"assetBalance": 1838
},
{
"year": 2034,
"age": 55,
"income": {
"total": 1020
},
"expenses": {
"living": 257,
"housing": 183,
"education": 0,
"tax": 175,
"event": 100
},
"totalExpense": 715,
"netCashFlow": 305,
"assetBalance": 2143
},
{
"year": 2035,
"age": 56,
"income": {
"total": 1030
},
"expenses": {
"living": 259,
"housing": 184,
"education": 0,
"tax": 177,
"event": 0
},
"totalExpense": 620,
"netCashFlow": 410,
"assetBalance": 2553
},
{
"year": 2036,
"age": 57,
"income": {
"total": 1040
},
"expenses": {
"living": 262,
"housing": 184,
"education": 0,
"tax": 178,
"event": 0
},
"totalExpense": 624,
"netCashFlow": 416,
"assetBalance": 2969
},
{
"year": 2037,
"age": 58,
"income": {
"total": 1050
},
"expenses": {
"living": 265,
"housing": 185,
"education": 0,
"tax": 180,
"event": 0
},
"totalExpense": 630,
"netCashFlow": 420,
"assetBalance": 3389
},
{
"year": 2038,
"age": 59,
"income": {
"total": 1060
},
"expenses": {
"living": 267,
"housing": 185,
"education": 0,
"tax": 181,
"event": 0
},
"totalExpense": 633,
"netCashFlow": 427,
"assetBalance": 3816
},
{
"year": 2039,
"age": 60,
"income": {
"total": 2350
},
"expenses": {
"living": 270,
"housing": 186,
"education": 0,
"tax": 375,
"event": 500
},
"totalExpense": 1331,
"netCashFlow": 1019,
"assetBalance": 4835
},
{
"year": 2040,
"age": 61,
"income": {
"total": 350
},
"expenses": {
"living": 273,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 534,
"netCashFlow": -184,
"assetBalance": 4651
},
{
"year": 2041,
"age": 62,
"income": {
"total": 350
},
"expenses": {
"living": 276,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 537,
"netCashFlow": -187,
"assetBalance": 4464
},
{
"year": 2042,
"age": 63,
"income": {
"total": 350
},
"expenses": {
"living": 279,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 540,
"netCashFlow": -190,
"assetBalance": 4274
},
{
"year": 2043,
"age": 64,
"income": {
"total": 350
},
"expenses": {
"living": 282,
"housing": 186,
"education": 0,
"tax": 75,
"event": 0
},
"totalExpense": 543,
"netCashFlow": -193,
"assetBalance": 4081
},
{
"year": 2044,
"age": 65,
"income": {
"total": 130
},
"expenses": {
"living": 285,
"housing": 61,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 488,
"netCashFlow": -358,
"assetBalance": 3723
},
{
"year": 2045,
"age": 66,
"income": {
"total": 130
},
"expenses": {
"living": 288,
"housing": 62,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 392,
"netCashFlow": -262,
"assetBalance": 3461
},
{
"year": 2046,
"age": 67,
"income": {
"total": 130
},
"expenses": {
"living": 291,
"housing": 63,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 396,
"netCashFlow": -266,
"assetBalance": 3195
},
{
"year": 2047,
"age": 68,
"income": {
"total": 130
},
"expenses": {
"living": 294,
"housing": 63,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 399,
"netCashFlow": -269,
"assetBalance": 2926
},
{
"year": 2048,
"age": 69,
"income": {
"total": 130
},
"expenses": {
"living": 297,
"housing": 64,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 403,
"netCashFlow": -273,
"assetBalance": 2653
},
{
"year": 2049,
"age": 70,
"income": {
"total": 130
},
"expenses": {
"living": 300,
"housing": 64,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 506,
"netCashFlow": -376,
"assetBalance": 2277
},
{
"year": 2050,
"age": 71,
"income": {
"total": 130
},
"expenses": {
"living": 303,
"housing": 65,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 410,
"netCashFlow": -280,
"assetBalance": 1997
},
{
"year": 2051,
"age": 72,
"income": {
"total": 130
},
"expenses": {
"living": 306,
"housing": 66,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 414,
"netCashFlow": -284,
"assetBalance": 1713
},
{
"year": 2052,
"age": 73,
"income": {
"total": 130
},
"expenses": {
"living": 309,
"housing": 66,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 417,
"netCashFlow": -287,
"assetBalance": 1426
},
{
"year": 2053,
"age": 74,
"income": {
"total": 130
},
"expenses": {
"living": 312,
"housing": 67,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 421,
"netCashFlow": -291,
"assetBalance": 1135
},
{
"year": 2054,
"age": 75,
"income": {
"total": 130
},
"expenses": {
"living": 315,
"housing": 67,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 524,
"netCashFlow": -394,
"assetBalance": 741
},
{
"year": 2055,
"age": 76,
"income": {
"total": 130
},
"expenses": {
"living": 318,
"housing": 68,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 428,
"netCashFlow": -298,
"assetBalance": 443
},
{
"year": 2056,
"age": 77,
"income": {
"total": 130
},
"expenses": {
"living": 321,
"housing": 69,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 432,
"netCashFlow": -302,
"assetBalance": 141
},
{
"year": 2057,
"age": 78,
"income": {
"total": 130
},
"expenses": {
"living": 324,
"housing": 69,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 435,
"netCashFlow": -305,
"assetBalance": -164
},
{
"year": 2058,
"age": 79,
"income": {
"total": 130
},
"expenses": {
"living": 327,
"housing": 70,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 439,
"netCashFlow": -309,
"assetBalance": -473
},
{
"year": 2059,
"age": 80,
"income": {
"total": 130
},
"expenses": {
"living": 330,
"housing": 70,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 542,
"netCashFlow": -412,
"assetBalance": -885
},
{
"year": 2060,
"age": 81,
"income": {
"total": 130
},
"expenses": {
"living": 333,
"housing": 71,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 446,
"netCashFlow": -316,
"assetBalance": -1201
},
{
"year": 2061,
"age": 82,
"income": {
"total": 130
},
"expenses": {
"living": 336,
"housing": 72,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 450,
"netCashFlow": -320,
"assetBalance": -1521
},
{
"year": 2062,
"age": 83,
"income": {
"total": 130
},
"expenses": {
"living": 339,
"housing": 72,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 453,
"netCashFlow": -323,
"assetBalance": -1844
},
{
"year": 2063,
"age": 84,
"income": {
"total": 130
},
"expenses": {
"living": 342,
"housing": 73,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 457,
"netCashFlow": -327,
"assetBalance": -2171
},
{
"year": 2064,
"age": 85,
"income": {
"total": 130
},
"expenses": {
"living": 345,
"housing": 73,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 560,
"netCashFlow": -430,
"assetBalance": -2601
},
{
"year": 2065,
"age": 86,
"income": {
"total": 130
},
"expenses": {
"living": 348,
"housing": 74,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 464,
"netCashFlow": -334,
"assetBalance": -2935
},
{
"year": 2066,
"age": 87,
"income": {
"total": 130
},
"expenses": {
"living": 351,
"housing": 75,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 468,
"netCashFlow": -338,
"assetBalance": -3273
},
{
"year": 2067,
"age": 88,
"income": {
"total": 130
},
"expenses": {
"living": 354,
"housing": 75,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 471,
"netCashFlow": -341,
"assetBalance": -3614
},
{
"year": 2068,
"age": 89,
"income": {
"total": 130
},
"expenses": {
"living": 357,
"housing": 76,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 475,
"netCashFlow": -345,
"assetBalance": -3959
},
{
"year": 2069,
"age": 90,
"income": {
"total": 130
},
"expenses": {
"living": 360,
"housing": 76,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 578,
"netCashFlow": -448,
"assetBalance": -4407
},
{
"year": 2070,
"age": 91,
"income": {
"total": 130
},
"expenses": {
"living": 363,
"housing": 77,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 482,
"netCashFlow": -352,
"assetBalance": -4759
},
{
"year": 2071,
"age": 92,
"income": {
"total": 130
},
"expenses": {
"living": 366,
"housing": 78,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 486,
"netCashFlow": -356,
"assetBalance": -5115
},
{
"year": 2072,
"age": 93,
"income": {
"total": 130
},
"expenses": {
"living": 369,
"housing": 78,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 489,
"netCashFlow": -359,
"assetBalance": -5474
},
{
"year": 2073,
"age": 94,
"income": {
"total": 130
},
"expenses": {
"living": 372,
"housing": 79,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 493,
"netCashFlow": -363,
"assetBalance": -5837
},
{
"year": 2074,
"age": 95,
"income": {
"total": 130
},
"expenses": {
"living": 375,
"housing": 79,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 596,
"netCashFlow": -466,
"assetBalance": -6303
},
{
"year": 2075,
"age": 96,
"income": {
"total": 130
},
"expenses": {
"living": 378,
"housing": 80,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 500,
"netCashFlow": -370,
"assetBalance": -6673
},
{
"year": 2076,
"age": 97,
"income": {
"total": 130
},
"expenses": {
"living": 381,
"housing": 81,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 504,
"netCashFlow": -374,
"assetBalance": -7047
},
{
"year": 2077,
"age": 98,
"income": {
"total": 130
},
"expenses": {
"living": 384,
"housing": 81,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 507,
"netCashFlow": -377,
"assetBalance": -7424
},
{
"year": 2078,
"age": 99,
"income": {
"total": 130
},
"expenses": {
"living": 387,
"housing": 82,
"education": 0,
"tax": 42,
"event": 0
},
"totalExpense": 511,
"netCashFlow": -381,
"assetBalance": -7805
},
{
"year": 2079,
"age": 100,
"income": {
"total": 130
},
"expenses": {
"living": 390,
"housing": 82,
"education": 0,
"tax": 42,
"event": 100
},
"totalExpense": 614,
"netCashFlow": -484,
"assetBalance": -8289
}
]
}

61
src/mock-data.json Normal file
View File

@ -0,0 +1,61 @@
{
"meta": {
"currencyUnit": "万円",
"startYear": 2027
},
"timeline": [
{ "year": 2027, "ages": { "rep": 48, "spouse": 38 }, "financials": { "income": 1083, "expenditure": 1038, "assetBalance": 852 } },
{ "year": 2028, "ages": { "rep": 49, "spouse": 39 }, "financials": { "income": 1092, "expenditure": 1038, "assetBalance": 936 } },
{ "year": 2029, "ages": { "rep": 50, "spouse": 40 }, "financials": { "income": 1099, "expenditure": 1496, "assetBalance": 605 } },
{ "year": 2030, "ages": { "rep": 51, "spouse": 41 }, "financials": { "income": 1106, "expenditure": 986, "assetBalance": 714 } },
{ "year": 2031, "ages": { "rep": 52, "spouse": 42 }, "financials": { "income": 1111, "expenditure": 992, "assetBalance": 853 } },
{ "year": 2032, "ages": { "rep": 53, "spouse": 43 }, "financials": { "income": 1116, "expenditure": 1125, "assetBalance": 911 } },
{ "year": 2033, "ages": { "rep": 54, "spouse": 44 }, "financials": { "income": 1119, "expenditure": 1111, "assetBalance": 982 } },
{ "year": 2034, "ages": { "rep": 55, "spouse": 45 }, "financials": { "income": 1122, "expenditure": 1117, "assetBalance": 1083 } },
{ "year": 2035, "ages": { "rep": 56, "spouse": 46 }, "financials": { "income": 1123, "expenditure": 1059, "assetBalance": 1176 } },
{ "year": 2036, "ages": { "rep": 57, "spouse": 47 }, "financials": { "income": 1123, "expenditure": 1162, "assetBalance": 1208 }, "events": ["Overseas Travel"] },
{ "year": 2037, "ages": { "rep": 58, "spouse": 48 }, "financials": { "income": 1122, "expenditure": 1308, "assetBalance": 1192 } },
{ "year": 2038, "ages": { "rep": 59, "spouse": 49 }, "financials": { "income": 1108, "expenditure": 1142, "assetBalance": 1229 } },
{ "year": 2039, "ages": { "rep": 60, "spouse": 50 }, "financials": { "income": 1354, "expenditure": 984, "assetBalance": 1671 }, "events": ["Retirement"] },
{ "year": 2040, "ages": { "rep": 61, "spouse": 51 }, "financials": { "income": 411, "expenditure": 867, "assetBalance": 1223 } },
{ "year": 2041, "ages": { "rep": 62, "spouse": 52 }, "financials": { "income": 360, "expenditure": 437, "assetBalance": 1146 } },
{ "year": 2042, "ages": { "rep": 63, "spouse": 53 }, "financials": { "income": 360, "expenditure": 443, "assetBalance": 1063 } },
{ "year": 2043, "ages": { "rep": 64, "spouse": 54 }, "financials": { "income": 360, "expenditure": 449, "assetBalance": 974 } },
{ "year": 2044, "ages": { "rep": 65, "spouse": 55 }, "financials": { "income": 360, "expenditure": 455, "assetBalance": 879 } },
{ "year": 2045, "ages": { "rep": 66, "spouse": 56 }, "financials": { "income": 360, "expenditure": 462, "assetBalance": 777 } },
{ "year": 2046, "ages": { "rep": 67, "spouse": 57 }, "financials": { "income": 360, "expenditure": 466, "assetBalance": 671 } },
{ "year": 2047, "ages": { "rep": 68, "spouse": 58 }, "financials": { "income": 360, "expenditure": 473, "assetBalance": 558 } },
{ "year": 2048, "ages": { "rep": 69, "spouse": 59 }, "financials": { "income": 360, "expenditure": 480, "assetBalance": 438 } },
{ "year": 2049, "ages": { "rep": 70, "spouse": 60 }, "financials": { "income": 245, "expenditure": 487, "assetBalance": 196 } },
{ "year": 2050, "ages": { "rep": 71, "spouse": 61 }, "financials": { "income": 245, "expenditure": 494, "assetBalance": -53 }, "isCritical": true, "events": ["Deficit Starts"] },
{ "year": 2051, "ages": { "rep": 72, "spouse": 62 }, "financials": { "income": 245, "expenditure": 501, "assetBalance": -309 } },
{ "year": 2052, "ages": { "rep": 73, "spouse": 63 }, "financials": { "income": 245, "expenditure": 508, "assetBalance": -572 } },
{ "year": 2053, "ages": { "rep": 74, "spouse": 64 }, "financials": { "income": 245, "expenditure": 515, "assetBalance": -842 } },
{ "year": 2054, "ages": { "rep": 75, "spouse": 65 }, "financials": { "income": 204, "expenditure": 523, "assetBalance": -1161 } },
{ "year": 2055, "ages": { "rep": 76, "spouse": 66 }, "financials": { "income": 204, "expenditure": 531, "assetBalance": -1488 } },
{ "year": 2056, "ages": { "rep": 77, "spouse": 67 }, "financials": { "income": 204, "expenditure": 539, "assetBalance": -1823 } },
{ "year": 2057, "ages": { "rep": 78, "spouse": 68 }, "financials": { "income": 204, "expenditure": 547, "assetBalance": -2166 } },
{ "year": 2058, "ages": { "rep": 79, "spouse": 69 }, "financials": { "income": 204, "expenditure": 555, "assetBalance": -2517 } },
{ "year": 2059, "ages": { "rep": 80, "spouse": 70 }, "financials": { "income": 360, "expenditure": 563, "assetBalance": -2720 } },
{ "year": 2060, "ages": { "rep": 81, "spouse": 71 }, "financials": { "income": 360, "expenditure": 571, "assetBalance": -2931 } },
{ "year": 2061, "ages": { "rep": 82, "spouse": 72 }, "financials": { "income": 360, "expenditure": 580, "assetBalance": -3151 } },
{ "year": 2062, "ages": { "rep": 83, "spouse": 73 }, "financials": { "income": 360, "expenditure": 589, "assetBalance": -3380 } },
{ "year": 2063, "ages": { "rep": 84, "spouse": 74 }, "financials": { "income": 360, "expenditure": 598, "assetBalance": -3618 } },
{ "year": 2064, "ages": { "rep": 85, "spouse": 75 }, "financials": { "income": 360, "expenditure": 607, "assetBalance": -3865 } },
{ "year": 2065, "ages": { "rep": 86, "spouse": 76 }, "financials": { "income": 360, "expenditure": 616, "assetBalance": -4121 } },
{ "year": 2066, "ages": { "rep": 87, "spouse": 77 }, "financials": { "income": 360, "expenditure": 626, "assetBalance": -4387 } },
{ "year": 2067, "ages": { "rep": 88, "spouse": 78 }, "financials": { "income": 360, "expenditure": 636, "assetBalance": -4663 } },
{ "year": 2068, "ages": { "rep": 89, "spouse": 79 }, "financials": { "income": 360, "expenditure": 645, "assetBalance": -4948 } },
{ "year": 2069, "ages": { "rep": 90, "spouse": 80 }, "financials": { "income": 360, "expenditure": 655, "assetBalance": -5243 } },
{ "year": 2070, "ages": { "rep": 91, "spouse": 81 }, "financials": { "income": 360, "expenditure": 666, "assetBalance": -5549 } },
{ "year": 2071, "ages": { "rep": 92, "spouse": 82 }, "financials": { "income": 360, "expenditure": 676, "assetBalance": -5865 } },
{ "year": 2072, "ages": { "rep": 93, "spouse": 83 }, "financials": { "income": 360, "expenditure": 687, "assetBalance": -6192 } },
{ "year": 2073, "ages": { "rep": 94, "spouse": 84 }, "financials": { "income": 360, "expenditure": 698, "assetBalance": -6530 } },
{ "year": 2074, "ages": { "rep": 95, "spouse": 85 }, "financials": { "income": 360, "expenditure": 709, "assetBalance": -6879 } },
{ "year": 2075, "ages": { "rep": 96, "spouse": 86 }, "financials": { "income": 360, "expenditure": 720, "assetBalance": -7239 } },
{ "year": 2076, "ages": { "rep": 97, "spouse": 87 }, "financials": { "income": 360, "expenditure": 732, "assetBalance": -7611 } },
{ "year": 2077, "ages": { "rep": 98, "spouse": 88 }, "financials": { "income": 360, "expenditure": 743, "assetBalance": -7994 } },
{ "year": 2078, "ages": { "rep": 99, "spouse": 89 }, "financials": { "income": 360, "expenditure": 755, "assetBalance": -8389 } },
{ "year": 2079, "ages": { "rep": 100, "spouse": 90 }, "financials": { "income": 360, "expenditure": 767, "assetBalance": -8796 }, "events": ["Simulation End"] }
]
}

29
src/types.ts Normal file
View File

@ -0,0 +1,29 @@
import { z } from "zod";
// Define the single year data structure
export interface YearlySnapshot {
year: number;
ages: {
rep: number;
spouse?: number;
};
financials: {
income: number;
expenditure: number;
assetBalance: number;
};
events?: string[];
isCritical?: boolean;
}
// Define the props for the Chart component
export interface LifePlanChartProps {
timeline?: YearlySnapshot[];
meta?: {
currencyUnit: string;
startYear: number;
};
hideHeader?: boolean;
externalDataIndex?: number;
externalFraction?: number;
}

1
tests/fixtures/.gitkeep vendored Normal file
View File

@ -0,0 +1 @@
# Keep this directory in git

48
tests/fixtures/README.md vendored Normal file
View File

@ -0,0 +1,48 @@
# Test Fixtures Directory
This directory should contain sample files for testing:
## Required Files
1. **sample.mp4** - A short (10-30 seconds) video with Japanese dialogue
- Used for testing video processing pipeline
- Should contain clear speech for transcription testing
- Recommended: Record a simple self-introduction or use royalty-free content
2. **expected_output.json** - Expected analysis output for `sample.mp4`
- Used for regression testing
- Generated by running: `python tests/test_video_cli.py -i tests/fixtures/sample.mp4 -o tests/fixtures/expected_output.json`
## Creating Test Fixtures
### Option 1: Record Your Own
```bash
# Use FFmpeg to create a test video from webcam (Windows)
ffmpeg -f dshow -i video="Integrated Camera" -t 10 sample.mp4
# Or use screen recording tools
```
### Option 2: Use Sample Content
Download a short royalty-free video with Japanese dialogue from:
- https://www.pexels.com/videos/
- https://pixabay.com/videos/
### Generate Expected Output
Once you have `sample.mp4`:
```bash
python tests/test_video_cli.py -i tests/fixtures/sample.mp4 -o tests/fixtures/expected_output.json
```
## File Structure
```
tests/
fixtures/
.gitkeep
sample.mp4 # Your test video (git-ignored)
expected_output.json # Expected result (committed to repo)
README.md # This file
```
## Note
`sample.mp4` is git-ignored (large file). Only commit `expected_output.json` for CI/CD testing.

160
tests/test_video_cli.py Normal file
View File

@ -0,0 +1,160 @@
#!/usr/bin/env python3
"""
Video Processor Test CLI
Standalone testing tool for video analysis module.
Usage:
# Test audio-first strategy
python test_video_cli.py --input sample.mp4 --strategy audio
# Compare strategies (when direct is implemented)
python test_video_cli.py --input sample.mp4 --compare
# Output to JSON file
python test_video_cli.py --input sample.mp4 --output result.json
"""
import argparse
import json
import sys
from pathlib import Path
# Add parent directory to path for imports
sys.path.insert(0, str(Path(__file__).parent.parent))
from backend.video_processor import analyze_video
from backend.video_processor.utils import check_ffmpeg_installed
def main():
parser = argparse.ArgumentParser(
description="Test video processing module independently"
)
parser.add_argument(
"--input",
"-i",
help="Path to input video file"
)
parser.add_argument(
"--strategy",
"-s",
choices=["audio", "direct"],
default="audio",
help="Analysis strategy (default: audio)"
)
parser.add_argument(
"--output",
"-o",
help="Path to output JSON file (default: print to console)"
)
parser.add_argument(
"--language",
"-l",
default="ja",
help="Video language code (default: ja)"
)
parser.add_argument(
"--compare",
action="store_true",
help="Compare both strategies (when available)"
)
parser.add_argument(
"--check-deps",
action="store_true",
help="Check if required dependencies are installed"
)
args = parser.parse_args()
# Check dependencies (can run without --input)
if args.check_deps:
print("🔍 Checking dependencies...")
if check_ffmpeg_installed():
print("✅ FFmpeg is installed")
else:
print("❌ FFmpeg is NOT installed")
print(" Install: https://ffmpeg.org/download.html")
if not args.check_deps:
sys.exit(1)
try:
import openai
print("✅ OpenAI library is installed")
except ImportError:
print("❌ OpenAI library is NOT installed")
print(" Install: pip install openai")
if not args.check_deps:
sys.exit(1)
if args.check_deps:
return
# Validate input file is provided for analysis
if not args.input:
print("❌ Error: --input is required for video analysis")
print(" Use --check-deps to only check dependencies")
sys.exit(1)
if not Path(args.input).exists():
print(f"❌ Error: Video file not found: {args.input}")
sys.exit(1)
try:
# Run analysis
print(f"\n{'='*60}")
print(f"🎬 Video Processor Test CLI")
print(f"{'='*60}")
print(f"Input: {args.input}")
print(f"Strategy: {args.strategy}")
print(f"Language: {args.language}\n")
config = {
"language": args.language
}
result = analyze_video(
video_path=args.input,
strategy=args.strategy,
config=config
)
# Convert to dict
result_dict = result.to_dict()
# Output results
if args.output:
with open(args.output, "w", encoding="utf-8") as f:
json.dump(result_dict, f, ensure_ascii=False, indent=2)
print(f"\n✅ Results saved to {args.output}")
else:
print(f"\n{'='*60}")
print("📊 Analysis Results")
print(f"{'='*60}\n")
print(json.dumps(result_dict, ensure_ascii=False, indent=2))
# Print summary
print(f"\n{'='*60}")
print("📈 Summary")
print(f"{'='*60}")
print(f"Duration: {result.duration:.2f}s")
print(f"Segments: {len(result.segments)}")
print(f"Key Moments: {len(result.key_moments)}")
print(f"High Importance: {len(result.get_high_importance_segments())}")
# Show first few segments
if result.segments:
print(f"\n🎯 First 3 Segments:")
for i, seg in enumerate(result.segments[:3], 1):
print(f"\n {i}. [{seg.start_time:.1f}s - {seg.end_time:.1f}s]")
print(f" {seg.transcript}")
except Exception as e:
print(f"\n❌ Error: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()

View File

@ -0,0 +1,142 @@
"""
Unit tests for video_processor module
"""
import os
import pytest
from pathlib import Path
# Add project root to path
import sys
sys.path.insert(0, str(Path(__file__).parent.parent))
from backend.video_processor import analyze_video
from backend.video_processor.utils import (
check_ffmpeg_installed,
get_video_duration,
extract_audio
)
from backend.video_processor.schemas import (
VideoSegment,
KeyMoment,
VideoAnalysisResult
)
class TestSchemas:
"""Test data structure definitions."""
def test_video_segment_creation(self):
seg = VideoSegment(
start_time=0.0,
end_time=5.5,
description="Test segment",
transcript="こんにちは",
importance="high",
tags=["greeting"]
)
assert seg.duration == 5.5
assert seg.importance == "high"
seg_dict = seg.to_dict()
assert seg_dict["startTime"] == 0.0
assert seg_dict["transcript"] == "こんにちは"
def test_key_moment_creation(self):
moment = KeyMoment(
timestamp=10.5,
type="quote",
content="重要な発言",
context="冒頭の挨拶"
)
moment_dict = moment.to_dict()
assert moment_dict["timestamp"] == 10.5
assert moment_dict["type"] == "quote"
def test_video_analysis_result(self):
result = VideoAnalysisResult(
source_video="test.mp4",
duration=60.0,
strategy="audio-first",
segments=[
VideoSegment(0, 10, "Seg 1", importance="high"),
VideoSegment(10, 20, "Seg 2", importance="low"),
],
key_moments=[
KeyMoment(5.0, "quote", "Test")
]
)
assert len(result.get_high_importance_segments()) == 1
assert len(result.get_quotes()) == 1
result_dict = result.to_dict()
assert result_dict["duration"] == 60.0
assert len(result_dict["segments"]) == 2
class TestUtils:
"""Test utility functions."""
def test_check_ffmpeg_installed(self):
# This should pass if FFmpeg is in PATH
is_installed = check_ffmpeg_installed()
assert isinstance(is_installed, bool)
@pytest.mark.skipif(
not os.path.exists("tests/fixtures/sample.mp4"),
reason="Sample video not found"
)
def test_get_video_duration(self):
duration = get_video_duration("tests/fixtures/sample.mp4")
assert duration > 0
@pytest.mark.skipif(
not os.path.exists("tests/fixtures/sample.mp4"),
reason="Sample video not found"
)
def test_extract_audio(self):
audio_path = extract_audio(
"tests/fixtures/sample.mp4",
output_path="tests/fixtures/test_audio.wav"
)
assert os.path.exists(audio_path)
# Cleanup
if os.path.exists(audio_path):
os.remove(audio_path)
@pytest.mark.skipif(
not os.getenv("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set"
)
@pytest.mark.skipif(
not os.path.exists("tests/fixtures/sample.mp4"),
reason="Sample video not found"
)
class TestAudioFirstStrategy:
"""Test audio-first strategy (requires API key and sample video)."""
def test_analyze_video(self):
result = analyze_video(
video_path="tests/fixtures/sample.mp4",
strategy="audio",
config={"language": "ja"}
)
assert result.strategy == "audio-first"
assert result.duration > 0
assert len(result.segments) > 0
# Check that segments have transcripts
for seg in result.segments:
assert seg.transcript is not None
assert len(seg.transcript) > 0
if __name__ == "__main__":
pytest.main([__file__, "-v"])

View File

@ -9,7 +9,9 @@
"esModuleInterop": true, "esModuleInterop": true,
"skipLibCheck": true, "skipLibCheck": true,
"forceConsistentCasingInFileNames": true, "forceConsistentCasingInFileNames": true,
"noUnusedLocals": true "noUnusedLocals": false,
"resolveJsonModule": true,
"moduleResolution": "node"
}, },
"exclude": ["remotion.config.ts"] "exclude": ["remotion.config.ts"]
} }

135
verify_structure.py Normal file
View File

@ -0,0 +1,135 @@
#!/usr/bin/env python3
"""
Quick Test Script - Verify Project Structure
Run this to verify the restructuring was successful.
"""
import sys
from pathlib import Path
def check_structure():
"""Verify all required directories and files exist"""
root = Path(__file__).parent
print("🔍 Checking project structure...\n")
required_items = {
"Directories": [
"assets/0_input",
"assets/1_processed",
"assets/2_output",
"backend",
"frontend/src",
"frontend/src/components",
],
"Python Files": [
"main.py",
"requirements.txt",
"backend/__init__.py",
"backend/extractor.py",
"backend/scripter.py",
"backend/media_fetcher.py",
"backend/data_builder.py",
],
"TypeScript Files": [
"frontend/src/render-props-types.ts",
],
"Config Files": [
".env.example",
"remotion.config.ts",
"package.json",
],
"Documentation": [
"PROJECT_README.md",
"MIGRATION_GUIDE.md",
"RESTRUCTURING_SUMMARY.md",
],
}
all_ok = True
for category, items in required_items.items():
print(f"📁 {category}:")
for item in items:
path = root / item
exists = path.exists()
symbol = "✅" if exists else "❌"
print(f" {symbol} {item}")
if not exists:
all_ok = False
print()
if all_ok:
print("✅ All required files and directories exist!\n")
print("📋 Next Steps:")
print("1. Review RESTRUCTURING_SUMMARY.md")
print("2. Follow MIGRATION_GUIDE.md to move existing files")
print("3. Install dependencies: pip install -r requirements.txt")
print("4. Configure .env: cp .env.example .env")
print("5. Test backend: python main.py --pdf test.pdf --dry-run")
else:
print("❌ Some files are missing. Please check the structure.\n")
sys.exit(1)
def test_imports():
"""Test that backend modules can be imported"""
print("\n🔬 Testing Python imports...\n")
try:
import backend
print("✅ backend package")
except ImportError as e:
print(f"❌ backend package: {e}")
return False
modules = ["extractor", "scripter", "media_fetcher", "data_builder"]
for module_name in modules:
try:
__import__(f"backend.{module_name}")
print(f"✅ backend.{module_name}")
except ImportError as e:
print(f"❌ backend.{module_name}: {e}")
return False
print("\n✅ All imports successful!")
return True
def test_cli():
"""Test main.py CLI"""
print("\n🚀 Testing main.py CLI...\n")
import subprocess
try:
result = subprocess.run(
[sys.executable, "main.py", "--help"],
capture_output=True,
text=True,
timeout=5
)
if result.returncode == 0:
print("✅ main.py --help works")
return True
else:
print(f"❌ main.py --help failed: {result.stderr}")
return False
except Exception as e:
print(f"❌ CLI test failed: {e}")
return False
if __name__ == "__main__":
print("=" * 60)
print("Auto Life Plan Video Generator - Structure Verification")
print("=" * 60)
print()
check_structure()
if test_imports():
test_cli()
print("\n" + "=" * 60)
print("✅ Verification complete!")
print("=" * 60)