276 lines
12 KiB
Markdown
276 lines
12 KiB
Markdown
# Project Overview: Life Plan Video Generator (PoC)
|
|
|
|
## 1\. Ultimate Goal
|
|
|
|
To build an automated pipeline that transforms a static **Life Plan Report (PDF)** and raw **Interview Footage** into a **3-5 minute dynamic explanatory video (MP4)**. The system is designed to help insurance consultants visualize complex financial data (e.g., asset depletion, inflation risks) for clients, creating an emotional and logical narrative to support decision-making.
|
|
|
|
## 2\. Core Architecture
|
|
|
|
The system follows a "Hybrid Local/Cloud" architecture to handle sensitive data and heavy video processing efficiently.
|
|
|
|
### **Backend (Python / Logic Layer)**
|
|
|
|
- **Role:** The "Brain". Handles data extraction, logic calculation, and asset preparation.
|
|
- **Tech Stack:** Python 3.10+, LangChain, Google Gemini 1.5 Pro (via API).
|
|
- **Key Responsibilities:**
|
|
1. **Ingestion:** Parse complex Japanese PDFs (Sony Life Reports) using Gemini 1.5 Pro.
|
|
2. **Structured Output:** Convert unstructured text into strict JSON (Zod Schema compliant).
|
|
3. **Simulation Engine:** Calculate future asset projections (inflation, compound interest on deficits) to fill missing data points (Linear Interpolation).
|
|
4. **Asset Prep:** Use `FFmpeg` (stream copy) to slice raw interview videos based on timestamps.
|
|
|
|
### **Frontend / Rendering (Remotion / Visual Layer)**
|
|
|
|
- **Role:** The "Renderer". Converts JSON data into video frames.
|
|
- **Tech Stack:** TypeScript, React, Remotion, Recharts, Zod.
|
|
- **Key Responsibilities:**
|
|
1. **Programmatic Video:** Render UI components as video frames.
|
|
2. **Dynamic Charts:** Visualize "Asset Depletion" and "Sufficiency Rates" using `Recharts`.
|
|
3. **Frame Synchronization:** Drive animations strictly via `useCurrentFrame()` (no CSS transitions).
|
|
|
|
---
|
|
|
|
## 3\. Remotion Specifics (Copilot Instructions)
|
|
|
|
### **What does Remotion do in this project?**
|
|
|
|
Remotion is **NOT** just a video editor; it is a **code-based rendering engine**. It treats the video timeline as a function of state `f(frame) = UI`.
|
|
|
|
### **Critical Implementation Rules**
|
|
|
|
1. **No Internal Animations:**
|
|
- When using `Recharts`, always set `isAnimationActive={false}`.
|
|
- **Reason:** Remotion renders frame-by-frame. Internal library animations rely on real-time timers, which break during rendering, causing flickering or static frames.
|
|
- **Solution:** Drive data slicing using `interpolate(frame, [start, end], [0, dataLength])`.
|
|
|
|
2. **Asset Handling:**
|
|
- Do **NOT** import huge video files directly into React. Use URL references or local paths managed by the Python backend.
|
|
- Use `<Video startFrom={x} durationInFrames={y} />` for virtual cutting.
|
|
|
|
3. **The "3-Layer X-Axis" Component:**
|
|
- The Life Plan chart requires a custom X-Axis showing: `Year`, `Representative Age`, and `Spouse Age` vertically stacked.
|
|
- Use Recharts' `<XAxis tick={<CustomTick />} />` pattern.
|
|
|
|
---
|
|
|
|
## 4\. Data Structures (The "Contract")
|
|
|
|
The bridge between Python and Remotion is `src/mock-data.json`.
|
|
|
|
### **Core Schema (`YearlySnapshot`)**
|
|
|
|
TypeScript
|
|
|
|
interface YearlySnapshot {
|
|
year: number; // 西暦 (e.g., 2027)
|
|
ages: {
|
|
rep: number; // 世帯主年齢
|
|
spouse?: number; // 配偶者年齢
|
|
};
|
|
financials: {
|
|
income: number; // 年間収入 (HUD Display)
|
|
expenditure: number; // 年間支出 (HUD Display)
|
|
assetBalance: number; // 資産残高 (Y-Axis, Red if < 0)
|
|
};
|
|
events?: string[]; // Life Events (e.g., "Retirement", "Buy House")
|
|
isCritical?: boolean; // Trigger for "Red Alert" visual effect
|
|
}
|
|
|
|
---
|
|
|
|
## 5\. Business Logic & Domain Knowledge
|
|
|
|
### **The "400 Million JPY" Logic**
|
|
|
|
- **Context:** The report often predicts a massive deficit (e.g., -400M JPY) at age 100, even if the deficit at age 70 is small (-1.7M JPY).
|
|
- **Why:** The simulation engine applies a **high borrowing interest rate** (e.g., 15%) once assets dip below zero. This creates a "compound interest debt trap."
|
|
- **Visual Requirement:** The chart must turn **RED** immediately when `assetBalance < 0` and grow exponentially downwards to visualize this risk.
|
|
|
|
### **The "Sufficiency Rate" (充足率)**
|
|
|
|
- **Formula:** `(Current Assets + Insurance) / Future Needs * 100%`.
|
|
- **Visual:** Displayed as a Donut Chart.
|
|
- < 100%: **Danger** (Red/Orange).
|
|
- > \= 100%: **Safe** (Green/Blue).
|
|
|
|
### **Hidden Costs**
|
|
|
|
- **Housing:** Even after loan repayment, maintenance costs & taxes often rise to ~1.5M JPY/year due to inflation (2% YoY).
|
|
- **Life Expectancy:** Simulations default to **Age 100** to account for longevity risk.
|
|
|
|
---
|
|
|
|
## 6\. Directory Structure
|
|
|
|
**This Repository (Remotion Frontend)**
|
|
|
|
/src
|
|
index.ts # Entry point - registers RemotionRoot
|
|
Root.tsx # Composition registry - defines video specs
|
|
Composition.tsx # Main composition component (currently placeholder)
|
|
LifePlanChart.tsx # Core animated chart component
|
|
mock-data.json # Sample data matching YearlySnapshot schema
|
|
types.ts # (MISSING) TypeScript interfaces for data contract
|
|
|
|
/public # Static assets (if needed)
|
|
remotion.config.ts # Global Remotion settings (frame format, overwrite)
|
|
package.json # Dependencies: Remotion 4.0.400, Recharts, React 19
|
|
|
|
**Full System Context** (Python backend not in this repo):
|
|
|
|
/0_input_raw # User uploads (PDF, MP4) - handled by Python
|
|
/1_processed_data # JSONs extracted by Gemini - consumed by this repo
|
|
/scripts # Python orchestration (extraction, simulation)
|
|
|
|
---
|
|
|
|
## 7\. Essential Development Workflows
|
|
|
|
### **Local Development**
|
|
|
|
```bash
|
|
npm run dev # Opens Remotion Studio (interactive preview)
|
|
npm run lint # ESLint + TypeScript type check
|
|
npx remotion render # Render final MP4
|
|
```
|
|
|
|
### **Project Standards**
|
|
|
|
- **Frame Rate**: All compositions use **60 fps** for smooth animations
|
|
- **Duration**: Typically 15 seconds (900 frames at 60fps)
|
|
- **Resolution**: 1920x1080 (Full HD)
|
|
|
|
### **Debugging Animation Issues**
|
|
|
|
1. Use Remotion Studio's timeline scrubber to inspect specific frames
|
|
2. Check `useCurrentFrame()` values with console.log (visible in Studio)
|
|
3. Verify `interpolate()` input/output ranges match data array bounds
|
|
4. If chart appears static: confirm `isAnimationActive={false}` on Recharts components
|
|
|
|
### **Data Integration from Python Backend**
|
|
|
|
- Python scripts generate JSON files → Place in `src/mock-data.json` or import dynamically
|
|
- Validate against `YearlySnapshot` schema before rendering
|
|
- **Critical**: Asset balance can go deeply negative (-400M JPY) due to compound interest simulation
|
|
|
|
---
|
|
|
|
## 8\. Remotion-Specific Patterns in This Project
|
|
|
|
### **Frame-Driven Animation Pattern**
|
|
|
|
```typescript
|
|
const frame = useCurrentFrame();
|
|
const { durationInFrames } = useVideoConfig();
|
|
|
|
// Map frame to data index (with end buffer to prevent abrupt cutoff)
|
|
const dataEndIndex = Math.floor(
|
|
interpolate(frame, [0, durationInFrames - 30], [0, timeline.length - 1], {
|
|
extrapolateRight: "clamp", // Prevent out-of-bounds access
|
|
}),
|
|
);
|
|
|
|
const visibleData = timeline.slice(0, dataEndIndex + 1);
|
|
```
|
|
|
|
### **Custom Recharts Components**
|
|
|
|
- **3-Row X-Axis**: Implemented via `<XAxis tick={<CustomXAxisTick />} />`
|
|
- Row 1: Year (西暦)
|
|
- Row 2: Representative age (bold)
|
|
- Row 3: Spouse age (parentheses, or "-")
|
|
- **Bi-Directional Gradient**: Uses `linearGradient` with dynamic offset calculation
|
|
```typescript
|
|
const gradientOffset = () => {
|
|
const dataMax = Math.max(...timeline.map((i) => i.financials.assetBalance));
|
|
const dataMin = Math.min(...timeline.map((i) => i.financials.assetBalance));
|
|
if (dataMax <= 0) return 0;
|
|
if (dataMin >= 0) return 1;
|
|
return dataMax / (dataMax - dataMin);
|
|
};
|
|
```
|
|
|
|
### **Visual Convention Rules**
|
|
|
|
- **Colors**: Green income (#38A169), Red expenses/deficit (#E53E3E), Blue positive balance (#4299E1)
|
|
- **Currency**: Large values formatted as `億` (÷10,000), e.g., `-400億` for -4,000,000 万円
|
|
- **HUD Panel**: Shows "current frame" snapshot (not entire dataset aggregation)
|
|
|
|
---
|
|
|
|
## 9\. Common Pitfalls & Solutions
|
|
|
|
| Issue | Why It Happens | Solution |
|
|
| --------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------- |
|
|
| Chart animation flickers/jumps | Recharts internal animations conflict with frame rendering | Set `isAnimationActive={false}` on all Recharts components |
|
|
| Data extends beyond visible range | `interpolate()` extrapolation defaults allow overflow | Use `extrapolateRight: "clamp"` option |
|
|
| Type errors on `YearlySnapshot` | `types.ts` file doesn't exist yet | Create `src/types.ts` with interfaces matching mock-data.json |
|
|
| Root.tsx renders blank video | Uses placeholder `MyComposition` instead of `LifePlanChart` | Update `<Composition component={LifePlanChart} />` |
|
|
| Video cuts off abruptly | No buffer at end of timeline | Use `[0, durationInFrames - 30]` instead of full duration |
|
|
|
|
---
|
|
|
|
## 10\. Repeated Mistakes (AI Assistant Learning)
|
|
|
|
### 🚨 CRITICAL: Gradient Offset Calculation
|
|
|
|
**MISTAKE (Repeated Multiple Times):**
|
|
|
|
```typescript
|
|
// ❌ WRONG - Uses full timeline data
|
|
const gradientOffset = React.useMemo(() => {
|
|
const dataMax = Math.max(...timeline.map((i) => i.financials.assetBalance));
|
|
const dataMin = Math.min(...timeline.map((i) => i.financials.assetBalance));
|
|
return dataMax / (dataMax - dataMin);
|
|
}, [timeline]);
|
|
```
|
|
|
|
**PROBLEM:**
|
|
|
|
- When timeline contains future negative values (e.g., -400M at age 100)
|
|
- But current animation only shows positive years (ages 30-50)
|
|
- The gradient calculates offset based on full range [-400M to +500M]
|
|
- Result: Zero-line appears **above** the visible positive data
|
|
- Visual impact: Blue area appears in wrong location
|
|
|
|
**CORRECT SOLUTION:**
|
|
|
|
```typescript
|
|
// ✅ CORRECT - Uses only visible data
|
|
const gradientOffset = React.useMemo(() => {
|
|
const dataMax = Math.max(
|
|
...visibleData.map((i) => i.financials.assetBalance),
|
|
);
|
|
const dataMin = Math.min(
|
|
...visibleData.map((i) => i.financials.assetBalance),
|
|
);
|
|
|
|
if (dataMin >= 0) return 1; // All positive: fully blue
|
|
if (dataMax <= 0) return 0; // All negative: fully red
|
|
|
|
return dataMax / (dataMax - dataMin); // Mixed: calculate zero crossing
|
|
}, [visibleData]); // ← Depends on visibleData, NOT timeline
|
|
```
|
|
|
|
**WHY THIS MATTERS:**
|
|
|
|
- The gradient must match what's **currently visible** on screen
|
|
- Animation progressively reveals data over time
|
|
- Early frames show only positive balances → should be fully blue
|
|
- Later frames reveal deficit → gradient transitions to red
|
|
- Using `timeline` breaks this progressive revelation
|
|
|
|
**RULE FOR AI:**
|
|
|
|
> When calculating visual properties (gradients, colors, domains) for animated charts, ALWAYS use `visibleData` (the currently rendered slice), NOT the full `timeline` dataset, unless you explicitly need stable axis ranges for Y-axis domains.
|
|
|
|
---
|
|
|
|
## 11\. Next Steps / Known TODOs
|
|
|
|
- [ ] Create `src/types.ts` with `LifePlanChartProps`, `YearlySnapshot`, `LifePlanMeta` interfaces
|
|
- [ ] Wire `LifePlanChart` into `Root.tsx` composition
|
|
- [ ] Add Zod schema validation for runtime data checking
|
|
- [ ] Implement "Sufficiency Rate" donut chart component
|
|
- [ ] Add video interview footage integration using `<Video>` components
|
|
- [ ] Create transition sequences between chart and interview clips
|