12 KiB

Raw Blame History

Project Overview: Life Plan Video Generator (PoC)

1. Ultimate Goal

To build an automated pipeline that transforms a static Life Plan Report (PDF) and raw Interview Footage into a 3-5 minute dynamic explanatory video (MP4). The system is designed to help insurance consultants visualize complex financial data (e.g., asset depletion, inflation risks) for clients, creating an emotional and logical narrative to support decision-making.

2. Core Architecture

The system follows a "Hybrid Local/Cloud" architecture to handle sensitive data and heavy video processing efficiently.

Backend (Python / Logic Layer)

Role: The "Brain". Handles data extraction, logic calculation, and asset preparation.
Tech Stack: Python 3.10+, LangChain, Google Gemini 1.5 Pro (via API).
Key Responsibilities:
1. Ingestion: Parse complex Japanese PDFs (Sony Life Reports) using Gemini 1.5 Pro.
2. Structured Output: Convert unstructured text into strict JSON (Zod Schema compliant).
3. Simulation Engine: Calculate future asset projections (inflation, compound interest on deficits) to fill missing data points (Linear Interpolation).
4. Asset Prep: Use FFmpeg (stream copy) to slice raw interview videos based on timestamps.

Frontend / Rendering (Remotion / Visual Layer)

Role: The "Renderer". Converts JSON data into video frames.
Tech Stack: TypeScript, React, Remotion, Recharts, Zod.
Key Responsibilities:
1. Programmatic Video: Render UI components as video frames.
2. Dynamic Charts: Visualize "Asset Depletion" and "Sufficiency Rates" using Recharts.
3. Frame Synchronization: Drive animations strictly via useCurrentFrame() (no CSS transitions).

3. Remotion Specifics (Copilot Instructions)

What does Remotion do in this project?

Remotion is NOT just a video editor; it is a code-based rendering engine. It treats the video timeline as a function of state f(frame) = UI.

Critical Implementation Rules

No Internal Animations:
- When using Recharts, always set isAnimationActive={false}.
- Reason: Remotion renders frame-by-frame. Internal library animations rely on real-time timers, which break during rendering, causing flickering or static frames.
- Solution: Drive data slicing using interpolate(frame, [start, end], [0, dataLength]).
Asset Handling:
- Do NOT import huge video files directly into React. Use URL references or local paths managed by the Python backend.
- Use <Video startFrom={x} durationInFrames={y} /> for virtual cutting.
The "3-Layer X-Axis" Component:
- The Life Plan chart requires a custom X-Axis showing: Year, Representative Age, and Spouse Age vertically stacked.
- Use Recharts' <XAxis tick={<CustomTick />} /> pattern.

4. Data Structures (The "Contract")

The bridge between Python and Remotion is src/mock-data.json.

Core Schema (`YearlySnapshot`)

TypeScript

interface YearlySnapshot {
  year: number;           // 西暦 (e.g., 2027)
  ages: {
    rep: number;          // 世帯主年齢
    spouse?: number;      // 配偶者年齢
  };
  financials: {
    income: number;       // 年間収入 (HUD Display)
    expenditure: number;  // 年間支出 (HUD Display)
    assetBalance: number; // 資産残高 (Y-Axis, Red if < 0)
  };
  events?: string[];      // Life Events (e.g., "Retirement", "Buy House")
  isCritical?: boolean;   // Trigger for "Red Alert" visual effect
}

5. Business Logic & Domain Knowledge

The "400 Million JPY" Logic

Context: The report often predicts a massive deficit (e.g., -400M JPY) at age 100, even if the deficit at age 70 is small (-1.7M JPY).
Why: The simulation engine applies a high borrowing interest rate (e.g., 15%) once assets dip below zero. This creates a "compound interest debt trap."
Visual Requirement: The chart must turn RED immediately when assetBalance < 0 and grow exponentially downwards to visualize this risk.

The "Sufficiency Rate" (充足率)

Formula: (Current Assets + Insurance) / Future Needs * 100%.
Visual: Displayed as a Donut Chart.
- < 100%: Danger (Red/Orange).
- = 100%: Safe (Green/Blue).

Hidden Costs

Housing: Even after loan repayment, maintenance costs & taxes often rise to ~1.5M JPY/year due to inflation (2% YoY).
Life Expectancy: Simulations default to Age 100 to account for longevity risk.

6. Directory Structure

This Repository (Remotion Frontend)

/src
  index.ts              # Entry point - registers RemotionRoot
  Root.tsx              # Composition registry - defines video specs
  Composition.tsx       # Main composition component (currently placeholder)
  LifePlanChart.tsx     # Core animated chart component
  mock-data.json        # Sample data matching YearlySnapshot schema
  types.ts              # (MISSING) TypeScript interfaces for data contract

/public                 # Static assets (if needed)
remotion.config.ts      # Global Remotion settings (frame format, overwrite)
package.json            # Dependencies: Remotion 4.0.400, Recharts, React 19

Full System Context (Python backend not in this repo):

/0_input_raw            # User uploads (PDF, MP4) - handled by Python
/1_processed_data       # JSONs extracted by Gemini - consumed by this repo
/scripts                # Python orchestration (extraction, simulation)

7. Essential Development Workflows

Local Development

npm run dev          # Opens Remotion Studio (interactive preview)
npm run lint         # ESLint + TypeScript type check
npx remotion render  # Render final MP4

Project Standards

Frame Rate: All compositions use 60 fps for smooth animations
Duration: Typically 15 seconds (900 frames at 60fps)
Resolution: 1920x1080 (Full HD)

Debugging Animation Issues

Use Remotion Studio's timeline scrubber to inspect specific frames
Check useCurrentFrame() values with console.log (visible in Studio)
Verify interpolate() input/output ranges match data array bounds
If chart appears static: confirm isAnimationActive={false} on Recharts components

Data Integration from Python Backend

Python scripts generate JSON files → Place in src/mock-data.json or import dynamically
Validate against YearlySnapshot schema before rendering
Critical: Asset balance can go deeply negative (-400M JPY) due to compound interest simulation

8. Remotion-Specific Patterns in This Project

Frame-Driven Animation Pattern

const frame = useCurrentFrame();
const { durationInFrames } = useVideoConfig();

// Map frame to data index (with end buffer to prevent abrupt cutoff)
const dataEndIndex = Math.floor(
  interpolate(frame, [0, durationInFrames - 30], [0, timeline.length - 1], {
    extrapolateRight: "clamp", // Prevent out-of-bounds access
  }),
);

const visibleData = timeline.slice(0, dataEndIndex + 1);

Custom Recharts Components

3-Row X-Axis: Implemented via <XAxis tick={<CustomXAxisTick />} />
- Row 1: Year (西暦)
- Row 2: Representative age (bold)
- Row 3: Spouse age (parentheses, or "-")

Bi-Directional Gradient: Uses linearGradient with dynamic offset calculation

const gradientOffset = () => {
  const dataMax = Math.max(...timeline.map((i) => i.financials.assetBalance));
  const dataMin = Math.min(...timeline.map((i) => i.financials.assetBalance));
  if (dataMax <= 0) return 0;
  if (dataMin >= 0) return 1;
  return dataMax / (dataMax - dataMin);
};

Visual Convention Rules

Colors: Green income (#38A169), Red expenses/deficit (#E53E3E), Blue positive balance (#4299E1)
Currency: Large values formatted as 億 (÷10,000), e.g., -400億 for -4,000,000 万円
HUD Panel: Shows "current frame" snapshot (not entire dataset aggregation)

9. Common Pitfalls & Solutions

Issue	Why It Happens	Solution
Chart animation flickers/jumps	Recharts internal animations conflict with frame rendering	Set `isAnimationActive={false}` on all Recharts components
Data extends beyond visible range	`interpolate()` extrapolation defaults allow overflow	Use `extrapolateRight: "clamp"` option
Type errors on `YearlySnapshot`	`types.ts` file doesn't exist yet	Create `src/types.ts` with interfaces matching mock-data.json
Root.tsx renders blank video	Uses placeholder `MyComposition` instead of `LifePlanChart`	Update `<Composition component={LifePlanChart} />`
Video cuts off abruptly	No buffer at end of timeline	Use `[0, durationInFrames - 30]` instead of full duration

10. Repeated Mistakes (AI Assistant Learning)

🚨 CRITICAL: Gradient Offset Calculation

MISTAKE (Repeated Multiple Times):

// ❌ WRONG - Uses full timeline data
const gradientOffset = React.useMemo(() => {
  const dataMax = Math.max(...timeline.map((i) => i.financials.assetBalance));
  const dataMin = Math.min(...timeline.map((i) => i.financials.assetBalance));
  return dataMax / (dataMax - dataMin);
}, [timeline]);

PROBLEM:

When timeline contains future negative values (e.g., -400M at age 100)
But current animation only shows positive years (ages 30-50)
The gradient calculates offset based on full range [-400M to +500M]
Result: Zero-line appears above the visible positive data
Visual impact: Blue area appears in wrong location

CORRECT SOLUTION:

// ✅ CORRECT - Uses only visible data
const gradientOffset = React.useMemo(() => {
  const dataMax = Math.max(
    ...visibleData.map((i) => i.financials.assetBalance),
  );
  const dataMin = Math.min(
    ...visibleData.map((i) => i.financials.assetBalance),
  );

  if (dataMin >= 0) return 1; // All positive: fully blue
  if (dataMax <= 0) return 0; // All negative: fully red

  return dataMax / (dataMax - dataMin); // Mixed: calculate zero crossing
}, [visibleData]); // ← Depends on visibleData, NOT timeline

WHY THIS MATTERS:

The gradient must match what's currently visible on screen
Animation progressively reveals data over time
Early frames show only positive balances → should be fully blue
Later frames reveal deficit → gradient transitions to red
Using timeline breaks this progressive revelation

RULE FOR AI:

When calculating visual properties (gradients, colors, domains) for animated charts, ALWAYS use visibleData (the currently rendered slice), NOT the full timeline dataset, unless you explicitly need stable axis ranges for Y-axis domains.

11. Next Steps / Known TODOs

Create src/types.ts with LifePlanChartProps, YearlySnapshot, LifePlanMeta interfaces
Wire LifePlanChart into Root.tsx composition
Add Zod schema validation for runtime data checking
Implement "Sufficiency Rate" donut chart component
Add video interview footage integration using <Video> components
Create transition sequences between chart and interview clips

12 KiB Raw Blame History