# OpenScreen Administrator & Developer Guide

## 1. Architecture Overview

OpenScreen is a desktop screen recording and video editing application built on Electron with a React/TypeScript renderer. The application follows a two-process architecture: the Electron main process manages windows, system tray, file I/O, and OS permissions, while the renderer process handles the UI, recording logic, and video export pipeline.

### Architecture Diagram

```text
┌──────────────────────────────────────────────────────┐
│                   Electron Main Process              │
│  ┌──────────┐  ┌──────────┐  ┌────────────────────┐ │
│  │ main.ts  │  │windows.ts│  │ ipc/handlers.ts    │ │
│  │ (app     │  │ (window  │  │ (file I/O, source  │ │
│  │  entry,  │  │  factory │  │  selection, cursor │ │
│  │  tray,   │  │  + HUD,  │  │  telemetry, project│ │
│  │  menus)  │  │  editor, │  │  persistence)      │ │
│  │          │  │  picker) │  │                    │ │
│  └──────────┘  └──────────┘  └────────────────────┘ │
│       │              │               │               │
│       └──────────────┴───────────────┘               │
│                      │ IPC (contextBridge)            │
│               ┌──────┴──────┐                        │
│               │ preload.ts  │                        │
│               └──────┬──────┘                        │
└──────────────────────┼───────────────────────────────┘
                       │
┌──────────────────────┼───────────────────────────────┐
│            Renderer Process (Vite + React)           │
│  ┌─────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │ App.tsx  │  │ LaunchWindow │  │  VideoEditor   │  │
│  │ (router) │  │ (HUD overlay)│  │  (full editor) │  │
│  └─────────┘  └──────────────┘  └───────┬────────┘  │
│                                         │            │
│  ┌──────────────────────────────────────┤            │
│  │  hooks/          contexts/     lib/  │            │
│  │  useScreen       I18nContext   shortcuts          │
│  │  Recorder        Shortcuts     userPrefs          │
│  │  useCamera       Context       customFonts        │
│  │  Devices                       compositeLayout    │
│  │  useEditor                     recordingSession   │
│  │  History                                          │
│  └──────────────────────────────────────┤            │
│                                         │            │
│  ┌──────────────────────────────────────┘            │
│  │  lib/exporter/                                    │
│  │  VideoExporter → FrameRenderer → VideoMuxer       │
│  │  GifExporter                                      │
│  │  StreamingVideoDecoder / VideoFileDecoder         │
│  │  AudioProcessor                                   │
│  └───────────────────────────────────────────────────│
└──────────────────────────────────────────────────────┘
```

### Component Inventory

| Component | Path | Role |
|---|---|---|
| **Electron Main** | `electron/main.ts` | App lifecycle, tray, menus, window management |
| **IPC Handlers** | `electron/ipc/handlers.ts` | All main↔renderer communication (28+ handlers) |
| **Window Factory** | `electron/windows.ts` | HUD overlay, editor, source selector window creation |
| **Preload Bridge** | `electron/preload.ts` | `contextBridge.exposeInMainWorld("electronAPI", ...)` |
| **Main i18n** | `electron/i18n.ts` | Main-process translation (menus, dialogs) |
| **App Router** | `src/App.tsx` | Routes by `windowType` query param |
| **LaunchWindow** | `src/components/launch/LaunchWindow.tsx` | HUD overlay recording controls |
| **SourceSelector** | `src/components/launch/SourceSelector.tsx` | Screen/window source picker |
| **VideoEditor** | `src/components/video-editor/VideoEditor.tsx` | Full editing interface |
| **Timeline** | `src/components/video-editor/timeline/` | Timeline rendering and interaction |
| **Export Pipeline** | `src/lib/exporter/` | Video/GIF encoding and muxing |
| **Recording Hook** | `src/hooks/useScreenRecorder.ts` | MediaRecorder management |
| **Editor History** | `src/hooks/useEditorHistory.ts` | Undo/redo state machine |
| **i18n System** | `src/i18n/` | Translation loading and interpolation |
| **UI Components** | `src/components/ui/` | shadcn/ui component library (Radix-based) |

### Technology Stack

| Layer | Technology |
|---|---|
| **Runtime** | Electron 39 (Chromium) |
| **Frontend** | React 18, TypeScript 5.2 |
| **Build** | Vite 5 + vite-plugin-electron |
| **Styling** | Tailwind CSS 3.4, CSS Modules |
| **UI Library** | shadcn/ui (Radix UI primitives, Lucide icons) |
| **Video Processing** | WebCodecs, MediaRecorder, PixiJS 8, mp4box |
| **Animation** | GSAP 3.13, Motion (Framer Motion) |
| **GIF** | gif.js |
| **Audio** | Web Audio API |
| **WebM Fix** | fix-webm-duration, @fix-webm-duration/fix |
| **Timeline** | dnd-timeline |
| **Testing** | Vitest, Playwright, Testing Library |
| **Linting** | Biome 2.3 |
| **Package** | electron-builder |

> [!NOTE]
> The renderer-side video pipeline uses PixiJS for GPU-accelerated frame compositing with filters (drop shadow, blur). The `web-demuxer` WASM module handles container demuxing in the browser context.

### Data Flow: Recording to Export

```text
1. User selects source → desktopCapturer.getSources() → thumbnail previews
2. Start recording → getUserMedia(chromeMediaSource: desktop) → MediaStream
3. Optional: microphone stream → Web Audio API mixer → combined stream
4. Optional: webcam stream → separate MediaRecorder
5. MediaRecorder(stream, {codec, bitrate}) → Blob chunks → fixWebmDuration
6. Stop → storeRecordedSession IPC → main writes files to RECORDINGS_DIR
7. Cursor telemetry: setInterval(100ms) → screen.getCursorScreenPoint()
8. Switch to editor → load video in VideoPlayback component
9. User edits: zoom/trim/speed/annotation regions on timeline
10. Export:
    a. StreamingVideoDecoder demuxes source → WebCodecs VideoDecoder
    b. FrameRenderer composites each frame (screen + webcam + annotations + effects)
    c. VideoMuxer encodes frames → mp4box → MP4 output
    d. OR GifExporter encodes frames → gif.js → GIF output
11. save-exported-video IPC → main shows save dialog → writes file
```

## 2. System Requirements

| Requirement | Minimum | Recommended |
|---|---|---|
| **Node.js** | 22.22.1 | 22.22.1 (exact match) |
| **npm** | 10.9.4 | 10.9.4 (exact match) |
| **OS** | macOS 12+, Windows 10+, Linux (X11) | macOS 14+, Windows 11 |
| **RAM** | 4 GB | 8+ GB (video export is memory-intensive) |
| **GPU** | Any (software rendering supported) | Hardware-accelerated GPU for PixiJS compositing |
| **Disk** | 500 MB for install | 10+ GB for recordings (4K@60fps ≈ 45 Mbps) |

### macOS-Specific Requirements

- **Screen Recording** permission in System Settings → Privacy & Security
- **Microphone** permission for audio recording
- **Camera** permission for webcam overlay
- Code signing identity and notary profile for distribution builds (see `scripts/build_macos.sh`)
- macOS entitlements file: `macos.entitlements`

> [!IMPORTANT]
> On macOS, OpenScreen disables `MacCatapLoopbackAudioForScreenShare` via `app.commandLine.appendSwitch` to use the standard Screen & System Audio Recording permissions instead of the CoreAudio Tap API. This is intentional — CoreAudio Tap requires `NSAudioCaptureUsageDescription` in the parent app's Info.plist, which breaks terminal/IDE development workflows.

## 3. Configuration Reference

### Environment Variables

| Variable | Context | Default | Description |
|---|---|---|---|
| `NODE_ENV` | Build | — | Set to `"test"` to disable `vite-plugin-electron-renderer` |
| `APP_ROOT` | Runtime | Computed | Set by main process: `path.join(__dirname, "..")` |
| `VITE_DEV_SERVER_URL` | Runtime | — | Set automatically by Vite dev server |
| `VITE_PUBLIC` | Runtime | Computed | Points to `public/` in dev or `dist/` in production |
| `HEADLESS` | Runtime | — | Set to `"true"` to create windows without showing them (testing) |

### Build Configuration Files

| File | Format | Purpose |
|---|---|---|
| `vite.config.ts` | TypeScript | Vite build config with electron plugin, path aliases, chunk splitting |
| `tsconfig.json` | JSON | TypeScript compiler options (strict, ESNext, bundler resolution) |
| `tsconfig.node.json` | JSON | TypeScript config for Node-side files (vite.config.ts) |
| `biome.json` | JSON | Linter and formatter configuration |
| `components.json` | JSON | shadcn/ui configuration (new-york style, stone base color) |
| `tailwind.config.cjs` | CJS | Tailwind CSS configuration |
| `postcss.config.cjs` | CJS | PostCSS with Tailwind and Autoprefixer |
| `electron-builder.json5` | JSON5 | Electron Builder packaging configuration |
| `playwright.config.ts` | TypeScript | E2E test config (120s timeout, `tests/e2e/` dir) |
| `vitest.config.ts` | TypeScript | Unit test config (jsdom environment) |
| `vitest.browser.config.ts` | TypeScript | Browser test config (Playwright Chromium with SwiftShader) |
| `.nvmrc` | Text | Node.js version pinning |
| `.editorconfig` | INI | Editor formatting preferences |
| `.env.example` | Text | Template for environment variables (macOS build signing) |

### Vite Build Configuration

The Vite config (`vite.config.ts`) defines:

- **Plugins**: `@vitejs/plugin-react`, `vite-plugin-electron/simple` (main entry: `electron/main.ts`, preload: `electron/preload.ts`)
- **Path alias**: `@` → `src/`
- **Build target**: `esnext`
- **Minification**: Terser with `drop_console`, `drop_debugger`, pure function removal
- **Manual chunks**:
  - `pixi` — `pixi.js`
  - `react-vendor` — `react`, `react-dom`
  - `video-processing` — `mediabunny`, `mp4box`, `@fix-webm-duration/fix`
- **Chunk size warning limit**: 1000 KB

> [!TIP]
> The manual chunk configuration ensures PixiJS (large) and video processing libraries are loaded separately from the main React bundle, improving initial load time.

### Biome Linter Configuration

Biome is configured with strict rules in `biome.json`:

- **Formatter**: Tab indentation, double quotes, LF line endings, 100-char line width
- **Linter**: No `recommended` preset — rules are individually enabled across `complexity`, `correctness`, `style`, and `suspicious` categories
- **TypeScript overrides**: Several `correctness` rules disabled for `.ts`/`.tsx` (handled by the TS compiler)
- **Assist**: Auto-organize imports enabled

### Electron Builder Configuration

Distribution builds are configured in `electron-builder.json5`. The macOS build script (`scripts/build_macos.sh`) extends this with:

- Dual-architecture builds: arm64 and x64
- Code signing via Apple Developer identity
- Notarization via `xcrun notarytool`
- Entitlements from `macos.entitlements`
- DMG output to `release/<version>/`

### i18n Configuration

Defined in `src/i18n/config.ts`:

```typescript
DEFAULT_LOCALE = "en"
SUPPORTED_LOCALES = ["en", "zh-CN", "es", "fr", "tr", "ko-KR"]
I18N_NAMESPACES = ["common", "dialogs", "editor", "launch", "settings", "shortcuts", "timeline"]
LOCALE_STORAGE_KEY = "openscreen-locale"
```

Translation files live at `src/i18n/locales/<locale>/<namespace>.json`. The loader uses `import.meta.glob` for eager loading at build time.

## 4. Deployment & Scaling

### Development Setup

```bash
git clone https://github.com/siddharthvaddem/openscreen.git
cd openscreen
nvm use           # switches to Node 22.22.1 via .nvmrc
npm install
npm run dev       # starts Vite dev server + Electron
```

### Production Build

The build pipeline compiles TypeScript, bundles with Vite, and packages with electron-builder:

```bash
npm run build           # all platforms
npm run build:mac       # macOS DMG
npm run build:win       # Windows installer
npm run build:linux     # Linux AppImage + deb
```

**Build artifacts structure:**

```text
├── dist/               # Vite renderer output
│   └── index.html
├── dist-electron/      # Compiled main process
│   ├── main.js
│   └── preload.mjs
└── release/<version>/  # Packaged distributables
```

### macOS Distribution

The `scripts/build_macos.sh` script automates the full macOS release process:

1. Validates prerequisites (Node.js, npm, signing identity, notary profile, entitlements)
2. Cleans previous build artifacts
3. Runs TypeScript compilation and Vite build
4. Builds for both arm64 and x64 architectures
5. Signs the `.app` bundle with hardened runtime
6. Notarizes via Apple's notary service
7. Staples the notarization ticket
8. Produces DMG files in `release/<version>/`

Required `.env` variables for macOS builds:

| Variable | Description |
|---|---|
| `APP_NAME` | Application display name |
| `SIGN_IDENTITY` | Apple Developer signing identity |
| `NOTARY_PROFILE` | Keychain profile for notarization |
| `APPLE_ID` | Apple Developer account email |
| `TEAM_ID` | Apple Developer Team ID |

> [!CAUTION]
> The macOS build script requires a valid Apple Developer account with code signing certificates installed. Running without proper credentials will fail at the signing step.

## 5. Operations

### Window Management

OpenScreen uses three window types, each created by factory functions in `electron/windows.ts`:

| Window | Size | Properties | Purpose |
|---|---|---|---|
| **HUD Overlay** | 600×160 (fixed) | Frameless, transparent, always-on-top, skip taskbar | Recording controls overlay |
| **Editor** | 1200×800 (starts maximized) | Hidden title bar on macOS, resizable | Video editing interface |
| **Source Selector** | 620×420 | Frameless, transparent, always-on-top, centered | Screen/window picker |

All windows use `contextIsolation: true` and `nodeIntegration: false`. The editor window additionally sets `webSecurity: false` to allow loading local video files via `file://` URLs.

> [!NOTE]
> On macOS, HUD and source selector windows use `setVisibleOnAllWorkspaces(true, { visibleOnFullScreen: true })` so they follow the user across Spaces (virtual desktops).

### System Tray

The tray icon changes based on recording state:

- **Idle**: Default OpenScreen icon with context menu: Open, Quit
- **Recording**: Red recording icon with tooltip showing source name, context menu: Stop Recording

### File System Layout

**Runtime directories (in Electron `userData`):**

```text
<userData>/
├── recordings/                    # Video recordings
│   ├── recording-<timestamp>.webm         # Screen recording
│   ├── recording-<timestamp>-webcam.webm  # Webcam recording (optional)
│   ├── recording-<timestamp>.session.json # Session manifest
│   ├── recording-<timestamp>.webm.cursor.json  # Cursor telemetry
│   └── *.openscreen                       # Project files
└── shortcuts.json                 # Custom keyboard shortcuts
```

### Recording Session Management

Each recording produces a session manifest (`*.session.json`) containing:

```json
{
  "screenVideoPath": "/path/to/recording-1234.webm",
  "webcamVideoPath": "/path/to/recording-1234-webcam.webm",
  "createdAt": 1234567890123
}
```

The IPC handler maintains state for the current recording session and current project path. Session manifests enable the editor to locate both screen and webcam files from a single reference.

### Cursor Telemetry

During recording, cursor position is sampled at 10Hz (100ms intervals) via `screen.getCursorScreenPoint()`. Samples are normalized to the source display's bounds as `(cx, cy)` fractions in `[0, 1]`.

| Setting | Value |
|---|---|
| Sample interval | 100ms |
| Max samples | 36,000 (1 hour at 10Hz) |
| Storage format | JSON: `{ version: 1, samples: [{timeMs, cx, cy}, ...] }` |
| File naming | `<video-path>.cursor.json` |

> [!TIP]
> Cursor telemetry enables cursor-following zoom effects. When multiple displays are connected, the cursor position is normalized to the source display's bounds, not the global screen coordinate space.

## 6. Performance

### Recording Bitrate Scaling

| Resolution | Base Bitrate | With 60fps Boost (1.7×) |
|---|---|---|
| 4K (3840×2160+) | 45 Mbps | 76.5 Mbps |
| QHD (2560×1440+) | 28 Mbps | 47.6 Mbps |
| Below QHD | 18 Mbps | 30.6 Mbps |

Audio bitrate: 192 kbps (system audio) or 128 kbps (microphone only).

### Codec Selection Priority

```text
1. video/webm;codecs=av1        (best compression)
2. video/webm;codecs=h264       (wide compatibility)
3. video/webm;codecs=vp9        (good compression)
4. video/webm;codecs=vp8        (legacy fallback)
5. video/webm                   (bare minimum)
```

### Export Pipeline Performance

The export pipeline processes each frame sequentially:

1. **Decode**: `StreamingVideoDecoder` demuxes via `web-demuxer` WASM and decodes via WebCodecs `VideoDecoder`
2. **Render**: `FrameRenderer` composites screen + webcam + annotations + effects using Canvas 2D (with PixiJS for filters)
3. **Encode**: WebCodecs `VideoEncoder` → `mp4box` muxer

For GIF export, `gif.js` handles frame quantization and LZW encoding.

> [!NOTE]
> The `AsyncVideoFrameQueue` manages backpressure between the decoder and renderer with a configurable queue depth, preventing memory exhaustion on long videos.

### Build Optimization

- **Terser**: Production builds drop `console.log`, `console.debug`, and `debugger` statements
- **Manual chunks**: PixiJS, React, and video processing libraries are split into separate bundles
- **Background throttling**: Disabled (`backgroundThrottling: false`) for both HUD and editor windows to ensure recording and playback aren't paused when unfocused

## 7. Security

### IPC Security Model

OpenScreen follows Electron's security best practices:

- **Context isolation**: `contextIsolation: true` on all windows
- **No node integration**: `nodeIntegration: false` on all windows
- **Preload bridge**: All renderer↔main communication goes through `contextBridge.exposeInMainWorld`

### File System Access Control

The IPC handler implements a path allowlist for file reads:

1. **Default allowed directory**: `RECORDINGS_DIR` (Electron `userData/recordings/`)
2. **Runtime-approved paths**: Added when the user explicitly selects files via OS file picker dialogs
3. **Project-scoped approval**: When loading a project, only paths within the project's directory or `RECORDINGS_DIR` are auto-approved

> [!WARNING]
> The `read-binary-file` IPC handler validates paths against the allowlist before reading. Paths outside allowed directories are rejected with "Access denied: path outside allowed directories". This prevents malicious project files from reading arbitrary filesystem locations.

### Recording File Name Validation

The `resolveRecordingOutputPath()` function enforces:

- File names cannot be empty
- Path traversal segments (`..`) are rejected
- Absolute paths are rejected
- Directory separators (`/`, `\`) are rejected
- Output is always within `RECORDINGS_DIR`

### Project File Trust Model

When loading `.openscreen` project files:

1. Media paths are validated against trusted directories (project's own directory + `RECORDINGS_DIR`)
2. Only files with allowed video extensions (`.webm`, `.mp4`, `.mov`, `.avi`, `.mkv`) are approved for read access
3. `file://` URLs are normalized to filesystem paths before validation
4. The `currentProjectPath` is used to determine if "Save" should overwrite without prompting

### Permission Handling

| Permission | Platform | Handling |
|---|---|---|
| Screen recording | All | Via `desktopCapturer` (Chromium-managed) |
| Microphone | All | `navigator.mediaDevices.getUserMedia` + macOS `systemPreferences.askForMediaAccess` |
| Camera | macOS | `systemPreferences.getMediaAccessStatus("camera")` + `askForMediaAccess("camera")` |
| Camera | Other | Browser-level `getUserMedia` (auto-granted in Electron) |

> [!CAUTION]
> The editor window sets `webSecurity: false` to allow loading local video files via `file://` URLs. This disables same-origin policy for the editor renderer process. This is necessary for the video playback pipeline but means the editor window should not load untrusted remote content.

### Unsaved Changes Protection

The editor tracks unsaved changes via `editorHasUnsavedChanges` flag. On close:

1. If no unsaved changes → window closes immediately
2. If unsaved changes → native dialog with three options: Save & Close, Discard & Close, Cancel
3. Save & Close sends `request-save-before-close` to renderer, waits for `save-before-close-done` confirmation before closing

## 8. Developer Internals

### Repository Structure

```text
openscreen/
├── electron/                     # Electron main process
│   ├── main.ts                   # App entry, tray, menus, window lifecycle
│   ├── preload.ts                # contextBridge API surface
│   ├── windows.ts                # Window factory (HUD, editor, source selector)
│   ├── i18n.ts                   # Main-process translations
│   └── ipc/
│       └── handlers.ts           # All IPC handler registrations (28+)
├── src/                          # Renderer process (React)
│   ├── main.tsx                  # React entry point
│   ├── App.tsx                   # Window type routing
│   ├── components/
│   │   ├── launch/               # HUD overlay + source picker
│   │   ├── ui/                   # shadcn/ui components (19 components)
│   │   └── video-editor/         # Core editor (20 files, 277K chars)
│   │       ├── VideoEditor.tsx   # Main editor component
│   │       ├── VideoPlayback.tsx # Video preview + webcam composite
│   │       ├── PlaybackControls.tsx
│   │       ├── SettingsPanel.tsx  # Wallpaper, padding, aspect ratio, webcam
│   │       ├── AnnotationOverlay.tsx
│   │       ├── AnnotationSettingsPanel.tsx
│   │       ├── BlurSettingsPanel.tsx
│   │       ├── CropControl.tsx
│   │       ├── ExportDialog.tsx  # Format, quality, GIF options
│   │       ├── FormatSelector.tsx
│   │       ├── GifOptionsPanel.tsx
│   │       ├── KeyboardShortcutsHelp.tsx
│   │       ├── ShortcutsConfigDialog.tsx
│   │       ├── TutorialHelp.tsx
│   │       ├── AddCustomFontDialog.tsx
│   │       ├── ArrowSvgs.tsx     # Arrow annotation SVG components
│   │       ├── projectPersistence.ts  # Save/load project state
│   │       ├── types.ts          # Editor type definitions
│   │       ├── timeline/         # Timeline components (7 files)
│   │       └── videoPlayback/    # Playback utilities (9 files)
│   ├── contexts/
│   │   ├── I18nContext.tsx       # Translation provider
│   │   └── ShortcutsContext.tsx  # Keyboard shortcut provider
│   ├── hooks/
│   │   ├── useScreenRecorder.ts  # MediaRecorder lifecycle (1250 lines)
│   │   ├── useEditorHistory.ts   # Undo/redo state machine
│   │   ├── useCameraDevices.ts   # Camera enumeration
│   │   ├── useMicrophoneDevices.ts  # Microphone enumeration
│   │   └── useAudioLevelMeter.ts # Real-time audio level visualization
│   ├── lib/
│   │   ├── compositeLayout.ts    # Webcam overlay layout computation
│   │   ├── customFonts.ts        # Google Fonts management
│   │   ├── shortcuts.ts          # Shortcut binding system
│   │   ├── userPreferences.ts    # localStorage preferences
│   │   ├── recordingSession.ts   # Session/project media types
│   │   ├── requestCameraAccess.ts
│   │   ├── frameStep.ts          # Frame-accurate navigation
│   │   ├── assetPath.ts          # Dev vs. production asset resolution
│   │   ├── webcamMaskShapes.ts   # CSS/Canvas clip path generation
│   │   ├── utils.ts              # cn() Tailwind merge utility
│   │   └── exporter/             # Export pipeline (17 files, 121K chars)
│   │       ├── videoExporter.ts  # Orchestrates full MP4 export
│   │       ├── gifExporter.ts    # GIF export with gif.js
│   │       ├── frameRenderer.ts  # Canvas 2D frame compositing
│   │       ├── annotationRenderer.ts  # Text/blur/arrow rendering
│   │       ├── muxer.ts          # mp4box wrapper
│   │       ├── streamingDecoder.ts  # WebCodecs video decoder
│   │       ├── videoDecoder.ts   # File-based video decoder
│   │       ├── audioEncoder.ts   # Audio re-encoding
│   │       ├── asyncVideoFrameQueue.ts  # Backpressure queue
│   │       ├── gradientParser.ts # CSS gradient to canvas
│   │       └── types.ts          # Export config types
│   ├── i18n/                     # Internationalization
│   │   ├── config.ts             # Locale/namespace definitions
│   │   ├── loader.ts             # Translation file loader
│   │   └── locales/              # 6 locale directories × 7 namespaces
│   └── utils/                    # Utility functions
│       ├── aspectRatioUtils.ts   # Aspect ratio calculations
│       ├── platformUtils.ts      # macOS detection
│       ├── timeUtils.ts          # Time formatting
│       └── getTestId.ts          # Test ID generation
├── tests/
│   ├── e2e/                      # Playwright E2E tests
│   │   └── gif-export.spec.ts
│   └── fixtures/
│       └── sample.webm
├── public/                       # Static assets
│   ├── wallpapers/               # 18 background wallpapers
│   └── wasm/
│       └── web-demuxer.wasm      # Video demuxer WASM module
├── icons/                        # App icons (mac .icns, win .ico, PNG set)
└── scripts/
    ├── build_macos.sh            # macOS signing + notarization
    └── i18n-check.mjs            # Translation key coverage checker
```

### Module Reference

#### `electron/ipc/handlers.ts` (28K chars)

The largest single file, implementing all IPC communication. Key handler groups:

**Source management:**
- `get-sources` — wraps `desktopCapturer.getSources()` and serializes thumbnails to data URLs
- `select-source` / `get-selected-source` — store and retrieve the selected screen/window

**Recording:**
- `set-recording-state` — toggles cursor telemetry capture
- `store-recorded-session` — writes screen + webcam video blobs to disk with session manifest
- `store-recorded-video` — legacy single-file store (delegates to session store)
- `get-recorded-video-path` — returns current session video or latest recording

**File operations:**
- `read-binary-file` — reads files within allowed directories only
- `save-exported-video` — shows save dialog for MP4/GIF export
- `open-video-file-picker` — shows open dialog with video format filter
- `reveal-in-folder` — `shell.showItemInFolder()` with directory fallback

**Project persistence:**
- `save-project-file` — trusted-path write or save dialog
- `load-project-file` / `load-current-project-file` — open and validate project JSON
- Project media paths undergo `approveReadableVideoPath()` validation

**Window management:**
- `switch-to-editor` / `switch-to-hud` — window transitions
- `start-new-recording` — clears session state and switches to HUD
- `open-source-selector` — opens/focuses source picker window

**System:**
- `request-camera-access` — macOS-specific `systemPreferences.askForMediaAccess`
- `get-platform` — returns `process.platform`
- `get-asset-base-path` — resolves packaged vs. development asset paths
- `set-locale` — updates main-process locale for menus/dialogs
- `get-shortcuts` / `save-shortcuts` — reads/writes `shortcuts.json`
- `get-cursor-telemetry` — loads cursor samples from `.cursor.json`
- `open-external-url` — `shell.openExternal()`

#### `src/hooks/useScreenRecorder.ts` (1250 lines)

The recording lifecycle hook managing:

- **Stream acquisition**: `getUserMedia` with `chromeMediaSource: "desktop"`, constraint negotiation for 4K/60fps
- **Audio mixing**: Web Audio API `AudioContext` combining system audio + microphone with gain boost
- **MediaRecorder**: Codec selection (AV1 → H.264 → VP9 → VP8), adaptive bitrate
- **Webcam**: Separate `MediaRecorder` for webcam stream at 720p/30fps
- **Pause/Resume**: Accumulates duration across segments for accurate timing
- **Finalization**: `fixWebmDuration` correction, session storage via IPC
- **Restart**: Discards current recording and starts a new one

#### `src/lib/exporter/frameRenderer.ts` (FrameRenderer class)

Core rendering class that composites each export frame:

- Maintains three canvases: composite (output), raster (screen content), shadow (effects)
- Applies crop region, padding, background wallpaper
- Renders webcam overlay with mask shapes (rectangle, rounded, circle, square)
- Renders annotations (text, image, figure/arrow, blur) via `annotationRenderer.ts`
- Applies zoom regions with keyframe interpolation
- Handles motion blur overlay

#### `src/lib/exporter/videoExporter.ts` (VideoExporter class)

Orchestrates the full export pipeline:

- Initializes `StreamingVideoDecoder`, `FrameRenderer`, `VideoMuxer`, and `AudioProcessor`
- Decodes source frames, applies editor state transformations
- Handles trim regions (skip frames) and speed regions (adjust timing)
- Reports progress via callback
- Supports cancellation

#### `src/lib/compositeLayout.ts`

Computes webcam overlay positioning for three layout modes:

- **Picture-in-picture**: Webcam sized as fraction of geometric mean of canvas dimensions, positioned with margin or at custom `(cx, cy)` coordinates
- **Vertical stack**: Webcam full-width at bottom, screen fills remaining space (cover mode)
- **Dual frame**: 2:1 horizontal split within max content bounds

Shape-specific behavior: circle and square masks force equal width/height; rounded masks apply 30% border radius.

#### `src/hooks/useEditorHistory.ts`

Undo/redo implementation using an immutable state stack:

- **MAX_HISTORY**: 80 states
- **pushState**: Creates a checkpoint and clears the future stack
- **updateState**: Live updates during slider drags — first call creates checkpoint, subsequent calls update in-place
- **commitState**: Marks the end of a live-update series
- **EditorState**: Includes zoom/trim/speed/annotation regions, crop, wallpaper, shadow, blur, border radius, padding, aspect ratio, webcam settings

#### `src/lib/shortcuts.ts`

Keyboard shortcut system with:

- 8 configurable actions: `addZoom`, `addTrim`, `addSpeed`, `addAnnotation`, `addBlur`, `addKeyframe`, `deleteSelected`, `playPause`
- 9 fixed shortcuts (undo, redo, cycle annotations, delete, pan/zoom timeline, frame step)
- Conflict detection across configurable and fixed bindings
- Platform-aware display formatting (Cmd vs. Ctrl, macOS symbols)
- Persistence via Electron IPC to `shortcuts.json`

### Code Patterns and Conventions

**State management**: React hooks with `useState`/`useRef` for local state. Context providers (`I18nContext`, `ShortcutsContext`) for cross-component state. No Redux or external state library.

**IPC pattern**: All communication uses `ipcMain.handle()` / `ipcRenderer.invoke()` (async request-response) or `ipcMain.on()` / `ipcRenderer.send()` (fire-and-forget for state notifications like `set-has-unsaved-changes`).

**Error handling**: Try-catch with `{ success: boolean, error?: string }` return objects for IPC handlers. Toast notifications (`sonner`) for user-facing errors. Console logging for developer diagnostics.

**Type safety**: Strict TypeScript throughout. Union types for configuration enums (`AspectRatio`, `ExportFormat`, `WebcamMaskShape`). `as const` for literal arrays with derived types.

**Styling**: Tailwind CSS with `cn()` utility (clsx + tailwind-merge). CSS Modules for component-scoped styles (`.module.css`). Dark theme by default.

**File naming**: Components use PascalCase (`.tsx`), utilities use camelCase (`.ts`), tests use `.test.ts` or `.spec.ts` suffix.

**Exports**: Barrel exports via `index.ts` files in `video-editor/`, `videoPlayback/`, and `exporter/` directories.

### Testing Strategy

| Test Type | Tool | Config | Scope |
|---|---|---|---|
| **Unit** | Vitest | `vitest.config.ts` (jsdom) | Hooks, utilities, layout computation |
| **Browser** | Vitest + Playwright | `vitest.browser.config.ts` | Export pipeline (requires WebGL via SwiftShader) |
| **E2E** | Playwright | `playwright.config.ts` | GIF export end-to-end |

**npm scripts:**

```bash
npm test              # Unit tests (single run)
npm run test:watch    # Unit tests (watch mode)
npm run test:browser  # Browser tests with headless Chromium
npm run test:e2e      # Playwright E2E tests
```

**Browser test configuration**: Uses Playwright with `--enable-unsafe-swiftshader` and `--use-gl=swiftshader` flags to enable software WebGL for PixiJS in headless CI. Test timeout: 120 seconds. Hook timeout: 30 seconds.

> [!NOTE]
> The `tests/fixtures/sample.webm` file serves as the input for E2E GIF export tests. Browser tests include pattern: `src/**/*.browser.test.{ts,tsx}`.

**Pre-commit**: Husky + lint-staged runs `biome check --no-errors-on-unmatched` on staged TypeScript, JavaScript, and JSON files.

### Known Issues (from Community)

| Issue | Status | Description |
|---|---|---|
| #157 | Open | Slow rendering/export on Linux (potentially Windows), 23 comments |
| #276 | Open | Video decode ended early |
| #376 | Open | Background not shown in exported video, 11 comments |
| #411 | Open | Force closing and not starting |
| #428 | Open | Can't export video on Windows |
| #433 | Open | Audio misaligned with speed + trim regions |
| #437 | Open | Blurry screen display |
| #439 | Open | Very slow export for 8-minute video at High quality |
| #379 | Open | Fail to read binary file |

### Potential Improvements

- **CLI / Agent interface** (issue #349): Open feature request for headless CLI and agent API
- **Stream Deck support** (issue #440): Hardware control integration
- **Transition controls** (issue #446): Zoom and annotation transition easing
- **Webcam overlay editing** (issue #412): In-place webcam overlay manipulation
- **Auto-hide toolbar** (issue #435): Toolbar hiding during recording
- **Zoom speed adjustment** (issue #265): Configurable zoom animation speed

## 9. Integration Points

### Electron APIs Used

| API | Purpose |
|---|---|
| `desktopCapturer` | Screen and window source enumeration |
| `systemPreferences` | macOS permission checks (microphone, camera) |
| `dialog` | File open/save dialogs |
| `shell` | Open URLs, reveal files in explorer |
| `Tray` | System tray icon and context menu |
| `Menu` | Application menu bar |
| `nativeImage` | Tray icon creation and resizing |
| `screen` | Display information, cursor position |
| `session` | Permission handlers for media access |
| `app` | Lifecycle, paths, command-line switches |

### Browser APIs Used

| API | Purpose |
|---|---|
| `MediaRecorder` | Recording media streams |
| `navigator.mediaDevices.getUserMedia` | Stream acquisition |
| `navigator.mediaDevices.enumerateDevices` | Camera/microphone enumeration |
| `AudioContext` / `Web Audio API` | Audio mixing, level metering |
| `VideoDecoder` / `VideoEncoder` (WebCodecs) | Frame-level video processing |
| `OffscreenCanvas` | GPU frame rendering |
| `CSS Font Loading API` | Custom font verification |
| `localStorage` | User preferences, custom fonts, locale |

### External Dependencies (Key)

| Package | Version | Purpose |
|---|---|---|
| `pixi.js` | 8.14 | GPU-accelerated rendering, drop shadow filters |
| `mp4box` | 2.2 | MP4 container muxing |
| `gif.js` | 0.2 | GIF encoding with web workers |
| `mediabunny` | 1.25 | Media processing utilities |
| `web-demuxer` | 4.0 | WASM-based video container demuxing |
| `fix-webm-duration` | 1.0 | WebM duration header correction |
| `gsap` | 3.13 | Animation library |
| `dnd-timeline` | 2.2 | Drag-and-drop timeline component |
| `react-rnd` | 10.5 | Resizable/draggable annotation overlays |
| `emoji-picker-react` | 4.16 | Emoji picker for annotations |

### Contributing

**Linting and formatting:**

```bash
npm run lint          # Check with Biome
npm run lint:fix      # Auto-fix linting issues
npm run format        # Auto-format with Biome
```

**i18n coverage check:**

```bash
npm run i18n:check    # Verify all locales have matching translation keys
```

**Code style**: The project uses Biome (not ESLint/Prettier). Tab indentation, double quotes, 100-character line width. See `biome.json` for full rules.

**PR guidelines**: The `CONTRIBUTING.md` file exists in the repository root (contents not in briefing). See the User Guide for basic usage context.

---

_Generated May 2026 from siddharthvaddem/openscreen at 899504f. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._