diff --git a/docs/superpowers/plans/2026-10-25-outlier-saver.md b/docs/superpowers/plans/2026-10-25-outlier-saver.md new file mode 100644 index 0000000..ba15df4 --- /dev/null +++ b/docs/superpowers/plans/2026-10-25-outlier-saver.md @@ -0,0 +1,2499 @@ +# Outlier Saver Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A Chrome MV3 extension that watches YouTube pages, finds videos whose views are anomalously high relative to the channel's subscribers (plus videos flagged by VidIQ), and stores them in IndexedDB with JSON/CSV export. + +**Architecture:** A content script collects video IDs (and an optional VidIQ badge value) from YouTube pages and sends them to a service worker. The worker batches IDs, asks the YouTube Data API v3 for video and channel stats, computes `score = views / subscribers`, decides whether to keep the video, and upserts it into IndexedDB. A popup lists and exports the data; an options page holds the API key and thresholds. All decision logic lives in pure, dependency-injected modules covered by `node:test`. + +**Tech Stack:** Chrome Manifest V3, plain JavaScript ES modules, IndexedDB, YouTube Data API v3, `node:test` (Node 22). No build step, no npm dependencies. + +## Global Constraints + +Copied from `docs/superpowers/specs/2026-10-25-outlier-saver-design.md`; every task implicitly includes them. + +- Manifest V3, plain JS ES modules, **no build step and no runtime or dev dependencies**; load as an unpacked extension. +- Tests use the built-in `node:test` runner only (`npm test` → `node --test`). +- Own score: `score = views / max(subscribers, 1)`, rounded to 2 decimals; `null` when subscribers are hidden. +- Save rule: `views >= minViews && score >= threshold`, **or** `vidiqScore !== null` (independent of own threshold and `minViews`). A video that is already saved is always updated. +- Settings defaults (`chrome.storage.local`, key `settings`): `apiKey` empty, `threshold` 5, `minViews` 10 000. +- API calls: `videos.list` (`part=snippet,statistics,contentDetails`) and `channels.list` (`part=statistics,snippet`), batches of 50 IDs, from the service worker only; the key never goes to a content script. +- Retries: 3 retries (4 attempts) with delays 500 ms, 1000 ms, 2000 ms on network errors and 5xx/429; quota and key errors are never retried. +- Quota resets at midnight America/Los_Angeles; on `quotaExceeded` collection pauses until then. +- `seen` and `channels` entries are rechecked after 24 hours. +- IndexedDB database `outlier-saver` v1 with stores `outliers` (key `videoId`, index `updatedAt`), `channels` (key `channelId`), `seen` (key `videoId`). +- Saved `url` is always `https://www.youtube.com/watch?v=`, including Shorts. `isShort` is true if the video was ever seen via a `/shorts/` link. +- Supported pages: home `/`, watch `/watch`, search `/results`, channel `/@…`, `/channel/…`, `/c/…`, `/user/…`. +- User-facing strings are in Russian. Untrusted text (titles, channel names) is rendered with `textContent`, never `innerHTML`. +- Commit messages end with the trailer `Co-Authored-By: Claude Sonnet 5.5 ` (passed as a second `-m`). + +--- + +## File Structure + +``` +manifest.json +package.json +.gitignore +README.md +src/background.js # queue, flush loop, status/badge, message + storage listeners +src/content.js # DOM scanner (classic script, dynamic-imports the two lib files it needs) +src/lib/video-id.js # parseVideoLink, pageFromPath, isVideoId +src/lib/score.js # computeScore, shouldSave +src/lib/vidiq.js # VIDIQ_BADGE_SELECTOR, parseVidiqScore, readVidiqScore +src/lib/export.js # COLUMNS, toJson, toCsv +src/lib/storage.js # DEFAULT_SETTINGS, normalizeSettings, get/saveSettings, get/setStatus +src/lib/api.js # ApiError, helpers, fetchVideos, fetchChannels, nextQuotaReset +src/lib/process.js # processBatch (pure orchestration with injected db + api) +src/lib/db.js # IndexedDB wrapper +src/popup/popup.{html,css,js} +src/options/options.{html,css,js} +test/*.test.js +``` + +Data shapes used across tasks: + +- **Queue item** (content → background → `processBatch`): `{ videoId: string, vidiqScore: number|null, viaShortsUrl: boolean, page: 'home'|'watch'|'search'|'channel' }` +- **Video** (from `fetchVideos`): `{ videoId, title, thumbnailUrl: string|null, channelId, channelTitle, views: number, publishedAt: string|null, durationSec: number|null }` +- **Channel** (from `fetchChannels`): `{ channelId, title, url, subscribers: number|null }`; stored in `channels` with an added `fetchedAt` ISO string. +- **Seen record**: `{ videoId, checkedAt: ISO string, status: 'ok'|'unavailable' }` +- **Outlier record**: see `COLUMNS` in Task 4 (17 fields). +- **Status** (`chrome.storage.local` key `status`): `{ state: 'ok'|'no-key'|'invalid-key'|'quota', message?: string, until?: number }` + +--- + +### Task 1: Video link and page parsing (+ project scaffold) + +**Files:** +- Create: `package.json`, `.gitignore` +- Create: `src/lib/video-id.js` +- Test: `test/video-id.test.js` + +**Interfaces:** +- Consumes: nothing. +- Produces: + - `isVideoId(value: unknown): boolean` — true for 11-char `[A-Za-z0-9_-]` strings + - `parseVideoLink(href: string|null|undefined, base?: string): { videoId: string, isShort: boolean } | null` + - `pageFromPath(pathname: string): 'home'|'watch'|'search'|'channel'|null` + +- [ ] **Step 1: Create the scaffold files** + +`package.json`: + +```json +{ + "name": "outlier-saver-extension", + "version": "0.1.0", + "private": true, + "type": "module", + "scripts": { + "test": "node --test" + } +} +``` + +`.gitignore`: + +``` +node_modules/ +.DS_Store +``` + +- [ ] **Step 2: Write the failing test** + +`test/video-id.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { isVideoId, pageFromPath, parseVideoLink } from '../src/lib/video-id.js'; + +test('isVideoId accepts only 11-character IDs', () => { + assert.equal(isVideoId('dQw4w9WgXcQ'), true); + assert.equal(isVideoId('abc_DEF-123'), true); + for (const value of ['short', 'waytoolongvideoid', 'has space!!', '', null, undefined, 12345678901]) { + assert.equal(isVideoId(value), false, String(value)); + } +}); + +test('parseVideoLink reads an absolute watch URL', () => { + assert.deepEqual(parseVideoLink('https://www.youtube.com/watch?v=dQw4w9WgXcQ'), { + videoId: 'dQw4w9WgXcQ', + isShort: false, + }); +}); + +test('parseVideoLink reads a relative watch href with extra params', () => { + assert.deepEqual(parseVideoLink('/watch?v=dQw4w9WgXcQ&t=42s&list=PL123'), { + videoId: 'dQw4w9WgXcQ', + isShort: false, + }); +}); + +test('parseVideoLink marks /shorts/ links as shorts', () => { + assert.deepEqual(parseVideoLink('/shorts/abcdefghijk'), { videoId: 'abcdefghijk', isShort: true }); + assert.deepEqual(parseVideoLink('/shorts/abcdefghijk?feature=share'), { + videoId: 'abcdefghijk', + isShort: true, + }); +}); + +test('parseVideoLink ignores links that are not a single video', () => { + for (const href of ['/', '/@mrbeast', '/playlist?list=PL123', '/feed/subscriptions', '/results?search_query=x']) { + assert.equal(parseVideoLink(href), null, href); + } +}); + +test('parseVideoLink rejects malformed IDs', () => { + for (const href of [ + '/watch', + '/watch?v=', + '/watch?v=short', + '/watch?v=waytoolongvideoid', + '/shorts/toolongidtoolong', + ]) { + assert.equal(parseVideoLink(href), null, href); + } +}); + +test('parseVideoLink rejects other hosts and garbage', () => { + assert.equal(parseVideoLink('https://evil.example.com/watch?v=dQw4w9WgXcQ'), null); + for (const href of [null, undefined, '', 'http://[bad']) { + assert.equal(parseVideoLink(href), null, String(href)); + } +}); + +test('pageFromPath maps the four supported page types', () => { + assert.equal(pageFromPath('/'), 'home'); + assert.equal(pageFromPath('/watch'), 'watch'); + assert.equal(pageFromPath('/results'), 'search'); + for (const path of ['/@mrbeast', '/@mrbeast/videos', '/channel/UC123/videos', '/c/name', '/user/name']) { + assert.equal(pageFromPath(path), 'channel', path); + } +}); + +test('pageFromPath returns null for unsupported pages', () => { + for (const path of ['/feed/subscriptions', '/shorts/abcdefghijk', '/playlist', '/premium']) { + assert.equal(pageFromPath(path), null, path); + } +}); +``` + +- [ ] **Step 3: Run the test to verify it fails** + +Run: `node --test test/video-id.test.js` +Expected: FAIL with `ERR_MODULE_NOT_FOUND` (cannot find `src/lib/video-id.js`). + +- [ ] **Step 4: Write the implementation** + +`src/lib/video-id.js`: + +```js +const ID_RE = /^[A-Za-z0-9_-]{11}$/; +const YOUTUBE_HOSTS = new Set(['www.youtube.com', 'youtube.com', 'm.youtube.com']); + +export function isVideoId(value) { + return typeof value === 'string' && ID_RE.test(value); +} + +// Returns { videoId, isShort } for a link to a single video, or null for anything else. +export function parseVideoLink(href, base = 'https://www.youtube.com') { + if (!href) return null; + let url; + try { + url = new URL(href, base); + } catch { + return null; + } + if (!YOUTUBE_HOSTS.has(url.hostname)) return null; + + if (url.pathname === '/watch') { + const id = url.searchParams.get('v'); + return isVideoId(id) ? { videoId: id, isShort: false } : null; + } + const shorts = url.pathname.match(/^\/shorts\/([^/]+)/); + if (shorts && isVideoId(shorts[1])) return { videoId: shorts[1], isShort: true }; + return null; +} + +export function pageFromPath(pathname) { + if (pathname === '/') return 'home'; + if (pathname === '/watch') return 'watch'; + if (pathname === '/results') return 'search'; + if (/^\/(@[^/]+|channel\/[^/]+|c\/[^/]+|user\/[^/]+)/.test(pathname)) return 'channel'; + return null; +} +``` + +- [ ] **Step 5: Run the test to verify it passes** + +Run: `node --test test/video-id.test.js` +Expected: PASS, 9 tests, 0 failures. + +- [ ] **Step 6: Commit** + +```bash +git add package.json .gitignore src/lib/video-id.js test/video-id.test.js +git commit -m "feat: parse YouTube video links and page types" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 2: Score formula and save decision + +**Files:** +- Create: `src/lib/score.js` +- Test: `test/score.test.js` + +**Interfaces:** +- Consumes: nothing. +- Produces: + - `computeScore(views: number, subscribers: number|null|undefined): number|null` + - `shouldSave({ views: number, score: number|null, vidiqScore: number|null }, { threshold: number, minViews: number }): boolean` + +- [ ] **Step 1: Write the failing test** + +`test/score.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { computeScore, shouldSave } from '../src/lib/score.js'; + +test('computeScore divides views by subscribers', () => { + assert.equal(computeScore(100000, 20000), 5); + assert.equal(computeScore(2500, 1000), 2.5); +}); + +test('computeScore rounds to two decimals', () => { + assert.equal(computeScore(10000, 3000), 3.33); +}); + +test('computeScore never divides by zero', () => { + assert.equal(computeScore(5000, 0), 5000); +}); + +test('computeScore returns null when subscribers are unknown', () => { + assert.equal(computeScore(5000, null), null); + assert.equal(computeScore(5000, undefined), null); +}); + +const settings = { threshold: 5, minViews: 10000 }; + +test('shouldSave accepts a video exactly at both limits', () => { + assert.equal(shouldSave({ views: 10000, score: 5, vidiqScore: null }, settings), true); +}); + +test('shouldSave rejects a score below the threshold', () => { + assert.equal(shouldSave({ views: 50000, score: 4.99, vidiqScore: null }, settings), false); +}); + +test('shouldSave rejects views below the minimum even with a huge score', () => { + assert.equal(shouldSave({ views: 9999, score: 50, vidiqScore: null }, settings), false); +}); + +test('shouldSave rejects a null score without a VidIQ score', () => { + assert.equal(shouldSave({ views: 500000, score: null, vidiqScore: null }, settings), false); +}); + +test('shouldSave accepts any video that has a VidIQ score', () => { + assert.equal(shouldSave({ views: 100, score: null, vidiqScore: 7.5 }, settings), true); + assert.equal(shouldSave({ views: 100, score: 0.01, vidiqScore: 3 }, settings), true); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `node --test test/score.test.js` +Expected: FAIL with `ERR_MODULE_NOT_FOUND`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/score.js`: + +```js +export function computeScore(views, subscribers) { + if (subscribers === null || subscribers === undefined) return null; + return Math.round((views / Math.max(subscribers, 1)) * 100) / 100; +} + +export function shouldSave({ views, score, vidiqScore }, { threshold, minViews }) { + if (vidiqScore !== null && vidiqScore !== undefined) return true; + return score !== null && score !== undefined && views >= minViews && score >= threshold; +} +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `node --test test/score.test.js` +Expected: PASS, 9 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/score.js test/score.test.js +git commit -m "feat: add outlier score formula and save decision" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 3: VidIQ badge reader (selector unset) + +**Files:** +- Create: `src/lib/vidiq.js` +- Test: `test/vidiq.test.js` + +**Interfaces:** +- Consumes: nothing. +- Produces: + - `VIDIQ_BADGE_SELECTOR: string|null` (null until Task 15) + - `parseVidiqScore(text: string|null|undefined): number|null` + - `readVidiqScore(card: { querySelector(selector: string): { textContent: string } | null } | null, selector?: string|null): number|null` + +- [ ] **Step 1: Write the failing test** + +`test/vidiq.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { VIDIQ_BADGE_SELECTOR, parseVidiqScore, readVidiqScore } from '../src/lib/vidiq.js'; + +test('parseVidiqScore extracts the first number', () => { + assert.equal(parseVidiqScore('12.5x'), 12.5); + assert.equal(parseVidiqScore('5,2x'), 5.2); + assert.equal(parseVidiqScore('Outlier 8'), 8); +}); + +test('parseVidiqScore returns null when there is no number', () => { + assert.equal(parseVidiqScore('no digits here'), null); + assert.equal(parseVidiqScore(''), null); + assert.equal(parseVidiqScore(null), null); + assert.equal(parseVidiqScore(undefined), null); +}); + +test('readVidiqScore returns null while no selector is configured', () => { + assert.equal(VIDIQ_BADGE_SELECTOR, null); + const card = { querySelector: () => ({ textContent: '9x' }) }; + assert.equal(readVidiqScore(card), null); +}); + +test('readVidiqScore reads the badge inside a card', () => { + const card = { querySelector: (selector) => (selector === '.badge' ? { textContent: '7.5x' } : null) }; + assert.equal(readVidiqScore(card, '.badge'), 7.5); +}); + +test('readVidiqScore returns null when the card has no badge or no card is given', () => { + assert.equal(readVidiqScore({ querySelector: () => null }, '.badge'), null); + assert.equal(readVidiqScore(null, '.badge'), null); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `node --test test/vidiq.test.js` +Expected: FAIL with `ERR_MODULE_NOT_FOUND`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/vidiq.js`: + +```js +// CSS selector of VidIQ's outlier badge on a video card. It stays null until the real +// markup is captured (Task 15); until then readVidiqScore returns null for every card. +export const VIDIQ_BADGE_SELECTOR = null; + +export function parseVidiqScore(text) { + const match = /(\d+(?:[.,]\d+)?)/.exec(text ?? ''); + return match ? Number(match[1].replace(',', '.')) : null; +} + +export function readVidiqScore(card, selector = VIDIQ_BADGE_SELECTOR) { + if (!selector || !card) return null; + const badge = card.querySelector(selector); + return badge ? parseVidiqScore(badge.textContent) : null; +} +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `node --test test/vidiq.test.js` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/vidiq.js test/vidiq.test.js +git commit -m "feat: add VidIQ badge reader with unset selector" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 4: JSON and CSV export + +**Files:** +- Create: `src/lib/export.js` +- Test: `test/export.test.js` + +**Interfaces:** +- Consumes: nothing. +- Produces: + - `COLUMNS: string[]` — `videoId, title, url, thumbnailUrl, channelId, channelTitle, channelUrl, subscribers, views, publishedAt, durationSec, isShort, score, vidiqScore, foundOn, firstSeenAt, updatedAt` + - `toJson(records: object[]): string` + - `toCsv(records: object[]): string` — header + one row per record, `\r\n` line endings, trailing `\r\n`, no BOM + +- [ ] **Step 1: Write the failing test** + +`test/export.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { COLUMNS, toCsv, toJson } from '../src/lib/export.js'; + +const record = (over = {}) => ({ + videoId: 'dQw4w9WgXcQ', + title: 'Hello', + url: 'https://www.youtube.com/watch?v=dQw4w9WgXcQ', + thumbnailUrl: 'https://i.ytimg.com/vi/dQw4w9WgXcQ/hqdefault.jpg', + channelId: 'UC1', + channelTitle: 'Chan', + channelUrl: 'https://www.youtube.com/@chan', + subscribers: 1000, + views: 50000, + publishedAt: '2026-10-01T00:00:00Z', + durationSec: 600, + isShort: false, + score: 50, + vidiqScore: null, + foundOn: 'home', + firstSeenAt: '2026-10-25T10:00:00.000Z', + updatedAt: '2026-10-25T10:00:00.000Z', + ...over, +}); + +test('toCsv writes a header and one row per record', () => { + const csv = toCsv([record()]); + assert.equal( + csv, + COLUMNS.join(',') + + '\r\n' + + 'dQw4w9WgXcQ,Hello,https://www.youtube.com/watch?v=dQw4w9WgXcQ,' + + 'https://i.ytimg.com/vi/dQw4w9WgXcQ/hqdefault.jpg,UC1,Chan,https://www.youtube.com/@chan,' + + '1000,50000,2026-10-01T00:00:00Z,600,false,50,,home,' + + '2026-10-25T10:00:00.000Z,2026-10-25T10:00:00.000Z\r\n', + ); +}); + +test('toCsv quotes commas, quotes and newlines', () => { + const csv = toCsv([record({ title: 'He said "hi", then\nleft' })]); + assert.ok(csv.includes('"He said ""hi"", then\nleft"')); +}); + +test('toCsv writes null and undefined as empty cells and keeps booleans readable', () => { + const csv = toCsv([record({ subscribers: null, vidiqScore: 7.5, isShort: true, channelUrl: undefined })]); + const row = csv.split('\r\n')[1]; + assert.ok(row.includes(',true,')); + assert.ok(row.includes(',7.5,')); + assert.ok(row.includes('Chan,,,50000')); // channelUrl and subscribers empty +}); + +test('toCsv of no records is just the header', () => { + assert.equal(toCsv([]), COLUMNS.join(',') + '\r\n'); +}); + +test('toJson pretty-prints and round-trips', () => { + const records = [record(), record({ videoId: 'abcdefghijk' })]; + const text = toJson(records); + assert.ok(text.includes('\n ')); + assert.deepEqual(JSON.parse(text), records); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `node --test test/export.test.js` +Expected: FAIL with `ERR_MODULE_NOT_FOUND`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/export.js`: + +```js +export const COLUMNS = [ + 'videoId', + 'title', + 'url', + 'thumbnailUrl', + 'channelId', + 'channelTitle', + 'channelUrl', + 'subscribers', + 'views', + 'publishedAt', + 'durationSec', + 'isShort', + 'score', + 'vidiqScore', + 'foundOn', + 'firstSeenAt', + 'updatedAt', +]; + +export function toJson(records) { + return JSON.stringify(records, null, 2); +} + +function csvCell(value) { + if (value === null || value === undefined) return ''; + const text = String(value); + return /[",\r\n]/.test(text) ? `"${text.replaceAll('"', '""')}"` : text; +} + +export function toCsv(records) { + const lines = [COLUMNS.join(','), ...records.map((r) => COLUMNS.map((c) => csvCell(r[c])).join(','))]; + return lines.join('\r\n') + '\r\n'; +} +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `node --test test/export.test.js` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/export.js test/export.test.js +git commit -m "feat: add JSON and CSV export" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 5: Settings and status storage + +**Files:** +- Create: `src/lib/storage.js` +- Test: `test/storage.test.js` + +**Interfaces:** +- Consumes: global `chrome.storage.local` (only inside the async functions, never at import time). +- Produces: + - `DEFAULT_SETTINGS: { apiKey: '', threshold: 5, minViews: 10000 }` + - `normalizeSettings(raw?: object): { apiKey: string, threshold: number, minViews: number }` + - `getSettings(): Promise`, `saveSettings(raw: object): Promise` + - `getStatus(): Promise` (defaults to `{ state: 'ok' }`), `setStatus(status: Status): Promise` + +- [ ] **Step 1: Write the failing test** + +`test/storage.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { DEFAULT_SETTINGS, normalizeSettings } from '../src/lib/storage.js'; + +test('normalizeSettings falls back to defaults for missing input', () => { + assert.deepEqual(normalizeSettings(), DEFAULT_SETTINGS); + assert.deepEqual(normalizeSettings({}), DEFAULT_SETTINGS); + assert.deepEqual(DEFAULT_SETTINGS, { apiKey: '', threshold: 5, minViews: 10000 }); +}); + +test('normalizeSettings trims the key and coerces numeric strings', () => { + assert.deepEqual(normalizeSettings({ apiKey: ' abc123 ', threshold: '7.5', minViews: '20000' }), { + apiKey: 'abc123', + threshold: 7.5, + minViews: 20000, + }); +}); + +test('normalizeSettings replaces invalid thresholds with the default', () => { + for (const threshold of [-1, 0, 'abc', '', null, NaN, Infinity]) { + assert.equal(normalizeSettings({ threshold }).threshold, 5, String(threshold)); + } +}); + +test('normalizeSettings replaces invalid minViews with the default and floors decimals', () => { + for (const minViews of [-5, 'abc', '', null, NaN]) { + assert.equal(normalizeSettings({ minViews }).minViews, 10000, String(minViews)); + } + assert.equal(normalizeSettings({ minViews: 1500.7 }).minViews, 1500); + assert.equal(normalizeSettings({ minViews: 0 }).minViews, 0); +}); + +test('normalizeSettings ignores a non-string key', () => { + assert.equal(normalizeSettings({ apiKey: 12345 }).apiKey, ''); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `node --test test/storage.test.js` +Expected: FAIL with `ERR_MODULE_NOT_FOUND`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/storage.js`: + +```js +export const DEFAULT_SETTINGS = { apiKey: '', threshold: 5, minViews: 10000 }; + +const toNumber = (value) => (value === null || value === undefined || value === '' ? NaN : Number(value)); + +export function normalizeSettings(raw = {}) { + const threshold = toNumber(raw.threshold); + const minViews = toNumber(raw.minViews); + return { + apiKey: typeof raw.apiKey === 'string' ? raw.apiKey.trim() : DEFAULT_SETTINGS.apiKey, + threshold: Number.isFinite(threshold) && threshold > 0 ? threshold : DEFAULT_SETTINGS.threshold, + minViews: Number.isFinite(minViews) && minViews >= 0 ? Math.floor(minViews) : DEFAULT_SETTINGS.minViews, + }; +} + +export async function getSettings() { + const { settings } = await chrome.storage.local.get('settings'); + return normalizeSettings(settings); +} + +export async function saveSettings(raw) { + const settings = normalizeSettings(raw); + await chrome.storage.local.set({ settings }); + return settings; +} + +// Status is { state: 'ok' | 'no-key' | 'invalid-key' | 'quota', message?, until? }. +export async function getStatus() { + const { status } = await chrome.storage.local.get('status'); + return status ?? { state: 'ok' }; +} + +export async function setStatus(status) { + await chrome.storage.local.set({ status }); +} +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `node --test test/storage.test.js` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/storage.js test/storage.test.js +git commit -m "feat: add settings normalization and status storage" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 6: API helpers (pure functions) + +**Files:** +- Create: `src/lib/api.js` +- Test: `test/api-helpers.test.js` + +**Interfaces:** +- Consumes: nothing. +- Produces (all exported from `src/lib/api.js`; Task 7 appends to the same file): + - `class ApiError extends Error` with `kind: 'network'|'server'|'quota'|'invalid-key'|'other'`, `status: number|null` + - `chunk(items: T[], size: number): T[][]` + - `parseIsoDuration(iso: string|null|undefined): number|null` (seconds) + - `pickThumbnail(thumbnails: object|null|undefined): string|null` + - `errorReasons(body: object|null): string[]` + - `classifyError(status: number, body: object|null): 'quota'|'invalid-key'|'server'|'other'` + - `nextQuotaReset(now?: Date): number` (epoch ms of the next midnight in America/Los_Angeles) + +- [ ] **Step 1: Write the failing test** + +`test/api-helpers.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { + ApiError, + chunk, + classifyError, + errorReasons, + nextQuotaReset, + parseIsoDuration, + pickThumbnail, +} from '../src/lib/api.js'; + +test('chunk splits into batches and keeps the remainder', () => { + assert.deepEqual(chunk([1, 2, 3, 4, 5], 2), [[1, 2], [3, 4], [5]]); + assert.deepEqual(chunk([], 50), []); + assert.equal(chunk(Array.from({ length: 120 }, (_, i) => i), 50).map((b) => b.length).join(), '50,50,20'); +}); + +test('parseIsoDuration converts ISO 8601 durations to seconds', () => { + assert.equal(parseIsoDuration('PT1H2M3S'), 3723); + assert.equal(parseIsoDuration('PT45S'), 45); + assert.equal(parseIsoDuration('PT10M'), 600); + assert.equal(parseIsoDuration('P1DT1H'), 90000); + assert.equal(parseIsoDuration('P0D'), 0); + assert.equal(parseIsoDuration('PT0S'), 0); +}); + +test('parseIsoDuration returns null for garbage', () => { + assert.equal(parseIsoDuration('garbage'), null); + assert.equal(parseIsoDuration(undefined), null); + assert.equal(parseIsoDuration(null), null); +}); + +test('pickThumbnail prefers the largest available size', () => { + const all = { + default: { url: 'd' }, + medium: { url: 'm' }, + high: { url: 'h' }, + standard: { url: 's' }, + maxres: { url: 'x' }, + }; + assert.equal(pickThumbnail(all), 'x'); + delete all.maxres; + assert.equal(pickThumbnail(all), 's'); + assert.equal(pickThumbnail({ default: { url: 'd' }, medium: { url: 'm' } }), 'm'); + assert.equal(pickThumbnail({}), null); + assert.equal(pickThumbnail(undefined), null); + assert.equal(pickThumbnail(null), null); +}); + +test('errorReasons collects reasons from errors[] and details[]', () => { + const body = { error: { errors: [{ reason: 'badRequest' }], details: [{ reason: 'API_KEY_INVALID' }] } }; + assert.deepEqual(errorReasons(body), ['badRequest', 'API_KEY_INVALID']); + assert.deepEqual(errorReasons(null), []); + assert.deepEqual(errorReasons({}), []); +}); + +test('classifyError recognises quota, key and server problems', () => { + assert.equal(classifyError(403, { error: { errors: [{ reason: 'quotaExceeded' }] } }), 'quota'); + assert.equal(classifyError(403, { error: { errors: [{ reason: 'dailyLimitExceeded' }] } }), 'quota'); + assert.equal( + classifyError(400, { error: { errors: [{ reason: 'badRequest' }], details: [{ reason: 'API_KEY_INVALID' }] } }), + 'invalid-key', + ); + assert.equal(classifyError(403, { error: { errors: [{ reason: 'accessNotConfigured' }] } }), 'invalid-key'); + assert.equal(classifyError(500, null), 'server'); + assert.equal(classifyError(503, {}), 'server'); + assert.equal(classifyError(429, { error: { errors: [{ reason: 'rateLimitExceeded' }] } }), 'server'); + assert.equal(classifyError(404, null), 'other'); +}); + +test('ApiError carries its kind and status', () => { + const err = new ApiError('quota', 'out of quota', 403); + assert.ok(err instanceof Error); + assert.equal(err.name, 'ApiError'); + assert.equal(err.kind, 'quota'); + assert.equal(err.status, 403); + assert.equal(err.message, 'out of quota'); +}); + +test('nextQuotaReset points at the next Pacific midnight (summer time)', () => { + const noon = new Date('2026-10-25T20:00:00Z'); // 13:00 PDT + assert.equal(new Date(nextQuotaReset(noon)).toISOString(), '2026-10-26T07:00:00.000Z'); + const justBefore = new Date('2026-10-26T06:59:59Z'); // 23:59:59 PDT + assert.equal(new Date(nextQuotaReset(justBefore)).toISOString(), '2026-10-26T07:00:00.000Z'); + const exactly = new Date('2026-10-26T07:00:00Z'); // 00:00:00 PDT + assert.equal(new Date(nextQuotaReset(exactly)).toISOString(), '2026-10-27T07:00:00.000Z'); +}); + +test('nextQuotaReset points at the next Pacific midnight (winter time)', () => { + const noon = new Date('2026-12-15T12:00:00Z'); // 04:00 PST + assert.equal(new Date(nextQuotaReset(noon)).toISOString(), '2026-12-16T08:00:00.000Z'); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `node --test test/api-helpers.test.js` +Expected: FAIL with `ERR_MODULE_NOT_FOUND`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/api.js`: + +```js +export class ApiError extends Error { + constructor(kind, message, status = null) { + super(message); + this.name = 'ApiError'; + this.kind = kind; + this.status = status; + } +} + +export function chunk(items, size) { + const batches = []; + for (let i = 0; i < items.length; i += size) batches.push(items.slice(i, i + size)); + return batches; +} + +export function parseIsoDuration(iso) { + const match = /^P(?:(\d+)D)?(?:T(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?)?$/.exec(iso ?? ''); + if (!match) return null; + const [, days = 0, hours = 0, minutes = 0, seconds = 0] = match; + return Number(days) * 86400 + Number(hours) * 3600 + Number(minutes) * 60 + Number(seconds); +} + +const THUMBNAIL_ORDER = ['maxres', 'standard', 'high', 'medium', 'default']; + +export function pickThumbnail(thumbnails) { + for (const size of THUMBNAIL_ORDER) { + const url = thumbnails?.[size]?.url; + if (url) return url; + } + return null; +} + +const QUOTA_REASONS = new Set(['quotaExceeded', 'dailyLimitExceeded']); +const KEY_REASONS = new Set([ + 'keyInvalid', + 'API_KEY_INVALID', + 'accessNotConfigured', + 'ipRefererBlocked', + 'API_KEY_HTTP_REFERRER_BLOCKED', + 'API_KEY_SERVICE_BLOCKED', + 'SERVICE_DISABLED', +]); + +export function errorReasons(body) { + const error = body?.error; + return [...(error?.errors ?? []), ...(error?.details ?? [])].map((entry) => entry.reason).filter(Boolean); +} + +export function classifyError(status, body) { + const reasons = errorReasons(body); + if (reasons.some((reason) => QUOTA_REASONS.has(reason))) return 'quota'; + if (reasons.some((reason) => KEY_REASONS.has(reason))) return 'invalid-key'; + if (status >= 500 || status === 429) return 'server'; + return 'other'; +} + +// The Data API quota resets at midnight Pacific Time. +export function nextQuotaReset(now = new Date()) { + const parts = new Intl.DateTimeFormat('en-US', { + timeZone: 'America/Los_Angeles', + hourCycle: 'h23', + hour: 'numeric', + minute: 'numeric', + second: 'numeric', + }).formatToParts(now); + const part = (type) => Number(parts.find((p) => p.type === type).value); + const sinceMidnightMs = + ((part('hour') * 60 + part('minute')) * 60 + part('second')) * 1000 + now.getMilliseconds(); + return now.getTime() - sinceMidnightMs + 24 * 60 * 60 * 1000; +} +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `node --test test/api-helpers.test.js` +Expected: PASS, 8 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/api.js test/api-helpers.test.js +git commit -m "feat: add YouTube API helper functions" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 7: API requests with retries + +**Files:** +- Modify: `src/lib/api.js` (append below the Task 6 code) +- Test: `test/api-requests.test.js` + +**Interfaces:** +- Consumes (same file): `ApiError`, `chunk`, `parseIsoDuration`, `pickThumbnail`, `classifyError`. +- Produces: + - `normalizeVideo(item): Video`, `normalizeChannel(item): Channel` + - `fetchVideos(ids: string[], opts: { apiKey: string, fetchImpl?: Function, sleep?: (ms) => Promise }): Promise<{ videos: Video[], missingIds: string[] }>` + - `fetchChannels(ids: string[], opts): Promise` + - Throws `ApiError` (kinds as in Task 6) after retries are exhausted. + +- [ ] **Step 1: Write the failing test** + +`test/api-requests.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { ApiError, fetchChannels, fetchVideos } from '../src/lib/api.js'; + +const jsonResponse = (status, body) => ({ ok: status >= 200 && status < 300, status, json: async () => body }); +const noSleep = async () => {}; + +const videoItem = (id) => ({ + id, + snippet: { + title: `Title ${id}`, + channelId: 'UC1', + channelTitle: 'Chan', + publishedAt: '2026-10-01T00:00:00Z', + thumbnails: { high: { url: `https://i.ytimg.com/vi/${id}/hqdefault.jpg` } }, + }, + statistics: { viewCount: '12345' }, + contentDetails: { duration: 'PT10M' }, +}); + +test('fetchVideos splits ids into batches of 50 and normalizes items', async () => { + const ids = Array.from({ length: 120 }, (_, i) => `v${String(i).padStart(10, '0')}`); + const seen = []; + const fetchImpl = async (url) => { + const u = new URL(url); + const batch = u.searchParams.get('id').split(','); + seen.push({ path: u.pathname, size: batch.length, key: u.searchParams.get('key'), part: u.searchParams.get('part') }); + return jsonResponse(200, { items: batch.map(videoItem) }); + }; + + const { videos, missingIds } = await fetchVideos(ids, { apiKey: 'KEY', fetchImpl, sleep: noSleep }); + + assert.deepEqual(seen.map((s) => s.size), [50, 50, 20]); + assert.ok(seen.every((s) => s.path === '/youtube/v3/videos' && s.key === 'KEY')); + assert.ok(seen.every((s) => s.part === 'snippet,statistics,contentDetails')); + assert.equal(videos.length, 120); + assert.deepEqual(missingIds, []); + assert.deepEqual(videos[0], { + videoId: ids[0], + title: `Title ${ids[0]}`, + thumbnailUrl: `https://i.ytimg.com/vi/${ids[0]}/hqdefault.jpg`, + channelId: 'UC1', + channelTitle: 'Chan', + views: 12345, + publishedAt: '2026-10-01T00:00:00Z', + durationSec: 600, + }); +}); + +test('fetchVideos reports ids the API did not return', async () => { + const fetchImpl = async (url) => { + const batch = new URL(url).searchParams.get('id').split(','); + return jsonResponse(200, { items: batch.filter((id) => id !== 'missing0001').map(videoItem) }); + }; + const { videos, missingIds } = await fetchVideos(['abcdefghijk', 'missing0001'], { + apiKey: 'K', + fetchImpl, + sleep: noSleep, + }); + assert.equal(videos.length, 1); + assert.deepEqual(missingIds, ['missing0001']); +}); + +test('requests are retried on 5xx with exponential backoff', async () => { + const delays = []; + let calls = 0; + const fetchImpl = async () => { + calls += 1; + return calls < 3 ? jsonResponse(503, {}) : jsonResponse(200, { items: [videoItem('abcdefghijk')] }); + }; + const { videos } = await fetchVideos(['abcdefghijk'], { + apiKey: 'K', + fetchImpl, + sleep: async (ms) => delays.push(ms), + }); + assert.equal(videos.length, 1); + assert.equal(calls, 3); + assert.deepEqual(delays, [500, 1000]); +}); + +test('requests give up after three retries', async () => { + let calls = 0; + const fetchImpl = async () => { + calls += 1; + return jsonResponse(500, {}); + }; + await assert.rejects( + fetchVideos(['abcdefghijk'], { apiKey: 'K', fetchImpl, sleep: noSleep }), + (err) => err instanceof ApiError && err.kind === 'server' && err.status === 500, + ); + assert.equal(calls, 4); +}); + +test('network failures are retried and reported as kind network', async () => { + let calls = 0; + const fetchImpl = async () => { + calls += 1; + throw new TypeError('Failed to fetch'); + }; + await assert.rejects( + fetchVideos(['abcdefghijk'], { apiKey: 'K', fetchImpl, sleep: noSleep }), + (err) => err instanceof ApiError && err.kind === 'network', + ); + assert.equal(calls, 4); +}); + +test('quota errors are not retried', async () => { + let calls = 0; + const fetchImpl = async () => { + calls += 1; + return jsonResponse(403, { error: { message: 'quota', errors: [{ reason: 'quotaExceeded' }] } }); + }; + await assert.rejects( + fetchVideos(['abcdefghijk'], { apiKey: 'K', fetchImpl, sleep: noSleep }), + (err) => err instanceof ApiError && err.kind === 'quota' && err.message === 'quota', + ); + assert.equal(calls, 1); +}); + +test('invalid key errors are not retried', async () => { + let calls = 0; + const fetchImpl = async () => { + calls += 1; + return jsonResponse(400, { + error: { message: 'API key not valid.', errors: [{ reason: 'badRequest' }], details: [{ reason: 'API_KEY_INVALID' }] }, + }); + }; + await assert.rejects( + fetchVideos(['abcdefghijk'], { apiKey: 'bad', fetchImpl, sleep: noSleep }), + (err) => err instanceof ApiError && err.kind === 'invalid-key', + ); + assert.equal(calls, 1); +}); + +test('fetchChannels normalizes subscriber counts and channel urls', async () => { + let params; + const fetchImpl = async (url) => { + params = new URL(url).searchParams; + return jsonResponse(200, { + items: [ + { id: 'UC1', snippet: { title: 'One', customUrl: '@one' }, statistics: { subscriberCount: '1500' } }, + { id: 'UC2', snippet: { title: 'Two' }, statistics: { hiddenSubscriberCount: true } }, + { id: 'UC3', snippet: { title: 'Three', customUrl: 'legacyname' }, statistics: { subscriberCount: '7' } }, + ], + }); + }; + + const channels = await fetchChannels(['UC1', 'UC2', 'UC3'], { apiKey: 'K', fetchImpl, sleep: noSleep }); + + assert.equal(params.get('part'), 'statistics,snippet'); + assert.equal(params.get('maxResults'), '50'); + assert.equal(params.get('id'), 'UC1,UC2,UC3'); + assert.deepEqual(channels, [ + { channelId: 'UC1', title: 'One', url: 'https://www.youtube.com/@one', subscribers: 1500 }, + { channelId: 'UC2', title: 'Two', url: 'https://www.youtube.com/channel/UC2', subscribers: null }, + { channelId: 'UC3', title: 'Three', url: 'https://www.youtube.com/channel/UC3', subscribers: 7 }, + ]); +}); + +test('fetchChannels of an empty list makes no request', async () => { + const fetchImpl = async () => { + throw new Error('should not be called'); + }; + assert.deepEqual(await fetchChannels([], { apiKey: 'K', fetchImpl, sleep: noSleep }), []); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `node --test test/api-requests.test.js` +Expected: FAIL — `fetchVideos` / `fetchChannels` are not exported (SyntaxError: does not provide an export named …). + +- [ ] **Step 3: Append the implementation to `src/lib/api.js`** + +```js +const API_BASE = 'https://www.googleapis.com/youtube/v3'; +const BATCH_SIZE = 50; +const RETRIES = 3; +const BASE_DELAY_MS = 500; + +const defaultSleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +async function requestOnce(path, params, { apiKey, fetchImpl = fetch }) { + const url = `${API_BASE}/${path}?${new URLSearchParams({ ...params, key: apiKey })}`; + let response; + try { + response = await fetchImpl(url); + } catch (err) { + throw new ApiError('network', err.message); + } + if (response.ok) { + try { + return await response.json(); + } catch { + throw new ApiError('server', 'Invalid JSON in API response', response.status); + } + } + let body = null; + try { + body = await response.json(); + } catch { + // error responses are not always JSON + } + throw new ApiError(classifyError(response.status, body), body?.error?.message ?? `HTTP ${response.status}`, response.status); +} + +async function request(path, params, opts) { + const sleep = opts.sleep ?? defaultSleep; + for (let attempt = 0; ; attempt += 1) { + try { + return await requestOnce(path, params, opts); + } catch (err) { + const retryable = err instanceof ApiError && (err.kind === 'network' || err.kind === 'server'); + if (!retryable || attempt >= RETRIES) throw err; + await sleep(BASE_DELAY_MS * 2 ** attempt); + } + } +} + +export function normalizeVideo(item) { + const { snippet = {}, statistics = {}, contentDetails = {} } = item; + return { + videoId: item.id, + title: snippet.title ?? '', + thumbnailUrl: pickThumbnail(snippet.thumbnails), + channelId: snippet.channelId, + channelTitle: snippet.channelTitle ?? '', + views: Number(statistics.viewCount ?? 0), + publishedAt: snippet.publishedAt ?? null, + durationSec: parseIsoDuration(contentDetails.duration), + }; +} + +export function normalizeChannel(item) { + const customUrl = item.snippet?.customUrl; + const stats = item.statistics ?? {}; + return { + channelId: item.id, + title: item.snippet?.title ?? '', + url: customUrl?.startsWith('@') + ? `https://www.youtube.com/${customUrl}` + : `https://www.youtube.com/channel/${item.id}`, + subscribers: stats.hiddenSubscriberCount || stats.subscriberCount === undefined ? null : Number(stats.subscriberCount), + }; +} + +export async function fetchVideos(ids, opts) { + const videos = []; + const found = new Set(); + for (const batch of chunk(ids, BATCH_SIZE)) { + const data = await request( + 'videos', + { part: 'snippet,statistics,contentDetails', id: batch.join(',') }, + opts, + ); + for (const item of data.items ?? []) { + videos.push(normalizeVideo(item)); + found.add(item.id); + } + } + return { videos, missingIds: ids.filter((id) => !found.has(id)) }; +} + +export async function fetchChannels(ids, opts) { + const channels = []; + for (const batch of chunk(ids, BATCH_SIZE)) { + const data = await request( + 'channels', + { part: 'statistics,snippet', id: batch.join(','), maxResults: String(BATCH_SIZE) }, + opts, + ); + for (const item of data.items ?? []) channels.push(normalizeChannel(item)); + } + return channels; +} +``` + +`maxResults` is deliberately sent for `channels` only: `videos.list` rejects `maxResults` together with `id`, while `channels.list` defaults to 5 results. + +- [ ] **Step 4: Run all tests to verify they pass** + +Run: `npm test` +Expected: PASS for every file, 0 failures. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/api.js test/api-requests.test.js +git commit -m "feat: fetch videos and channels from the YouTube Data API" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 8: Batch processing (core logic) + +**Files:** +- Create: `src/lib/process.js` +- Test: `test/process.test.js` + +**Interfaces:** +- Consumes: `computeScore`, `shouldSave` from `./score.js`. +- Produces: + - `RECHECK_MS = 24 * 60 * 60 * 1000` + - `processBatch(items: QueueItem[], ctx: { settings: { threshold, minViews }, db, api, now?: Date }): Promise<{ checked: number, saved: number }>` + - `ctx.db` must provide: `getSeen(ids): Promise`, `putSeen(records)`, `getChannels(ids): Promise`, `putChannels(records)`, `getOutliers(ids): Promise`, `putOutliers(records)`. + - `ctx.api` must provide: `fetchVideos(ids): Promise<{ videos, missingIds }>`, `fetchChannels(ids): Promise`. + - Rejects (writing nothing) if an API call throws. + +Behavior: items whose `seen` record is younger than 24 h are skipped, **except** items carrying a `vidiqScore` (a VidIQ badge may render after the first scan). A video already stored in `outliers` is always rewritten with fresh stats. + +- [ ] **Step 1: Write the failing test** + +`test/process.test.js`: + +```js +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { processBatch } from '../src/lib/process.js'; + +const NOW = new Date('2026-10-25T10:00:00.000Z'); +const NOW_ISO = NOW.toISOString(); +const HOUR = 3600 * 1000; +const ago = (hours) => new Date(NOW.getTime() - hours * HOUR).toISOString(); +const SETTINGS = { threshold: 5, minViews: 10000 }; +const ID = 'vid00000001'; + +function memoryDb() { + const stores = { seen: new Map(), channels: new Map(), outliers: new Map() }; + const getter = (name) => async (keys) => + new Map(keys.filter((key) => stores[name].has(key)).map((key) => [key, stores[name].get(key)])); + const putter = (name, keyField) => async (records) => { + for (const record of records) stores[name].set(record[keyField], record); + }; + return { + stores, + getSeen: getter('seen'), + getChannels: getter('channels'), + getOutliers: getter('outliers'), + putSeen: putter('seen', 'videoId'), + putChannels: putter('channels', 'channelId'), + putOutliers: putter('outliers', 'videoId'), + }; +} + +function fakeApi({ videos = [], channels = [] } = {}) { + const calls = { videos: [], channels: [] }; + return { + calls, + fetchVideos: async (ids) => { + calls.videos.push(ids); + const found = videos.filter((v) => ids.includes(v.videoId)); + return { videos: found, missingIds: ids.filter((id) => !found.some((v) => v.videoId === id)) }; + }, + fetchChannels: async (ids) => { + calls.channels.push(ids); + return channels.filter((c) => ids.includes(c.channelId)); + }, + }; +} + +const item = (videoId = ID, over = {}) => ({ videoId, vidiqScore: null, viaShortsUrl: false, page: 'home', ...over }); +const video = (videoId = ID, over = {}) => ({ + videoId, + title: 'Title', + thumbnailUrl: 'https://i.ytimg.com/t.jpg', + channelId: 'UC1', + channelTitle: 'Chan', + views: 60000, + publishedAt: '2026-10-01T00:00:00Z', + durationSec: 600, + ...over, +}); +const channel = (channelId, subscribers) => ({ + channelId, + title: 'Chan', + url: 'https://www.youtube.com/@chan', + subscribers, +}); +const run = (items, db, api) => processBatch(items, { settings: SETTINGS, db, api, now: NOW }); + +test('saves a video whose score reaches the threshold', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 10000)] }); + + const result = await run([item(ID, { page: 'search' })], db, api); + + assert.deepEqual(result, { checked: 1, saved: 1 }); + assert.deepEqual(db.stores.outliers.get(ID), { + videoId: ID, + title: 'Title', + url: `https://www.youtube.com/watch?v=${ID}`, + thumbnailUrl: 'https://i.ytimg.com/t.jpg', + channelId: 'UC1', + channelTitle: 'Chan', + channelUrl: 'https://www.youtube.com/@chan', + subscribers: 10000, + views: 60000, + publishedAt: '2026-10-01T00:00:00Z', + durationSec: 600, + isShort: false, + score: 6, + vidiqScore: null, + foundOn: 'search', + firstSeenAt: NOW_ISO, + updatedAt: NOW_ISO, + }); + assert.deepEqual(db.stores.seen.get(ID), { videoId: ID, checkedAt: NOW_ISO, status: 'ok' }); + assert.deepEqual(db.stores.channels.get('UC1'), { ...channel('UC1', 10000), fetchedAt: NOW_ISO }); +}); + +test('marks a below-threshold video as seen without saving it', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 100000)] }); + const result = await run([item()], db, api); + assert.deepEqual(result, { checked: 1, saved: 0 }); + assert.equal(db.stores.outliers.size, 0); + assert.equal(db.stores.seen.get(ID).status, 'ok'); +}); + +test('does not save a video below minViews even with a huge score', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [video(ID, { views: 9000 })], channels: [channel('UC1', 100)] }); + await run([item()], db, api); + assert.equal(db.stores.outliers.size, 0); +}); + +test('saves a VidIQ-flagged video even when subscribers are hidden', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', null)] }); + await run([item(ID, { vidiqScore: 7.5 })], db, api); + const saved = db.stores.outliers.get(ID); + assert.equal(saved.score, null); + assert.equal(saved.subscribers, null); + assert.equal(saved.vidiqScore, 7.5); +}); + +test('does not save a video with hidden subscribers and no VidIQ score', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', null)] }); + await run([item()], db, api); + assert.equal(db.stores.outliers.size, 0); +}); + +test('treats a channel missing from the API response as unknown subscribers', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [video()], channels: [] }); + await run([item(ID, { vidiqScore: 4 })], db, api); + assert.equal(db.stores.outliers.get(ID).subscribers, null); + assert.equal(db.stores.outliers.get(ID).channelUrl, 'https://www.youtube.com/channel/UC1'); +}); + +test('skips ids checked less than 24 hours ago', async () => { + const db = memoryDb(); + db.stores.seen.set(ID, { videoId: ID, checkedAt: ago(1), status: 'ok' }); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 10000)] }); + const result = await run([item()], db, api); + assert.deepEqual(result, { checked: 0, saved: 0 }); + assert.equal(api.calls.videos.length, 0); +}); + +test('rechecks ids older than 24 hours', async () => { + const db = memoryDb(); + db.stores.seen.set(ID, { videoId: ID, checkedAt: ago(25), status: 'ok' }); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 10000)] }); + const result = await run([item()], db, api); + assert.deepEqual(result, { checked: 1, saved: 1 }); +}); + +test('an item with a VidIQ score bypasses a fresh seen record', async () => { + const db = memoryDb(); + db.stores.seen.set(ID, { videoId: ID, checkedAt: ago(1), status: 'ok' }); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 1000000)] }); + const result = await run([item(ID, { vidiqScore: 4 })], db, api); + assert.deepEqual(result, { checked: 1, saved: 1 }); + assert.equal(db.stores.outliers.get(ID).vidiqScore, 4); +}); + +test('a repeat sighting refreshes stats and keeps the original metadata', async () => { + const db = memoryDb(); + db.stores.outliers.set(ID, { + videoId: ID, + firstSeenAt: '2026-10-20T00:00:00.000Z', + foundOn: 'home', + isShort: true, + vidiqScore: 9, + }); + db.stores.seen.set(ID, { videoId: ID, checkedAt: ago(30), status: 'ok' }); + const api = fakeApi({ videos: [video(ID, { views: 70000 })], channels: [channel('UC1', 10000)] }); + + await run([item(ID, { page: 'search', viaShortsUrl: false })], db, api); + + const saved = db.stores.outliers.get(ID); + assert.equal(saved.firstSeenAt, '2026-10-20T00:00:00.000Z'); + assert.equal(saved.foundOn, 'home'); + assert.equal(saved.isShort, true); + assert.equal(saved.vidiqScore, 9); + assert.equal(saved.views, 70000); + assert.equal(saved.score, 7); + assert.equal(saved.updatedAt, NOW_ISO); +}); + +test('an already saved video is updated even when its score fell below the threshold', async () => { + const db = memoryDb(); + db.stores.outliers.set(ID, { videoId: ID, firstSeenAt: ago(100), foundOn: 'home', isShort: false, vidiqScore: null }); + db.stores.seen.set(ID, { videoId: ID, checkedAt: ago(30), status: 'ok' }); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 1000000)] }); + const result = await run([item()], db, api); + assert.deepEqual(result, { checked: 1, saved: 1 }); + assert.equal(db.stores.outliers.get(ID).score, 0.06); +}); + +test('a short found via a /shorts/ link is flagged', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 10000)] }); + await run([item(ID, { viaShortsUrl: true })], db, api); + assert.equal(db.stores.outliers.get(ID).isShort, true); +}); + +test('records unavailable videos in seen and does not save them', async () => { + const db = memoryDb(); + const api = fakeApi({ videos: [], channels: [] }); + const result = await run([item()], db, api); + assert.deepEqual(result, { checked: 1, saved: 0 }); + assert.deepEqual(db.stores.seen.get(ID), { videoId: ID, checkedAt: NOW_ISO, status: 'unavailable' }); + assert.equal(db.stores.outliers.size, 0); +}); + +test('reuses a cached channel younger than 24 hours', async () => { + const db = memoryDb(); + db.stores.channels.set('UC1', { ...channel('UC1', 10000), fetchedAt: ago(1) }); + const api = fakeApi({ videos: [video()], channels: [] }); + const result = await run([item()], db, api); + assert.equal(api.calls.channels.length, 0); + assert.deepEqual(result, { checked: 1, saved: 1 }); +}); + +test('refetches a cached channel older than 24 hours', async () => { + const db = memoryDb(); + db.stores.channels.set('UC1', { ...channel('UC1', 10000), fetchedAt: ago(25) }); + const api = fakeApi({ videos: [video()], channels: [channel('UC1', 20000)] }); + const result = await run([item()], db, api); + assert.deepEqual(api.calls.channels, [['UC1']]); + assert.equal(db.stores.channels.get('UC1').subscribers, 20000); + assert.deepEqual(result, { checked: 1, saved: 0 }); // score 3 < 5 +}); + +test('an API failure writes nothing and does not mark ids as seen', async () => { + const db = memoryDb(); + const api = { + ...fakeApi(), + fetchVideos: async () => { + throw new Error('boom'); + }, + }; + await assert.rejects(run([item()], db, api), /boom/); + assert.equal(db.stores.seen.size, 0); + assert.equal(db.stores.outliers.size, 0); + assert.equal(db.stores.channels.size, 0); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `node --test test/process.test.js` +Expected: FAIL with `ERR_MODULE_NOT_FOUND`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/process.js`: + +```js +import { computeScore, shouldSave } from './score.js'; + +export const RECHECK_MS = 24 * 60 * 60 * 1000; + +const watchUrl = (videoId) => `https://www.youtube.com/watch?v=${videoId}`; + +function isFresh(checkedAt, nowMs) { + return checkedAt !== undefined && nowMs - Date.parse(checkedAt) < RECHECK_MS; +} + +export async function processBatch(items, { settings, db, api, now = new Date() }) { + const nowMs = now.getTime(); + const nowIso = now.toISOString(); + const byId = new Map(items.map((item) => [item.videoId, item])); + + const seen = await db.getSeen([...byId.keys()]); + const due = [...byId.values()].filter( + (item) => item.vidiqScore !== null || !isFresh(seen.get(item.videoId)?.checkedAt, nowMs), + ); + if (due.length === 0) return { checked: 0, saved: 0 }; + + const { videos, missingIds } = await api.fetchVideos(due.map((item) => item.videoId)); + + const channelIds = [...new Set(videos.map((video) => video.channelId))]; + const cachedChannels = await db.getChannels(channelIds); + const staleIds = channelIds.filter((id) => !isFresh(cachedChannels.get(id)?.fetchedAt, nowMs)); + const fetchedChannels = staleIds.length > 0 ? await api.fetchChannels(staleIds) : []; + const channelRecords = fetchedChannels.map((channel) => ({ ...channel, fetchedAt: nowIso })); + const channels = new Map(cachedChannels); + for (const record of channelRecords) channels.set(record.channelId, record); + + const existing = await db.getOutliers(videos.map((video) => video.videoId)); + const outliers = []; + for (const video of videos) { + const item = byId.get(video.videoId); + const previous = existing.get(video.videoId); + const channel = channels.get(video.channelId); + const subscribers = channel?.subscribers ?? null; + const score = computeScore(video.views, subscribers); + const vidiqScore = item.vidiqScore ?? previous?.vidiqScore ?? null; + + if (!previous && !shouldSave({ views: video.views, score, vidiqScore }, settings)) continue; + + outliers.push({ + videoId: video.videoId, + title: video.title, + url: watchUrl(video.videoId), + thumbnailUrl: video.thumbnailUrl, + channelId: video.channelId, + channelTitle: video.channelTitle, + channelUrl: channel?.url ?? `https://www.youtube.com/channel/${video.channelId}`, + subscribers, + views: video.views, + publishedAt: video.publishedAt, + durationSec: video.durationSec, + isShort: item.viaShortsUrl || previous?.isShort || false, + score, + vidiqScore, + foundOn: previous?.foundOn ?? item.page, + firstSeenAt: previous?.firstSeenAt ?? nowIso, + updatedAt: nowIso, + }); + } + + // Results first, `seen` last: if anything above throws, the ids are retried later. + await db.putChannels(channelRecords); + await db.putOutliers(outliers); + await db.putSeen([ + ...videos.map((video) => ({ videoId: video.videoId, checkedAt: nowIso, status: 'ok' })), + ...missingIds.map((videoId) => ({ videoId, checkedAt: nowIso, status: 'unavailable' })), + ]); + return { checked: due.length, saved: outliers.length }; +} +``` + +- [ ] **Step 4: Run all tests to verify they pass** + +Run: `npm test` +Expected: PASS for every file, 0 failures. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/process.js test/process.test.js +git commit -m "feat: add batch processing with injected db and api" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 9: IndexedDB wrapper + +IndexedDB does not exist in Node and the project allows no dependencies, so this module is syntax-checked here and exercised for real in Tasks 12–13. + +**Files:** +- Create: `src/lib/db.js` + +**Interfaces:** +- Consumes: global `indexedDB`. +- Produces (exactly the names `processBatch` expects, plus popup helpers): + - `openDb(): Promise` + - `getSeen(ids: string[]): Promise>`, `putSeen(records: Seen[]): Promise` + - `getChannels(ids: string[]): Promise>`, `putChannels(records): Promise` + - `getOutliers(ids: string[]): Promise>`, `putOutliers(records): Promise` + - `listOutliers(limit?: number): Promise` — newest `updatedAt` first + - `countOutliers(): Promise` + +- [ ] **Step 1: Write the module** + +`src/lib/db.js`: + +```js +const DB_NAME = 'outlier-saver'; +const DB_VERSION = 1; + +let dbPromise = null; + +export function openDb() { + dbPromise ??= new Promise((resolve, reject) => { + const request = indexedDB.open(DB_NAME, DB_VERSION); + request.onupgradeneeded = () => { + const db = request.result; + db.createObjectStore('outliers', { keyPath: 'videoId' }).createIndex('updatedAt', 'updatedAt'); + db.createObjectStore('channels', { keyPath: 'channelId' }); + db.createObjectStore('seen', { keyPath: 'videoId' }); + }; + request.onsuccess = () => resolve(request.result); + request.onerror = () => reject(request.error); + }).catch((err) => { + dbPromise = null; + throw err; + }); + return dbPromise; +} + +function promisify(request) { + return new Promise((resolve, reject) => { + request.onsuccess = () => resolve(request.result); + request.onerror = () => reject(request.error); + }); +} + +async function getMany(storeName, keys) { + const db = await openDb(); + const store = db.transaction(storeName, 'readonly').objectStore(storeName); + const records = await Promise.all(keys.map((key) => promisify(store.get(key)))); + return new Map(records.filter(Boolean).map((record) => [record[store.keyPath], record])); +} + +async function putMany(storeName, records) { + if (records.length === 0) return; + const db = await openDb(); + await new Promise((resolve, reject) => { + const tx = db.transaction(storeName, 'readwrite'); + const store = tx.objectStore(storeName); + for (const record of records) store.put(record); + tx.oncomplete = () => resolve(); + tx.onerror = () => reject(tx.error); + tx.onabort = () => reject(tx.error); + }); +} + +export const getSeen = (videoIds) => getMany('seen', videoIds); +export const putSeen = (records) => putMany('seen', records); +export const getChannels = (channelIds) => getMany('channels', channelIds); +export const putChannels = (records) => putMany('channels', records); +export const getOutliers = (videoIds) => getMany('outliers', videoIds); +export const putOutliers = (records) => putMany('outliers', records); + +export async function listOutliers(limit = Infinity) { + const db = await openDb(); + return new Promise((resolve, reject) => { + const results = []; + const request = db + .transaction('outliers', 'readonly') + .objectStore('outliers') + .index('updatedAt') + .openCursor(null, 'prev'); + request.onsuccess = () => { + const cursor = request.result; + if (!cursor || results.length >= limit) { + resolve(results); + return; + } + results.push(cursor.value); + cursor.continue(); + }; + request.onerror = () => reject(request.error); + }); +} + +export async function countOutliers() { + const db = await openDb(); + return promisify(db.transaction('outliers', 'readonly').objectStore('outliers').count()); +} +``` + +- [ ] **Step 2: Syntax-check** + +Run: `node --check src/lib/db.js` +Expected: no output, exit code 0. + +- [ ] **Step 3: Commit** + +```bash +git add src/lib/db.js +git commit -m "feat: add IndexedDB wrapper" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 10: Service worker and manifest + +**Files:** +- Create: `manifest.json`, `src/background.js` + +**Interfaces:** +- Consumes: `getSettings`, `getStatus`, `setStatus` (storage.js); every export of `db.js`; `fetchVideos`, `fetchChannels`, `ApiError`, `nextQuotaReset` (api.js); `processBatch` (process.js); `isVideoId` (video-id.js). +- Produces: a worker that + - accepts `chrome.runtime` messages `{ type: 'videos-seen', items: QueueItem[] }`, + - flushes the queue in batches of 50 about 1 s after the last enqueue, + - writes `status` to `chrome.storage.local` and shows/clears a `!` badge, + - resets status and flushes when the API key changes. + +- [ ] **Step 1: Write `manifest.json`** + +```json +{ + "manifest_version": 3, + "name": "Outlier Saver", + "version": "0.1.0", + "description": "Находит на YouTube видео с аномальными просмотрами и сохраняет их локально.", + "permissions": ["storage"], + "host_permissions": ["https://www.googleapis.com/youtube/v3/*"], + "background": { "service_worker": "src/background.js", "type": "module" }, + "action": { "default_title": "Outlier Saver" } +} +``` + +- [ ] **Step 2: Write `src/background.js`** + +```js +import { ApiError, fetchChannels, fetchVideos, nextQuotaReset } from './lib/api.js'; +import * as db from './lib/db.js'; +import { processBatch } from './lib/process.js'; +import { getSettings, getStatus, setStatus } from './lib/storage.js'; +import { isVideoId } from './lib/video-id.js'; + +const BATCH_SIZE = 50; +const FLUSH_DELAY_MS = 1000; +const MAX_QUEUE = 5000; +const PAGES = new Set(['home', 'watch', 'search', 'channel']); + +// videoId -> queue item. In memory only: if the worker is killed, the content script +// re-sends visible ids after the next navigation, and `seen` is written only after success. +const queue = new Map(); +let flushTimer = null; +let flushing = false; + +function sanitizeItem(raw) { + if (!raw || !isVideoId(raw.videoId)) return null; + return { + videoId: raw.videoId, + vidiqScore: typeof raw.vidiqScore === 'number' && Number.isFinite(raw.vidiqScore) ? raw.vidiqScore : null, + viaShortsUrl: raw.viaShortsUrl === true, + page: PAGES.has(raw.page) ? raw.page : 'home', + }; +} + +function enqueue(rawItems) { + for (const raw of rawItems) { + const item = sanitizeItem(raw); + if (!item) continue; + const queued = queue.get(item.videoId); + if (queued) { + queued.vidiqScore = item.vidiqScore ?? queued.vidiqScore; + queued.viaShortsUrl = queued.viaShortsUrl || item.viaShortsUrl; + } else if (queue.size < MAX_QUEUE) { + queue.set(item.videoId, item); + } + } + scheduleFlush(); +} + +function requeue(items) { + for (const item of items) if (!queue.has(item.videoId)) queue.set(item.videoId, item); +} + +function takeBatch() { + const batch = []; + for (const [videoId, item] of queue) { + if (batch.length >= BATCH_SIZE) break; + batch.push(item); + queue.delete(videoId); + } + return batch; +} + +function scheduleFlush() { + if (flushTimer !== null || flushing || queue.size === 0) return; + flushTimer = setTimeout(flush, FLUSH_DELAY_MS); +} + +async function updateBadge(status) { + if (status.state === 'ok') { + await chrome.action.setBadgeText({ text: '' }); + return; + } + await chrome.action.setBadgeBackgroundColor({ color: '#d93025' }); + await chrome.action.setBadgeText({ text: '!' }); +} + +async function reportStatus(status) { + await setStatus(status); + await updateBadge(status); +} + +// Returns true when the flush loop may continue, false when collection must stop. +async function handleError(err, batch) { + if (err instanceof ApiError && err.kind === 'quota') { + requeue(batch); + await reportStatus({ state: 'quota', until: nextQuotaReset(), message: err.message }); + return false; + } + if (err instanceof ApiError && err.kind === 'invalid-key') { + requeue(batch); + await reportStatus({ state: 'invalid-key', message: err.message }); + return false; + } + console.warn(`Outlier Saver: dropped a batch of ${batch.length} ids`, err); + return true; +} + +async function flush() { + flushTimer = null; + if (flushing) return; + flushing = true; + try { + while (queue.size > 0) { + const settings = await getSettings(); + if (!settings.apiKey) { + await reportStatus({ state: 'no-key' }); + return; + } + const status = await getStatus(); + if (status.state === 'invalid-key') return; + if (status.state === 'quota' && Date.now() < status.until) return; + if (status.state !== 'ok') await reportStatus({ state: 'ok' }); + + const batch = takeBatch(); + const apiKey = settings.apiKey; + const api = { + fetchVideos: (ids) => fetchVideos(ids, { apiKey }), + fetchChannels: (ids) => fetchChannels(ids, { apiKey }), + }; + try { + await processBatch(batch, { settings, db, api }); + } catch (err) { + if (!(await handleError(err, batch))) return; + } + } + } finally { + flushing = false; + } +} + +chrome.runtime.onMessage.addListener((message) => { + if (message?.type === 'videos-seen' && Array.isArray(message.items)) enqueue(message.items); +}); + +// A new API key clears a previous "no key / invalid key / quota" pause. +chrome.storage.onChanged.addListener((changes, area) => { + if (area !== 'local' || !changes.settings) return; + if (changes.settings.oldValue?.apiKey === changes.settings.newValue?.apiKey) return; + reportStatus({ state: 'ok' }).then(scheduleFlush); +}); + +getStatus().then(updateBadge); +``` + +- [ ] **Step 3: Verify** + +Run: `node --check src/background.js && npm test` +Expected: no syntax errors, all tests pass. + +Manual (needs Chrome): open `chrome://extensions`, enable Developer mode, **Load unpacked** → select the project folder. Expected: the card shows no "Errors" button; clicking **service worker** opens a DevTools console with no red errors. + +- [ ] **Step 4: Commit** + +```bash +git add manifest.json src/background.js +git commit -m "feat: add service worker with batching and status handling" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 11: Options page + +**Files:** +- Create: `src/options/options.html`, `src/options/options.css`, `src/options/options.js` +- Modify: `manifest.json` (add `options_ui`) + +**Interfaces:** +- Consumes: `getSettings`, `saveSettings` from `../lib/storage.js`. +- Produces: an options page that persists `{ apiKey, threshold, minViews }` under `settings`. + +- [ ] **Step 1: Write `src/options/options.html`** + +```html + + + + + + Outlier Saver — настройки + + + +
+

Outlier Saver

+
+ + + +
+ + +
+
+
+ Как получить API-ключ +
    +
  1. Откройте Google Cloud Console и создайте проект.
  2. +
  3. В разделе «API и сервисы → Библиотека» включите YouTube Data API v3.
  4. +
  5. В «Учётные данные» создайте API-ключ.
  6. +
  7. В ограничениях ключа выберите «Ограничить API → YouTube Data API v3». Ограничение по HTTP-источникам (referrer) не включайте: запросы идут из расширения.
  8. +
  9. Вставьте ключ в поле выше и нажмите «Сохранить».
  10. +
+
+
+ + + +``` + +- [ ] **Step 2: Write `src/options/options.css`** + +```css +:root { + color-scheme: light dark; + --line: color-mix(in srgb, currentColor 20%, transparent); + --muted: color-mix(in srgb, currentColor 60%, transparent); +} +body { margin: 0; font: 15px/1.5 system-ui, sans-serif; } +main { max-width: 520px; margin: 0 auto; padding: 24px 16px; } +h1 { font-size: 20px; margin: 0 0 16px; } +form { display: grid; gap: 16px; } +label { display: grid; gap: 4px; font-weight: 600; } +input { padding: 8px 10px; font: inherit; border: 1px solid var(--line); border-radius: 6px; background: transparent; color: inherit; } +.actions { display: flex; align-items: center; gap: 12px; } +button { padding: 8px 16px; font: inherit; border: 0; border-radius: 6px; background: #1a73e8; color: #fff; cursor: pointer; } +#saved { color: var(--muted); } +details { margin-top: 24px; color: var(--muted); } +summary { cursor: pointer; } +``` + +- [ ] **Step 3: Write `src/options/options.js`** + +```js +import { getSettings, saveSettings } from '../lib/storage.js'; + +const form = document.getElementById('form'); +const apiKey = document.getElementById('apiKey'); +const threshold = document.getElementById('threshold'); +const minViews = document.getElementById('minViews'); +const savedLabel = document.getElementById('saved'); + +function fill(settings) { + apiKey.value = settings.apiKey; + threshold.value = settings.threshold; + minViews.value = settings.minViews; +} + +fill(await getSettings()); + +form.addEventListener('submit', async (event) => { + event.preventDefault(); + fill(await saveSettings({ apiKey: apiKey.value, threshold: threshold.value, minViews: minViews.value })); + savedLabel.textContent = 'Сохранено'; + setTimeout(() => { + savedLabel.textContent = ''; + }, 2000); +}); +``` + +- [ ] **Step 4: Add `options_ui` to `manifest.json`** + +Replace the whole file with: + +```json +{ + "manifest_version": 3, + "name": "Outlier Saver", + "version": "0.1.0", + "description": "Находит на YouTube видео с аномальными просмотрами и сохраняет их локально.", + "permissions": ["storage"], + "host_permissions": ["https://www.googleapis.com/youtube/v3/*"], + "background": { "service_worker": "src/background.js", "type": "module" }, + "action": { "default_title": "Outlier Saver" }, + "options_ui": { "page": "src/options/options.html", "open_in_tab": true } +} +``` + +- [ ] **Step 5: Verify (manual, needs Chrome)** + +1. `chrome://extensions` → reload the extension → **Details → Extension options**. +2. The form shows threshold `5`, minimum views `10000`, empty key. +3. Enter a key, set threshold `7.5`, save → "Сохранено" appears. Close and reopen the page: values persist. +4. Enter threshold `-3` and save: the browser blocks the form (min 0.1). Clear minimum views and save: the field is required. + +- [ ] **Step 6: Commit** + +```bash +git add manifest.json src/options +git commit -m "feat: add options page for API key and thresholds" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 12: Popup (list, status, export) + +**Files:** +- Create: `src/popup/popup.html`, `src/popup/popup.css`, `src/popup/popup.js` +- Modify: `manifest.json` (add `default_popup`) + +**Interfaces:** +- Consumes: `countOutliers`, `listOutliers` (db.js); `toCsv`, `toJson` (export.js); `getStatus` (storage.js). +- Produces: a popup showing the status line, the saved-video count, the 50 most recently updated outliers, and buttons to download JSON/CSV or open options. + +- [ ] **Step 1: Write `src/popup/popup.html`** + +```html + + + + + Outlier Saver + + + +
+ Outlier Saver + +
+

+
    +
    + + + +
    + + + +``` + +- [ ] **Step 2: Write `src/popup/popup.css`** + +```css +:root { + color-scheme: light dark; + --line: color-mix(in srgb, currentColor 15%, transparent); + --muted: color-mix(in srgb, currentColor 60%, transparent); + --accent: #1a73e8; +} +body { width: 440px; margin: 0; font: 13px/1.4 system-ui, sans-serif; } +header, footer { display: flex; align-items: center; gap: 8px; padding: 10px 12px; } +header { justify-content: space-between; border-bottom: 1px solid var(--line); } +footer { border-top: 1px solid var(--line); } +footer button:last-child { margin-left: auto; } +button { padding: 5px 10px; font: inherit; } +#status { margin: 0; padding: 8px 12px; color: var(--muted); } +#status.warn { color: #d93025; } +#list { list-style: none; margin: 0; padding: 0; max-height: 380px; overflow-y: auto; } +#list li { display: grid; grid-template-columns: 96px 1fr; gap: 10px; padding: 8px 12px; border-bottom: 1px solid var(--line); } +#list img { width: 96px; height: 54px; object-fit: cover; border-radius: 4px; background: var(--line); } +#list a { color: inherit; font-weight: 600; text-decoration: none; overflow-wrap: anywhere; } +#list a:hover { text-decoration: underline; } +.meta { margin-top: 2px; color: var(--muted); } +.badge { display: inline-block; margin-right: 6px; padding: 0 6px; border-radius: 8px; background: var(--accent); color: #fff; font-size: 11px; } +.empty { padding: 24px 12px; text-align: center; color: var(--muted); } +``` + +- [ ] **Step 3: Write `src/popup/popup.js`** + +```js +import { countOutliers, listOutliers } from '../lib/db.js'; +import { toCsv, toJson } from '../lib/export.js'; +import { getStatus } from '../lib/storage.js'; + +const LIST_LIMIT = 50; +const compact = new Intl.NumberFormat('ru-RU', { notation: 'compact', maximumFractionDigits: 1 }); +const format = (n) => (n === null || n === undefined ? '—' : compact.format(n)); + +const countEl = document.getElementById('count'); +const statusEl = document.getElementById('status'); +const listEl = document.getElementById('list'); +const jsonButton = document.getElementById('export-json'); +const csvButton = document.getElementById('export-csv'); + +function statusText(status) { + switch (status.state) { + case 'no-key': + return { text: 'Не задан API-ключ — откройте настройки.', warn: true }; + case 'invalid-key': + return { text: `API-ключ отклонён: ${status.message ?? 'неизвестная ошибка'}`, warn: true }; + case 'quota': + if (Date.now() >= status.until) return { text: 'Сбор работает.', warn: false }; + return { + text: `Квота YouTube API исчерпана, сбор возобновится после ${new Date(status.until).toLocaleString('ru-RU')}.`, + warn: true, + }; + default: + return { text: 'Сбор работает.', warn: false }; + } +} + +function renderRow(outlier) { + const li = document.createElement('li'); + + const thumb = document.createElement('img'); + thumb.alt = ''; + thumb.loading = 'lazy'; + if (outlier.thumbnailUrl) thumb.src = outlier.thumbnailUrl; + + const title = document.createElement('a'); + title.textContent = outlier.title; + title.href = outlier.url; + title.target = '_blank'; + title.rel = 'noreferrer'; + + const badges = document.createElement('div'); + if (outlier.score !== null) badges.append(badge(`×${outlier.score}`)); + if (outlier.vidiqScore !== null) badges.append(badge(`VidIQ ${outlier.vidiqScore}`)); + + const meta = document.createElement('div'); + meta.className = 'meta'; + meta.textContent = `${outlier.channelTitle} · ${format(outlier.subscribers)} подп. · ${format(outlier.views)} просм.`; + + const body = document.createElement('div'); + body.append(title, badges, meta); + li.append(thumb, body); + return li; +} + +function badge(text) { + const span = document.createElement('span'); + span.className = 'badge'; + span.textContent = text; + return span; +} + +function download(filename, parts, type) { + const url = URL.createObjectURL(new Blob(parts, { type: `${type};charset=utf-8` })); + const link = Object.assign(document.createElement('a'), { href: url, download: filename }); + document.body.append(link); + link.click(); + link.remove(); + setTimeout(() => URL.revokeObjectURL(url), 1000); +} + +async function exportAs(kind) { + const records = await listOutliers(); + const day = new Date().toISOString().slice(0, 10); + if (kind === 'json') { + download(`outliers-${day}.json`, [toJson(records)], 'application/json'); + } else { + // The BOM makes Excel read the CSV as UTF-8 (Cyrillic titles). + download(`outliers-${day}.csv`, ['', toCsv(records)], 'text/csv'); + } +} + +const [count, recent, status] = await Promise.all([countOutliers(), listOutliers(LIST_LIMIT), getStatus()]); + +countEl.textContent = `Сохранено: ${count}`; +const { text, warn } = statusText(status); +statusEl.textContent = text; +statusEl.classList.toggle('warn', warn); + +if (recent.length === 0) { + const empty = document.createElement('li'); + empty.className = 'empty'; + empty.textContent = 'Пока ничего не найдено. Откройте YouTube и пролистайте ленту.'; + listEl.append(empty); + jsonButton.disabled = true; + csvButton.disabled = true; +} else { + listEl.append(...recent.map(renderRow)); +} + +jsonButton.addEventListener('click', () => exportAs('json')); +csvButton.addEventListener('click', () => exportAs('csv')); +document.getElementById('open-options').addEventListener('click', () => chrome.runtime.openOptionsPage()); +``` + +- [ ] **Step 4: Add `default_popup` to `manifest.json`** + +Replace the whole file with: + +```json +{ + "manifest_version": 3, + "name": "Outlier Saver", + "version": "0.1.0", + "description": "Находит на YouTube видео с аномальными просмотрами и сохраняет их локально.", + "permissions": ["storage"], + "host_permissions": ["https://www.googleapis.com/youtube/v3/*"], + "background": { "service_worker": "src/background.js", "type": "module" }, + "action": { "default_title": "Outlier Saver", "default_popup": "src/popup/popup.html" }, + "options_ui": { "page": "src/options/options.html", "open_in_tab": true } +} +``` + +- [ ] **Step 5: Verify (manual, needs Chrome)** + +1. Reload the extension, click its toolbar icon. Expected: "Сохранено: 0", the empty-state text, the status line ("Не задан API-ключ…" if no key is saved yet), export buttons disabled. +2. Open `chrome-extension:///src/popup/popup.html` in a normal tab (the id is on `chrome://extensions`), open DevTools → Console and seed two records: + +```js +const db = await import('/src/lib/db.js'); +const base = { + thumbnailUrl: null, channelId: 'UC1', channelTitle: 'Тест, "канал"', channelUrl: 'https://www.youtube.com/@t', + subscribers: 1000, publishedAt: '2026-10-01T00:00:00Z', durationSec: 600, isShort: false, + foundOn: 'home', firstSeenAt: new Date().toISOString(), updatedAt: new Date().toISOString(), +}; +await db.putOutliers([ + { ...base, videoId: 'seed0000001', title: 'Первый ролик, с запятой', url: 'https://www.youtube.com/watch?v=seed0000001', views: 50000, score: 50, vidiqScore: null }, + { ...base, videoId: 'seed0000002', title: 'Второй ролик', url: 'https://www.youtube.com/watch?v=seed0000002', views: 90000, score: null, vidiqScore: 8.2 }, +]); +``` + +3. Reload the tab. Expected: "Сохранено: 2"; both rows render, `` shows literally (no markup), badges `×50` and `VidIQ 8.2` appear. +4. Click **Скачать JSON** and **Скачать CSV**. Expected: two files named `outliers-YYYY-MM-DD.*`; the CSV opens in a spreadsheet with correct columns, the comma/quote title intact, and Cyrillic readable. +5. Remove the seed data when done: `indexedDB.deleteDatabase('outlier-saver')` in the same console, then reload the extension. + +- [ ] **Step 6: Commit** + +```bash +git add manifest.json src/popup +git commit -m "feat: add popup with outlier list and JSON/CSV export" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 13: Content script + +**Files:** +- Create: `src/content.js` +- Modify: `manifest.json` (add `content_scripts`, `web_accessible_resources`) + +**Interfaces:** +- Consumes: `parseVideoLink`, `pageFromPath` (video-id.js), `readVidiqScore` (vidiq.js), both loaded with dynamic `import()` because content scripts cannot be ES modules. +- Produces: `chrome.runtime.sendMessage({ type: 'videos-seen', items: QueueItem[] })`. + +Behavior: +- Scans only on the four supported page types. Collects IDs from `a[href*="/watch?v="]` and `a[href*="/shorts/"]`; on `/watch` also the open video itself. +- Scans at most once per 500 ms (a trailing throttle — YouTube mutates the DOM constantly while a video plays, so a debounce would never fire). +- Remembers sent IDs per navigation: `true` = sent with a VidIQ score, `false` = sent without one. An ID sent without a score is sent again if a score appears later. +- After `yt-navigate-finish` the memory is cleared and the page is rescanned. +- Stops itself if the extension was reloaded (`sendMessage` throws). + +- [ ] **Step 1: Write `src/content.js`** + +```js +(async () => { + const { parseVideoLink, pageFromPath } = await import(chrome.runtime.getURL('src/lib/video-id.js')); + const { readVidiqScore } = await import(chrome.runtime.getURL('src/lib/vidiq.js')); + + const LINK_SELECTOR = 'a[href*="/watch?v="], a[href*="/shorts/"]'; + // Only used to find the element that contains a VidIQ badge; ID collection does not depend on it. + const CARD_SELECTOR = [ + 'ytd-rich-item-renderer', + 'ytd-video-renderer', + 'ytd-compact-video-renderer', + 'ytd-grid-video-renderer', + 'yt-lockup-view-model', + 'ytm-shorts-lockup-view-model', + 'ytd-reel-item-renderer', + ].join(','); + const SCAN_INTERVAL_MS = 500; + + // videoId -> true if already sent together with a VidIQ score, false if sent without one. + let sent = new Map(); + let scanTimer = null; + + function collectItems(page) { + const items = new Map(); + const consider = (parsed, card) => { + const state = sent.get(parsed.videoId); + if (state === true) return; + const vidiqScore = readVidiqScore(card); + if (state === false && vidiqScore === null) return; + const item = items.get(parsed.videoId) ?? { + videoId: parsed.videoId, + vidiqScore: null, + viaShortsUrl: false, + page, + }; + item.vidiqScore ??= vidiqScore; + item.viaShortsUrl ||= parsed.isShort; + items.set(parsed.videoId, item); + }; + + for (const link of document.querySelectorAll(LINK_SELECTOR)) { + const parsed = parseVideoLink(link.getAttribute('href')); + if (parsed) consider(parsed, link.closest(CARD_SELECTOR) ?? link.parentElement); + } + if (page === 'watch') { + const current = parseVideoLink(location.href); + if (current) consider(current, null); + } + return [...items.values()]; + } + + function scan() { + scanTimer = null; + const page = pageFromPath(location.pathname); + if (!page) return; + const items = collectItems(page); + if (items.length === 0) return; + for (const item of items) sent.set(item.videoId, item.vidiqScore !== null); + try { + chrome.runtime.sendMessage({ type: 'videos-seen', items }).catch(() => {}); + } catch { + observer.disconnect(); // the extension was reloaded; this script is orphaned + } + } + + function scheduleScan() { + if (scanTimer === null) scanTimer = setTimeout(scan, SCAN_INTERVAL_MS); + } + + const observer = new MutationObserver(scheduleScan); + observer.observe(document.documentElement, { childList: true, subtree: true }); + document.addEventListener('yt-navigate-finish', () => { + sent = new Map(); + scheduleScan(); + }); + scheduleScan(); +})(); +``` + +- [ ] **Step 2: Add the content script to `manifest.json`** + +Replace the whole file with: + +```json +{ + "manifest_version": 3, + "name": "Outlier Saver", + "version": "0.1.0", + "description": "Находит на YouTube видео с аномальными просмотрами и сохраняет их локально.", + "permissions": ["storage"], + "host_permissions": ["https://www.googleapis.com/youtube/v3/*"], + "background": { "service_worker": "src/background.js", "type": "module" }, + "action": { "default_title": "Outlier Saver", "default_popup": "src/popup/popup.html" }, + "options_ui": { "page": "src/options/options.html", "open_in_tab": true }, + "content_scripts": [ + { + "matches": ["https://www.youtube.com/*"], + "js": ["src/content.js"], + "run_at": "document_idle" + } + ], + "web_accessible_resources": [ + { + "resources": ["src/lib/video-id.js", "src/lib/vidiq.js"], + "matches": ["https://www.youtube.com/*"] + } + ] +} +``` + +- [ ] **Step 3: Syntax-check and run the unit tests** + +Run: `node --check src/content.js && npm test` +Expected: no syntax errors, all tests pass. + +- [ ] **Step 4: End-to-end check on live YouTube (manual, needs Chrome and a real API key)** + +1. Reload the extension. In **Options** save a valid key and set threshold `1` and minimum views `1000` so that results appear quickly. +2. Open `https://www.youtube.com/`, scroll the feed for ~15 seconds. Open the service worker console from `chrome://extensions`: no errors. +3. Click the toolbar icon. Expected: "Сохранено: N" with N > 0, rows with thumbnail, title, channel, subscriber and view counts, and a `×score` badge; the status line says "Сбор работает." +4. Repeat on a `/watch?v=…` page (right sidebar plus the open video), a `/results?search_query=…` page, and a channel `/@…/videos` page. Navigate between them without reloading: new videos keep appearing. +5. In DevTools → Application → IndexedDB → `outlier-saver`, confirm the three stores exist and `seen` contains more IDs than `outliers`. Spot-check one record: `subscribers` is a number and `score ≈ views / subscribers`. +6. Remove the key in Options and scroll: the badge shows `!` and the popup says "Не задан API-ключ…". Restore the key: the badge clears and collection resumes. +7. Save a deliberately wrong key (`abc`) and scroll: the badge shows `!` and the popup says the key was rejected. Restore the real key: it recovers. +8. If nothing is collected: check the page's DevTools console for content-script errors (the extension's `web_accessible_resources` import is the usual suspect) and the service worker console for `Outlier Saver: dropped a batch…` warnings. + +- [ ] **Step 5: Commit** + +```bash +git add manifest.json src/content.js +git commit -m "feat: add content script that collects video ids from YouTube pages" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 14: README + +**Files:** +- Create: `README.md` + +- [ ] **Step 1: Write `README.md`** + +````markdown +# Outlier Saver + +Расширение для Chrome: пока вы смотрите YouTube, оно находит видео с аномально высокими просмотрами относительно размера канала («outliers») и сохраняет их локально. Данные можно выгрузить в JSON или CSV. + +## Установка + +1. Откройте `chrome://extensions` и включите «Режим разработчика». +2. «Загрузить распакованное расширение» → выберите эту папку. +3. Откройте настройки расширения и вставьте API-ключ YouTube Data API v3 (инструкция — в самих настройках). + +## Как это работает + +- Расширение собирает ID видео на главной, странице просмотра, в поиске и на страницах каналов. +- Название, превью, просмотры и подписчиков берёт из YouTube Data API v3 пачками по 50 (1 единица квоты за запрос, лимит — 10 000 в сутки). +- `score = просмотры ÷ подписчики`. Видео сохраняется, если `просмотры ≥ минимума` и `score ≥ порога` (по умолчанию 10 000 и 5), либо если на карточке есть значок VidIQ. +- Данные лежат в IndexedDB браузера. Кнопки «Скачать JSON» и «Скачать CSV» в popup выгружают всё сохранённое. + +## Разработка + +Сборки нет. Тесты (Node 22, без зависимостей): + +``` +npm test +``` + +После изменения кода нажмите «Обновить» у расширения на `chrome://extensions`. + +## Ручная проверка на живом YouTube + +- Подгрузка карточек при скролле на главной. +- Переход между страницами без перезагрузки. +- Работа на `/results` и `/@канал/videos`. +- Запись появляется в popup; скачиваются JSON и CSV. +- Без ключа и с неверным ключом на иконке бейдж `!`, в popup понятная причина. +```` + +- [ ] **Step 2: Commit** + +```bash +git add README.md +git commit -m "docs: add README" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` + +--- + +### Task 15: VidIQ badge selector (needs input from the user) + +This task cannot be completed without a sample of VidIQ's real markup, which only the user can capture from their Chrome with VidIQ installed. Until then `vidiqScore` stays `null` and everything else works. + +**Files:** +- Modify: `src/lib/vidiq.js` (`VIDIQ_BADGE_SELECTOR`, possibly `parseVidiqScore`) +- Modify: `test/vidiq.test.js` +- Modify: `src/content.js` (`CARD_SELECTOR`, only if the badge is not inside one of the listed card elements) + +**Interfaces:** +- Consumes: the markup sample from the user. +- Produces: a working `VIDIQ_BADGE_SELECTOR` and a `parseVidiqScore` test on real badge text. + +- [ ] **Step 1: Ask the user for the markup** + +Ask the user to open YouTube with VidIQ active, find a video card that shows a VidIQ outlier badge, right-click the badge → **Inspect**, then right-click the highlighted element → **Copy → Copy outerHTML**, and paste it together with the tag names of its three nearest ancestors. Also ask what the badge shows as text (for example `12.5x`). + +- [ ] **Step 2: Choose the selector** + +Pick the most stable selector from the sample: prefer a VidIQ-specific `data-*` attribute, an `id`, or a readable class name; avoid hashed or generated class names. Confirm that the badge element sits inside one of the elements listed in `CARD_SELECTOR` in `src/content.js`; if it does not, add the real ancestor's tag name to `CARD_SELECTOR`. + +- [ ] **Step 3: Update the test first** + +In `test/vidiq.test.js`, replace the test `readVidiqScore returns null while no selector is configured` with a test that asserts `VIDIQ_BADGE_SELECTOR` equals the chosen selector string, and add a `parseVidiqScore` assertion using the exact text from the sample. Run `node --test test/vidiq.test.js` and confirm the new assertions fail. + +- [ ] **Step 4: Implement** + +Set `export const VIDIQ_BADGE_SELECTOR = '';` in `src/lib/vidiq.js` and remove the "stays null" comment. If the sample text does not parse with the generic number regexp, adjust `parseVidiqScore` accordingly. Run `npm test`; expect all tests to pass. + +- [ ] **Step 5: Verify on live YouTube (manual)** + +Reload the extension and scroll a feed where VidIQ shows badges. Expected: the popup shows rows with a `VidIQ ` badge, including videos whose own score is below the threshold; the number matches the badge on the card. + +- [ ] **Step 6: Commit** + +```bash +git add src/lib/vidiq.js test/vidiq.test.js src/content.js +git commit -m "feat: read the VidIQ outlier badge from video cards" -m "Co-Authored-By: Claude Sonnet 5.5 " +``` diff --git a/docs/superpowers/specs/2026-10-25-outlier-saver-design.md b/docs/superpowers/specs/2026-10-25-outlier-saver-design.md index 33df964..07f2fba 100644 --- a/docs/superpowers/specs/2026-10-25-outlier-saver-design.md +++ b/docs/superpowers/specs/2026-10-25-outlier-saver-design.md @@ -22,9 +22,11 @@ | Часть | Файл | Ответственность | |---|---|---| | Content script | `src/content.js` | Следит за DOM на `youtube.com`, достаёт ID видео из ссылок, читает значок VidIQ, шлёт пачки в background | -| Service worker | `src/background.js` | Очередь ID, вызовы API, расчёт score, решение «сохранять или нет», запись в БД, бейдж на иконке | +| Service worker | `src/background.js` | Очередь ID, цикл сброса пачками, статус и бейдж на иконке, реакция на смену ключа | +| Обработка пачки | `src/lib/process.js` | `processBatch`: проверка `seen`, запросы к API, расчёт score, решение «сохранять или нет», запись в БД. Зависимости (БД и API) передаются параметрами, поэтому логика покрыта тестами | +| Настройки и статус | `src/lib/storage.js` | Значения по умолчанию, `normalizeSettings`, чтение и запись настроек и статуса в `chrome.storage.local` | | Формула и решение | `src/lib/score.js` | Чистые функции: `computeScore`, `shouldSave` | -| Разбор ссылок | `src/lib/video-id.js` | Извлечение ID из `/watch?v=`, `/shorts/` и отсев мусора | +| Разбор ссылок | `src/lib/video-id.js` | Извлечение ID из `/watch?v=`, `/shorts/`, отсев мусора, определение типа страницы по пути | | Значок VidIQ | `src/lib/vidiq.js` | Селектор значка (одна константа) и `parseVidiqScore(text)` | | API | `src/lib/api.js` | `videos.list`, `channels.list`, разбиение на пачки по 50, разбор ошибок | | База | `src/lib/db.js` | Обёртка над IndexedDB | @@ -36,14 +38,14 @@ ## Поток данных -1. Content script на `youtube.com` реагирует на событие `yt-navigate-finish` (YouTube — SPA, страницы не перезагружаются) и на `MutationObserver` по документу. Срабатывание дебаунсится на 500 мс. +1. Content script на `youtube.com` реагирует на событие `yt-navigate-finish` (YouTube — SPA, страницы не перезагружаются) и на `MutationObserver` по документу. Сканирование запускается не чаще раза в 500 мс (троттлинг, а не дебаунс: пока играет видео, DOM меняется постоянно, и дебаунс не сработал бы). После навигации список уже отправленных ID сбрасывается. 2. Из ссылок `a[href*="/watch?v="]` и `a[href*="/shorts/"]` через `video-id.js` берутся ID. Для каждой карточки пробуется прочитать значок VidIQ. Уже отправленные с этой страницы ID запоминаются в `Set`, чтобы не спамить сообщениями. 3. Content script шлёт background сообщение `{videoId, vidiqScore | null, viaShortsUrl, page}`, где `page` — `home | watch | search | channel`. -4. Background копит ID и сбрасывает очередь раз в секунду либо при накоплении 50 штук. ID, проверенные менее 24 часов назад (`seen.checkedAt`), пропускаются. +4. Background копит ID и сбрасывает очередь раз в секунду либо при накоплении 50 штук. ID, проверенные менее 24 часов назад (`seen.checkedAt`), пропускаются, кроме ID со значком VidIQ: значок может отрисоваться позже первого сканирования, поэтому content script отправляет ID повторно, когда у него появился score, а проверка `seen` для таких ID не применяется. 5. `videos.list` (`part=snippet,statistics,contentDetails`) возвращает название, превью, просмотры, дату публикации, длительность и `channelId`. 6. Для каналов, которых нет в кэше `channels` или чья запись старше 24 часов, вызывается `channels.list` (`part=statistics,snippet`). 7. `score = views / max(subscribers, 1)`. Если подписчики скрыты, `score = null`. -8. Решение о сохранении (`shouldSave`): видео сохраняется, если `views ≥ minViews` и `score ≥ threshold`, **или** если `vidiqScore` не `null`. Условие по VidIQ действует независимо от своего порога и от `minViews`. +8. Решение о сохранении (`shouldSave`): видео сохраняется, если `views ≥ minViews` и `score ≥ threshold`, **или** если `vidiqScore` не `null`. Условие по VidIQ действует независимо от своего порога и от `minViews`. Уже сохранённое видео обновляется при каждой проверке, даже если его score опустился ниже порога. 9. Результат пишется upsert-ом в `outliers`, ID с временем проверки — в `seen` (в том числе несохранённые, чтобы не запрашивать их повторно). ## Модель данных @@ -94,7 +96,7 @@ | Скрытые подписчики | `score = null`, видео сохраняется только при наличии `vidiqScore` | | Очень мало подписчиков | Делитель `max(subs, 1)`, мусор отсекается через `minViews` | | Приватное или удалённое видео | API его не возвращает; в `seen` пишется `status: unavailable`, повторная проверка после TTL | -| Service worker выгрузили | Очередь живёт в памяти и может потеряться. Content script пришлёт ID снова при следующем сканировании, так как `seen` пишется только после результата | +| Service worker выгрузили | Очередь живёт в памяти и может потеряться. `seen` пишется только после результата, а content script присылает ID снова после следующей навигации по YouTube или перезагрузки страницы | | Повторная встреча видео | Upsert: обновляются `views`, `subscribers`, `score`, `vidiqScore` (если он есть сейчас), `updatedAt`; `isShort` может только стать `true` | ## Расход квоты