samson-art/transcriptor-mcp releases

Added

• MCP HTTP edge guidance: Added documentation and examples for deploying stdio mcp-proxy behind an external edge (reverse proxy or API gateway) with token auth and traffic control.
• Edge/operator guides: Added docs/edge-smithery-gate.md and docs/mcp-edge-rate-limit.md with concrete policies for X-MCP-Api-Token, Smithery-shaped traffic gating, and reverse-proxy rate-limit strategies.
• Build-time server-card generation: Added scripts/generate-server-card.mjs and npm scripts (generate:server-card, postbuild) to produce .well-known/mcp/server-card.json automatically after build for SEP-1649/Smithery discovery.
• MCP config schema support for apiToken: .well-known/mcp-config now documents and maps apiToken (X-MCP-Api-Token) in addition to authToken.

Changed

• Smithery session config contract: smithery.yaml now separates authToken (Authorization/Bearer for self-hosted edge auth) from apiToken (X-MCP-Api-Token for token pools/quotas), with explicit header mapping metadata.
• Docs alignment around MCP architecture: README and docs now consistently describe this repo’s MCP model as stdio + external mcp-proxy, clarify that Node app RATE_LIMIT_* applies to REST API only, and move MCP auth/rate-limit responsibilities to infrastructure edge layers.
• Monitoring documentation scope: docs/monitoring.md clarifies that /metrics is exposed by the REST API only, while MCP-over-HTTP observability belongs to proxy/WAF metrics, logs, or Sentry.
• Quick-start and public-url guidance: MCP quick-start/public URL docs now include stronger guidance for edge auth, /mcp and /sse protection, and safer .well-known behavior for catalog discovery.
• Pre-commit checks: .husky/pre-commit now runs make prepare && make check-no-smoke.

Security

• Safer MCP auth signaling in server card: Generated server card keeps authentication.required: false to avoid advertising unsupported OAuth schemes while relying on edge-enforced X-MCP-Api-Token/Bearer policies documented for operators.

Added

• MCP_TRUST_PROXY: Parsed in src/env.ts (parseMcpTrustProxyEnv) and passed to Fastify trustProxy in src/mcp-http.ts so request.ip reflects the client behind reverse proxies (X-Forwarded-For). Supports boolean-ish strings, hop counts, or proxy-addr-style strings (unset/empty defaults to true).
• Prometheus mcp_http_requests_by_client_ip_total: Counter with route, method, client_ip (src/metrics.ts, recordMcpHttpRequestByClientIp); incremented on onResponse with stable route labels (routeLabelForMcpHttpMetrics). Optional disable via MCP_METRICS_HTTP_REQUESTS_BY_CLIENT_IP (isMcpMetricsHttpRequestsByClientIpEnabled) to avoid high cardinality.
• Anonymous MCP quota by client (HTTP): When there is no X-Api-Key, resolveLimit / enforceMcpToolQuota accept optional anonymous material (hashed as anon:<material>); HTTP supplies normalized client IP via McpRequestContext.anonymousQuotaMaterial and createMcpServer({ getAnonymousQuotaMaterial }). Stdio MCP omits the resolver and keeps the legacy single global anonymous bucket (__mcp_quota_anonymous_v1__).
• IP helpers for quota/metrics: normalizeIpStringForQuota, normalizeMcpClientIp in src/mcp-http.ts (trim, bracketed IPv6, zone id strip, lowercase).

Changed

• MCP HTTP: GET /sse session setup runs inside runWithMcpRequestContext(buildMcpHttpRequestContext(...)) so SSE tool calls see the same API key and anonymous quota material as streamable /mcp and /message.

Tests

• src/env.test.ts: parseMcpTrustProxyEnv, isMcpMetricsHttpRequestsByClientIpEnabled.
• src/mcp-http.test.ts: IP normalization and route label helpers.
• src/mcp-quota.test.ts: Distinct anonymous buckets, enforceMcpToolQuota with anonymous material vs stdio global bucket.
• src/mcp-request-context.test.ts: anonymousQuotaMaterial in context.
• src/metrics.test.ts: mcp_http_requests_by_client_ip_total export.
• E2E src/e2e/api-smoke.ts: Asserts mcp_http_requests_by_client_ip_total on GET /metrics when MCP quota metrics are checked.

Added

• MCP tool call quota (optional): Per-client limits keyed by X-Api-Key on MCP HTTP (src/mcp-quota.ts, src/mcp-core.ts). Enable with MCP_QUOTA_ENABLED; defaults MCP_QUOTA_DEFAULT_MAX / MCP_QUOTA_DEFAULT_WINDOW; optional strict mode MCP_QUOTA_REJECT_UNREGISTERED; customizable messages via MCP_QUOTA_CONTACT_MESSAGE, MCP_QUOTA_MESSAGE_NO_KEY, MCP_QUOTA_MESSAGE_INVALID_KEY.
• Client API key registry (hashed secrets only): JSON file or inline env — MCP_CLIENT_API_KEYS_FILE (preferred) or MCP_CLIENT_API_KEYS_JSON, plus MCP_CLIENT_API_KEY_PEPPER. Validation, prefix keys, and lookup in src/api-key-registry.ts; map-based registry builder for fast hash lookup; src/mcp-quota-registry.ts re-exports loader helpers.
• HTTP request context for quota: src/mcp-request-context.ts (AsyncLocalStorage) so streamable /mcp, POST /sse, and /message expose the client key to createMcpServer({ getClientApiKey }) (src/mcp-http.ts).
• Prometheus metrics (MCP): mcp_quota_checks_total, mcp_quota_exceeded_total, mcp_quota_tool_calls_blocked_total, mcp_quota_http_429_total, mcp_quota_check_duration_seconds in src/metrics.ts; resetMetricsRegistryForTests() for unit tests.
• Quota counter store: Fixed-window buckets in src/mcp-quota-store.ts — in-memory (MemoryQuotaCounterStore) for single-process; RedisQuotaCounterStore (Lua INCR + PEXPIRE) for shared Redis when wired in.
• Duration parsing: parseQuotaWindowMs() in src/env.ts for quota windows (e.g. 24h, 30m, 1 minute).
• MCP session config / Smithery: Optional apiKey in MCP_SESSION_CONFIG_SCHEMA and .well-known/mcp-config — gateway maps form field to X-Api-Key (distinct from MCP_AUTH_TOKEN / Bearer).
• Monitoring stack (repo): monitoring/prometheus.yml, Grafana provisioning (monitoring/grafana/provisioning/...) and MCP quota dashboard JSON monitoring/grafana/provisioning/dashboards/files/mcp-quota.json. docs/monitoring.md — new quota metrics, PromQL snippets, dashboard mount notes.
• Docs and repo hygiene: CONTRIBUTING.md (dev setup, make prepare / make check); SECURITY.md (supported versions, private reporting via GitHub Security Advisories). docs/configuration.md and .env.example — full quota and registry variable list.
• Tests: src/api-key-registry.test.ts, src/mcp-quota.test.ts, src/mcp-quota-store.test.ts, src/mcp-request-context.test.ts, src/metrics.test.ts; MCP quota scenarios in src/mcp-core.test.ts; schema assertions in src/mcp-http.test.ts. E2E src/e2e/api-smoke.ts — optional MCP_QUOTA_ENABLED / high default max, asserts quota-related series on GET /metrics (skip with SMOKE_SKIP_MCP_QUOTA_METRICS).

Changed

• Dockerfile: Build stage node:20-alpine; runtime base node:20-bookworm-slim (Debian, apt-get for yt-dlp/ffmpeg stack); comment clarifying Alpine vs Debian for system packages.
• npm test: Jest runs with --forceExit to avoid hanging on open handles in CI.

Security

• .gitignore: Ignore secrets/ to reduce risk of committing local key material.

Added

• get_playlist_transcripts hardening (downloadPlaylistSubtitles in src/youtube.ts): Returns a discriminated DownloadPlaylistSubtitlesOutcome (ok + results or failure) instead of null on error. On yt-dlp failure, still scans the temp directory and returns partial results when any subtitle files were written (aligned with single-video runYtDlpAndExtractSubtitles).
• --ignore-errors for playlist subtitle runs so one bad entry does not abort the batch. Opt out with YT_DLP_PLAYLIST_IGNORE_ERRORS=0. Documented in docs/configuration.md and .env.example.
• YT_DLP_VERBOSE_ON_ERROR: When set to 1, after a failed playlist run with no partial files, runs yt-dlp once more with -v and without --quiet/--no-progress and logs stderr for diagnostics. Documented in docs/configuration.md and .env.example.
• collectExecFileErrorDetails() and ExecFileErrorDetails: Normalized fields from failed execFile / yt-dlp runs (message, exitCode, signal, cmd, stdout, stderr) for structured logs.
• formatPlaylistDownloadFailureMessage(): Builds the MCP/API-facing error string (message, exit code, stderr tail, operational hints).
• appendYtDlpEnvArgs options: Optional third argument AppendYtDlpEnvArgsOptions with quiet: false to omit --no-progress and --quiet (used for verbose replay).

Changed

• Playlist failure logging: Logs exitCode, signal, and cmd when present, not only empty stdout/stderr under --quiet.
• MCP get_playlist_transcripts: On full failure, throws an error whose message comes from formatPlaylistDownloadFailureMessage instead of the generic Failed to fetch playlist subtitles.

Tests

• src/youtube.test.ts: Coverage for collectExecFileErrorDetails, formatPlaylistDownloadFailureMessage, --ignore-errors / YT_DLP_PLAYLIST_IGNORE_ERRORS=0, failure outcome shape, and appendYtDlpEnvArgs with quiet: false.

Added

• Background Whisper jobs and late cache write: When the client hits WHISPER_TIMEOUT but Whisper finishes afterward, the transcript is still saved to Redis (same subtitle cache keys as a normal success) so the next request for that video can be a cache hit. Implemented via deduplicated in-flight jobs in src/whisper-jobs.ts (startOrReuseWhisperJob), Promise.race against getWhisperConfig().timeout in src/validation.ts for auto-discovery and explicit type/lang flows, and optional timeoutMs on transcribeWithWhisper / local+API helpers (0 = no fetch abort).
• WHISPER_BACKGROUND_TIMEOUT: Env var for the long-running Whisper HTTP client used by background jobs (unset = max(1800000, 3 × WHISPER_TIMEOUT); 0 = no client-side abort). Documented in docs/configuration.md, docs/caching.md, .env.example, and docker-compose.example.yml.
• Prometheus gauge whisper_background_jobs_active: Tracks in-flight deduplicated background Whisper jobs; setWhisperBackgroundJobsActive() in src/metrics.ts.
• Tests: src/whisper-jobs.test.ts; src/whisper.test.ts asserts fetch is called without signal when timeoutMs === 0; src/validation.test.ts covers cache.set after simulated timeout for auto-discover and explicit lang.

Changed

• WHISPER_TIMEOUT semantics (docs): Clarified as the per-request wait before returning 404 to the client; background transcription may continue for cache population when Redis is enabled.
• docs/monitoring.md: Documented whisper_background_jobs_active for API and MCP metrics tables.

Added

• VTT word-by-word deduplication: parseVTT now groups and deduplicates consecutive cues with identical text, fixing duplicated words in word-level VTT subtitles (e.g. from YouTube auto-generated captions).
• Reddit support: Reddit (reddit.com, old.reddit.com, v.redd.it) added as a supported platform for video transcripts and metadata. Documentation, validation, and MCP tool descriptions updated.
• extractPlatformFromUrl(): New helper extracts platform identifier from input URL hostname (youtube, reddit, vimeo, etc.) for flexible source reporting.

Changed

• Subtitle validation and download logic: Refactored validateAndDownloadSubtitles — introduced throwNoSubtitlesError for centralized "subtitles not found" handling; split auto-discovery and explicit-request flows into handleAutoDiscoverFlow and handleExplicitRequestFlow for clearer structure and maintainability.
• Error messages: Improved guidance for subtitle availability and Whisper fallback attempts when subtitles are not found.
• README and documentation: Refined introduction and connection options; added "supported platforms" section emphasizing multi-platform support; clarified Whisper fallback and Redis caching in docs/configuration.md; streamlined quick-start with Smithery and Glama no-install options.
• source field: Response schemas and validation now accept generic strings for source (e.g. youtube, whisper, reddit) instead of fixed literals, allowing new platforms without schema changes.
• Publish Docker workflow: Extracts and outputs built image tags; removed unused release trigger.

Added

• Publish Docker workflow: Manual run via workflow_dispatch with optional version input (e.g. 0.6.5 or v0.6.5) and latest_only flag. When latest_only=true, builds from default branch and pushes only :latest (no version tag).
• Docker image verification: Publish workflow logs the yt-dlp version installed in the built image for easier debugging.
• docs/configuration.md: Section on container memory limits for long Whisper transcriptions — deploy.resources.limits.memory (e.g. 4–6 GB) to avoid OOM kills on CPU.

Changed

• WHISPER_TIMEOUT default: Increased from 2 minutes (120000 ms) to 10 minutes (600000 ms) to better support long videos.
• 404 error messages: When Whisper fallback fails, the "Subtitles not found" response now explicitly mentions that Whisper was attempted and suggests increasing WHISPER_TIMEOUT (e.g. 3600000 for 1-hour videos).
• Whisper error logging: Local and API modes now distinguish timeout (AbortError) vs network/service error in log messages.
• docs/configuration.md: Updated WHISPER_TIMEOUT description and flow text for 1-hour videos on CPU; .env.example and docker-compose.example.yml use 600000 as the example value.

Added

• MCP tool get_playlist_transcripts: Fetch cleaned subtitles for multiple videos from a playlist in one call. Parameters: url (playlist or watch with list=), optional playlistItems (yt-dlp -I spec, e.g. "1:5", "1,3,7", "-1"), maxItems, type, lang. New downloadPlaylistSubtitles() in src/youtube.ts; tool registered in mcp-core.ts and server card.
• Configurable subtitle format (srt, vtt, ass, lrc): New env YT_DLP_SUB_FORMAT and optional format parameter for MCP tools get_transcript, get_raw_subtitles, and get_playlist_transcripts. REST API POST /subtitles accepts format. Default remains srt. Cache keys include format. Exported SubtitleFormat and resolveSubtitleFormat() in src/youtube.ts; get_raw_subtitles output schema and server card include ass and lrc.
• search_videos extended with yt-dlp filters: Optional dateBefore (e.g. "now-1year"), date (exact date), matchFilter (e.g. "!is_live", "duration < 3600"). searchVideos() in src/youtube.ts now accepts these in SearchVideosOptions; yt-dlp receives --datebefore, --date, --match-filter when set.
• search_videos extended: Optional offset (pagination), uploadDateFilter (hour | today | week | month | year), and response_format (json | markdown). searchVideos() in src/youtube.ts now accepts SearchVideosOptions (offset, dateAfter); yt-dlp receives --dateafter when filter is set. Server card and README tool reference updated.
• Dynamic transcript resource: New MCP resource template transcriptor://transcript/{videoId}. Clients can read a video transcript by URI (e.g. transcriptor://transcript/dQw4w9WgXcQ) without calling a tool. Uses ResourceTemplate from the MCP SDK; handler fetches and parses subtitles and returns JSON (videoId, type, lang, text, optional source).
• MCP prompt search_and_summarize: New prompt with args query (required) and url (optional). Builds a user message that asks the model to search YouTube for the query and summarize the first result's transcript, or to summarize the given video URL. Exposed in server card and in transcriptor://info.
• Discoverable info resource: New MCP resource transcriptor://info (Smithery discoverable) returning JSON with server message, availableResources (info, transcript template, supported-platforms, usage), tools, and prompts. Registered in mcp-core.ts and listed in server card.
• Use-case documentation: Four new guides in docs/: IDE and AI assistants (Cursor, Claude, VS Code), No-code automation (n8n), Researchers and batch processing, Self-hosted and enterprise. Earlier guides: summarize video, search and transcript. Linked from main README and docs/README.md.
• 404 response available field: When subtitles are not found, the API now returns available: { official, auto } in the 404 payload so clients can show supported languages without an extra /subtitles/available call.
• GET /changelogs: REST API and MCP HTTP servers now expose GET /changelogs, returning CHANGELOG.md as text/markdown for programmatic access.
• CORS for MCP HTTP discovery: @fastify/cors enabled for MCP HTTP server (origin: true, methods: GET). Allows Smithery and other registries to fetch /.well-known/mcp/server-card.json and /.well-known/mcp/config-schema.json from cross-origin requests (SEP-1649).
• SEP-1649 server card fields: Server card now includes $schema, version, protocolVersion, transport (streamable-http /mcp), and capabilities. Improves compatibility with MCP Server Cards spec and Smithery tool discovery.
• Quick Start reordered: Smithery URL is now the first option ("no install"); Docker and local Node follow. Explicit "Connect by URL — no local install" messaging. README links to Smithery server page in header, Quick Start, Features, and "When to use".
• Smithery badge and VS Code install badges (README): Smithery badge added to the badge row; Overview now states "Optimized for Smithery with resources, prompts, and flexible configuration". Quick Start includes one-click install badges for VS Code and VS Code Insiders (URL-based config for the Smithery server).
• README "When to use Transcriptor MCP": New section describing when to choose transcriptor-mcp (transcripts/metadata without downloads, multi-platform, Whisper fallback, remote/HTTP, monitoring).
• yt-dlp --no-playlist for single-video tools: downloadSubtitles, fetchYtDlpJson, and downloadAudio now pass --no-playlist so URLs like watch?v=X&list=Y process only the single video instead of the full playlist.
• yt-dlp env filters: New optional env vars: YT_DLP_MAX_FILESIZE (e.g. "50M") for Whisper audio; YT_DLP_DOWNLOAD_ARCHIVE (path) and --break-on-existing for get_playlist_transcripts; YT_DLP_AGE_LIMIT for search_videos. Documented in docs/configuration.md and .env.example.
• YT_DLP_AUDIO_TIMEOUT: Separate timeout for audio download (Whisper fallback). Falls back to YT_DLP_TIMEOUT when unset. Enables processing videos up to 5 hours at slow download speeds (e.g. at ~420 KiB/s, 5 h audio needs ~15 min; set 900000 ms). Documented in docs/configuration.md and .env.example.
• Optimal audio quality for Whisper: When downloading audio via yt-dlp for Whisper fallback, the app now prefers smaller streams to reduce download time without hurting speech recognition. Format selector bestaudio[abr<=192]/bestaudio (prefer streams ≤192 kbps; fallback to best audio) and --audio-quality 5 (~128 kbps VBR for m4a) are used by default. Configurable via YT_DLP_AUDIO_FORMAT and YT_DLP_AUDIO_QUALITY (0–9, default: 5). Documented in docs/configuration.md and .env.example.
• Sentry Performance / tracing: Optional SENTRY_TRACES_SAMPLE_RATE (0–1, default 0.1) and SENTRY_SEND_DEFAULT_PII env vars. Performance monitoring is active when running via start / start:mcp / start:mcp:http. Documented in docs/sentry.md and .env.example.
• Sentry: beforeSend filters out expected client errors (NotFoundError, ValidationError) to reduce noise; expected 404s are monitored via Prometheus instead.
• Prometheus metric http_404_expected_total: Counter for expected 404 responses (NotFoundError) with labels method and route.
• Load scenario "10 VU, 1 min": New k6 script load/ten-users-1min.js — 10 concurrent users, each requests one video at a time until 1 minute; uses VIDEO_POOL. Make target load-test-10vu-1min, npm script load-test:10vu-1min. Documented in load/load-testing.md with thresholds.
• Load scenario "100 VU, 2h podcasts": New k6 script load/podcast-2h-100vu.js — 100 VU at once, one 2h podcast per VU; uses PODCAST_2H_POOL and getPodcast2hRequest() in load/config.js. Make target load-test-podcast-2h, npm script load-test:podcast-2h.
• Load test result report: load/load-test-result-2025-02-15.md — results for subtitles, mixed, 10 VU 1 min, and podcast 2h scenarios (k6, BASE_URL, thresholds, metrics).
• yt-dlp retries and extra args (all calls): YT_DLP_RETRIES (-R), YT_DLP_RETRY_SLEEP (e.g. linear=1::2), YT_DLP_EXTRA_ARGS (space-separated). Documented in docs/configuration.md and .env.example.
• yt-dlp sleep options (rate limits): YT_DLP_SLEEP_REQUESTS, YT_DLP_SLEEP_INTERVAL, YT_DLP_MAX_SLEEP_INTERVAL, YT_DLP_SLEEP_SUBTITLES. Documented in docs/configuration.md and .env.example.
• yt-dlp subtitle encoding: YT_DLP_ENCODING (e.g. utf-8, cp1251) for subtitle downloads (--encoding).
• yt-dlp audio download options (Whisper only): YT_DLP_AUDIO_CONCURRENT_FRAGMENTS, YT_DLP_AUDIO_LIMIT_RATE, YT_DLP_AUDIO_THROTTLED_RATE, retries, buffer, downloader args for DASH/HLS reliability and speed. Documented in docs/configuration.md and .env.example.
• YT_DLP_NO_WARNINGS: When set to 1, pass --no-warnings to all yt-dlp calls. Reduces log noise.
• YT_DLP_IGNORE_NO_FORMATS: When not set to 0, pass --ignore-no-formats-error when fetching video metadata (info, chapters, available subtitles), so region-locked or undownloadable videos still return metadata. Set to 0 to fail on "No video formats" (default yt-dlp behavior). Documented in docs/configuration.md.
• Unit tests for Tool Quality: In mcp-http.test.ts, tests for server card: "includes title for each tool (Tool Quality)" asserts every tool has the expected title; "includes parameter descriptions for get_raw_subtitles (Tool Quality)"; "includes SEP-1649 fields" for $schema, version, protocolVersion, transport, and capabilities.

Changed

• README: Use-case section now references docs/README.md with the full list of guides (summarize video, search and transcript, IDE/Cursor/Claude, n8n, researchers/batch, self-hosted).
• Tool Quality (Smithery): Server card now includes title for every tool (e.g. "Get video transcript", "Get raw video subtitles") and description for every parameter of get_raw_subtitles (type, lang, response_limit, next_cursor). In mcp-core.ts, optional fields of subtitleInputSchema now have .describe() so live MCP tools/list returns parameter descriptions. Improves Smithery Tool Quality score.
• Error messages: NotFoundError messages now hint at /subtitles/available and auto-discovery (omit type/lang).
• README Features: First bullet is "Connect by URL (Smithery)" — use the server without installing Docker or Node. MCP quick start section retitled to "Docker and self-hosted" with a pointer to Smithery for one-click connection.
• MCP config schema and .well-known/mcp-config: Enriched documentation with expanded gettingStarted (three steps including Smithery URL and tool names), apiLink (GitHub readme), and updated security text. Applied in both MCP_SESSION_CONFIG_SCHEMA in mcp-http.ts and .well-known/mcp-config.
• Server card: Resources list now includes info (transcriptor://info) and transcript (template transcriptor://transcript/{videoId}); prompts list includes search_and_summarize with arguments query and url. Server-card test updated to allow resources with either uri or uriTemplate.
• yt-dlp headless behavior: All yt-dlp calls now pass --quiet and --no-progress for cleaner server logs. appendYtDlpSubtitleArgs() in src/youtube.ts applies common subtitle args; unit tests in youtube.test.ts and validation.test.ts updated for format and new env vars.
• Long videos (5+ hours): docs/configuration.md — added YT_DLP_AUDIO_TIMEOUT and guidance for 5-hour videos (use local Whisper, WHISPER_TIMEOUT=3600000; Whisper API max 25 MB).
• Audio download (Whisper): downloadAudio() in src/youtube.ts now passes -f, --audio-quality, and the chosen format/quality to yt-dlp. Flow description in docs/configuration.md (Whisper section) updated to mention the default format and quality.
• Sentry context: 5xx events include requestUrl from body for subtitle endpoints; route tag added; MCP errors include transport type and sessionId.
• docs/sentry.md: Documented beforeSend filtering, Prometheus for expected 404s, SENTRY_RELEASE recommendation; Performance / Tracing section.
• load/config.js: Added PODCAST_2H_POOL and getPodcast2hRequest(iter, vu) for the 2h podcast load scenario.
• load/load-testing.md: New scenario table rows for ten-users-1min.js and podcast-2h-100vu.js; PODCAST_2H_POOL description; thresholds for ten-users-1min; dedicated sections for both scenarios.
• Unit tests optimized for real usage scenarios: mcp-core.test.ts — added scenario tests "Use case: Search and transcript" and "Use case: Pagination for long transcripts"; consolidated duplicate "invalid URL" error tests. mcp-http.test.ts — consolidated five server-card tests into one comprehensive test; grouped resolvePublicBaseUrlForRequest tests by scenario.
• mcp-http.test.ts: Simplified idempotentHint assertion for server-card tools; corrected env var cleanup order in resolvePublicBaseUrlForRequest test.
• smithery.yaml: Comment added with public URL and "Connect by URL — no local install".

Removed

• Obsolete load test result: Removed load/load-test-subtitles-result-2026-02-13.md; results consolidated in load/load-test-result-2025-02-15.md.

Added

• MCP tool search_videos: Search videos on YouTube via yt-dlp (ytsearch). No required parameters; provide query and optional limit (default 10, max 50). Returns list of videos with metadata (id, title, url, duration, uploader, viewCount, thumbnail). New searchVideos(query, limit, log) in src/youtube.ts; tool registered in mcp-core.ts and exposed in server card.
• Sentry breadcrumbs from Pino logs: When a 4xx or 5xx error is sent to Sentry, the event now includes a full trail of log calls (debug, info, warn, error) that led up to the error. REST API and MCP HTTP use a Pino logger that writes each log line to stdout and adds a Sentry breadcrumb; maxBreadcrumbs set to 100 in Sentry init. New module src/logger-sentry-breadcrumbs.ts (createLoggerWithSentryBreadcrumbs()); docs/sentry.md updated with a Breadcrumbs section.
• MCP_PUBLIC_URLS: Comma-separated list of public base URLs for multi-origin MCP deployments (e.g. Smithery + direct domain). The server selects the matching URL per request using Host or X-Forwarded-Host. When set, takes precedence over MCP_PUBLIC_URL. Backward compatible: single MCP_PUBLIC_URL still works.
• POST /sse compatibility: Some MCP clients (e.g. Cursor via Smithery) POST to /sse for streamable HTTP. The server now accepts POST on /sse and delegates to the streamable handler; canonical endpoint remains POST /mcp.

Added

• Unit tests: ensureWritableCookiesFile — returns original path when writable; copies to temp and cleans up when read-only.
• CI (GitHub Actions): .github/workflows/ci.yml runs on push/PR to main: npm ci, make check-no-smoke (format-check, lint, typecheck, test, build). On push to main, optional smoke job runs REST API smoke with SMOKE_SKIP_MCP=1. .github/workflows/publish-docker.yml runs on tag push v*: build and push REST API and MCP images to Docker Hub (multi-arch linux/amd64, linux/arm64). Requires DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets.
• Readiness and metrics (REST API): GET /health/ready — when CACHE_MODE=redis, pings Redis; returns 503 if Redis is unreachable (for Kubernetes readiness). GET /metrics — Prometheus text format with counters: http_requests_total, http_request_errors_total, cache_hits_total, cache_misses_total. New src/metrics.ts; validation layer records cache hit/miss; REST error handler and onResponse hook record errors and requests.
• Cache: cache.ping() in src/cache.ts for Redis liveness. Unit tests for ping() when cache off and when Redis URL unset.
• Documentation: README — repo/package name note (yt-captions-downloader vs transcriptor-mcp), Versioning subsection (version from package.json, tagging), Security section (do not commit or log WHISPER_API_KEY, CACHE_REDIS_URL, MCP_AUTH_TOKEN, cookies path; use env or secret manager). docs/configuration.md — Health and metrics (health, health/ready, metrics), Recommended values for production table. docs/caching.md — section “When Redis is unavailable” (graceful degradation: request still served via yt-dlp).
• E2E smoke: MCP streamable HTTP smoke now includes checkMcpStreamableGetTranscript: after initialize, calls tools/call for get_transcript and asserts content or structuredContent. load/load-testing.md — “Recommended thresholds for regression” (e.g. http_req_failed rate<0.05, p95<120s; k6 run --throw for CI).
• Pre-commit (Husky): husky devDependency and prepare script; .husky/pre-commit runs npm run format-check && npm run lint.
• verify-pool script: npm run verify-pool (and Make target) runs load/verify-pool.js to validate the k6 load-test video ID pool.
• Optional Redis cache: Responses for subtitles, video info, available subtitles, and chapters can be cached in Redis to reduce repeated yt-dlp calls. Configure via env: CACHE_MODE (off or redis), CACHE_REDIS_URL (required when redis), CACHE_TTL_SUBTITLES_SECONDS (default 7 days for subtitles), CACHE_TTL_METADATA_SECONDS (default 1 hour for video info, available subtitles, chapters). New src/cache.ts with getCacheConfig(), get(), set(), close(). Both REST API and MCP use the cache when enabled. Documented in docs/caching.md, docs/configuration.md, and .env.example.
• MCP uses validation layer: MCP tools now call validateAndDownloadSubtitles, validateAndFetchAvailableSubtitles, validateAndFetchVideoInfo, and validateAndFetchVideoChapters instead of calling youtube/whisper directly, so MCP benefits from the same cache and validation as the REST API. Removed private fetchSubtitlesContent from mcp-core.ts; tools catch ValidationError and NotFoundError and return tool errors.
• Unit tests: cache.test.ts for getCacheConfig (mode, TTLs from env), get/set when CACHE_MODE=off, and close(). validation.test.ts mocks ./cache.js so existing tests run with cache disabled. mcp-core.test.ts updated to mock validation’s validateAnd* and expect corresponding calls.
• REST/MCP error types: src/errors.ts exports HttpError, ValidationError, and NotFoundError with status codes and error labels. Validation helpers throw these; REST global error handler maps them to 4xx/5xx and consistent JSON (error, message).
• MCP HTTP auth module: src/mcp-auth.ts provides ensureAuth(request, reply, authToken) and getHeaderValue(); MCP HTTP server uses them when MCP_AUTH_TOKEN is set. Token comparison is timing-safe to prevent timing attacks.
• Unit tests: mcp-auth.test.ts for getHeaderValue and ensureAuth (no auth, missing/ malformed Bearer, wrong token, correct token). mcp-http.test.ts for 401 on /mcp when auth required and no/ invalid header, and that /health remains allowed without auth when token is set.
• Load testing: docs/load-testing.md documents k6-based load tests for the REST API (health, subtitles, mixed). Make targets: load-test, load-test-health, load-test-subtitles, load-test-mixed (Docker k6); npm scripts: load-test, load-test:subtitles, load-test:mixed. Configurable via LOAD_BASE_URL / BASE_URL and RATE_LIMIT_MAX for throughput.
• Export: YtDlpVideoInfo type is now exported from youtube.ts for callers that pass pre-fetched data into fetchVideoChapters.
• Unit tests: youtube.test.ts — fetchVideoChapters with preFetchedData (no execFile call, correct chapter mapping; null handling). validation.test.ts — fetchYtDlpJson called once and data passed to fetchVideoChapters; Vimeo test expects three-argument call. mcp-core.test.ts — chapters tool expectations updated for fetch order and three-argument fetchVideoChapters call.
• yt-dlp proxy (optional): All yt-dlp requests (subtitle download, video info, chapters, audio for Whisper) can be routed through a proxy. Set YT_DLP_PROXY to a URL; supported schemes: http://, https://, socks5:// (e.g. http://user:[email protected]:8080, socks5://127.0.0.1:9050 for Tor). Documented in docs/configuration.md and .env.example; in Docker, set the variable in the container environment if needed.
• Unit tests: mcp-core.test.ts — Whisper fallback with omitted lang (auto-detect). whisper.test.ts — no language param when lang is empty.
• yt-dlp cookies file logging: When COOKIES_FILE_PATH is set, the app now logs cookies file status before each yt-dlp call (subtitle download, audio download, video info/chapters). Logs include path, existence, file size, or access error message (no cookie contents). Helps diagnose "Sign in to confirm you're not a bot" and other YouTube auth issues when running in Docker or with mounted cookies.

Changed

• Graceful shutdown: REST API (src/index.ts) and MCP HTTP (src/mcp-http.ts) now call closeCache() after closing the server so the Redis connection is closed cleanly.
• yt-dlp-check: Fallback logger uses console.warn instead of console.info for the info-level message to satisfy the no-console lint rule.
• Dependencies: Bumped Fastify plugins (@fastify/cors ^11.2.0, @fastify/multipart ^9.4.0, @fastify/rate-limit ^10.3.0, @fastify/swagger ^9.7.0, @fastify/swagger-ui ^5.2.5, @fastify/type-provider-typebox ^6.1.0), @sinclair/typebox ^0.34.48, ioredis ^5.9.3. Dev: @types/jest ^30.0.0, @types/node ^25.2.3, @typescript-eslint/* and typescript-eslint ^8.55.0, eslint ^9.18.0, jest ^30.2.0, prettier ^3.8.1, ts-jest ^29.4.6, typescript ^5.9.3, husky ^9.1.7.
• Dependency: Added ioredis for Redis cache backend (used only when CACHE_MODE=redis).
• MCP: Shared logic for subtitle fetch and Whisper fallback is now in a private fetchSubtitlesContent(resolved, log) in mcp-core.ts. Tools get_transcript and get_raw_subtitles call it and only handle final processing (parse + paginate vs raw + paginate). Removes duplication of resolveSubtitleArgs, downloadSubtitles, and Whisper fallback between the two tools.
• Docker: single Dockerfile with shared base. One Dockerfile now builds both REST API and MCP images via multi-stage build. Stages: builder (Node, npm ci, build) → base (node, python3, pip, curl, unzip, ffmpeg, Deno, yt-dlp -U, YT_DLP_JS_RUNTIMES) → api (REST, port 3000) and mcp (MCP, port 4200). Build with docker build -f Dockerfile --target api . or --target mcp .. Dockerfile.mcp removed; Makefile targets docker-build-api and docker-build-mcp (and buildx variants) use the same Dockerfile with the appropriate target. README and docs/quick-start.mcp.md updated to use --target api / --target mcp.
• Chapters: single yt-dlp fetch. validateAndFetchVideoChapters (REST /video-info/chapters) and MCP tool get_video_chapters now perform one yt-dlp network call instead of two. fetchVideoChapters in youtube.ts accepts an optional third argument preFetchedData; when provided, it reuses that data and skips the internal fetchYtDlpJson call. Validation and MCP handlers fetch once and pass the result into fetchVideoChapters, so video ID and chapters are derived from the same response.
• Unit tests: youtube.test.ts — getYtDlpEnv and appendYtDlpEnvArgs now cover YT_DLP_PROXY / proxyFromEnv (trim, presence of --proxy in args, omission when unset).
• MCP tools get_transcript and get_raw_subtitles: Parameter lang is now optional. When omitted, subtitle download still uses en for yt-dlp; when Whisper fallback is used, language is auto-detected (no language query param sent to Whisper). Tool descriptions updated to mention optional lang and auto-detect behavior.

Security

• MCP HTTP auth: Bearer token validation uses crypto.timingSafeEqual so comparison time does not depend on the token value.

Fixed

• yt-dlp cookies on read-only volume: When COOKIES_FILE_PATH points to a read-only file (e.g. Docker volume mounted without write access), yt-dlp failed with PermissionError while saving cookies at exit, even when the download succeeded. The app now copies the cookies file to a writable temp location before passing it to yt-dlp; the temp file is removed after each call. New ensureWritableCookiesFile() in youtube.ts checks read/write access and returns either the original path or a temp copy. Used by downloadSubtitles, downloadAudio, and fetchYtDlpJson.

Changed

• Project rename: yt-captions-downloader → transcriptor-mcp. Package name, GitHub repo, Docker images, and docker-compose service names have been updated.
• Package: transcriptor-mcp (was yt-captions-downloader-mcp).
• GitHub: samson-art/transcriptor-mcp.
• Docker images: artsamsonov/transcriptor-mcp (MCP), artsamsonov/transcriptor-mcp-api (REST API).
• docker-compose services: transcriptor-mcp (MCP), transcriptor-mcp-api (REST API).
• MCP server name: transcriptor-mcp (reported in MCP initialize).
• User-Agent: transcriptor-mcp (for yt-dlp requests).
• MCP config key: Use transcriptor in claude_desktop_config.json / Cursor MCP settings (shorter UX).
• docs/configuration.md: Documented YT_DLP_SKIP_VERSION_CHECK and YT_DLP_REQUIRED; startup checks reference src/yt-dlp-check.ts. Added "Whisper fallback" section for all WHISPER_* variables and usage (local container vs API).

Added

• Multi-platform support: Subtitles, available subtitles, video info, and chapters work with URLs from YouTube, Twitter/X, Instagram, TikTok, Twitch, Vimeo, Facebook, Bilibili, VK, and Dailymotion (via yt-dlp). Bare video IDs are supported for YouTube only.
• Whisper fallback: When YouTube subtitles cannot be obtained (yt-dlp returns none), the app can transcribe video audio via Whisper. Configurable with WHISPER_MODE (off, local, api). Local mode uses a self-hosted HTTP service (e.g. whisper-asr-webservice in Docker); API mode uses an OpenAI-compatible transcription endpoint. New env vars: WHISPER_BASE_URL, WHISPER_TIMEOUT, WHISPER_API_KEY, WHISPER_API_BASE_URL. REST responses for /subtitles and /subtitles/raw include optional source: "youtube" | "whisper"; MCP tools get_transcript and get_raw_subtitles use the same fallback and expose source in structured content.
• Audio download: downloadAudio(videoId, logger) in youtube.ts downloads audio-only via yt-dlp for Whisper input; uses same cookies and timeout as subtitle download.
• Docker: docker-compose.example.yml adds a whisper service (image onerahmet/openai-whisper-asr-webservice:latest) and example WHISPER_* env for transcriptor-mcp-api and transcriptor-mcp. .env.example and docs/configuration.md document all Whisper options.
• Unit tests: src/whisper.test.ts for getWhisperConfig and transcribeWithWhisper; validation.test.ts extended with Whisper fallback success and 404 when Whisper returns null.
• yt-dlp startup check: REST API, MCP HTTP, and MCP stdio servers run a yt-dlp availability check at startup. If yt-dlp is missing or fails to run, the app logs an ERROR and exits (unless YT_DLP_REQUIRED=0). If the installed version is older than the latest on GitHub, a WARNING is logged.
• Environment variables: YT_DLP_SKIP_VERSION_CHECK — when set to 1, skips the GitHub version check and WARNING; YT_DLP_REQUIRED — when set to 0, logs ERROR but does not exit when yt-dlp is missing or fails.
• Unit tests: src/yt-dlp-check.test.ts for version parsing, comparison, GitHub fetch, and startup check behavior.

Added

• REST API: GET /health endpoint returning { "status": "ok" } for liveness/readiness and Docker HEALTHCHECK.
• REST API: Optional CORS allowlist via CORS_ALLOWED_ORIGINS (comma-separated origins); when unset, all origins remain allowed.
• MCP HTTP server: Rate limiting configurable via MCP_RATE_LIMIT_MAX and MCP_RATE_LIMIT_TIME_WINDOW.
• MCP HTTP server: Session TTL and periodic cleanup via MCP_SESSION_TTL_MS and MCP_SESSION_CLEANUP_INTERVAL_MS.
• Version: src/version.ts reads version from package.json; REST API and MCP server use it for responses and server info.
• E2E smoke test: MCP coverage — starts MCP container and verifies stdio (initialize over stdin/stdout), streamable HTTP (POST /mcp), and SSE (GET /sse). New env vars: SMOKE_SKIP_MCP, SMOKE_MCP_IMAGE, SMOKE_MCP_URL / SMOKE_MCP_PORT, SMOKE_MCP_AUTH_TOKEN, plus API-related overrides.
• Docs: docs/configuration.md — CORS, MCP rate limit/session/cleanup, health endpoint, and E2E smoke test env vars. .env.example updated with CORS_ALLOWED_ORIGINS and MCP HTTP options.

Changed

• REST API and MCP server now derive version from package.json instead of hardcoded values.
• REST API: global Fastify error handler returns 500 with error and message; route handlers no longer wrap in try/catch so validation/parsing errors are handled consistently.
• E2E smoke test flow: single entry npm run test:e2e:api with optional MCP checks; README updated with env var table and simplified run instructions.
• docker-compose.example.yml: reordered keys (ports after environment), added restart: unless-stopped for MCP service; API service no longer includes build (image-only).
• Jest: exclude src/e2e/api-smoke.ts from coverage (top-level await).

[0.3.6] - 2026-02-05

Changed

• Upgraded Fastify to v5 and related plugins (@fastify/cors, @fastify/multipart, @fastify/rate-limit, @fastify/swagger, @fastify/swagger-ui, @fastify/type-provider-typebox) to compatible major versions.
• Bumped @modelcontextprotocol/sdk to ^1.26.0.

[0.3.5] - 2026-02-04

Added

• OpenAPI/Swagger documentation at /docs with request/response schemas for all REST endpoints (subtitles, raw subtitles, available subtitles, video info, chapters).
• E2E smoke test now verifies that Swagger UI at /docs is reachable.

Changed

• REST routes registered with @fastify/swagger and @fastify/swagger-ui; each endpoint documents body and response schemas for generated OpenAPI spec.

[0.3.4] - 2026-02-04

[0.3.4] - 2026-02-04

Added

• Docker-based e2e smoke test for the REST API (src/e2e/api-smoke.ts) that builds a local image, starts a container and verifies POST /subtitles against a real YouTube video.
• Documentation in README for running Docker smoke tests locally and as part of the make publish workflow.
• Additional unit tests for validation helpers and yt-dlp integration (URL / video ID / language sanitization, video info and chapter extraction, environment-driven yt-dlp flags).
• Dedicated test suite for MCP tools (src/mcp-core.test.ts) covering success and error paths for transcripts, raw subtitles, available subtitles, video info and chapters.

Changed

• Hardened validation.ts helpers to provide more explicit 4xx errors for invalid URLs, video IDs and language codes across subtitles, available subtitles, video info and chapters endpoints.
• Improved youtube.ts helpers to map more yt-dlp metadata, expose chapter markers, and sort official vs auto subtitle language codes for stable output.
• Refined MCP core implementation to use stricter validation and add pagination/error handling tests for all tools.
• Updated Jest configuration to collect coverage from src, exclude entrypoints (REST + MCP) and enable verbose output.

[0.3.3] - 2026-02-04

Added

• New /subtitles/available REST endpoint that returns the video ID and sorted lists of official vs auto-generated subtitle language codes.
• Validation helper validateAndFetchAvailableSubtitles for safely extracting and sanitizing YouTube video IDs before fetching available subtitles.
• Unit test fetchAvailableSubtitles.test.ts covering fetchAvailableSubtitles behavior (official vs auto subtitles).
• Documentation for using the MCP server as an n8n MCP client over streamable HTTP, including guidance on N8N_PROXY_HOPS.

Changed

• Production Dockerfile now installs Deno as a JS runtime for yt-dlp, updates yt-dlp to the latest stable release, and configures YT_DLP_JS_RUNTIMES="deno,node".
• MCP core now imports Zod via zod/v3 to improve JSON Schema compatibility with strict MCP clients (such as n8n).
• Jest configuration adds a moduleNameMapper rule to map .js imports back to TypeScript sources under NodeNext/ESM.
• Updated API documentation in README to cover the new /subtitles/available endpoint with request/response examples.
• Updated .gitignore to also ignore Makefile.

[0.3.1] - 2026-02-03

Added

• MCP server over HTTP transports:
  • Streamable HTTP endpoint at /mcp (src/mcp-http.ts)
  • SSE endpoint at /sse with message handler at /message (src/mcp-http.ts)
• Optional auth for HTTP MCP via MCP_AUTH_TOKEN (Bearer token)
• Optional SSE allowlists via MCP_ALLOWED_HOSTS / MCP_ALLOWED_ORIGINS
• Extracted reusable MCP server core into src/mcp-core.ts
• New script: start:mcp:http

Changed

• Updated docker-compose.example.yml MCP service to run HTTP mode and expose port 4200
• Updated Dockerfile.mcp to expose 4200 for HTTP mode
• Streamlined src/mcp.ts to be stdio-only entrypoint
• Bumped package version to 0.3.1

[0.3.0] - 2026-02-03

Added

• MCP server (Cursor) over stdio (src/mcp.ts) with tools:
  • get_transcript (plain text transcript, paginated)
  • get_raw_subtitles (raw SRT/VTT, paginated)
  • get_available_subtitles (official vs auto language codes)
  • get_video_info (basic metadata via yt-dlp)
• Docker image for MCP server (Dockerfile.mcp)
• docker-compose.example.yml with an additional MCP service example
• REPOSITORY_OVERVIEW.md project overview document

Changed

• Switched project to ESM:
  • package.json now uses "type": "module"
  • TypeScript config updated to module/moduleResolution: nodenext
  • Local imports updated to use .js extensions for NodeNext compatibility
• Added MCP-related scripts:
  • start:mcp, dev:mcp
• yt-dlp integration extended with:
  • fetchVideoInfo
  • fetchAvailableSubtitles
  • shared handling for yt-dlp env flags (--cookies, --js-runtimes, --remote-components)
• Jest config renamed to jest.config.cjs

Security

• Added cookies.txt to .gitignore to avoid accidental commits of sensitive cookies

Added

• Docker Compose configuration for easier container orchestration and deployment
• Cookie support for accessing age-restricted or region-locked YouTube videos
• COOKIES_FILE_PATH environment variable for persistent cookie file management
• @fastify/multipart dependency for handling file uploads in cookie requests
• Comprehensive cookie handling with proper sanitization and temporary file management

Changed

• Refactored API routes in src/index.ts for improved code organization
• Updated README with detailed cookie usage examples and new environment variables
• Simplified validation logic by removing redundant cookie validation code
• Enhanced subtitle download function to support optional cookie parameters

Removed

• Health check endpoint from Dockerfile (moved to application-level routing)

Full Changelog: https://github.com/samson-art/yt-captions-downloader/compare/0.1.0...0.2.0