Teaching an AI Agent to Hold the Remote

Smart TV development has the worst inner loop in web development. Your app is a web app, but it lives inside two vendor SDKs, behind a certificate ceremony, on a screen that is physically across the room. Change a line of CSS and the ritual begins: package, sign, install over the network, walk to the TV, squint, walk back. Samsung and LG each have their own CLI, their own signing story, their own ways of failing silently.

Meanwhile, AI coding agents got good. An agent can edit the code, run the build, read the logs — but it can't see the TV. So I built tv-mcp, an open-source Model Context Protocol server that hands the whole loop to the agent: build, sign, install, launch, screenshot, read the console, press remote keys. On real Samsung Tizen and LG webOS panels. This is how it's put together, and what one afternoon with a real hospitality TV taught it.

The one insight the whole design hangs on

A Tizen .wgt and a webOS .ipk look like alien artifacts, but inside both wrappers sits a Chromium webview — and both platforms expose that webview's remote inspector. That's the Chrome DevTools Protocol, the same thing your desktop browser speaks.

So the server splits into two planes. A platform plane per vendor handles everything genuinely different: packaging, signing, installing, launching, via tizen/sdb on Samsung and ares-* on LG. And a single shared debug plane rides CDP: screenshots via Page.captureScreenshot, console capture, JavaScript evaluation inside the running app. Write it once, and it works identically on both brands, because underneath it's just Chromium.

Thirteen tools, three at a time

The lazy way to build an MCP server is to dump every tool schema on the model at session start. Thirteen tools' worth of JSON schema is context the agent pays for on every request, whether it's debugging a focus bug or just asking what devices exist.

tv-mcp discloses progressively instead. A fresh session sees exactly three tools: list_devices, connect_device, docs. Connect to a TV and the app-lifecycle tier appears — build, install, launch, logs. Launch with debug: true and the inspector tier unlocks: screenshot, console_logs, eval_js. The mechanism is vanilla MCP: tools register disabled, and tool.enable() fires a tools/list_changed notification so the client refreshes its list.

The deep platform knowledge follows the same rule. Tizen certificate internals, webOS dev-mode expiry, pairing protocols — none of it lives in tool descriptions. It ships as MCP resources the agent fetches only when it hits the corresponding wall. Tool descriptions stay one line each.

Errors are UI for agents

Every error the server throws carries a mandatory remedy field, because an agent can only fix what the error tells it to fix. "Install failed" is a dead end. This is a next move:

Install rejected by TV (certificate):
  Check certificate error : Invalid certificate chain

Remedy: The distributor certificate does not cover this TV —
  use a Samsung-issued certificate profile that includes this
  device's DUID (sdb shell 0 getduid).
  Read tvmcp://docs/tizen-signing.

That rule is enforced in the contributing guide, not just observed in the code. It's the difference between an agent that retries blindly and one that fixes the signing profile and moves on.

Shipping without secrets

The release pipeline has no long-lived credentials anywhere. npm publishing uses trusted publishing over OIDC — timely, since npm's 2FA-bypass tokens lose publishing rights in 2027 — and the official MCP Registry authenticates the GitHub Actions run itself via OIDC to authorize the io.github.* namespace. One gh release create and CI builds, tests, publishes to npm with provenance attestations, and updates the registry entry. Nothing to rotate, nothing to leak.

Then a real TV had opinions

Everything above worked in tests and against the SDK. Then I pointed it at the lab TV, which turned out to be a Samsung HG32F800 — a hospitality panel, the hotel-room kind. For a project pitched at hospitality and IPTV teams, the test hardware could not have been more on brand. In one afternoon it broke three of my clean abstractions.

First: the official install command fails silently. tizen install reported "Failed to install Tizen application" with an empty platform log. Pushing the package with sdb and installing via vd_appinstall instead produced progress percentages and the actual failure: an invalid certificate chain. My signing profile used the generic Tizen distributor certificate, which real TVs reject — panels want a Samsung-issued certificate with the TV's own device ID (DUID) baked in. Switched profiles, install completed. The driver now uses the sdb path permanently, because an error you can read beats an official command that shrugs.

Second: the relaunch deadlock. sdb shell 0 debug hangs forever if the app is already running. The driver dutifully killed the app first — except pkgcmd takes the package id (TvMcpTest0), not the full app id (TvMcpTest0.tvmcptest), so the kill silently failed and every second launch froze. Now it kills by package id, clears stale port forwards, and times the debug call out with a remedy instead of hanging.

Third: hotel firmware locks the remote. The Samsung remote-control WebSocket API on port 8002 — which I'd implemented with the whole one-time pairing dance — is disabled outright on hospitality firmware. No prompt, no port, EHOSTUNREACH. The fallback saved the feature: since the debug plane can evaluate JavaScript in the app, it can dispatch synthetic KeyboardEvents, which spatial-navigation frameworks treat exactly like real remote presses. The app logged last key: 39 and the navigation test passed, on a panel where the "proper" API doesn't exist.

The loop the project promised held up end to end on that panel: the agent connected, packaged, signed, installed, launched with the inspector attached, pulled a screenshot over CDP, read its own console.log heartbeats, evaluated state inside the running app, and pressed keys. Nobody walked to the TV.

What the panel taught me

• Find the shared substrate. Two hostile SDKs shrank to two thin drivers the moment I noticed both TVs are Chromium underneath. The 80% that's identical deserves one implementation.
• Context is a budget. Agents pay for every tool schema on every request. Disclose tools when they become usable, not when they're conceivable.
• Write errors for the reader that acts on them. For an agent that means machine-legible failure plus a concrete next step, every time.
• Real hardware is the review you can't skip. Three bugs in one afternoon, none visible in tests, all obvious on a panel. Ship the abstraction, then let the device mark it up.

tv-mcp is MIT-licensed on GitHub, on npm (npx tv-mcp), and in the official MCP Registry as io.github.skdonthi/tv-mcp. If you ship web apps to fleets of hotel or cruise TVs and want commercial-panel support (Pro:Centric, SSSP) to exist, the repo explains how to help — hardware access moves that roadmap faster than money. And if you're building MCP servers for hardware nobody thought agents would touch, I'd love to compare notes: email or LinkedIn.