[MCP]: Extension-backed visible-tab screenshot fallback after target loss / 扩展后台截图兜底

English

Feature request: extension-backed visible-tab screenshot fallback when an existing-browser Playwright target is lost

When using the Playwright Chrome Extension with an existing, user-profile Chrome session, some sign-in flows can invalidate the Playwright/CDP page target after the user clicks Sign in. The browser tab itself remains open and the login UI (including a QR code) is visibly rendered in that same tab, but subsequent calls such as page.screenshot() fail because the previously attached automation target is gone.

This is not a request for a site-specific adapter. I reproduced the pattern with a QR-code login modal in a Chinese web app, but the underlying case appears applicable to SPAs and sign-in flows that recreate their renderer/document or otherwise replace the attached target.

Observed behavior

1. Connect to an existing Chrome tab via Playwright Extension.
2. Use Playwright MCP to click the page's Sign in button.
3. The current tab shows an in-page login modal with a QR code; no new tab is created.
4. The original Playwright page / automation target is no longer usable, so page.screenshot() cannot complete.
5. The Chrome extension can still enumerate the original tab, and Chrome's extension background APIs can capture its visible surface.

The user's Chrome profile, cookies, and login state remain in the existing browser. The failure is the lifetime of the attached automation target, not the browser state itself.

Proposed direction

Please consider an explicit, extension-backed visible-tab capture fallback for the existing-browser extension mode.

A possible shape (names are only illustrative):

• Retain the identity of the user-approved connected tab independently of a single Playwright page object.
• Expose an opt-in MCP operation such as browser_capture_visible_tab, backed by the extension Service Worker and chrome.tabs.captureVisibleTab.
• Allow the operation to work when the originally attached Playwright target has detached, provided the approved tab still exists.
• Return a PNG/JPEG payload or an explicit status such as target_lost, timeout, or unsupported.
• Do not read or export cookies, passwords, local storage, or hidden page content.

This would be a fallback for an explicitly approved tab, not a replacement for normal Playwright screenshots. It could help agents keep an interactive login workflow alive long enough to show a human the visible QR code, then re-locate the page after login completes.

Security / UX considerations

• Capture should be explicit and limited to the tab the user approved for the extension connection.
• The tool should not silently scan arbitrary tabs or persist image data.
• The browser window/tab may need to be focused for captureVisibleTab; the API should document this behavior.
• The feature should remain generic and should not encode site-specific selectors, QR recognition, or automatic authentication.

Local prototype and willingness to contribute

I have a local proof of concept that separates this fallback from the unstable Playwright page: the extension background stores the source tabId, observes the login modal, calls chrome.tabs.captureVisibleTab, and returns an in-memory PNG through a local MCP relay. It has been tested with a real current-Chrome QR-login modal after the original Playwright target became unusable.

I would be happy to prepare a focused PR with tests and documentation if the maintainers agree that this belongs in the extension/MCP design and can advise on the desired API and security model.

---

中文说明

功能建议：当现有浏览器中的 Playwright 目标丢失时，提供由扩展后台完成的可见标签页截图兜底

在使用 Playwright Chrome Extension 连接用户当前 Chrome 配置文件时，部分网页登录流程会在点击“登录”后使原先附着的 Playwright/CDP 页面目标失效。Chrome 标签页本身仍然存在，同一标签页内仍正常渲染登录弹层和二维码，但后续 page.screenshot() 会因原自动化目标已丢失而失败。

这不是某个网站的专用适配需求。虽然我是在一个中文网站的二维码登录弹层中复现的，但根因更通用：SPA 或登录流程可能重建文档／渲染上下文，或替换已经附着的自动化目标。

现象

1. 通过 Playwright Extension 连接一个当前 Chrome 标签页。
2. 使用 Playwright MCP 点击页面“登录”。
3. 当前标签页内出现二维码登录弹层，没有新建标签页。
4. 原先的 Playwright page／自动化目标已不可用，page.screenshot() 无法完成。
5. Chrome 扩展仍可枚举原标签页，扩展后台 API 仍可以截取其当前可见内容。

用户的 Chrome 配置、Cookie 和登录状态都还留在原浏览器中；失效的是已附着自动化目标的生命周期，而不是浏览器状态本身。

建议方向

建议在 existing-browser extension 模式中增加一个显式调用、由扩展后台执行的可见标签页截图兜底能力：

• 在单一 Playwright page 对象之外，保留用户已授权连接标签页的身份信息；
• 提供类似 browser_capture_visible_tab 的 MCP 操作（名称仅为示例），由扩展 Service Worker 调用 chrome.tabs.captureVisibleTab；
• 当原 Playwright 目标已经 detached、但已授权标签页仍存在时，该操作依然可以执行；
• 返回 PNG/JPEG 或明确状态，例如 target_lost、timeout、unsupported；
• 不读取或导出 Cookie、密码、Local Storage 或不可见页面内容。

该能力应只是用户明确授权标签页的兜底方案，而不是替代正常的 Playwright 截图。它可以让代理在交互式登录期间把当前可见二维码交给用户，然后在扫码完成后重新定位页面继续操作。

安全与体验边界

• 截图必须显式调用，并限制在用户已批准连接的标签页。
• 不应静默扫描任意标签页，也不应长期持久化图片。
• captureVisibleTab 可能需要目标窗口／标签页处于前台，API 需要明确这一行为。
• 保持通用能力，不引入网站专用选择器、二维码识别或自动认证逻辑。

本地原型与贡献意愿

我已有本地最小原型：扩展后台保存源 tabId、观察登录弹层、调用 chrome.tabs.captureVisibleTab，并经本地 MCP Relay 返回仅在内存中传递的 PNG。已在真实当前 Chrome 的二维码登录弹层中验证：原 Playwright 目标不可用后，截图仍能成功。

如果维护者认可该能力属于扩展／MCP 的设计范围，并能指导期望的 API 形态与安全边界，我愿意按要求准备聚焦的 PR、测试与文档。