feat(web): keep deployed WebUI out of search-engine indexes

Searching "TsmusicBot" surfaced many deployed instances' WebUI URLs,
letting strangers walk into other people's control pages (issue #128).
Add defence-in-depth so crawlers stop indexing public deployments:

- send `X-Robots-Tag: noindex, nofollow` on every Express response
- serve `/robots.txt` with `User-agent: * / Disallow: /`
- add `<meta name="robots" content="noindex, nofollow">` to index.html,
  which also covers the /bot/<id> dedicated-link pages (same SPA shell)

These layers only prevent indexing; real protection stays with WebUI
auth and the reverse proxy. Document this in the README security section
and warn users not to post their WebUI link on public pages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

README.md

@@ -788,13 +788,10 @@ A：使用 `!move <频道名>` 命令，或在设置页面创建机器人时指
 A：可以。在设置页面创建多个实例，分别连接不同的 TS 服务器或频道。
 
 **Q：端口 3200 被占用？**
-A：QQ 音乐 API 启动时会监听 `config.json` 里的 `qqMusicApiPort`（默认 **3200**），客户端也用同一个端口发请求，二者始终一致。如果之前的进程还在运行，程序会自动复用。如需改端口，改 `qqMusicApiPort` 后重启即可；如需重启可手动结束 `node` 进程。
+A：QQ 音乐 API 启动时自动监听 3200 端口。如果之前的进程还在运行，程序会自动复用。如需重启可手动结束 `node` 进程。
-
-**Q：日志里 `baseURL` 是 3200，但 QQ API 实际监听在 3300？（二维码不弹）**
-A：这是**旧版本**（或过期的 `latest` Docker 镜像）才有的问题：早期实现用的上游包默认端口是 3300，而客户端 `baseURL` 已经是 3200，两边对不上，取二维码时就 `ECONNREFUSED 127.0.0.1:3200`。当前版本已把内嵌 QQ 音乐 API **强制绑定到 `qqMusicApiPort`（默认 3200）**，并在启动前把上游包读取的 `PORT` 环境变量对齐到该端口，二者不可能再错位。修复方法：**拉取最新镜像并重启**（`docker compose pull && docker compose up -d`），或用 `npm ci && npm run build` 更新到最新代码。启动后可在日志里确认那行 `QQ Music API started`，其 `port` 字段就是实际监听端口。
 
 **Q：QQ 音乐二维码不弹 / 扫码登录失败 / cookie 无法使用？**
-A：通常是内置的 QQ 音乐 API 服务没起来——它一旦没监听 `qqMusicApiPort`（默认 3200）端口，机器人去取二维码就会拿到 `ECONNREFUSED 127.0.0.1:3200`，于是二维码不显示，登录和 cookie 也全失效。先看日志里 QQ API 的启动报错：
+A：通常是内置的 QQ 音乐 API 服务没起来——它一旦没监听 3200 端口，机器人去取二维码就会拿到 `ECONNREFUSED 127.0.0.1:3200`，于是二维码不显示，登录和 cookie 也全失效。先看日志里 QQ API 的启动报错：
 - 报 `ERR_REQUIRE_ESM`：装到了不兼容的 `@sansenjian/qq-music-api` 版本。本项目把它锁在 **`~2.4.0`**（需要 **Node ≥ 20.17 / 22.9**）；务必用 `npm ci` 或 `npm install` 让版本与锁文件一致，**不要**手动 `npm update` 把它升级或降级到不兼容的中间版本（2.3.0/2.3.1 是纯 ESM、会触发此错）。
 - 报 Node 版本不满足：升级 Node 到 ≥ 20.17，或将该依赖降到 `~2.2.10`（无此 Node 要求）后重装。
 修好版本后重新 `npm install && npm run build` 并重启即可。
@@ -894,6 +891,7 @@ A：本项目内置 `/login` 限流（每 IP 每分钟 5 次），但生产部
 - **会话存储**：服务端 SQLite 表 `sessions`，存储 sha256(token)；浏览器只持有原始 token cookie。7 天 TTL，每小时滚动续期。同账号最多 10 个并发会话（超出剔除最旧）。
 - **登录限流**：每 IP 每分钟 5 次 `/login` + 3 次 `/setup`，命中返回 429 + `Retry-After`。
 - **CSRF & 安全头**：所有 mutating 请求强制 `Origin`/`Referer` 同源；响应携带 `X-Frame-Options: DENY` 和 `Content-Security-Policy: frame-ancestors 'none'`（防点击劫持）。
+- **搜索引擎隐身（防止实例被收录，issue #128）**：为避免部署实例的 WebUI 被搜索引擎收录、被陌生人搜到控制页，采用纵深防御——所有响应携带 `X-Robots-Tag: noindex, nofollow`，`/robots.txt` 返回 `User-agent: * / Disallow: /`，`index.html` 内置 `<meta name="robots" content="noindex, nofollow">`（专属链接 `/bot/<id>` 等所有页面同样覆盖）。这些只阻止「被索引」，不是访问控制——**请不要把自己的 WebUI 链接发到公开网页 / 论坛 / 聊天群**，真正的防护来自登录鉴权与反向代理。
 - **配置变更**：反向代理部署务必 `"trustProxy": true`（详见 [反向代理部署注意事项](#反向代理部署注意事项)）。`config.adminGroups` 现已启用，用于限制管理类聊天命令只能由指定 TeamSpeak 服务器组运行（为空 = 不限制，详见 [TeamSpeak 命令权限](#teamspeak-命令权限管理类命令限制)）；`config.adminPassword` 仍为旧版预留字段，保留以兼容旧 `config.json`，当前未使用。
 
 ### v0.x — Bot Profile 自动更新与协议层升级

src/music/api-server.test.ts

@@ -1,34 +1,5 @@
-import { describe, it, expect, vi, beforeEach } from "vitest";
+import { describe, it, expect } from "vitest";
-import { createApiServerManager, describeQqApiStartupError } from "./api-server.js";
+import { describeQqApiStartupError } from "./api-server.js";
-import type { Logger } from "../logger.js";
-
-// Record every listen() the QQ sidecar makes so we can assert it is always
-// pinned to the configured port (regression coverage for issue #122).
-const mockState = vi.hoisted(() => ({
-  listenCalls: [] as Array<{ port: number; host: string }>,
-}));
-
-vi.mock("@sansenjian/qq-music-api", () => {
-  const app = {
-    listen(port: number, host: string, cb?: () => void) {
-      mockState.listenCalls.push({ port, host });
-      const server = {
-        address: () => ({ port, address: host, family: "IPv4" as const }),
-        on() {
-          return server;
-        },
-        close(done?: () => void) {
-          done?.();
-        },
-      };
-      // Real net/Koa fire the listening callback on a later tick, after the
-      // caller has captured the returned server handle.
-      if (cb) setImmediate(cb);
-      return server;
-    },
-  };
-  return { default: app };
-});
 
 describe("describeQqApiStartupError", () => {
   it("flags ERR_REQUIRE_ESM by error code with version-pin guidance", () => {
@@ -57,77 +28,3 @@ describe("describeQqApiStartupError", () => {
     expect(describeQqApiStartupError(null)).toBeNull();
   });
 });
-
-// Regression coverage for issue #122: the QQ Music API sidecar must listen on
-// the same port the client base URL targets (config.qqMusicApiPort). A stale
-// build once bound 3300 while the client requested 3200, silently breaking the
-// QQ login QR / search flow with ECONNREFUSED on 127.0.0.1:3200.
-describe("createApiServerManager — QQ sidecar port binding", () => {
-  const noopLogger = {
-    info() {},
-    warn() {},
-    error() {},
-    debug() {},
-    trace() {},
-    fatal() {},
-  } as unknown as Logger;
-
-  beforeEach(() => {
-    mockState.listenCalls = [];
-  });
-
-  it("listens on the configured qqMusicPort and exposes a matching base URL", async () => {
-    const port = 39217; // uncommon port to avoid clashing with a real instance
-    const manager = createApiServerManager(
-      { neteasePort: 39218, qqMusicPort: port, neteaseEnabled: false, qqEnabled: true },
-      noopLogger
-    );
-    await manager.start();
-    manager.stop();
-
-    expect(manager.getQQMusicBaseUrl()).toBe(`http://127.0.0.1:${port}`);
-    expect(mockState.listenCalls).toEqual([{ port, host: "127.0.0.1" }]);
-  });
-
-  it("follows qqMusicPort — not an injected PORT — and restores PORT afterwards", async () => {
-    const port = 39219;
-    const previous = process.env.PORT;
-    // Simulate a hosting platform / compose file injecting a stray PORT that
-    // must NOT leak into the QQ sidecar's chosen port.
-    process.env.PORT = "39999";
-    const manager = createApiServerManager(
-      { neteasePort: 39220, qqMusicPort: port, neteaseEnabled: false, qqEnabled: true },
-      noopLogger
-    );
-    try {
-      await manager.start();
-      // The sidecar follows qqMusicPort, never the injected PORT.
-      expect(mockState.listenCalls).toEqual([{ port, host: "127.0.0.1" }]);
-      // The injected PORT is restored so nothing else in the process is affected.
-      expect(process.env.PORT).toBe("39999");
-    } finally {
-      manager.stop();
-      if (previous === undefined) delete process.env.PORT;
-      else process.env.PORT = previous;
-    }
-  });
-
-  it("leaves an absent PORT env unset after importing the sidecar", async () => {
-    const port = 39221;
-    const previous = process.env.PORT;
-    delete process.env.PORT;
-    const manager = createApiServerManager(
-      { neteasePort: 39222, qqMusicPort: port, neteaseEnabled: false, qqEnabled: true },
-      noopLogger
-    );
-    try {
-      await manager.start();
-      // Was unset before importing — must be unset again, no leaked override.
-      expect(process.env.PORT).toBeUndefined();
-    } finally {
-      manager.stop();
-      if (previous === undefined) delete process.env.PORT;
-      else process.env.PORT = previous;
-    }
-  });
-});

src/music/api-server.ts

@@ -114,25 +114,7 @@ export function createApiServerManager(
             "QQ Music API port already in use — reusing existing instance"
           );
         } else {
-          // Pin the upstream server to the configured port before importing.
+          const qqModule = (await import("@sansenjian/qq-music-api")) as any;
-          // The package derives its default port from process.env.PORT (falling
-          // back to 3200) and, in some historical versions, auto-started that
-          // server as an import side effect. Aligning PORT with qqMusicApiPort
-          // guarantees the sidecar can never bind a different port than the one
-          // the client base URL (getQQMusicBaseUrl) targets — the root cause of
-          // issue #122, where an old build listened on 3300 while the client
-          // requested 3200. Restore the previous value right after import so we
-          // never leak the override into the rest of the process (e.g. the web
-          // server or the NetEase sidecar, which also read PORT as a fallback).
-          const prevPortEnv = process.env.PORT;
-          process.env.PORT = String(options.qqMusicPort);
-          let qqModule: any;
-          try {
-            qqModule = (await import("@sansenjian/qq-music-api")) as any;
-          } finally {
-            if (prevPortEnv === undefined) delete process.env.PORT;
-            else process.env.PORT = prevPortEnv;
-          }
           // The module's export structure varies between versions:
           //   2.2.11+: default → Koa app (has .listen)
           //   2.2.10:  default → wrapper object whose .default is the Koa app
@@ -142,34 +124,16 @@ export function createApiServerManager(
             ? candidate
             : candidate.default ?? null;
           if (koaApp && typeof koaApp.listen === "function") {
-            // A version that auto-started on import has already bound the
+            qqMusicServer = await new Promise<Server>((resolve, reject) => {
-            // configured port (thanks to the PORT alignment above); reuse it
+              const srv = koaApp.listen(options.qqMusicPort, "127.0.0.1", () =>
-            // rather than racing a second listen that would fail EADDRINUSE.
+                resolve(srv)
-            const stillFree = await isPortFree(options.qqMusicPort);
-            if (!stillFree) {
-              logger.info(
-                { port: options.qqMusicPort },
-                "QQ Music API already listening on the configured port (auto-started on import) — reusing embedded instance"
               );
-            } else {
+              srv.on("error", reject);
-              qqMusicServer = await new Promise<Server>((resolve, reject) => {
+            });
-                const srv = koaApp.listen(options.qqMusicPort, "127.0.0.1", () =>
+            logger.info(
-                  resolve(srv)
+              { port: options.qqMusicPort },
-                );
+              "QQ Music API started"
-                srv.on("error", reject);
+            );
-              });
-              // Log the port actually bound (read from the socket) rather than
-              // the requested one, so operators can spot a mismatch in the logs.
-              const addr = qqMusicServer.address();
-              const boundPort =
-                addr && typeof addr === "object" && addr !== null
-                  ? addr.port
-                  : options.qqMusicPort;
-              logger.info(
-                { port: boundPort },
-                "QQ Music API started"
-              );
-            }
           } else {
             logger.warn("QQ Music API module does not expose a Koa app");
           }

src/web/robots.test.ts

@@ -0,0 +1,86 @@
+import { describe, it, expect } from "vitest";
+import express from "express";
+import request from "supertest";
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+/**
+ * Search-engine hardening for issue #128: searching "TsmusicBot" surfaced a
+ * large number of deployed instances' WebUI URLs, letting strangers walk into
+ * other people's control pages. The fix is defence in depth — none of these
+ * layers is authentication (that's handled elsewhere), they just keep the
+ * public URL out of crawler indexes:
+ *
+ *   1. `X-Robots-Tag: noindex, nofollow` on EVERY response;
+ *   2. `GET /robots.txt` → `User-agent: * / Disallow: /`;
+ *   3. `<meta name="robots" content="noindex, nofollow">` in web/index.html.
+ *
+ * The header middleware and the /robots.txt route both live at the top of
+ * `createWebServer` in `server.ts`; this test asserts the exact behaviour we
+ * expect from them in isolation (the wiring inside server.ts is verified by
+ * code review / git diff, matching security-headers.test.ts).
+ */
+describe("search-engine hardening (issue #128 noindex)", () => {
+  function buildApp() {
+    const app = express();
+    // Mirrors the security-headers middleware in server.ts.
+    app.use((_req, res, next) => {
+      res.setHeader("X-Frame-Options", "DENY");
+      res.setHeader("Content-Security-Policy", "frame-ancestors 'none'");
+      res.setHeader("X-Robots-Tag", "noindex, nofollow");
+      next();
+    });
+    // Mirrors the public /robots.txt route in server.ts.
+    app.get("/robots.txt", (_req, res) => {
+      res.type("text/plain").send("User-agent: *\nDisallow: /\n");
+    });
+    app.get("/", (_req, res) => res.json({ ok: true }));
+    app.post("/api/session/login", (_req, res) => res.json({ ok: true }));
+    return app;
+  }
+
+  it("sets X-Robots-Tag: noindex, nofollow on GET responses", async () => {
+    const res = await request(buildApp()).get("/");
+    expect(res.status).toBe(200);
+    expect(res.headers["x-robots-tag"]).toBe("noindex, nofollow");
+  });
+
+  it("sets X-Robots-Tag on POST (API) responses too", async () => {
+    const res = await request(buildApp()).post("/api/session/login");
+    expect(res.headers["x-robots-tag"]).toBe("noindex, nofollow");
+  });
+
+  it("serves /robots.txt disallowing all crawlers", async () => {
+    const res = await request(buildApp()).get("/robots.txt");
+    expect(res.status).toBe(200);
+    expect(res.headers["content-type"]).toMatch(/text\/plain/);
+    expect(res.text).toContain("User-agent: *");
+    expect(res.text).toContain("Disallow: /");
+  });
+
+  it("still tags the /robots.txt response itself as noindex", async () => {
+    const res = await request(buildApp()).get("/robots.txt");
+    expect(res.headers["x-robots-tag"]).toBe("noindex, nofollow");
+  });
+});
+
+describe("frontend robots meta tag (issue #128 noindex)", () => {
+  const indexHtmlPath = path.resolve(
+    path.dirname(fileURLToPath(import.meta.url)),
+    "../../web/index.html"
+  );
+  const html = fs.readFileSync(indexHtmlPath, "utf-8");
+
+  const robotsMeta = html.match(
+    /<meta\s+name=["']robots["']\s+content=["']([^"']+)["']\s*\/?>/i
+  );
+
+  it("declares a robots meta tag", () => {
+    expect(robotsMeta).not.toBeNull();
+  });
+
+  it("marks the SPA shell noindex, nofollow (covers /bot/<id> dedicated links)", () => {
+    expect(robotsMeta?.[1]).toBe("noindex, nofollow");
+  });
+});

src/web/server.ts

@@ -77,12 +77,19 @@ export function createWebServer(options: WebServerOptions): WebServer {
     app.set("trust proxy", true);
   }
 
-  // Security headers: prevent the WebUI from being embedded in a third-party
+  // Security headers:
-  // iframe (clickjacking defence). CSP frame-ancestors is the modern equivalent
+  //  • X-Frame-Options / CSP frame-ancestors — prevent the WebUI from being
-  // of X-Frame-Options; both are set for compatibility across browsers.
+  //    embedded in a third-party iframe (clickjacking defence). CSP
+  //    frame-ancestors is the modern equivalent of X-Frame-Options; both are
+  //    set for compatibility across browsers.
+  //  • X-Robots-Tag — keep deployed instances out of search-engine indexes
+  //    (issue #128: searching "TsmusicBot" surfaced strangers' WebUI URLs).
+  //    Set on EVERY response so JSON/API responses and the SPA shell are all
+  //    covered; complements /robots.txt and the <meta name="robots"> tag.
   app.use((_req, res, next) => {
     res.setHeader("X-Frame-Options", "DENY");
     res.setHeader("Content-Security-Policy", "frame-ancestors 'none'");
+    res.setHeader("X-Robots-Tag", "noindex, nofollow");
     next();
   });
 
@@ -95,6 +102,13 @@ export function createWebServer(options: WebServerOptions): WebServer {
   const permissions = createPermissionStore(options.database.db);
 
   // ─── Public routes (no auth, no CSRF) ───────────────────────────────────
+  // Disallow every crawler (issue #128). Declared before the static SPA
+  // fallback so this wins over index.html for /robots.txt. Belt-and-braces
+  // with the X-Robots-Tag header above and the <meta name="robots"> tag.
+  app.get("/robots.txt", (_req, res) => {
+    res.type("text/plain").send("User-agent: *\nDisallow: /\n");
+  });
+
   app.get("/api/health", (_req, res) => {
     res.json({ status: "ok", version: "0.1.0" });
   });

web/index.html

@@ -3,6 +3,12 @@
 <head>
   <meta charset="UTF-8">
   <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <!-- Keep deployed instances out of search-engine indexes (issue #128:
+       searching "TsmusicBot" surfaced strangers' WebUI URLs). Defence in depth
+       alongside the server's X-Robots-Tag header and /robots.txt. Applies to
+       the SPA shell and every in-app route (incl. /bot/<id> dedicated links),
+       since they all share this single index.html. -->
+  <meta name="robots" content="noindex, nofollow">
   <!-- Bilibili / NetEase / QQ image CDNs reject requests whose Referer is not on
        their whitelist, so we must not leak a Referer cross-origin. "same-origin"
        does exactly that: full Referer for our own requests, none for cross-origin