feat(web): keep deployed WebUI out of search-engine indexes

Searching "TsmusicBot" surfaced many deployed instances' WebUI URLs,
letting strangers walk into other people's control pages (issue #128).
Add defence-in-depth so crawlers stop indexing public deployments:

- send `X-Robots-Tag: noindex, nofollow` on every Express response
- serve `/robots.txt` with `User-agent: * / Disallow: /`
- add `<meta name="robots" content="noindex, nofollow">` to index.html,
  which also covers the /bot/<id> dedicated-link pages (same SPA shell)

These layers only prevent indexing; real protection stays with WebUI
auth and the reverse proxy. Document this in the README security section
and warn users not to post their WebUI link on public pages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
saopig1andClaude Fable 5 committed 2026-07-16 23:31:31 +08:00
1 parent 051171b019
commit ea0f7c17b5
4 files changed
+110 -3

No files matched your search

+6
View File
@@ -3,6 +3,12 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<!-- Keep deployed instances out of search-engine indexes (issue #128:
searching "TsmusicBot" surfaced strangers' WebUI URLs). Defence in depth
alongside the server's X-Robots-Tag header and /robots.txt. Applies to
the SPA shell and every in-app route (incl. /bot/<id> dedicated links),
since they all share this single index.html. -->
<meta name="robots" content="noindex, nofollow">
<!-- Bilibili / NetEase / QQ image CDNs reject requests whose Referer is not on
their whitelist, so we must not leak a Referer cross-origin. "same-origin"
does exactly that: full Referer for our own requests, none for cross-origin