fix(local): remux aac into .m4a so the extracted audio is bit-exact (#149)

Extraction always used Matroska (.mka) because it takes essentially any
audio codec. That is right for most codecs but wrong for AAC: MP4 records
the AAC encoder priming (the ~1000 warm-up samples every AAC encoder emits)
in an edit list, and the edit list does not survive into Matroska. The
remuxed track then decodes ~23 ms longer than the source, with the priming
samples played at the head instead of discarded.

Measured on a 5s 640x480 fixture: source audio decodes to 962980 bytes of
PCM, the .mka to 967440 — 4460 bytes / ~23 ms extra, peaking at -66 dBFS.
Inaudible in practice, but it also puts the track fractionally out of step
with its own reported duration, for no reason.

Pick the container by codec instead: aac -> .m4a (keeps the edit list),
everything else -> .mka as before. If the preferred container refuses the
codec, retry into .mka before falling back to keeping the whole video. AAC
is worth the special case because mp4 / mov / m4v — what people actually
upload — almost always carry it.

Adds the strongest available test of the "lossless" claim: decode the audio
straight out of the source mp4, decode the stored extract, assert the PCM is
byte-for-byte equal. Forcing .mka fails it with exactly the 4460-byte delta.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
saopig1andClaude Opus 5 committed 2026-08-14 01:50:08 +08:00
1 parent 9fdc164f98
commit af1dac848d
6 files changed
+855 -10

No files matched your search

+33 -1
View File
@@ -392,7 +392,9 @@ describe("LocalMusicProvider video upload, end to end (#149)", () => {
const resolved = await p.getSongUrl(song.id);
expect(resolved).not.toBeNull();
// The video container is gone; what remains is the extracted audio track.
expect(resolved!.url.endsWith(".mka")).toBe(true);
// AAC (what libx264+aac mp4s carry) goes to .m4a so the encoder-priming
// edit list survives — see extractedAudioExt.
expect(resolved!.url.endsWith(".m4a")).toBe(true);
expect(existsSync(join(dir, `${song.id}.mp4`))).toBe(false);
expect(existsSync(resolved!.url)).toBe(true);
expect(statSync(resolved!.url).size).toBeGreaterThan(0);
@@ -418,6 +420,36 @@ describe("LocalMusicProvider video upload, end to end (#149)", () => {
expect(decoded.stdout.length).toBeGreaterThan(300000);
}, 60000);
it.runIf(have)("aac extraction decodes bit-for-bit identically to the audio inside the video", async () => {
// The strongest statement of "lossless": decode the audio track straight
// out of the source mp4, decode the stored extract, compare the PCM.
// A Matroska remux would NOT pass this — it loses the MP4 edit list that
// discards AAC encoder priming, so it decodes ~23 ms longer.
const p = new LocalMusicProvider(dir);
const bytes = render("bitexact.mp4", withAudio(4, "libx264", "aac"));
const sourceCopy = join(dir, "source-kept.mp4");
writeFileSync(sourceCopy, bytes);
const song = await p.uploadAudio({
buffer: bytes, originalName: "bitexact.mp4", mimeType: "video/mp4",
});
const url = (await p.getSongUrl(song.id))!.url;
const toPcm = (input: string, pre: string[] = []) => spawnSync(
ffmpeg!,
["-hide_banner", "-loglevel", "error", "-i", input, ...pre,
"-f", "s16le", "-ar", "48000", "-ac", "2", "-acodec", "pcm_s16le", "-"],
{ maxBuffer: 128 * 1024 * 1024 },
);
const fromVideo = toPcm(sourceCopy, ["-vn", "-map", "0:a:0"]);
const fromExtract = toPcm(url);
expect(fromVideo.status).toBe(0);
expect(fromExtract.status).toBe(0);
expect(fromExtract.stdout.length).toBe(fromVideo.stdout.length);
expect(fromExtract.stdout.equals(fromVideo.stdout)).toBe(true);
}, 90000);
it.runIf(have)("refuses a video that genuinely has no audio track", async () => {
const p = new LocalMusicProvider(dir);
const silent = render("silent.mp4", [