Earshot Gives Coding Agents an Open-Source Audio Feedback Loop
Earshot gives coding agents calibrated hearing and a code-based sound forge, running fully on CPU with a CLI, Python API, and MCP server.
- Earshot gives coding agents calibrated hearing: describe, match, find, critique sounds via CLI, Python API, and MCP.
- Runs CPU-only on two open models (CLAP plus AudioSet AST), roughly 1.4 GB, offline after first download.
- Forge command synthesizes sounds from physical-synthesis recipes and improves them via an evolutionary critic loop.
- Validation: 98% kind-of-sound accuracy on strong matches, 0.92 ROC-AUC real vs synthesized, 93% event detection in mixes.
- v0.3 adds 201 code-generated sounds, a kit builder, and frame-accurate SFX placement.
- MIT licensed; limits include no taste, licensing, speech, or above-14 kHz judgments.
Earshot gives coding agents a local audio feedback loop
Coding agents can write and render product films, game demos, and interface prototypes. Their toolchains rarely reveal whether a key press matches its brief, an impact lands on the intended frame, or a musical swell aligns with a transition. Earshot is an open-source toolkit that lets agents describe, match, locate, critique, and synthesize sounds from code on a CPU.
The v0.3 announcement adds 201 programmatically generated sounds, a kit builder, and tighter frame-accurate sound-effect placement. The MIT-licensed project ships as a command-line interface, Python API, and Model Context Protocol server, which exposes its tools to compatible agents such as Claude Code.
Waveforms become evidence
Earshot converts an audio file into structured observations that an agent can use in later steps. Its outputs cover descriptive tags, text-to-audio similarity, event timing, estimated source realism, audiovisual synchronization, and comparisons with reference films. Every command returns a plain-language assessment and a JSON payload suitable for scripts, tests, and agent tool calls.
Six ways to listen
| Command | Output | Typical use |
|---|---|---|
describe FILE |
Calibrated tags, AudioSet labels, raw measurements, alternatives, and z-scores | Summarize a sound and surface plausible competing interpretations. A z-score measures distance from the calibration baseline in standard deviations. |
match FILE QUERY |
A similarity score and a verdict of strong, plausible, weak, or no match | Check whether a sound fits a written brief. |
find FILE QUERY |
Ranked moments and sample-level trim points | Locate an event inside a longer recording. |
realness FILE |
Per-window estimates of recorded or code-synthesized origin | Flag conspicuously synthetic sections and identify uncertain regions. |
timeline and sheet |
A second-by-second transcript or annotated PNG containing frames, events, a spectrogram, and sync markers | Give text-oriented or vision-capable agents a compact view of a complete mix. |
critique FILE --video V |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
