Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models
1Music Technology Group, Universitat Pompeu Fabra; 2Deezer Research; 3Nantes Université, École Centrale Nantes, CNRS, LS2N
Layer-wise downstream performance of 17 music foundation models on 15 tasks. The label-free metrics of each layer and their correlation with performance are on the Metrics page.
Each cell reports the best layer of a model for a task, its score and the profile across layers. Select a cell to open the full curve, or a column header to sort. Headers marked ▸ expand to further metrics of the same datasets. Params counts the evaluated encoder (the audio tower of two-tower models, the decoder of autoregressive models).
For each model and task family, the three layers with the highest mean score after min-max normalization of each task's layer profile, with the best of the three, its relative depth and the mean profile. Hover over a cell for the best layer of each task.
| Model | Paradigm | Probed layers | Tonal | Rhythm | Timbre | Semantic | Similarity |
|---|---|---|---|---|---|---|---|
| MERT-v1-95M | Masked | 13 | L4, L10, L11best L11, 92% depth | L6–8best L7, 58% depth | L4, L6, L7best L4, 33% depth | L5–7best L6, 50% depth | L2–4best L2, 17% depth |
| MuQ (iter) | Masked | 13 | L1–3best L2, 17% depth | L10–12best L12, 100% depth | L2, L5, L6best L5, 42% depth | L5, L9, L10best L10, 83% depth | L0–2best L0, 0% depth |
| MERT-v1-330M | Masked | 25 | L2–4best L4, 17% depth | L7–9best L9, 38% depth | L7, L8, L12best L7, 29% depth | L6, L7, L10best L10, 42% depth | L5–7best L7, 29% depth |
| MusicFM (MSD) | Masked | 13 | L1, L3, L4best L3, 25% depth | L6, L8, L10best L10, 83% depth | L2, L3, L5best L3, 25% depth | L6–8best L7, 58% depth | L1–3best L2, 17% depth |
| OMAR-RQ (multifeature) | Masked | 24 | L2, L4, L5best L5, 22% depth | L14, L15, L18best L14, 61% depth | L9, L11, L15best L9, 39% depth | L14–16best L16, 70% depth | L7, L13, L14best L14, 61% depth |
| OMAR-RQ (base) | Masked | 24 | L1, L2, L4best L1, 4% depth | L14, L16, L18best L14, 61% depth | L4, L11, L14best L14, 61% depth | L7–9best L9, 39% depth | L1–3best L3, 13% depth |
| MERT-v2-30s | Masked | 25 | L2, L4, L5best L5, 21% depth | — | L9, L10, L13best L9, 38% depth | L20, L22, L23best L20, 83% depth | L7, L11, L19best L7, 29% depth |
| MERT-v2-FullSong | Masked | 25 | L1, L3, L4best L3, 12% depth | — | L12, L13, L21best L12, 50% depth | L22–24best L24, 100% depth | L6–8best L7, 29% depth |
| MusicGen-S | Autoregressive | 25 | L13, L18, L19best L19, 79% depth | L11, L12, L14best L11, 46% depth | — | L9, L14, L16best L9, 38% depth | L13, L15, L17best L17, 71% depth |
| YuE-s1-0.5B | Autoregressive | 25 | L22–24best L24, 100% depth | L11, L12, L15best L12, 50% depth | — | L11, L12, L14best L11, 46% depth | L6, L15, L16best L6, 25% depth |
| MusicGen-M | Autoregressive | 49 | L0, L1, L42best L0, 0% depth | L28, L32, L34best L32, 67% depth | L2, L17, L21best L17, 35% depth | L30, L32, L33best L33, 69% depth | L0–2best L1, 2% depth |
| MusicGen-L | Autoregressive | 49 | L41, L43, L44best L43, 90% depth | L33–35best L33, 69% depth | L29, L32, L40best L32, 67% depth | L33–35best L35, 73% depth | L0, L1, L33best L1, 2% depth |
| YuE-s1-7B | Autoregressive | 33 | L0, L2, L32best L32, 100% depth | L9, L10, L12best L9, 28% depth | — | L7, L12, L32best L7, 22% depth | L2, L3, L21best L2, 6% depth |
| MT2 | Contrastive | 13 | L4, L5, L8best L8, 67% depth | — | L8, L9, L11best L8, 67% depth | L9–11best L10, 83% depth | — |
| Myna-Base | Contrastive | 13 | L9–11best L11, 92% depth | — | L5, L8, L9best L9, 75% depth | L6, L9, L12best L12, 100% depth | L6, L7, L12best L12, 100% depth |
| LAION-CLAP | Contrastive | 22 | L6–8best L6, 29% depth | — | L17–19best L18, 86% depth | L18–20best L19, 90% depth | L9, L11, L12best L11, 52% depth |
| MAEST | Supervised | 12 | L1–3best L2, 18% depth | — | L4, L5, L9best L5, 45% depth | L4–6best L5, 45% depth | L4–6best L6, 55% depth |
| Column | Dataset | Metric |
|---|---|---|
| Key | GiantSteps | weighted score, mir_eval 0.8.2 |
| Pitch | NSynth | accuracy |
| Chords | Chords1217 | MIREX score, mir_eval 0.8.2 (root, thirds and triads under the expanded Tonal header) |
| Beat, Downbeat | GTZAN | F-measure, 70 ms tolerance |
| Instrument | NSynth | instrument family accuracy |
| Genre | GTZAN | accuracy |
| Structure | Harmonix | clip-level section accuracy |
| Emotion | Emomusic | R² (arousal and valence under the expanded Semantic header) |
| MTT | MagnaTagATune, 50 tags | macro-averaged AP (macro AUROC under the expanded Semantic header) |
| MTG genre, instrument, mood, top 50 | MTG-Jamendo, 87, 40, 56 and 50 tags | macro-averaged AP (macro AUROC under the expanded Semantic header) |
| Similarity | DimSim | triplet accuracy with centered cosine similarity (cosine and L2 under the expanded Similarity header) |
Every column, including the expanded ones, can be opened in Layer profiles.
Each layer of the frozen encoder is evaluated separately with a trained probe, an MLP with one hidden layer of 512 units or a linear probe (we will extend this to further probe architectures, such as attentive probes). Scores may differ from those reported in the original papers, which use other probes, checkpoints or splits. Per-layer results are available in the data folder.
@inproceedings{kanatas2026goodlayer,
title = {What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models},
author = {Kanatas, Angelos-Nikolaos and Kong, Yuexuan and Alonso-Jim{\'e}nez, Pablo and Serra, Xavier and Bogdanov, Dmitry},
booktitle = {Proceedings of the International Society for Music Information Retrieval Conference (ISMIR)},
year = {2026}
}
We continue to update these results with more models and downstream tasks.