FAKEFAKE BOOKBOOK

Liner notes

The problem

A jazz standard, stripped to its skeleton, is a chord chart: thirty-two bars of symbols like D-7 | G7 | C^7. Musicians read these from a fake book and improvise everything else. Charts are tiny, structured, rule-bound text — bars hold four beats, certain chords want to follow certain others, forms repeat in AABA.

That makes them a sharp test of fine-tuning. A pretrained language model has read the internet but has never studied harmony. Can a few thousand charts teach it the idiom? And because charts follow rules, every hyperparameter setting can be scored, not just eyeballed.

The data

The corpus is the iReal Pro community “Jazz 1460” playlist, the most widely shared collection of standards in the app's own chart format. Each chart is decoded from its irealb:// encoding, flattened (repeats, codas and endings expanded), filtered to 4/4, de-duplicated and repaired.

<style=Medium Up Swing> <key=F>
| F^7 Ab7 | Db^7 E7 | A^7 C7 | C-7 F7 |
| Bb^7 Db7 | Gb^7 A7 | D-7 G7 | G-7 C7 |

The symbols are iReal's own: ^7 major seventh, -7 minor seventh, h7 half-diminished, o7 diminished. Each chart is transposed into extra keys, which is musically exact augmentation, and the header lets you ask for a style and a key.

The method

The base model is GPT-2. It is fine-tuned with the assignment's reference recipe — learning rate 5e-4, five epochs, batch size 4, 128-token blocks — and then retrained from the pretrained weights with one knob changed at a time.

At the end of every epoch the model writes 48 charts and we measure chords that parse, bars that are well formed, ii–V–I cadences per eight bars, roots inside the requested key, and novelty: the share of eight-bar phrases that never appear in the training set. That last one catches memorisation even while the loss keeps falling.