Chordity

한국어 · English · 日本語 · 中文 · Français · Español

If you have collaborated on or contributed to this project, or would like to, please contact me personally. This project is source-available software that anyone can use free of charge. For personal and non-commercial purposes, you may use it as is at no cost. However, copyright and ownership belong to the original author, collaborators and contributors: without the original author’s permission you may not modify it or take any part of it and put it into your own program (even for your own use only), and you may not claim this program as your own or distribute or sell it elsewhere. Uploading modified versions to a GitHub fork is also not permitted. Use by organizations such as schools, private academies, clubs, churches, religious organizations and companies is not automatically free, so please contact me first. The full terms are in LICENSE.

Chordity v1.11.10

Chordity is a web page that writes down the chords bar by bar when you give it a song file, and plays a metronome locked exactly to the beat of the song.

At the moment a chord changes, a piano plays that chord, so you can check it by ear and play along. No installation, no sign-up and no internet connection are needed. Your music files never leave your computer.

The results can be wrong. The analysis principles, the reasoning behind its decisions, and how to check and correct the results are collected on the help page (in Korean). You can also open it by clicking the notice at the very bottom of the app.

Chordity: per-measure chord recognition, a beat-locked metronome and synthesized piano chord playback in a single offline HTML file. No libraries, no uploads.

The screen during playback: a yellow highlighter box surrounds the chord now sounding and fills with orange as far as it has played, the chord name is enlarged, and the slash of the current beat is drawn darker

Contents

Part 1. The easy overview

When it comes in handy

How to use it

  1. Open the file chordity.html in a browser such as Chrome, Edge, Safari or Firefox.
  2. Click Open file (“파일 열기”) or drag a music file onto the window. After a few seconds the chord chart appears.
  3. Press the Play (“재생”) button. Along with the music, the metronome sounds a “tock” on every beat (a high “tick” on the first beat of each bar, and when the chord change unit is ½ beat, a small “tik” between beats as well), a yellow highlighter box surrounds the chord now sounding, and that chord’s name gets bigger. Inside the box, orange fills up like a progress bar as far as it has played, and when the chord changes the box springs over to the next chord. The piano plays the chord wherever the original song strikes it (it strikes again when the chord changes or the same chord is struck anew, and keeps ringing where the chord is held over).

Any file the browser can play will do, such as MP3, WAV, M4A, FLAC or OGG.

How to read the screen

What you see Meaning
Yellow highlighter box The chord now sounding. It covers from the beat where the chord was struck up to where the next chord comes in (if a grey held-over chord continues into the next bar, up to there; if the line wraps, one piece per line). When the chord changes, it springs over to the next chord
Orange fill inside the box How much of that chord has been played so far (a progress bar). The end of the fill is the current playback position, and it reaches each beat’s slash at the moment that beat begins
Enlarged chord name The name of the chord now sounding (a newly struck chord). It pops up large as it comes in and slowly shrinks back to normal size once playback moves on to the next chord. It grows only as much as it can without touching the next name
Slash / One beat. The slash that turns darker is the current beat
Thick slash A beat where a chord is newly struck (a beat carrying a chord in black text). It shows the rhythm in which chords are struck. Grey chords and N.C. spots normally get an ordinary slash
C Am G7 Chord names. A name sitting right above a slash means the chord changes on that beat
G/B A G chord whose lowest note is B
Small text, chord names placed on two rows (upper and lower) When a bar has so many chords that a name would touch the next one, the text is made smaller; if it still does not fit, the names in that bar alternate between an upper and a lower row (the left edge of a name is the beat where that chord is struck). On a narrow screen, if a long name on the last beat of a bar does not fit even on two rows, the less important parts are dropped and … is added (Bm7/F♯ → Bm7…). The full name appears when you hover the mouse over it, or in the current-chord box at the top during playback
Grey chord The chord from the previous bar held over without being struck again. Even the same chord is shown in black text if it is struck anew on the first beat of the bar
N.C. A part with no chord (only drums, or silence)
Bar 0 The part where the song starts in the middle of a bar (a pickup, or anacrusis)
A label such as 2박 above a bar Only that bar has a different number of beats (2박 means “2 beats”)
Black label E 장조로 The key changes from that beat on (modulation); E 장조로 reads “to E major”, with the Korean particle 로/으로 (“to”). The key box at the top also shows it, like E♭ 장조 → E 장조. On the first bar of the song, the starting key and meter are attached without the particle (C 장조 4/4, like the key signature and time signature in sheet music)
Black label 3/4으로 The meter changes from that bar on (meter change); 3/4으로 reads “to 3/4”. The meter box at the top also shows it, like 4/4 → 3/4 → 4/4
White pill 72 132로 The tempo from that beat on (BPM, rounded), on the line below the slashes. On the beat where the song starts, the starting tempo is shown as a bare number; where the tempo changes suddenly, the new tempo gets the particle 로/으로 (“to”)
White hexagonal frame 여기까지 63 A section where the tempo changes gradually (on the line below the slashes); 여기까지 63 means “up to here: 63”. The length the frame spans is the changing section, and when the playback position is inside the frame, it fills with the highlighter color as far as playback has gone and shows the current tempo, like 현재 67 (“now 67”). When it is completely filled it fades to white, and the white pill on the beat after the end of the frame (the tempo arrived at) lights up briefly
Tempo (top box and playback bar) The tempo at the playback position. When the tempo changes gradually, it follows along, like 120, 119, 118 …. The playback bar stays fixed at the very top of the screen even when you scroll, so it is always visible
Names like 8B or 9A with small letters attached Chord names when Camelot notation is selected (see “Viewing in Camelot” below)
Roman numerals such as I, vi, V7/ii Chord names when Roman numeral notation is selected (see “Viewing in Roman numerals” below)
Circle indicators Which beat it is now. The largest circle is the first beat of the bar
“Next G” (“다음 G”) at the top The chord coming up soon
Metronome (“메트로놈”) and chord playback (“코드 연주”) buttons Toggle on and off each time you press them
Music (“음악”), click (“클릭”) and chord (“코드”) sliders Adjust the volume of the original song, the metronome and the piano chords separately

If the metronome sounds off

Most of the time it lines up on its own, but for some songs different people count the beat differently. Open Beat not lining up? Correct it (“박이 안 맞나요? 보정하기”) and try the buttons below. Each takes effect the moment you press it.

If it sounds like this Press this button
The clicks fall between the beats Shift ½ beat (“½박 옮기기”)
The clicks are too dense Halve ÷2 (“절반으로 ÷2”)
The clicks are too sparse Double speed ×2 (“두 배 빠르게 ×2”)
The song feels like 6/8 but there are too many clicks Group in threes ÷3 (“셋씩 묶기 ÷3”)
The start of the bar is off by one beat Move one beat earlier (“한 박 앞당기기”), move one beat later (“한 박 늦추기”)
The clicks feel very slightly early or late Click timing (“클릭 타이밍”)
You want to reset the beat corrections Reset beat corrections (“박 보정 되돌리기”)
You want to reset settings such as chord type Restore default settings (“설정 기본값으로”)

What you have changed so far is shown under Active (“적용 중”) in the same panel, split into “beat corrections” (“박 보정”) and “settings” (“설정”). Even when the panel is collapsed, a summary is visible next to its title.

If you want simpler chords, choose “basic” (“기본”) under Chord type (“코드 종류”); if you also want to see chords that change within a beat, choose “half-beat” (“½박 단위”). In half-beat mode the metronome also adds a small, high “tik” between beats (on the eighth notes), playing like “tick-tik-tock-tik-tock-tik-tock-tik”.

Viewing chord names differently

With the Chord notation (“코드 표기”) bar right above the chart, you can view the same chords in three ways. The chart, the current and next chord, and the text chart all change the moment you press it, and the chosen style is kept the next time you open the app. From the keyboard, focus the bar and switch with ← and →.

Button Example Good for
Note names (“음이름”) E♭, B♭/D, Cm7 Playing along right away while looking at the chords, or passing chords between players
Camelot (“카멜롯”) 5B, 6B, 3A⁷ Finding keys and chords that go well together, or choosing DJ mix points
Roman numerals (“로마 숫자”) I, V⁶, ii⁷/IV Analyzing a song’s harmonic progression, or transposing it to another key

The three images below show the same 11 bars in E♭ major in each style.

Viewing in note names

Note name notation: E♭, B♭/D, Cm, B♭m7, E♭7, A♭maj7 …

These are the most familiar chord names. The root is written large, qualities such as m, sus and dim a little smaller, and extensions such as 7 and maj7 as superscripts, so even complex chords can be read at a glance.

Viewing in Camelot

Camelot notation: 5B, 6B, 5A, 3A⁷, 5B⁷, 4Bmaj7 …

Chords are written with the numbers of the Camelot wheel, which DJs use when blending one track into the next. Keys are placed on positions 1 to 12 in circle-of-fifths order, with B added for the major family and A for the minor family. Keys and chords with the same number or numbers one apart sound good together.

Viewing in Roman numerals

Roman numeral notation: I, V⁶, vi, ii⁷/IV, V⁷/IV, IVᴹ⁷ …

This is the Roman numeral harmonic analysis used in harmony textbooks. Instead of note names, chords are written as the chord built on which degree of the key, which reveals the skeleton the song moves along. The same progression looks the same in any key, so it is handy for transposing to another key or comparing progressions across songs.

Notation Meaning
I, IV, V (uppercase) Chords of the major-triad family. The numeral is the degree of the key the chord is built on
i, ii, vi (lowercase) Chords of the minor-triad family
°, ø, +, M Diminished (°), half-diminished seventh (ø), augmented triad (+), major seventh (M7)
7 Seventh chord
6, ⁶₄ Inversions of a triad. 6 if the bass is the 3rd, ⁶₄ if it is the 5th
⁶₅, ⁴₃, ⁴₂ Inversions of a seventh chord, when the bass is the 3rd, 5th or 7th
V/x, vii°/x, ii/x A chord used briefly to lead to x (secondary chord). V7/vi is the dominant seventh leading to vi
♭VI, ♭VII, etc. Degrees outside the scale of the key, such as chords borrowed from the parallel minor (borrowed chords)

Hearing chords on the piano

With Chord playback (“코드 연주”) turned on, the piano plays each chord at the moment it changes. Listening to it layered over the original lets you check by ear right away whether the written chord is right, and it also guides you when playing along. The piano automatically matches the song’s tuning, so it never sounds out of tune against the original.

Keyboard shortcuts

Key What it does
Space Play, pause
M Metronome on, off
P Chord playback on, off
← → Previous bar, next bar
Home Go to the beginning

Clicking a bar jumps to that bar, and clicking or dragging on the waveform at the top also takes you wherever you want. The shortcuts work even when the keyboard is in Korean input mode.

Songs that work well, songs that are hard

Songs with a clear beat, such as pop, rock, ballads, dance, hip-hop, K-pop and acoustic pop with drums, work well. It also recognizes songs whose tempo changes greatly partway through, odd meters such as 5/4 and 7/4, chords whose bass is a note outside the chord such as F/G, and rock power chords (E5). It can be wrong on songs with many tensions and a walking bass such as jazz, songs felt in two-beat units such as bossa nova (bars are written twice as long), metal with very heavy distortion, and songs whose beat can be counted in two ways such as reggae. In those cases, use the correction buttons above.

Good to know

Phone screen (dark theme)

↑ Back to contents

Part 2. Principles

This program does not use a pretrained AI model. It works with signal processing and probabilistic models that turn the physical properties of sound and the rules of music into formulas. That is why every decision can be explained, and you can see the evidence yourself under Analysis info (“분석 정보”) at the bottom of the screen.

2-1. Turning sound into a picture

Sound is a vibration of the air, and a recording is a sequence of numbers measuring that vibration tens of thousands of times per second. Since it is hard to tell which notes are sounding just from those numbers, the sound is cut into very short pieces, and for each piece we compute how much sound of each pitch is mixed in (the spectrum). Laying these out in time order gives a picture of the sound with time on the horizontal axis and pitch on the vertical axis (a spectrogram). All of the analysis that follows is the work of reading this picture.

There is an unavoidable trade-off here. Cutting short pieces tells you precisely when a sound occurred but blurs which note it is; cutting long pieces does the opposite. So short pieces are used for finding the beat, and long pieces separately for finding chords. The lowest bass notes need the longest pieces, because the frequency difference between semitones there is very small.

2-2. Finding the beat: the pattern of moments when new sounds start

When a drum is hit or a key is pressed, the sound energy suddenly increases at that moment. Such a moment is called an onset. If you add up only the parts of the spectrogram that got louder than just before, you get a curve that spikes at every onset.

The beat is the skeleton of these onsets repeating at regular intervals. First, the curve is slid along in time and overlapped with itself (autocorrelation) to find the interval at which it overlaps best, that is, the tempo. Then a row of beats that “passes through as many strong onsets as possible while keeping the spacing close to the tempo” is found across the whole song at once (dynamic programming). Because the whole thing is optimized together rather than deciding each beat separately, the row does not break even if a few weak beats are mixed in, and it follows the tempo even when the performance wavers a little. Here the standard for “regular intervals” is not the tempo of the whole song but a local tempo computed every few seconds. So it is not dragged along by places where the rhythm briefly gets complicated, like fills, yet it follows when the song actually slows down or speeds up.

2-3. Checking that it is the real beat

Finding a regular row is not the end. There are two common mistakes.

Taking the offbeat as the beat. When the hi-hat strongly fills in between beats, the row can be caught shifted by half a beat. But music has rules. The moments when the kick, snare and bass come in and the moments when chords change mostly cluster on the beat. So the strength of such events is compared at the found beat positions and at positions shifted by half a beat, and if the shifted side is clearly stronger, the whole row is moved by half a beat.

Taking the tempo as double or half. When a slow ballad has dense eighth-note arpeggios, it is easy to mistake each eighth note for a beat. In that case the beats are split into two groups by taking every other beat. If the kick and snare land on the real beats with only keyboard sounds in between, one group comes out much stronger than the other. Then the weak group is discarded and the tempo is corrected to half. Conversely, if a snare comes regularly right in the middle between beats, the tempo was caught too slow, so it is corrected to double. Doubling, if wrong, immediately produces offbeat clicks, so it is done only under much stricter conditions.

In real recordings, hi-hats and keyboards fill the offbeats, so the difference in strength between the two groups can come out small. So it also looks at which group the kick and snare fall in. At the correct tempo, the kick (beats 1 and 3) and the snare (beats 2 and 4) fall in different groups; if the tempo was caught double, both fall in the same group (the real beats). When correcting the tempo to half, it does not drop every other beat but tracks again from the start at the new tempo. That is because if the eighth-note-level tracking slips by just one slot somewhere in the middle of the song, every beat dropped after that point would be off by half a beat.

Two shapes of a fast song caught at half tempo. A fast song of around 160–180 BPM is easily caught one level slower, that is, at half tempo. The shape then differs depending on where the slow beat sits. If the slow beat sits on the snare (beats 2 and 4), a snare comes evenly on every beat and the kick comes between beats. In a slow song at its correct tempo, the kick comes on the beat (beats 1 and 3) and the snare every other beat, so this shape does not occur. If the slow beat sits on the kick (a half-time feel), snares come between beats as often as on them. In both cases the tempo is doubled only when tempo estimation gives the double tempo as a candidate of almost equal strength. If the slow beat is left sitting on the snare, the first beat of the bar is also off, so two chords get mixed into one beat and many seventh chords and slash chords appear.

The halving side is also checked once more. If eighth notes had been taken as beats, the middle between beats would be a sixteenth-note position, where low-pitched hits such as kick or bass are rare. So if low sounds often come between beats too, the beat is taken to be correct and is not halved. When trying to halve based only on the strength difference between the even and odd groups, if the snares cluster in the group opposite the kick (a backbeat), the beat is also correct and is not halved.

The way the “snare” is recognized is physical. When the wires under a snare drum rattle, they make a noise that grows louder all at once from the middle range up to very high frequencies. A keyboard note grows louder only at a few pitches, and a hi-hat only at the high end. So by measuring “did the middle range and the high range get louder at the same time” as the product of the two values, only the snare can be picked out. If either one is small, the product becomes small.

2-4. Dividing into bars

To group beats into bars, you need to know where the first beat of the bar (the downbeat) is. The downbeat leaves traces: the chord changes, the bass note changes, the kick hits hard, and the sound sometimes gets louder. Each beat gets a score for such traces, and the grouping with the highest score is found while keeping the rule that “bars usually repeat at the same length”. Whether 4 beats or 3 beats fits the traces better is also decided this way. If the feeling of each beat splitting into three is strong, it is treated as a compound meter such as 6/8 or 12/8.

In songs where the drums play a backbeat, the even/odd shape of kick on beats 1 and 3, snare on beats 2 and 4 is also used as a trace of the downbeat. In syncopated riffs where the chord changes an eighth note before the downbeat, the trace of the chord change is pulled one beat earlier, but the even/odd shape of the drums stays the same, so the bar is not shifted by a beat. Also, 3, 5 and 7 beats per bar are chosen only when the downbeat traces actually repeat at that length. This is to prevent a short song from looking like 7 beats just because a few rare, large traces were picked.

2-5. Working out the chords

The same note in different octaves (a low C and a high C) plays the same role in a chord. So if every note is folded into the 12 slots C, C♯, D … B, you get a fingerprint of the notes sounding at that moment (chroma). For a C chord, the C, E and G slots stand out.

Real instruments do not produce just one tone. When you play C, harmonics such as the C an octave above, the G above that, C again, E … ring along with it. For example, a harmonic from the E of a C chord fills the B slot, so a comparison that ignores harmonics mistakes C for Cmaj7. That is why the chord fingerprints are also built to include harmonics before comparing.

The bass (the lowest note) is looked at separately. If the chord is C but the bass is E, it is written C/E.

Every recording has a slightly different reference pitch. Many songs do not have A at exactly 440 Hz. So first it measures how high or low the notes of the whole song are on average, corrects by that amount, and then puts them into the 12 slots.

2-6. The rules of music as probabilities

If you simply picked the most similar chord on every beat, the passing notes of the melody would make the chord flicker from beat to beat. So two pieces of common sense are added as probabilities: chords do not change often, and when they change, it is most likely on the downbeat and unlikely between beats. Weighing these rules together with how similar the sound is, it picks the most plausible chord sequence for the whole song at once (a hidden Markov model and the Viterbi algorithm). It also estimates the song’s key and gives chords that fit the key a small bonus.

2-7. A metronome that does not drift

This metronome does not tick at a fixed interval like “97 ticks per minute”; it ticks at the time of each beat it found. So it follows along even when the performance speeds up or slows down a little.

The timing is precise too. If a timer fires a click by saying “play now!”, it gets delayed by tens of ms whenever the computer is busy. So clicks are scheduled 0.2 seconds ahead on the clock of the audio device that plays the music. Because the music and the clicks use the same clock and the same output path, they mesh to the individual sample, and even on a device with delayed output, both come out late together and stay aligned with each other.

2-8. Playing the chords on the piano

Including recorded piano sounds would make the file large and require external files. So the piano sound is built directly from the laws of physics. A piano string is not an ideal string but a stiff steel wire, so its harmonics ring slightly higher than exact integer multiples (inharmonicity). The hammer strikes at about 1/7 of the string’s length, so harmonics that do not move at that point ring weakly. Two or three strings sound one note together, and because they differ very slightly in pitch, the sound undulates slowly (beating), and a two-stage decay arises that falls quickly at first and then lingers. Higher harmonics die away faster, so the sound gradually becomes softer. These properties are layered as formulas to synthesize each note.

A chord is played split into one left-hand bass note and a right-hand chord. For the right hand, it chooses the inversion that moves the hand the least from the previous chord, so the sound does not jump around when the chord changes. The piano is tuned to the song’s reference pitch found in the analysis and is scheduled on the same audio clock as the metronome, so it sounds exactly at the moment the chord changes.

2-9. Finding where the key changes

A song’s key shows in “which notes are mainly used”. A song in E♭ major mainly uses E♭, F, G, A♭, B♭, C and D, and if it moves to E major it uses E, F♯, G♯, A, B, C♯ and D♯. So for each beat, the note distribution of the few bars before and after is compared with the typical distributions of the 24 keys, and the rule that keys do not change often is added as a probability to find the most plausible flow of keys across the whole song. Chords from another key that pass by for a bar or two (e.g. E7 → Am in a song in C major) are not counted as modulation. Moves between relative major and minor keys that use the same notes (C major and A minor) are not counted as modulation either, because the key signature does not change.

The modulation position is refined to the beat. When the music passes through the dominant of the new key (e.g. B7 for E major), the position where the new tonic chord appears is taken as the modulation point, following the convention of changing the key signature in sheet music. The ♯/♭ spelling of chord names also follows the key at that point. In an E major section, it writes G♯m7, not A♭m7.

↑ Back to contents

Part 3. How it works

3-1. Overall flow

flowchart TD
  F["Audio file"] --> D["Browser decoding<br/>PCM, device sample rate"]
  D --> PB["Playback buffer"]
  D --> MO["Mono mixdown"]
  MO --> R22["Zero-delay resampling<br/>22,050 Hz"]
  R22 --> R11["Zero-delay resampling<br/>11,025 Hz"]
  R22 --> RF["Rhythm features<br/>per-band spectral flux, loudness"]
  R11 --> HF["Harmony features<br/>peak STFT, tuning, chroma"]
  RF --> TP["Tempo estimation<br/>autocorrelation comb score × preference prior"]
  TP --> DP["DP beat tracking<br/>refinement, cleanup, extension"]
  DP --> PC["On-beat phase check"]
  HF --> PC
  PC --> LC["Beat-level check<br/>double, half, 4:3, 3:2"]
  LC --> A[("Analysis result A")]
  HF --> A
  A --> BT["buildTimeline<br/>correction ops, meter and downbeat HMM, chord HMM"]
  U["User corrections, settings"] --> BT
  BT --> TL[("Timeline TL")]
  TL --> CH["Chord chart view"]
  TL --> SC["Metronome scheduler"]
  TL --> PS["Chord playback<br/>voicing, piano synthesis"]
  PB --> OUT["Limiter, AudioContext output"]
  SC --> OUT
  PS --> OUT

The analysis is split into a heavy stage and a lightweight stage.

3-2. Loading and preprocessing

  1. The browser’s built-in decoder decodes the file to PCM. This one buffer is used for both playback and analysis. Decoding separately or going through the browser’s resampler could make the time axes of the two results drift apart by several ms.
  2. The channels are averaged into mono, and a self-implemented resampler produces 22,050 Hz (for rhythm) and 11,025 Hz (for harmony) signals. The filter kernel is symmetric, so the signal is not shifted by even one sample.
  3. The peak amplitude is normalized to 1.

3-3. Rhythm features

Step What it does Key values
Short-time Fourier transform Turns each piece of the 22,050 Hz signal into a spectrum Window 1024 points (46.4 ms), hop 256 points (11.6 ms, 86.13 frames per second)
Log compression Makes changes in quiet sounds visible too log(1 + 1000·|X|)
Maximum filter Widens the spectrum from two frames earlier by ±1 bin along frequency before comparing Keeps vibrato and pitch glides from being mistaken for onsets
Per-band flux Averages only the increases, per band Low 21–200 Hz (kick, bass), mid 200–2,500 Hz (snare body, keyboards), high 2.5–10 kHz (hi-hat, snare wires)
Onset envelope Divides each band by its standard deviation and sums High band weighted by half (reduces the influence of offbeat hi-hats)
Loudness RMS per frame Used to decide sounding sections and to draw the waveform

3-4. Harmony features

Step What it does Key values
Treble peak STFT Spectral peaks for chord tones Window 2048 points (186 ms), 110–4,200 Hz, up to 36 per frame
Bass peak STFT For bass notes and tuning Window 4096 points (372 ms), 32–1,400 Hz, up to 24 per frame
Peak conditions Only distinct peaks At least −45 dB relative to the frame maximum, at least 2× the average of the surrounding ±6 bins
Log-parabolic interpolation Frequency and magnitude more precise than the bin spacing Uses the log magnitudes of the peak and its neighboring bins
Reference pitch estimation Averages the “fractional semitone part” of all peaks on a circle e.g. +18 cent → A4 = 444.6 Hz
Chroma Accumulates the corrected pitches into 12 slots Larger weight the closer to the semitone center (cos²). Treble MIDI 48–96, bass 28–57
Harmonic change curve Cosine distance between chroma of the 4 frames before and after Used in the on-beat phase check

3-5. Tempo and beats

  1. Sounding sections: sections at least −38 dB relative to the song’s loud level (top 5%) are treated as sound.
  2. Tempo: the onset curve is normalized by dividing by the ±3-second average loudness, and its autocorrelation is computed. Scanning 40–240 BPM in 0.25 BPM steps, a comb score that looks at 1, 2, 3 and 4 times the interval together is multiplied by a preference prior centered at 110 BPM, and the area around the maximum is interpolated with a parabola.
  3. DP beat tracking: the method of Ellis (2007) is used with two changes. The spacing is compared with a local tempo curve computed with 8-second windows (a 7-second median to filter out fills), and the penalty strength is made proportional to the square of the beat period, so that the same jitter in ms gets the same penalty even in slow songs.
  4. Refinement: each beat is moved to the onset peak within ±2 frames (±23 ms) and interpolated with a parabola. Beats with weak onsets are filled in by linear regression on neighboring beats. Finally, a detection delay of 6.5 ms is compensated.
  5. Cleanup: beats outside sounding sections are removed, and the same spacing is extended forward and backward to cover all sounding sections.
  6. Checks: the on-beat phase check and the beat-level check are done in turn (the principle in 2-3). If the beat level is corrected to half, steps 3–5 are redone at the new tempo and the phase is checked again. If beats appear to split into three, a tempo 4/3 times slower and a tempo 3/2 times faster are also checked (1.11.5, 1.11.6, (9) in 4-1).

3-6. Meter, bars, chords (the lightweight stage)

  1. Apply correction operations: the ½ beat, ×2, ÷2 and ÷3 the user pressed are applied to the beat list in the order they were pressed. For ÷2 and ÷3, which beats to keep is decided by beat accent.
  2. Per-beat pooling: the chroma within each beat interval is averaged with more weight in the middle. From the bass chroma, half of the whole-song median is subtracted to remove constantly present components such as a pitched kick drum.
  3. Key and modulation: for each beat, the note distribution of ±6 beats around it is compared with the Krumhansl–Kessler key profiles, and the key sections are divided by an HMM whose states are the 12 key signatures. Sections shorter than 24 beats are absorbed into a neighboring section. If there is no modulation, a single key chosen from the whole-song distribution is used. After the chords are recognized, the key of each section is decided again using the chords. It looks at which key’s scale the chords fit, the time the tonic chord occupies, V → I cadences, the first and last chords, and the chords on the first beats of 4-bar phrases, together with the note distribution.
  4. Preliminary chords: a quick pass using only major and minor triads is solved once and used as downbeat evidence.
  5. Subdivision decision: the onset strengths at the 1/2 point and at the 1/3 and 2/3 points within a beat are compared to see whether it is a compound meter.
  6. Downbeat HMM: the meter is chosen automatically (4 beats vs 3 beats, or 4 vs 2 for compound meters), and the user’s downbeat shifts are applied.
  7. Final chord HMM: solved by Viterbi with up to 289 states including inversions, in units of 1 beat or ½ beat.
  8. Building bars: bars are split at each downbeat, and each gets the positions and lengths of chord changes within the bar, whether a chord continues from the previous bar, the note names (♯ or ♭) that fit the key at that point, the Camelot code and the Roman numeral. The screen shows the one chosen in the chord notation bar above the chart, and changing the notation does not re-run the analysis. Modulation boundaries are refined to the beat with a slight preference for the downbeat (to the tonic chord’s position when passing through the dominant), and the bar and beat number are recorded.

3-7. Playback and metronome

3-8. What the correction buttons do

Button Internal operation Behavior
Shift ½ beat (“½박 옮기기”) half Builds a new beat list from the midpoints of neighboring beats and snaps them back to onset peaks. Beats that stick out before the music starts are discarded
Double speed ×2 (“두 배 빠르게 ×2”) x2 Adds a midpoint between every pair of beats
Halve ÷2 (“절반으로 ÷2”), group in threes ÷3 (“셋씩 묶기 ÷3”) d2, d3 Keeps one of every 2 (3). The position kept is the one with the larger sum of beat accents
Move one beat earlier (“한 박 앞당기기”), later (“늦추기”) downShift ∓1 Rotates the position-in-bar numbers by one
Meter (“박자”) meter Re-solves the downbeat HMM with the specified number of beats

Corrections are stored as an operation log on the beat list and are reapplied in order from the start each time. “Reset beat corrections” (“박 보정 되돌리기”) clears this log. Chord type, chord playback mode, downbeat accent, auto-scroll and click timing are settings saved in the browser rather than with the song, so they are reset separately with “Restore default settings” (“설정 기본값으로”). “Active” (“적용 중”) lists, for both kinds, only what differs from the defaults.

3-9. Chord playback

Step What it does
Chord segments For each chord event in a bar, the engine passes on the chord tones (0 = C) together with the bass note to play. When slash notation is turned off, the bass is the root.
Strike list Built according to when chords are struck. “Follow the song” (“곡 따라”) strikes where the chord changes and, even for the same chord, on downbeats where the original strikes anew (black text in the chart), and keeps ringing where the chord is held over (grey). “When the chord changes” strikes a same chord continuing across bars only once, “again every bar” strikes again on each downbeat, and “every beat” strikes on all beats. On N.C. nothing is struck and the previous chord also stops.
Voicing The bass is in D2–C♯3; for the right hand all chord tones are placed in close position, choosing the inversion closest to the center of the previous voicing.
Synthesis and cache The notes used in the song are synthesized one at a time in advance, in the order they first appear (once per note). If the song’s reference pitch changes, they are rebuilt.
Scheduling The same scheduler as the metronome schedules strikes 0.2 seconds ahead on the audio clock. The click timing correction is not applied; chords are played at the music’s time as is.
Release At the time a chord segment ends (the next strike or N.C.), the sound decays with a 70 ms time constant, as if the dampers came down.
Restart If playback starts or jumps into the middle of a chord, the chord in progress is sounded once immediately.
Mixing Music, click and chord each have their own volume; the piano gets a little 1.6-second room reverb mixed in, then a fixed make-up gain (+6.4 dB). The three sounds pass through a limiter (−1 dBFS) just before the output and are combined without clipping.

The metronome and chord playback are also resynchronized separately. Turning the metronome off and on or changing the click timing does not cut off a piano chord that is ringing.

↑ Back to contents

Part 4. A closer look

4-1. Algorithms and formulas

Symbols

Symbol Meaning
$F = 22050/256 \approx 86.13$ Rhythm frame rate (frames/second)
$X(t,k)$ STFT value at frame $t$, frequency bin $k$
$\tau$ Beat period (in frames)
$b_i$ Time of the $i$-th beat (seconds)
$m$ MIDI note number (real-valued); distinguished by context from the number of beats per bar

(1) Zero-delay resampling

Reducing the output/input sample rate ratio by the greatest common divisor to $L/D = f_\text{out}/f_\text{in}$, the input position of the $n$-th output sample is $nD/L = i_n + \phi_n$ ( $i_n$ the integer part, $\phi_n$ the fractional part).

y[n] = \sum_{j=0}^{2W-1} x[\,i_n - W + 1 + j\,]\; h(\phi_n + W - 1 - j)

h(u) = 2f_c\,\mathrm{sinc}(2f_c u)\,w\!\left(\frac{u}{W+1}\right),\qquad
w(r) = 0.42 + 0.5\cos\,\pi r + 0.08\cos\, 2\pi r

f_c = 0.46\,\min\!\left(1, \frac{f_\text{out}}{f_\text{in}}\right),\qquad W = \left\lceil \frac{10}{2f_c} \right\rceil

Since $h$ is an even function (symmetric), its phase is 0, that is, the delay is 0. The phase $\phi_n$ is precomputed in a table of up to 4,096 steps, and the input position is updated with an integer accumulator ( $D = qL + r$ ) so floating-point error does not accumulate.

(2) Real FFT

An $N$-point real signal is computed with a single $N/2$-point complex FFT. With $z[m] = x[2m] + i\,x[2m+1]$ and $Z = \mathrm{FFT}_{N/2}(z)$,

X[k] = \underbrace{\tfrac12\left(Z[k] + \overline{Z[N/2-k]}\right)}_{F_e[k]}
     + e^{-2\pi i k/N}\,\underbrace{\tfrac{-i}{2}\left(Z[k] - \overline{Z[N/2-k]}\right)}_{F_o[k]},\qquad 0 \le k \le N/2

Indices of $Z$ are taken with period $N/2$ ( $Z[N/2] = Z[0]$ ).

(3) Per-band spectral flux and the onset envelope

Y(t,k) = \ln\!\left(1 + \lambda \cdot \tfrac{4}{N}\,|X(t,k)|\right),\qquad \lambda = 1000

\Delta(t,k) = \max\!\Big(0,\; Y(t,k) - \max_{|\nu|\le 1} Y(t-2,\,k+\nu)\Big)

f_b(t-1) = \frac{1}{|B_b|}\sum_{k \in B_b} \Delta(t,k),\qquad b \in \{\text{low}, \text{mid}, \text{high}\}

The value is recorded at $t-1$, the midpoint of the two frames being compared ( $t$, $t-2$ ). The onset envelope for beat tracking is as follows.

O(t) = \frac{f_\text{low}(t)}{\sigma_\text{low}} + \frac{f_\text{mid}(t)}{\sigma_\text{mid}} + \frac12\,\frac{f_\text{high}(t)}{\sigma_\text{high}}

(4) Tempo estimation

It is normalized by the loudness of a ±3-second window and the mean is subtracted.

E(t) = \frac{O(t)}{\sqrt{\frac{1}{|W_t|}\sum_{s \in W_t} O(s)^2} + \epsilon} - \bar E,\qquad W_t = [\,t - 3F,\ t + 3F\,]

The autocorrelation is divided by the overlap length, normalized by $r(0)$, smoothed with $[\tfrac14, \tfrac12, \tfrac14]$, and written as $\tilde r$.

r(\ell) = \frac{1}{n-\ell}\sum_{t=0}^{n-\ell-1} E(t)\,E(t+\ell)

For each BPM candidate $v$, lags of 1 to 4 times the beat period $\tau_v = 60F/v$ are considered together.

J(v) = \pi(v)\cdot\max\!\Big(0,\ \sum_{k=1}^{4} c_k \max_{|\delta| \le \delta_k} \tilde r(k\tau_v + \delta)\Big),\qquad c = \left(1, \tfrac12, 0.33, \tfrac14\right)

\pi(v) = \exp\!\left(-\frac12 \log_2^2\, \frac{v}{110}\right),\qquad
\delta_1 = 0.5,\quad \delta_{k\ge 2} = 0.5 + 0.004\,k\tau_v

$v \in [40, 240]$ is scanned in steps of 0.25, and a fractional BPM is obtained from the maximum by three-point parabolic interpolation.

(5) DP beat tracking (Ellis, 2007)

The local score, which smooths the onsets with a Gaussian of width $\sigma = \tau/32$, and the cumulative score are as follows.

\ell(t) = \frac{1}{\sigma_O}\sum_{i} O(t+i)\,e^{-\frac12\left(\frac{32 i}{\tau}\right)^2}

C(t) = \ell(t) + \max_{\tau(t)/2 \,\le\, d \,\le\, 2\tau(t)}\Big[\,C(t-d) - \alpha \ln^2\,\frac{d}{\tau(t)}\,\Big],\qquad \alpha = \mathrm{clip}\!\left(2000\,T^2,\ 100,\ 1600\right)

$T = \tau/F$ is the beat period (seconds). Since $\alpha\ln^2(d/\tau) \approx 2000\,(\Delta t)^2$ ( $\Delta t$ being how many seconds the spacing is off), the penalty is based on absolute time regardless of tempo. The fixed value of 100 in the original method is percentage-based, so in slow songs the same percentage becomes many more ms, and the beat was dragged along by syncopation.

The local beat period $\tau(t)$ is obtained from the autocorrelation $r_w$ of 8-second windows (at 1-second intervals); momentary wobbles such as fills are filtered out with the median of 7 windows, and the result is linearly interpolated for each frame.

\tau_w = \arg\max_{\ell \in [0.85\tau,\ 1.15\tau]} \Big( r_w(\ell) + \tfrac12\, r_w(2\ell) \Big),\qquad
\tau(t) = \mathrm{clip}\!\Big(\mathrm{interp}_t\big(\mathrm{median}_{7}\,\tau_w\big),\ 0.85\tau,\ 1.15\tau\Big)

The backtracking starting point is the last of the local maxima of $C$ that exceeds half of their median. Beats at the start and end of the path whose smoothed local score is below half of the overall RMS are trimmed.

Intro grid (1.10.4). In an intro with few instruments, the anticipated accent at the end of a bar (bass or cymbal on the eighth note after beat 4) is much stronger than the onsets between beats, so dynamic programming can take that accent as a beat and drift onto a beat a few % slower than the main section, putting bar lines about half a beat off. So within the first 48 beats of the song, if a grid extended backward at a steady spacing equal to the period of the 8 beats following some beat diverges from the current beats by more than a quarter beat, the two grids are compared. The criterion is how well the onsets from the first sound up to that beat fit the eighth-note positions (both on-beat and offbeat), and the intro beats are replaced with that grid only when the backward grid is better by more than 0.15 (onset-weighted mean, 30 ms wide Gaussian). In that case the bar lines also continue the phase of the main section backward. If the intro really has a different tempo or free tempo, the original beats fit the onsets better and are left as they are. The intro of 「マリーゴールド」 (from the first sound at 0.55 s to the band’s entry at 7.34 s) was caught at 0.596-second spacing, as a 3-beat pickup bar + bar lines at 2.55 and 4.94 s, but it changed to three full bars at 0.566-second spacing (0.55, 2.82, 5.08 s). These bar lines match, within 5 ms, the starting points at which the intro riff repeats every 2.26 seconds (4 beats). Among the 1000 GTZAN songs, the rule kicked in for only one reggae song, and the score is unchanged.

Local misalignment in the intro (1.11.7). Even in a song made at a fixed tempo, if it starts on piano or the like without drums, notes anticipated by a sixteenth note in the intro can be stronger than the onsets between beats, so dynamic programming may take those notes as beats. In this case the total number of beats in the intro is correct, so the “diverges by more than a quarter beat” condition above is not triggered, but the beats are about 1/4 beat early for almost the whole intro, the spacing is uneven, and the chord segments are off too. So if the 8 beats following some beat have an even tempo (spacing within ±3% of the period), and when that grid is extended backward at a steady spacing some current beat deviates from it by more than 0.15 beat, the two grids are compared by how well the intro onsets fit the sixteenth-note positions. The intro beats are replaced with that grid only when the backward grid fits better and there is evidence that the song was made at a fixed tempo (at least 30 onset peaks in the intro, and sixteenth-note fit of the backward grid of at least 0.85).

(6) Beat refinement

Parabolic interpolation is done at the onset maximum $j$ within $\pm 2$ frames.

\hat t = j + \frac12\cdot\frac{O(j-1) - O(j+1)}{O(j-1) - 2O(j) + O(j+1)},\qquad b = \frac{\hat t}{F} + t_\text{cal},\quad t_\text{cal} = 6.5\ \text{ms}
For beats whose strength is below 0.35 times the median, the position is set by linear regression on the 4 neighbors on each side with weights $1/(1+ j-i )$. With sounding section $[t_s, t_e]$ and beat spacing $p$, beats are clipped to $[\,t_s - p/4,\ t_e + p/4\,]$, and beats are added at the same spacing at the front while $b_0 - p \ge \max(0, t_s - p/4)$ and at the back while $b_\text{last} + p \le t_e - p/2$.

(7) On-beat phase check

The harmonic change curve $\nu$ is built from chroma $\hat c$ that has been square-root compressed and L2-normalized.

\nu(t) = \left(1 - \cos\!\Big(\sum_{d=0}^{3}\hat c_{t+d},\ \sum_{d=1}^{4}\hat c_{t-d}\Big)\right)\cdot \min\!\left(1, \sqrt{\frac{\min(e_t, e_{t-1})}{e_{70\%}}}\right)

The beat accent curve is a weighted sum of four curves normalized by their mean $\mu$ over sounding sections.

S(t) = 1.0\,\frac{f_\text{low}}{\mu_\text{low}} + 0.7\,\frac{f_\text{mid}}{\mu_\text{mid}} + 0.2\,\frac{f_\text{high}}{\mu_\text{high}} + 0.8\,\frac{\nu}{\mu_\nu}

\text{on} = \frac1N\sum_i \max_{|\delta|\le 2} S(b_i + \delta),\qquad
\text{off} = \frac1N\sum_i \max_{|\delta|\le 2} S\!\left(\tfrac{b_i + b_{i+1}}{2} + \delta\right)

If $\text{off} > 1.1\cdot\text{on}$, all beats are moved by half a beat (when there are 8 or more beats).

(8) Beat-level check

Define the kick curve $K$ and the snare curve $S_n$. $S_n$ is the geometric mean of the mid and high bands, so it grows only when both bands rise at the same time.

K(t) = \frac{f_\text{low}(t)}{\sigma_\text{low}},\qquad
S_n(t) = \sqrt{\frac{f_\text{mid}(t)}{\sigma_\text{mid}}\cdot\frac{f_\text{high}(t)}{\sigma_\text{high}}}

For each beat, $p_i = K + S_n + \tfrac12 S$ (maximum over ±2 frames) is computed, and the ratio of the averages of the even-numbered and odd-numbered beats is examined.

r_2 = \frac{\max(\bar p_\text{even}, \bar p_\text{odd})}{\min(\bar p_\text{even}, \bar p_\text{odd})}
Correction Condition Action
To half ( $r_2 \ge 2.4$ and $s_{S_n} > -0.1$ , or same-side bias) and $\kappa \ge 2$, BPM ≥ 110, BPM/2 ≥ 55 Track again from the start with beat period $2\tau$
To double BPM ≤ 100, 2·BPM ≤ 210, $\sigma \ge 1.3$, $\kappa \ge 2$ Add midpoints and snap them to onsets
To double (backbeat type) BPM ≤ 100, 2·BPM ≤ 200, $c_2 \ge 0.8$, $\sigma \le 0.8$, $\kappa \le 0.95$, $\lvert d_{S_n}\rvert \le 0.25$ Same
To double (half-time type) BPM ≤ 100, 2·BPM ≤ 210, $c_2 \ge 0.85$, $\sigma \ge 0.9$, $1 \le \kappa \le 10$ Same 
\sigma = \frac{\sum S_n(\text{mid})}{\sum S_n(\text{beat})},\qquad
\kappa = \frac{\sum K(\text{beat})}{\sum K(\text{mid})},\qquad
d_{S_n} = \frac{S_{n,\text{even}} - S_{n,\text{odd}}}{S_{n,\text{even}} + S_{n,\text{odd}}}

mid is the midpoint between beats, and $c_2$ is the score of the tempo candidate at twice the current tempo (±8%) divided by the top score. The backbeat type is the case where the slow beat sits on the snare, so snares come evenly on every beat ( $d_{S_n} \approx 0$ ) and kicks come between beats; the half-time type is the case with the kick on the beat and snares between beats too. The $\kappa \le 10$ of the half-time type is there to exclude drumless piano songs with almost no low-pitched hits between beats. In the halving correction, $\kappa \ge 2$ checks whether the space between beats is a sixteenth-note position (where low-pitched hits are rare), and $s_{S_n} > -0.1$ checks that it is not a backbeat (1.10.0).

Same-side bias is the bias measured over even and odd beats, taking the side where the kick is stronger as +.

s_X = \mathrm{sgn}(K_\text{even} - K_\text{odd})\cdot\frac{X_\text{even} - X_\text{odd}}{X_\text{even} + X_\text{odd}},\qquad
\text{same side} \iff s_K \ge 0.15\ \land\ s_{S_n} \ge 0.15\ \land\ s_\nu > 0\ \land\ r_2 \ge 1.3

At the correct beat level, the backbeat puts the snare on the side opposite the kick, so $s_{S_n} < 0$ (−0.28 to −0.61 on synthetic songs); when the tempo is caught double, both fall on the real-beat side, so $s_{S_n} > 0$.

Doubling is much stricter because the risks are asymmetric. Wrongly halving only makes the clicks sparser, but wrongly doubling makes every added click an offbeat.

(9) Beat subdivision decision

From the profile $P(\varphi)$, the average of the onset maxima at each phase obtained by dividing a beat interval into 24 parts,

\text{triple} \iff \min\!\big(P(\tfrac13), P(\tfrac23)\big) > 1.15\,P(\tfrac12)\ \ \land\ \ \tfrac12\big(P(\tfrac13) + P(\tfrac23)\big) > 1.1\,\mathrm{median}(P)

4:3 beat-level check (1.11.5). A slow 4/4 song with dense sixteenth notes can look as if its onsets repeat every three sixteenths (a dotted eighth), so because of the tempo preference prior (centered at 110 BPM), 4/3 times the real tempo (e.g. 92 BPM instead of 69 BPM) can be picked, and the song can be written as 12/8 with each beat split into three. If the beat subdivision decision says triple, the beats are re-tracked at the tempo $b’$ with the largest comb score $C$ (before multiplying by the preference prior) near 3/4 (±4%) of the current beat tempo $b$, and the change is made only when all of the following hold.

\frac{C(b')}{C(b)} \ge 0.93,\qquad
\frac{C(b')\,\pi(b')}{C(b)\,\pi(b)} \ge 0.85,\qquad
\frac{P'(\tfrac12)}{\tfrac12\big(P'(\tfrac13) + P'(\tfrac23)\big)} \ge 1.4,\qquad
G = \rho(8) + \rho(16) - \rho(12) - \rho(24) \ge 0,\qquad
\pi(x) = e^{-\frac12\log_2^2(x/110)}

The first condition means that without the preference prior, the slower tempo explains the onset period about as well as or better than the current tempo; the second means that the two candidates were also nearly tied in the initial tempo estimate with the preference prior applied. $P’$ is the beat subdivision profile measured with the new beats, and the new beat must split clearly into two eighth notes rather than a triplet. $\rho(L)$ is the lag-$L$ autocorrelation of a harmonic-change sequence built from the treble and bass chroma distance over 4 slots before and after, on a grid dividing the current beat into three (sixteenth notes); so $G$ is positive if the harmony changes every 8 or 16 slots (half bar or bar of 4/4), and negative if it changes every 6 or 12 slots (half bar or bar of 12/8), since it then repeats at lags 12 and 24. The new tempo must be 50–140 BPM. If the re-tracked beats follow only the kick and have latched onto the offbeat (the snare in the middle between beats is stronger than on the beat while the kick is stronger on the beat: $S_n(\text{beat})/S_n(\text{mid}) < 0.9$ and $K(\text{beat})/K(\text{mid}) > 1$), they are moved by half a beat. At first only the first condition was used, but 8 songs got worse (guitar-accompaniment songs and very fast songs), so the third and then the second condition were added to narrow it down; the fourth condition is a safety margin that does not reduce the number of songs that change (the 1.11.5 verification in 4-6).

3:2 beat-level check (1.11.6). Conversely, songs with an even stream of eighth notes (solo guitar accompaniment, 8-beat rock, etc.) can look as if the accent repeats every three eighth notes (a dotted quarter), so they can be caught at 2/3 of the real tempo and written as 12/8 or 6/8. For songs not changed by the 4:3 check, the beats are re-tracked at the tempo $b’’$ with the largest $C$ near 3/2 (±4%) of the current tempo $b$, and the change is made only when all of the following hold.

\frac{C(b'')}{C(b)} \ge 0.93,\qquad
\frac{P''(\tfrac12)}{\tfrac12\big(P''(\tfrac13) + P''(\tfrac23)\big)} \ge 1.4,\qquad
G \ge 0.05,\qquad
\frac{\bar S(\text{beat})}{\bar S(\text{mid})} \ge 1.05,\qquad
90 \le b'' \le 240

$G$ is the same measure as in the 4:3 check (here the slots dividing the current beat into three are eighth notes, so it is positive if the harmony changes every 8 or 16 slots, i.e. every one or two bars of 4/4). The fourth condition requires the accent $S$ at the re-tracked beat positions to be more than 5% stronger than in the middle between beats, filtering out cases where the tempo is right but the beats have latched onto the offbeat. The half-beat shift rule is the same as in the 4:3 check. Cases where the new tempo is slower than 90 BPM (the current tempo below 60 BPM) are not considered, because they are mostly genuinely slow triplet ballads. It started with just the range and the $G$ condition, but 26 songs improved while 22 got worse, so the score ratio, accent ratio and eighth-note split conditions were added in turn until no song got worse; then one song came out with worse major/minor in chord scoring, so the tempo lower bound was raised to 90 BPM (the 1.11.6 verification in 4-6).

(10) Spectral peaks, tuning, chroma

With log magnitudes $L_-, L_0, L_+ = \ln X(k-1) , \ln X(k) , \ln X(k+1) $, 
\delta = \frac12\cdot\frac{L_- - L_+}{L_- - 2L_0 + L_+},\qquad
\hat f = (k + \delta)\frac{f_s}{N},\qquad
\hat A = \frac{4}{N}\exp\!\left(L_0 - \tfrac14(L_- - L_+)\,\delta\right),\qquad
m = 69 + 12\log_2\,\frac{\hat f}{440}

The reference pitch deviation (in semitones) is the circular mean of the fractional parts $d_j = m_j - \mathrm{round}(m_j)$ (peaks at MIDI 40–90 from the bass STFT, weights $\sqrt{A_j}$ ).

\theta = \frac{1}{2\pi}\mathrm{atan2}\!\Big(\sum_j \sqrt{A_j}\sin\, 2\pi d_j,\ \sum_j \sqrt{A_j}\cos\, 2\pi d_j\Big),\qquad
A4 = 440 \cdot 2^{\theta/12}\ \text{Hz}

Chroma is accumulated with the corrected pitch $\tilde m = m - \theta$. $w(\cdot)$ is the range window for treble and bass respectively.

C(t, q) = \sum_{j:\ \mathrm{round}(\tilde m_j) \bmod 12 = q} A_j\, w(\tilde m_j)\, \cos^2\!\big(\pi(\tilde m_j - \mathrm{round}(\tilde m_j))\big)

(11) Beat-synchronous pooling, bass floor, key

The frames in the beat interval $[t_0, t_1)$ are averaged with weight concentrated in the middle.

\bar C(q) = \frac{\sum_j \omega_j\,C(j, q)}{\sum_j \omega_j},\qquad \omega_j = 0.1 + \sin(\pi u_j),\quad u_j = \frac{j h - t_0}{t_1 - t_0}

Removing the constant component from the bass chroma and estimating the key are done as follows. $\vec{K}_\text{mode}$ are the Krumhansl–Kessler major and minor profiles, and $\mathrm{rot}_t$ is rotation to tonic $t$.

B'_s(q) = \max\!\Big(0,\ B_s(q) - \tfrac12\mathrm{median}_i B_i(q)\Big)

(\hat t, \widehat{\text{mode}}) = \arg\max_{t,\ \text{mode}}\ \mathrm{corr}\!\big(\vec{h},\ \mathrm{rot}_t(\vec{K}_\text{mode})\big),\qquad
h_q = \sum_i \sqrt{\frac{T_i(q)}{\sum_{q'} T_i(q')}} + \frac12\sqrt{\frac{B_i(q)}{\sum_{q'} B_i(q')}}

(12) Chord templates and emission scores

With harmonic offsets $d = (0, 12, 19, 24, 28, 31)$ semitones and weights $0.6^{h-1}$, the chord tones and bass leakage ( $\rho = 0.25$ ) are added.

\tilde T(q) = \sum_{p \in \text{chord}}\ \sum_{h=1}^{6} 0.6^{h-1}\,[\,q \equiv p + d_h\,] \;+\; \rho\sum_{h=1}^{6} 0.6^{h-1}\,[\,q \equiv p_\text{bass} + d_h\,]\pmod{12}

T = \frac{(\tilde T/\max\tilde T)^{\gamma}}{\left\|(\tilde T/\max\tilde T)^{\gamma}\right\|_2},\qquad \gamma = 0.45

The observation $o_s$ is built from the treble chroma in the same way (max normalization, power $\gamma$, L2 normalization). The emission score of chord $c$ is as follows.

e_s(c) = \beta\,\langle o_s, T_c\rangle + \pi_q + \pi_\text{inv} + \kappa\,[\,c \subset \text{key}\,]
+ \eta\,g_s\Big(\ln\big(0.06 + \hat b_s(p_\text{bass})\big) - \ln\big(0.06 + \tfrac{1}{12}\big)\Big)

$\beta = 30$, $\kappa = 0.6$, $\eta = 1.5$; $\hat b_s$ is the bass chroma summing to 1, and $g_s = \min!\big(1, E^\text{bass}s / (0.5\,E^\text{bass}{60\%})\big)$ is a factor that reduces the bass term when the bass is weak. The score of no chord (N) grows the flatter the chroma and the quieter the sound.

e_s(\mathrm{N}) = \beta\left(\frac{1}{\sqrt{12}}\sum_q o_s(q) - 0.12\right) + Q_s,\qquad
Q_s = \begin{cases} 12 & r_s < 0.02 \\ 6\,(0.08 - r_s)/0.06 & r_s < 0.08 \\ 0 & \text{otherwise}\end{cases}

$r_s$ is the slot’s treble energy divided by the overall 60th percentile. The prior scores $\pi_q$ and the inversion prior scores $\pi_\text{inv}$ are as follows.

Quality maj min 7 maj7 m7 sus4 sus2 dim aug m7♭5 dim7
$\pi_q$ 0 0 −0.4 −0.6 −0.4 −1.2 −1.4 −1.3 −1.8 −1.2 −1.6
Bass Root 3rd 5th 7th
$\pi_\text{inv}$ 0 −0.7 −1.0 −1.0

Inversions are allowed only for maj, min, 7, maj7 and m7. The number of states per vocabulary is 73 for basic, 241 for standard and 289 for extended (including N).

Reducing the vocal melody (1.11.0). The biggest cause of chord errors is the vocal melody mixing into the harmony band so that the chord is read as a neighboring chord with many common tones (C ↔ Am, G7 ↔ Gm7, etc.). Popular music recordings usually place the vocals in the center and spread the accompaniment left and right, so in the side signal $S = (L - R)/2$, left minus right, the vocals are almost erased while the accompaniment remains. So a second treble chroma is built from the side signal and added to the treble chroma for the final chord decision ($C_\text{treble} = C_M + C_S$). The bass is also in the center, so the harmonics of the bass notes drop out of the side signal; therefore the treble component of strong notes in the segment’s bass chroma is restored at half weight. This mixed chroma is used only for choosing the root and the major/minor branch; if the chosen root and branch match the chord read from the unmixed chroma, the seventh-chord and inversion names come from the unmixed side. It is not mixed into beat, downbeat, meter or key estimation. Mono files and files whose two channels are nearly identical (side/mid energy below −30 dB) do not get a side signal, so their results are unchanged. Stereo files take longer to analyze.

Matching repetitions (1.11.0). Popular songs repeat the same chord progression, as in verses 1 and 2 and the chorus, with only the melody and fills differing each time. For each beat, within-song similarity is measured with unit vectors of treble and bass chroma, and wherever the diagonal averaged over an 8-beat window (the same position at least 16 beats away) exceeds the top 3% within the song, the best-matching spot there becomes that beat’s “repetition partner” (only partners with the same beat position within the bar, up to 6 per beat). For songs whose last chorus goes up a semitone or whole tone, it also compares after shifting by the key difference. Before the final chord Viterbi, each beat’s chord scores are averaged with weights against its partners’ scores (weight 1 for itself, 0.7 × similarity per partner), but from partners it borrows only the best score per root, while qualities such as major, minor, 7 and sus are decided by that beat’s own sound. Another repetition corrects a beat that leaned toward a neighboring chord because of one instance of melody, without pulling in the quality of a chord deliberately played differently. The similarity table is the size of the number of beats squared, so it is not used for very long inputs over 3,000 slots.

Half-diminished names (1.11.1). The standard vocabulary does not include m7♭5 (adding it changes m7 and dim in other songs and makes things worse overall), so half-diminished seventh chords (ii°7 in minor, vii°7 in major) were written as dim. After the chords and key are all decided, for each segment written as dim, if the minor 7th (root + 10 semitones) in the unmixed treble chroma is at least 0.5 times the average of the triad tones, only the name is changed to m7♭5. The root, major/minor, boundaries and key stay the same. Against the chords played in GuitarSet, samples with the correct quality increased from 59.37% to 60.28% (+97 / −3; for popular music genres +69 / −0), and most of the changed samples were m7♭5 actually played by the performer.

(13) Chord Viterbi: O(T·K)

Since the transition score is “0 to stay, $-\lambda_s$ to change (regardless of the destination chord)”, the best transition into every destination state can be computed with just the single maximum at each step.

V_s(c) = e_s(c) + \max\!\Big(V_{s-1}(c),\ \max_{c'} V_{s-1}(c') - \lambda_s\Big)
Slot position Downbeat Middle beat of an even meter Other beats Between beats (½-beat resolution) Preliminary chords (all beats)
$\lambda_s$ 1.5 3.0 4.0 6.0 3.0

(14) Downbeat evidence and the meter HMM

Six features of beat $i$ are each standardized ( $Z$ ) and summed.

z_i = 1.0\,Z(h)_i + 1.0\,Z(l)_i + 0.5\,Z(g)_i + 0.6\,Z(k)_i + 0.3\,Z(e)_i + 0.4\,Z(o)_i
Symbol Feature
$h_i$ Treble chroma change $1 - \langle \hat t_i, \hat t_{i-1}\rangle$
$l_i$ Whether the preliminary chord changed
$g_i$ Bass chroma change
$k_i$ Low-band onset strength at the beat position
$e_i$ Loudness increase (dB, positive only)
$o_i$ Whether a new entry into the tonic chord occurred

The states are the positions within the bar $p \in {0, \dots, m-1}$. The emission score is $1.5\,z_i$ at $p = 0$, $0.45\,z_i$ at $p = m/2$ for even meters ( $m \ge 4$ ), and 0 otherwise.

P(p \to p+1) = 1 - 2\varepsilon,\qquad P(k \to 0) = \varepsilon\ \ (1 \le k \le m-2),\qquad P(m-1 \to m-1) = \varepsilon,\qquad \varepsilon = 3\times 10^{-4}

Allowing bars to be shortened or lengthened with small probability keeps the path from collapsing even for odd meters or truncated bars. The automatic meter choice uses the contrast $\Delta_m$ (mean $z$ of the beats chosen as downbeats − mean $z$ of the other beats). The candidates are 4 and 3 beats (4 and 2 for compound meters), and the other one is chosen only when its contrast is at least 0.25 (0.3 for compound meters) higher than that of 4 beats.

Drum even/odd prior (1.10.0). For each beat, the kick strength $K$ and snare strength $S_n$ are standardized, and a local consistency is computed by summing the values multiplied by $(-1)^j$ within 8 beats before and after and dividing by the sum of absolute values (with the sign flipped for the snare).

c^X_i = (-1)^i\,\frac{\sum_{|j-i| \le 8} (-1)^j X_j}{\sum_{|j-i| \le 8} |X_j|} \in [-1, 1],\qquad
z_i \leftarrow z_i + \mathrm{sgn}(c^K_i)\,\min\big(|c^K_i|, |c^{-S_n}_i|\big)\quad \text{if } |c^K_i|, |c^{-S_n}_i| \ge 0.3,\ c^K_i\,c^{-S_n}_i > 0

This is a prior that the downbeat is on the kick side (beats 1 and 3), not the snare side (beats 2 and 4). Distinguishing beat 1 from beat 3 is still decided by the harmonic evidence. It is used only when both the kick and the snare alternate clearly and on opposite sides, so it is 0 where only one side alternates, such as the strumming of a solo guitar. If this is confirmed on at least 20% of the song’s beats (a drum kit is present), it is also added using just the snare consistency (0.5 or more) where the low band is blurred by the bass or syncopated accents. On GTZAN pop, the even/odd indicated by this prior matched the even/odd of the reference downbeat 99.6% of the time.

Reinforcing the meter choice (1.10.0). 3, 5 and 7 beats are chosen, in addition to the contrast condition, only when the autocorrelation of the downbeat evidence at that bar length is larger than at 4 beats, and if a drum backbeat is confirmed in 30% or more of the song, the 3, 5 and 7 beat candidates are removed (the model for meter changes within a song stays as it is). The 4-beat vs 2-beat (half-bar) decision uses $z$ before the drum prior is added. This keeps a kick that raises beats 1 and 3 equally from making 4/4 read as 2/4.

\rho_z(m) + \rho_z(2m) > \rho_z(4) + \rho_z(8)

Revisiting bars of different length (1.11.4). In songs that strike the downbeat chord half a beat to one beat early, the per-beat harmonic changes $h, l, g$ are split between the fourth beat and the first beat, so the downbeat evidence tends to point to the third beat as if it were the downbeat, the bar lines shift by half a bar, and 2-beat bars creep in. If the fixed-meter solution contains bars of different length and there is no meter change within the song, it is solved again with evidence in which the per-beat changes are replaced by the treble and bass chroma distances $h^{(2)}_i, g^{(2)}_i$ between windows of the 2 beats before and the 2 beats after (so that anticipated changes also gather on the downbeat).

z'_i = z_i - Z(h)_i - Z(l)_i - 0.5\,Z(g)_i + Z(h^{(2)})_i + Z(g^{(2)})_i,\qquad
h^{(2)}_i = 1 - \cos\Big(\textstyle\sum_{j=i-2}^{i-1} \hat T_j,\ \sum_{j=i}^{i+1} \hat T_j\Big)
The re-solved result is used only when all three of the following hold (for 4/4 songs whose meter was chosen automatically and songs where 4/4 was chosen by hand; downbeat shifts move the corrected grid as is). ① There are no bars of different length at all (the whole song is in one bar phase). ② With both the original evidence $z$ and the new evidence $z’$, when the whole song is taken in one phase, that phase’s downbeat score is higher than the other phases by more than the cost of two bars of different length ($2\, \ln \varepsilon $). ③ When the same phrase repeats 4 or 8 bars later, the cases in which the beats at the same positions chosen by the new grid have more similar harmony than those chosen by the original grid are more numerous by a sign test with $z \ge 3$ (since the same phrase must also have the same bar phase, this is evidence that the phase flip in the original solution was spurious). Used unconditionally, it made some songs worse on GTZAN (34), though fewer than the songs whose downbeats improved (71), so the conditions were narrowed until no existing song got worse.

Phase at the start of the song (1.10.3). If the song is judged to start with a pickup bar, a short bar is wedged into the first two bars, and the number of beats from the start of the song to the end of that short bar is a multiple of the bar length, then this is a case where the downbeat phase was caught one beat late and then fixed up with a short bar (a fake pickup + a fake “3-beat” bar). In that case it counts again in full bars from the start of the song. Irregular bars in the 6 popular GTZAN genres dropped from 711 to 695, and the downbeat F stayed the same.

(15) Metronome scheduling

If the music started playing at $t_s$ on the audio clock from song position $o_s$, then with the user correction $\delta$ added, the scheduled time of the $i$-th click, the current playback position and the displayed position are as follows.

t^\text{click}_i = t_s + (b_i + \delta - o_s),\qquad
\text{pos} = o_s + \max(0,\ t_\text{now} - t_s),\qquad
\text{pos}_\text{display} = \text{pos} - L_\text{out}

The scheduler wakes with period $T_\text{tick} = 25$ ms and schedules clicks with $b_i + \delta \le \text{pos} + H$ ( $H = 0.2$ s, 1.5 s for background tabs). Since $H \gg T_\text{tick}$, clicks are already scheduled unless the timer is late by more than about 175 ms at a time. The click sound is a buffer with the following waveform precomputed (downbeat $f$ = 1,850 Hz, $A$ = 0.9; other beats 1,250 Hz, 0.6; linearly decaying noise added in the first 2 ms). When the chord change unit is ½ beat, scheduling beat $b_i$ also schedules a small click (2,600 Hz, $A$ = 0.3, 8 ms decay, half the noise) at the following between-beat point $(b_i + b_{i+1})/2 + \delta$. Since it sounds half a beat later, it is not late even in slow songs, and when the position is moved, scheduling restarts from the click after the current beat (1.10.0).

x(t) = A\Big[\,0.75\,\big(\sin\, 2\pi f t + 0.3\sin\, 2\pi (2.02 f) t\big)\,e^{-t/16\,\text{ms}}\Big]

(16) Piano synthesis

The fundamental frequency and inharmonic partials are set from the MIDI note $m$ and the song’s reference pitch deviation $\theta$ (in semitones).

f_0 = 440 \cdot 2^{(m - 69 + \theta)/12},\qquad f_k = k f_0\sqrt{1 + B k^2},\qquad B = 3.5\times 10^{-4}\cdot 2^{(m-60)/24}

The partial amplitudes are set by the hammer strike position $x_h = 1/7.5$ and the high-frequency damping of the hammer felt.

a_k = \frac{|\sin(\pi k x_h)|}{k^{0.8}}\cdot\frac{1}{1 + (f_k/f_b)^2},\qquad f_b = \mathrm{clip}\big(2600 + 40(m-60),\ 1800,\ 5000\big)\ \text{Hz}

Each partial is the sum of a fast decay (prompt sound) and a slow decay (aftersound), and the low partials are sounded together by several strings $s$ at slightly different pitches $d_s$.

y(t) = \sum_{k}\ \sum_{s=1}^{S_k} \frac{a_k}{S_k}\left(0.60\,e^{-t/\tau^{F}_k} + 0.40\,e^{-t/\tau^{S}_k}\right)\sin\!\left(2\pi f_k\, 2^{d_s/1200}\, t + \varphi_k\right)

\tau^{S}_k = \max\!\left(0.12,\ \tau_1\Big(\frac{f_0}{f_k}\Big)^{0.7}\right),\qquad \tau^{F}_k = 0.15\,\tau^{S}_k,\qquad \tau_1 = \mathrm{clip}\!\left(8.5\cdot 2^{-(m-38)/16},\ 1.0,\ 8.5\right)\ \text{s}

The number of strings $S_k$ for $k \le 6$ is 1 (m < 35), 2 (m < 47) or 3 (otherwise), and 1 for $k > 6$. The tuning offsets between strings are $d = (0, +1.0, -0.8)$ cent. Partials are used only up to the 32nd and 8 kHz, and are not computed after falling below −100 dB. The starting phase $\varphi_k$ differs for each partial, using a random number fixed per note, but the several strings of the same partial start in the same phase (see beating in 4-2). On top of this come 30 ms of hammer noise, a 2 ms attack ramp and a 60 ms fade at the end, and the level is set so that the RMS of the first 0.3 seconds is 0.12. The length is $\mathrm{clip}(5.6 - 0.07(m-38),\ 2.6,\ 5.6)$ seconds. The whole piano path is multiplied by a fixed make-up gain of +6.4 dB (×2.09). Sines are generated by a rotation recurrence instead of Math.sin, with the magnitude reset to 1 every 2,048 samples.

\begin{pmatrix} c_{i+1} \\ s_{i+1} \end{pmatrix} =
\begin{pmatrix} \cos\,\omega & -\sin\,\omega \\ \sin\,\omega & \cos\,\omega \end{pmatrix}
\begin{pmatrix} c_i \\ s_i \end{pmatrix},\qquad \omega = \frac{2\pi f}{f_s}

(17) Voicing and striking

The right-hand candidates are close-position voicings stacked upward, one for each inversion of the chord tones, with the lowest note placed in F3–E4 (53–64).

n_\text{bass} = 38 + \big((p_\text{bass} - 38) \bmod 12\big),\qquad
\hat V = \arg\min_{V}\ \big|\bar V - \mu\big| + 2\max(0,\ V_\text{top} - 77),\qquad
\mu \leftarrow 0.7\,\bar{\hat V} + 0.3\cdot 62

$\bar V$ is the mean pitch of the voicing, and $\mu$ starts at D4 (62) and follows the previous voicing while being pulled slightly toward the center. The velocity is 0.62 for the bass and $0.5\sqrt{3/n}$ for each right-hand note ( $n$ being the number of right-hand notes), and the stereo position is $\mathrm{clip}((m-60)/36,\ -0.5,\ 0.5)$. The strike time $t_j$ is mapped onto the audio clock in the same way as in (15), but without adding the user click correction $\delta$, and from the time $t^\text{end}_j$ when the chord segment ends, the volume decreases with a 70 ms time constant.

t^\text{piano}_j = t_s + (t_j - o_s),\qquad g(t) = g_0\,e^{-(t - t^\text{end}_j)/70\,\text{ms}}\quad (t \ge t^\text{end}_j)

(18) Modulation tracking

The note distribution $h_i$ of beat $i$ and $H_i$, gathered over ±6 beats around it with triangular weights, are built.

h_i(q) = \sqrt{\frac{T_i(q)}{\sum_{q'} T_i(q')}} + \frac12\sqrt{\frac{B_i(q)}{\sum_{q'} B_i(q')}},\qquad
H_i = \sum_{|j-i| \le 6}\Big(1 - \frac{|j-i|}{7}\Big)\,h_j

The score of key-signature class $g$ (major tonic $g$, relative minor $g+9$) is the larger of its correlations with the two key profiles.

c_i(g) = \max\Big(\mathrm{corr}\big(H_i,\ \vec K_\text{major}(g)\big),\ \mathrm{corr}\big(H_i,\ \vec K_\text{minor}(g+9)\big)\Big),\qquad g = 0, \dots, 11

A 12-state HMM that charges a penalty $\Lambda$ only when the key changes is solved in the same $O(T \cdot 12)$ way as the chord Viterbi.

V_i(g) = c_i(g) + \max\Big(V_{i-1}(g),\ \max_{g'} V_{i-1}(g') - \Lambda\Big),\qquad \Lambda = 6

Sections shorter than 24 beats are absorbed into the neighboring section with the larger $\sum c_i$. For each section, whichever of the major key and its relative minor correlates more with the whole section’s distribution is chosen. The boundaries are searched again within ±8 beats using per-beat scores $u_j$ that are not pooled over a window.

b^\ast = \arg\max_{|b - b_0| \le 8}\ \sum_{j=b-16}^{b-1}\big(u_j(A) - u_j(B)\big) + \sum_{j=b}^{b+15}\big(u_j(B) - u_j(A)\big) + 0.5\,[\,b\ \text{is a downbeat}\,]

If the chord at $b^\ast$ is V, V7 or Vsus4 of the new key and a tonic chord of the new key (I or Imaj7 in major, i or im7 in minor) follows within 8 beats, the boundary is moved to the position of that tonic chord. The keys of the sections divided this way are used for the key bonus in the chord emission score (12) and for note name spelling.

Once chord recognition is done, the key of each section is decided again from the chords. For the section’s chord sequence (identical consecutive chords merged into one, with length $\ell_j$ beats) and a candidate key $K$, the key with the largest of the following scores is chosen among the 24.

J(K) = \frac{\sum_j \ell_j\,[\,c_j \subset K\,]}{\sum_j \ell_j} + 0.5\,\frac{\sum_j \ell_j\,[\,c_j = \mathrm{I}_K\,]}{\sum_j \ell_j} + 0.3\min\!\Big(1, \frac{N_{V \to I}}{3}\Big) + 0.15\,[\,c_1 = \mathrm{I}_K\,] + 0.15\,[\,c_\text{last} = \mathrm{I}_K\,] + 0.4\,\frac{N_\text{phrase}(\mathrm{I}_K)}{N_\text{phrase}} + 1.2\,\mathrm{corr}\Big(\sum_i h_i,\ \vec K\Big)

If neighboring sections end up with the same key after this re-decision, they are merged into one section (1.10.0). The key-signature classes that split the sections may differ while the keys re-decided from chords are the same, and if left as is, a modulation would be written where the key did not change. Neighboring sections split into relative major and minor (G major and E minor) are also merged into one section, and the key is decided again from the chords of the merged section. This follows the convention of not writing a move between relative keys as a modulation; since songs split this way are ones where the major/minor judgment is ambiguous, the merged section is written as minor only when the minor score is more than 0.2 higher than the relative major (major is more common in popular music, 1.10.0).

Filtering out near-tie modulations (1.11.1). When re-deciding the key of each section from the chords, if the key chosen for a section leads the neighboring section’s key (or whichever of its relative keys scores higher in this section) by less than 0.1 in $J$, that section is also taken to be in the neighbor’s key. This prevents a modulation with almost no evidence from blocking the merging of relative keys and causing the key names of the whole song to go wrong in a chain. In 1.11.0, the intro of 「고민중독」 (Gomin jungdok) split into D major 1.788 vs G major 1.781, and the whole song was written as B minor → E minor → F minor; with this rule it returns to G major → A♭ major (correct). Across all the data tested, only three cases fell within this margin (0.007, 0.02, 0.036), and real modulations led by more than 0.7.

Key names when m7 and sixth chords overlap (1.11.4). In a major-key song, vi7 (D♯m7 in F♯ major) has the same notes as I6 (F♯6) and cannot be told apart by chords, but the tonic chord score goes only to the minor side (i7 of D♯ minor), so a major-key song with a lot of vi7 can be written with the name of its relative minor. Only when the key chosen from the chords is minor, the section’s note distribution fits the relative major best (first among the 24 keys), that margin is larger than 0.15 in $1.2\,\Delta r$ (1.5 times the near-tie margin of 0.1), and m7 on that minor root takes up at least 1/3 of the section’s chord time, is that m7 also counted as the tonic chord of the relative major and the comparison redone. It does not move to any other major key. Used unconditionally, it turned 29 songs on GTZAN whose key had been right into wrong ones (while 26 wrong ones became right), so the conditions were narrowed.

$[\,c \subset K\,]$ is whether all chord tones belong to the key’s scale (including the leading tone for minor), $\mathrm{I}K$ is the tonic chord (I or Imaj7 in major, i or im7 in minor), $N{V \to I}$ is the number of V or V7 → I progressions, and $N_\text{phrase}$ is the number of downbeats every 4 bars starting from the section’s first full bar. The weight of 1.2 in the last term was set using 18 songs with a clear key.

Refining modulation names (1.10.3). If the first key section of the song is 8 bars or shorter and is the subdominant or dominant key of the next section in the same mode (major or minor), it is merged into the next key. When an intro starts on iv or V, the first chord of the phrase makes only the intro come out in a different key, writing a false modulation such as “B♭ minor → F minor”. Also, in songs with a modulation, the key names are unified to the mode of the longest section. Relative major and minor have the same notes, so it is only a choice of name, but if chosen separately for each section, a modulation up a whole step would be written like “C minor → F major”, hiding what changed; so it is written as “C minor → D minor”.

Refining modulation boundaries (1.10.4). Modulation boundaries are set by a window over note distributions, so they tend to land a bar or two early, on the new key’s side, because of chords with many notes shared by both keys (such as V7 of the old key). After all chords are decided, for each downbeat within 2 bars after the boundary, “the fraction of chord tones in the old key’s scale − the fraction in the new key’s scale” is summed per beat, and the boundary is postponed to the downbeat that keeps the most old-key chords in the old key (only when it is better by more than 0.5 beat). It is never moved earlier. This avoids conflicting with the position set at the tonic chord after passing through the new key’s dominant. Also, if the first beat of the new key is N.C., at least half of that bar is N.C. (a break), and the tonic chord of the new key has not appeared yet, the sound of that bar is taken as the pickup start of the next phrase, and the new key is written from the next downbeat (conditions narrowed in 1.10.5: a bar that rests briefly and then immediately strikes the new tonic chord is in the new key from that bar). The B minor of 「夜に駆ける」 was moved from bar 102 (G7 of C minor), through bar 103 (a break whose first half bar is silent, with only the unaccompanied first note of the vocal), to bar 104, where the accompaniment enters in B minor. The A♭ major modulation of 「고민중독」 was moved from 2 bars early to 1 bar early (the transitional bar leading into the modulation).

(19) Camelot notation

The Camelot wheel places keys at positions 1 to 12 in circle-of-fifths order, adding B for major keys and A for minor keys. The number for major tonic $t$ (C = 0), and the number for the minor key that shares its number with its relative major (tonic $t+3$), are as follows.

n_\text{major}(t) = \big((7t + 7) \bmod 12\big) + 1,\qquad n_\text{minor}(t) = n_\text{major}\big((t + 3) \bmod 12\big)

$7t \bmod 12$ means how many fifths up from C the key is (since $7 \cdot 7 \equiv 1 \pmod{12}$, the key a fifth higher gets a number one greater). Adding 7 makes C major come out as 8B.

Number A (minor) B (major)
1 A♭m (G♯m) B
2 E♭m (D♯m) F♯ (G♭)
3 B♭m D♭
4 Fm A♭
5 Cm E♭
6 Gm B♭
7 Dm F
8 Am C
9 Em G
10 Bm D
11 F♯m A
12 C♯m (D♭m) E

A chord uses the number of the key with the same root and the same third, and the chord type is kept as a suffix. sus chords, which have no third, are placed on the B side.

Chord type Camelot code Examples
Major triad, 7, maj7, aug, sus2, sus4 Number of the major key on the root + B C → 8B, G7 → 9B7, Fmaj7 → 7Bmaj7, Esus4 → 12Bsus4
Minor triad, m7, dim, m7♭5, dim7 Number of the minor key on the root + A Am → 8A, Dm7 → 7A7, Bdim → 10Adim, Bm7♭5 → 10A7♭5

The diatonic chords of major key $n$B are I, IV, V at $n$B, $(n-1)$B, $(n+1)$B and ii, iii, vi at $(n-1)$A, $(n+1)$A, $n$A, all gathering within $n \pm 1$ (after 12 comes 1).

(20) Roman numeral harmonic analysis

Based on the key at that point (tonic $t$, major or minor), the interval of root $r$, $d = (r - t) \bmod 12$, is converted to a scale degree. Minor keys are based on the natural minor scale, and the diminished chord on the leading tone is conventionally written vii° without an accidental.

$d$ 0 1 2 3 4 5 6 7 8 9 10 11
Major I ♭II II ♭III III IV ♯IV V ♭VI VI ♭VII VII
Minor I ♭II II III ♯III IV ♯IV V VI ♯VI VII ♯VII
Rule Details
Upper/lowercase Chords whose third is a minor third (m, m7, dim, m7♭5, dim7) are lowercase
Quality symbols dim, dim7 → °, m7♭5 → ø, aug → +, maj7 → M
Inversion figures Triads: root position (none), 6, ⁶₄. Seventh chords: 7, ⁶₅, ⁴₃, ⁴₂. If slash (bass) notation is turned off, root position
Secondary dominants If a major triad or dominant seventh outside the key resolves down a perfect fifth to the very next chord, and that chord is a diatonic triad x other than the tonic, then V/x, V7/x. VII → III in minor is also V/III
Secondary leading-tone chords If a diminished chord outside the key moves to the diatonic triad x a semitone above, then vii°/x, vii°7/x, viiø7/x
Secondary ii If a minor triad, m7 or m7♭5 outside the key leads to V/x a perfect fifth below, then ii/x, ii7/x, iiø7/x
Just before a modulation If V or V7 of the next section’s key goes to that key’s tonic chord, it is V relative to the new key

(21) Reinforcements by song type

Tempo jumps. For each 8-second window (at 1-second intervals), a beat-period score $s_w(\ell)$ is computed over the range 0.6 to 1.65 times the overall beat period, and the flow is chosen by a Viterbi over tempo states. Scores near the original tempo (±4%) saturate to 1 if they are at least 60% of the window maximum, so the path does not leave easily, while other tempos are compared by their actual scores so the path does not move to spurious periods such as 2/3. The cost of moving between windows is as follows.

c(d) = \begin{cases} 0 & d \le 0.005 \\ -6 - 100\,(d - 0.005) & 0.005 < d \le 0.04 \\ -12 - 2d & d > 0.04 \end{cases},\qquad d = \left|\ln\frac{\ell_w}{\ell_{w-1}}\right|

A gradually changing tempo is followed at no cost within 0.5% per window, and a path that goes far by chaining small steps becomes more expensive than a single jump. If the path has no jump over 4%, the local tempo curve of (6) is used as is; if it does, the song is split into sections at the jump points, and a curve is built for each section in the same way (section center tempo ±15%, 7-window median).

Walking bass level. If the tempo is 85 BPM or below, the on-beat/between-beat ratio of low-band onsets is 2.5 or below, and there is no backbeat (snares clustered opposite the kick, $s_{S_n} \le -0.15$), the beat is raised one level; songs raised this way are checked once more at 150 BPM or below. Tempo candidates are 40–300 BPM.

Meter candidates. Simple-meter songs compare 4, 3, 5 and 7 beats. A meter other than 4 beats is chosen only when its downbeat contrast is higher than that of 4 beats by more than 0.25 (3 beats) or 0.3 (5 and 7 beats), and bars forced to fit by stretching or shrinking make up 15% or less. On the 4-beat path, if the downbeat evidence of the third beat is at least 85% of that of the first beat, it is treated as 2 beats.

Power chords. In the score of a power chord state, a penalty is applied for the amount by which the sum of the chroma 2, 3, 4 and 5 semitones above the root exceeds 0.7 times the root and fifth.

e_s(c_5) \leftarrow e_s(c_5) - 6\,\max\Big(0,\ \sum_{d \in \{2,3,4,5\}} o_s(r+d) - 0.7\,\big(o_s(r) + o_s(r+7)\big)\Big)

Also, power chords are written only when they span at least 16 beats and at least 8% of the whole song; if fewer, power chords are removed and the decision is redone. This keeps the sus4 chords of a ballad from being read as scattered power chords.

Bass a whole step above. There are states with a bass a whole step above the root of a major triad (F/G, B♭/C, prior score −1.6), accepted only when at least 30% of the slot’s bass chroma is that note (otherwise −3). In Roman numerals the bass is taken as the root, giving V11, and in Camelot 11 is appended to the bass key’s number.

Refining inversions. If the bass moves three or more times within the same chord (walking bass), inversions shorter than 2 beats are merged into their neighbors, and a fifth in the bass is treated as an alternating bass going back and forth with the root and merged (kept if the whole segment has the fifth in the bass). The one-beat descending bass of a ballad (V⁴₂ → iii7) is left as is.

Meter changes. The states of the downbeat model are extended to (meter, position in bar), allowing a switch to another meter (3, 4, 5 or 7 beats) only at the end of a bar, with a very small probability $\mu = 10^{-7}$ (changed from $10^{-6}$ → $10^{-7}$ in 1.10.0, so that places at a song’s ending where chords change one beat early are not read as 3/4). It is treated as a meter change only when the new meter lasts 4 bars or more (one- or two-bar ones are irregular bars), and if there is no such change, the result of this model is not used, so results for songs with a constant meter do not change.

Checking meter changes (1.11.4). In sections where the downbeat evidence is scattered (such as a chorus where the upper voices sustain while the harmony moves in the bass), noise can happen to line up with a different bar length, and the final chorus of a song that is 4/4 throughout can be written as 5/4 or 7/4. So for each section with a changed meter, it measures how similar the per-beat treble and bass chroma are between beats separated by the phrase spacing (4 bars or 8 bars) of bar length $m$, $S(m) = \max_{k \in {4, 8}} \overline{\cos}(i,\ i + k m)$, and if the $S$ of the main meter is more than 0.02 higher than the $S$ of the new meter, that meter change is removed and it is solved again. This is because popular music repeats in 4- and 8-bar phrases, and spacings of one or two bars cannot distinguish meters. It is used only when the section is at least 16 bars in the main meter, so that an 8-bar phrase repetition can be seen twice (with 30-second excerpts the sections were short, the repetition average wobbled, and some songs got worse downbeats).

Tempo display. For each beat, the tempo is computed as the median of the spacings of the 4 beats before and after, and written into the BPM at the top of the screen during playback. A label is placed on the song’s first beat with the tempo of the first 16 beats, and elsewhere where the tempo of the 8 beats before and after changes suddenly by more than 12% and the new tempo (the next 16 beats) differs by more than 8% from the previous label. The label position is the beat where the difference starts to exceed half its maximum, and if a downbeat lies within two beats, it is snapped there.

Gradually changing tempo. It looks for sections where the tempo, smoothed by the median of the spacings of the 8 beats before and after, keeps changing in one direction by more than 1% relative to 4 beats earlier, changing by more than 8% in total over at least 16 beats (sections containing a sudden change are excluded). The smoothed curve lags at the end, so if within 16 beats after the end of the section the tempo keeps changing by more than 1% per beat, the end is taken to be there. Since the start is extended 4 beats earlier and the end later, neighboring sections can overlap; if they overlap or touch in the same direction they are merged into one section, and if they overlap in opposite directions they are split at the tempo extremum within the overlapping beats, after which the conditions above (16 beats, 8%, no sudden change) are checked again (1.10.0). On the chart, a hexagonal frame spans from the section’s start beat to just before its end beat, and the end beat gets a white pill with the tempo arrived at. During playback the frame fills up to the playback position, and when it reaches the arrival beat, the fill fades over 0.8 seconds as $(1-u)^2$ ($u$ is the time elapsed since arrival ÷ 0.8 seconds). The arrival pill lights up at the arrival beat for the following time and then turns off immediately ($\Delta b$ is the length of the arrival beat).

T_{\mathrm{on}} = 1.1\,\Delta b + 0.06\,\mathrm{s}

The factor 1.1 makes how far it spills into the next beat proportional to the tempo, and the 0.06 seconds is a fixed tail so that the moment it turns off does not appear to coincide with the start of the next beat. At 100 BPM the tail is 0.12 seconds; it is slightly longer for slower songs and slightly shorter for faster ones.

r_j = \left|\ln\frac{\mathrm{med}\{\Delta b_{j-8}, \dots, \Delta b_{j-1}\}}{\mathrm{med}\{\Delta b_{j}, \dots, \Delta b_{j+7}\}}\right| \ge 0.12

(22) Held-over chords and change positions

Held over (grey). When the chord on a downbeat is the same as in the previous slot, the strength of that chord’s tones is compared between the average over about 0.12 seconds just before the beat and the maximum over about 0.12 seconds just after it (all chord tones in the treble chroma, the bass note in the bass chroma). If either jumps by more than 1.2×, it is treated as newly struck and written in black text; otherwise it is treated as held over and written in grey.

r = \frac{\max_{t \le \tau \le t + 0.12}\ \sum_{k \in \mathcal{C}} c_\tau(k)}{\operatorname{mean}_{t - 0.12 \le \tau < t}\ \sum_{k \in \mathcal{C}} c_\tau(k) + \epsilon} \ge 1.2

Refining changes on weak beats. If a chord is judged to change on a weak beat (beat 2 or 4) of an even meter, but on the strong beat just before it the notes found only in the old chord were not newly struck (the ratio above is below 1.3), and the new chord’s score on that strong beat is not lower than the old chord’s by 2.5 or more, the change is moved to the preceding strong beat. This is so that in a broken-chord accompaniment, Esus4 → E, whose third (G♯) only sounds on beat 4 and so was judged a beat late, is written on beat 3 following notation convention. Boundaries at downbeats, and boundaries where the old chord was struck again or keeps sounding, are not moved.

One-beat chords on the downbeat (1.11.8). The final HMM charges a penalty for changing chords within a bar (4.0 on the second beat), so a one-beat chord that sounds only on the downbeat can be merged into the chord of the following beats even if it scores higher on that beat. So when the same chord $Y$ is written from the downbeat (not a chord continued from the previous bar, and the second beat is the same chord), it finds the chord $Z$ with a different root that fits best looking at the downbeat alone, and if all of the following hold, the downbeat is written as $Z$. $E_b$ is the chord score at beat $b$, $r$ is the strength ratio from the held-over test above (maximum just after relative to the average just before the beat), and $\mathcal{C}_Z \setminus \mathcal{C}_Y$ are the notes found only in $Z$.

E_b(Z) - E_b(Y) \ge 3.5,\qquad
E_{b+1}(Y) > E_{b+1}(Z),\qquad
r_b(\mathcal{C}_Z \setminus \mathcal{C}_Y) \ge 1.2,\qquad
r_{b+1}(\mathcal{C}_Y \setminus \mathcal{C}_Z) \ge 1.2,\qquad
\mathrm{bass}(Z) \ne \mathrm{bass}(Y)

The first and second conditions say that $Z$ is better on the downbeat and $Y$ on the second beat; the third and fourth are evidence that the two chords really were struck separately (notes found only in $Z$ newly sound on the downbeat, and notes found only in $Y$ are struck again on the second beat). With only the score conditions, it would also change cases where the reverberation of the previous bar’s chord mixes with the new chord and looks like a different chord on the downbeat, and checking against the GuitarSet reference showed such cases were more common (the 1.11.8 verification in 4-6).

Refining chord names (1.10.0). After the key is fixed, only the names are refined, leaving the boundaries as they are. If a diatonic m7 continues with its third in the bass (Am7/C, Em7/G, Bm7/D), it is the same set of notes as a sixth chord on that bass (C6, G6, D6), so following notation convention it is written as the major triad on the bass (C, G, D). It is changed only when that note is the strongest in the bass chroma summed over the segment, and if the same m7 with a different bass is right before or after (Gm7/B♭ → Gm7), only the bass moved within one chord, so it is left as is. Places where the bass may have been misheard are also left as they are. If that note is less than half of the segment’s bass and the slots where it is the strongest note on each beat fall short of 3/4 (several notes mixed and blurry), it is not changed, and an m7 briefly wedged between identical chords (Em7 – F♯m7/A – Em7) is a passing chord and is not changed either. A sus4 is written as the triad on the same root if, in the segment’s treble chroma, the third (the larger of major or minor) sounds at least 0.7 times as strong as the fourth. This keeps a fourth mixed in from the melody or pitched drums from being read as sus4. In known segments whose reference is sus4 (synthetic songs, GuitarSet performance references, real recordings), this ratio never exceeded 0.65.

Refining anticipations at the end of a bar (1.10.3). If a chord judged to change on the last beat of a bar continues with the same root into the next downbeat, it is often an anticipation (a strum played an eighth note ahead) or a bass approach note. Sheet music writes such changes on the next downbeat, so unless the new chord’s score on that beat leads the old chord by more than 6.5, the change is moved to the next downbeat. The threshold was matched against aligned chord charts for 11 candidate spots in 4 real recordings (6 of the 7 moved spots matched the charts, and the 2 that really changed on that beat were left as is).

4-2. The physical world

Sound, pitch, equal temperament

Sound is a vibration of air pressure, and pitch is the frequency of that vibration. 12-tone equal temperament divides an octave (a doubling of frequency) into 12 equal ratios.

f(m) = 440 \cdot 2^{(m - 69)/12}\ \text{Hz},\qquad
m(f) = 69 + 12\log_2\,\frac{f}{440},\qquad
\text{cent}(f_1, f_2) = 1200\log_2\,\frac{f_1}{f_2}

The fact that an octave is a doubling of frequency is the basis for folding notes into 12 slots (chroma). A recording’s reference pitch often deviates from 440 Hz because of tape speed in old recordings, instrument tuning, or deliberate tuning. For example, +18 cent is $440 \cdot 2^{0.18/12} \approx 444.6$ Hz.

Harmonics

The vibration of a string or pipe also produces components at integer multiples $h f_0$ of the fundamental frequency $f_0$ (harmonics). The distance of a harmonic from the fundamental is $12\log_2\, h$ semitones.

Harmonic $h$ Distance in semitones Nearest equal-tempered Difference Template weight
1 0 0 (same note) 0 cent 1
2 12.00 12 (octave) 0 cent 0.6
3 19.02 19 (fifth above the octave) +2.0 cent 0.36
4 24.00 24 (two octaves) 0 cent 0.216
5 27.86 28 (major third above two octaves) −13.7 cent 0.13
6 31.02 31 (fifth above two octaves) +2.0 cent 0.078
7 33.69 34 −31.2 cent Not used

The 7th harmonic deviates by as much as 31 cents from equal temperament and would fall into the wrong note slot, so it is left out of the templates. Including harmonics in the model has a large effect. Even when only a C major triad (C, E, G) is played, the 3rd harmonic of E is B and the 3rd harmonic of G is D, so if harmonics were ignored it would look like Cmaj7 or Cadd9. In the minor triad Cm, the 5th harmonic of C produces E (a major third), making it look tilted toward major. The comparison is fair only if the templates also contain these components.

Frequency ranges by instrument

Sound Main range Physical characteristics Use in analysis
Kick drum about 40–100 Hz Short low boom, pitch drops quickly Low-band flux → beat, downbeat evidence
Bass about 41 Hz (E1)–400 Hz Integer-multiple harmonics, long sustain Bass chroma → inversions (slash chords)
Snare body about 150–250 Hz, wire noise about 2–10 kHz Noise spread over a wide band Simultaneous rise in mid and high bands → backbeat
Hi-hat about 5–15 kHz Metallic noise High-band flux (weighted down for beat decisions)
Piano, guitar, vocals fundamentals about 80 Hz–1 kHz, harmonics up to several kHz Integer-multiple harmonics Treble chroma → chords

The time–frequency trade-off

The frequency resolution of a window of length $T$ is roughly $1/T$, and the theoretical lower bound is $\Delta t\,\Delta f \ge 1/(4\pi)$ (with equality for a Gaussian window). Meanwhile, the semitone spacing is proportional to frequency.

\Delta f_\text{semitone}(f) = f\,(2^{1/12} - 1) \approx 0.0595\,f

At the lowest bass note E1 (41.2 Hz), the semitone spacing is only 2.45 Hz, so telling notes apart requires $T \gtrsim 1/2.45 \approx 0.41$ seconds. At A2 (110 Hz) it is 6.5 Hz, and at A4 (440 Hz) a comfortable 26.2 Hz. So a separate window is used for each purpose.

Purpose Sample rate Window Window length Hop Bin spacing
Rhythm (onsets) 22,050 Hz 1024 46.4 ms 11.6 ms 21.5 Hz
Treble (chords) 11,025 Hz 2048 185.8 ms 23.2 ms 5.38 Hz
Bass, tuning 11,025 Hz 4096 371.5 ms 46.4 ms 2.69 Hz

The bass window falls a little short of 0.41 seconds, but usually only one bass note sounds at a time, so this is compensated by estimating the peak position far more precisely than the bin spacing using log-parabolic interpolation.

Sampling and aliasing

A sample rate $f_s$ cannot hold components above $f_s/2$ (the Nyquist frequency), and if the rate is simply reduced, high components fold down into low frequencies (aliasing). So the resampler doubles as a low-pass filter cutting at 92% of the new Nyquist. The 22,050 Hz signal keeps up to about 10.1 kHz, covering the hi-hat band (2.5–10 kHz), and the 11,025 Hz signal keeps up to about 5.07 kHz, covering the treble peak range (up to 4.2 kHz). Since the needed information lies below these limits, reducing the number of samples greatly lowers the amount of computation.

Loudness, reverberation, logarithms

Human perception

Piano strings

These are the physical phenomena imitated by the piano synthesis used for chord playback.

Phenomenon Physics In the synthesis
Inharmonicity Because of the bending stiffness of a steel string, higher harmonics are sharper than integer multiples. $B = \pi^3 E d^4 / (64 T L^2)$ ( $E$ Young’s modulus, $d$ diameter, $T$ tension, $L$ length) $f_k = k f_0\sqrt{1 + Bk^2}$, with $B$ increasing in higher registers
Strike position When the hammer strikes at about 1/7 to 1/9 of the string’s length, harmonics that have a node (a point that does not move) at that spot barely sound $\lvert\sin(\pi k x_h)\rvert$ , $x_h = 1/7.5$
Hammer felt The soft felt touches briefly and filters out very high components $1/(1 + (f_k/f_b)^2)$
Double decay The several strings of one note exchange energy through the bridge, so the sound decays quickly at first and then lingers Prompt sound 60%, aftersound 40%
Beating The strings of one note are tuned slightly differently, around 1 cent, so the sound undulates slowly Low partials are synthesized with 2–3 strings, and the strings of the same partial start in the same phase so they gradually drift apart
Per-partial decay Higher harmonics lose energy faster, so the sound darkens over time $\tau_k \propto (f_0/f_k)^{0.7}$
Sustain by register Low strings are long and heavy and ring for a long time Fundamental aftersound 1.0–8.5 seconds

Latency and distance

4-3. System

Components

flowchart LR
  subgraph UI["UI script"]
    LD["Loading<br/>file picker, drag and drop, decoding"]
    CT["Correction and settings controls"]
    RD["Renderer<br/>chord chart, indicators, waveform"]
    PL["Player and scheduler"]
    PN["Chord playback<br/>voicing, piano synthesis cache"]
  end
  subgraph EN["Engine script Engine"]
    AN["analyze"]
    BT["buildTimeline"]
    TX["chartText"]
  end
  LD -->|PCM channels| AN
  AN -->|analysis result A| BT
  CT -->|settings| BT
  BT -->|timeline TL| RD
  BT -->|beats, downbeats| PL
  BT -->|chord segments| PN
  BT --> TX
  PL --> AO["Limiter, AudioContext output"]
  PN --> AO

Repository layout

.
├── chordity.html             the whole app: engine script + UI script + styles (single file, about 230 KB)
├── information.html          help page (Korean): evidence for decisions, analysis principles and formulas, verification results (single file)
├── LICENSE                   terms of use
├── README.md
├── README.{en,ja,zh,fr,es}.md  README translations: English, 日本語, 中文, Français, Español
└── docs/
    ├── screenshot-desktop.png
    ├── screenshot-mobile.png
    ├── notation-name.png
    ├── notation-camelot.png
    └── notation-roman.png

Engine API

The first <script> in chordity.html defines the global Engine (it is also exported via module.exports when loaded from Node). The following description is used for developing and verifying this project. Extracting the engine and loading it in your own program counts as modification or partial use and cannot be done without the original author’s permission (LICENSE section 2-a).

// buf: AudioBuffer (예: AudioContext.decodeAudioData 결과)
const channels = Array.from({ length: buf.numberOfChannels }, (_, c) => buf.getChannelData(c));

// 무거운 단계: 한 번만
const A = await Engine.analyze(
  { channels, sampleRate: buf.sampleRate },
  (progress, stage) => console.log(Math.round(progress * 100) + '%', stage ?? '')
);

// 가벼운 단계: 설정이 바뀔 때마다
const TL = Engine.buildTimeline(A, { vocab: 'extended', res: 2, slash: true });

console.log(Engine.chartText(TL, { title: '내 노래' }));
// # 내 노래  |  BPM 97.1  |  박자 4/4  |  조성 C 장조
//   1 | C . / . / . / . | G . / . / . / . | ...

// 카멜롯 표기로 내보내기: 조성은 C 장조(8B), 코드는 8B, 9A7 처럼
console.log(Engine.chartText(TL, { title: '내 노래', notation: 'camelot' }));

// 로마 숫자로 내보내기: I, V7/vi, ii65 처럼(자리바꿈 숫자는 이어서 적음)
console.log(Engine.chartText(TL, { title: '내 노래', notation: 'roman' }));

Settings (Engine.defaultSettings())

Key Value Default Meaning
ops Array of 'half', 'x2', 'd2', 'd3' [] Beat correction operations applied in order
meter 'auto' or an integer 'auto' Beats per bar
downShift Integer 0 Downbeat shift (positive means later)
vocab 'basic', 'standard', 'extended' 'standard' Chord vocabulary
res 1 or 2 1 Chord change unit (1 beat, ½ beat)
slash true, false true Bass note (inversion) notation

Main fields of the analysis result A

Field Contents
duration, sampleRate Original length (seconds), original sample rate
fl, fm, fh, rms, O Low/mid/high-band flux, loudness, onset envelope (per rhythm frame)
nov, S, drum Harmonic change, beat accent, kick and snare curves
audible, startT, endT Sounding sections
harm tuning (semitones), treble and bass chroma and their hops
tempo bpm, tau, top candidates cands
beats Automatic beat times (seconds)
phase, level Decision values and evidence of the phase check and beat-level check
level43 4:3 and 3:2 beat-level checks: triple (whether beats appeared to split into three), applied; if they appeared triple, curBpm and per-candidate decision values alts; if changed, ratio (4/3 = slower, 2/3 = faster), toBpm, flip (half-beat shift)
elapsed Analysis time (seconds)

Main fields of the timeline TL

Field Contents
empty Whether a chart could not be made because there were too few beats
beats, pos, barOfBeat Beat times, position within the bar (0 is the downbeat), the bar each beat belongs to
bars[] num, pickup, t0, t1, startBeat, nBeats, events[]
events[] off, dur (in beats), tied, pcs (chord tones, 0 = C), bassPc (bass to play), label (root, base, ext, bass, text, ascii, nc, cam: Camelot code, ext, text, ascii; rn: Roman numeral acc, num, qual, fig, sus, sec, text, ascii)
bpm Tempo computed from the median beat spacing
meter m, triple, label (e.g. 4/4, 12/8), contrast, alt
key First key: tonic, mode, score, name (e.g. E♭ 장조, “E♭ major”)
keys[] Key sections: name, cam (Camelot code), tonic, mode, startBeat, barNum (bar number), beat (which beat), t0 (seconds)

Three clocks

Clock Basis Used for
Sample time Sample index in the decoded buffer ÷ sample rate Analysis, all beat times (seconds)
Audio clock AudioContext.currentTime Music playback and click scheduling
Screen clock requestAnimationFrame Current-chord box and name enlargement, indicators, playback position display

Beat times are computed in sample time and mapped onto the audio clock with formula (15) in 4-1. The playback buffer and the analysis signal come from the same decoding result and the resampler delay is 0, so there is no offset between the two time axes.

Current-chord box and name enlargement (1.10.1). On every frame of the screen clock, the chord within the bar is found from the beat containing the playback position and the position within that beat (as a fraction of the beat length). During playback, the position has the output latency subtracted but is not allowed to go before where playback started, so the preceding chord does not flash briefly right after a jump. A chord unit is a newly struck chord (black text, N.C.) plus the grey held-over chords that follow it, and it can continue across bars. The screen is updated only when the unit changes. The first name grows within 0.1 seconds (a CSS transition; on leaving it shrinks over 0.7 seconds), and the scale is set to no more than 1.3× (1.15× for two-row bars) and smaller than the space left before the next name ÷ the name’s width, so the enlarged name never touches its neighbor. The growth anchor is the left edge of the name (keeping its beat position), and vertically it is at 45% for one-row bars (between the upper pill row and the slashes that lengthen during playback) and at the top edge for two-row bars. There is one highlighter box per chart row, and on each row the unit spans, it covers from 7px before the first name (or the left edge of the row if it continues from the previous row) to 2px before the next chord name (just before the bar line at the end of a bar, before the double bar line at the end of the song, or the right edge of the row if it continues onto the next row). The two ends $x$ of the box each move toward the target $x^\ast$ following a damped oscillator $\ddot x = \omega^2 (x^\ast - x) - 2\zeta\omega\dot x$. The leading end, with $\omega = 24\ \mathrm{rad/s},\ \zeta = 0.42$, overshoots the target by about 23% and comes back, while the other end, with $\omega = 16\ \mathrm{rad/s},\ \zeta = 1$ (critical damping), follows without overshooting and stops at about 0.6 seconds. Because the trailing end does not overshoot, the two ends do not cross even when moving to a short chord. A box on a newly appearing row pops out from the start of the chord, and the box on a row the unit has left shrinks toward the direction of travel while fading, disappearing when its two ends meet. Past the end of the song, it shrinks forward. Right after the chart is redrawn (notation or screen width change) and with the reduced-motion setting, the box and name go straight to their places. Inside the box, the progress fill (orange) fills from the left end of the piece up to the playback position. The playback position’s x is the point that divides the bar’s beat positions (slash positions) by the position within the beat (as a fraction of the beat length) (1.10.2), and if the unit continues from the previous row, the piece on the previous row is filled to the end. The fill is drawn in the box’s own coordinates, so its end stays at the playback position even while the box is moving on its spring. The box color is #ffed85 in the light theme and #4b4219 in the dark theme, and the fill is #ffc061 in the light theme and #5e3110 in the dark theme; even the grey of grey chords and N.C. (light #4e5662, dark #aab1bb) reads at 4.5:1 or better anywhere on the fill, the box or the chart surface. So the text color is not changed while the box is moving, nor in two-row bars where a name is longer than the box.

Audio graph

flowchart LR
  M["Music buffer"] --> GM["Music volume"]
  C["Click buffer"] --> GC["Click volume"]
  P["Piano notes<br/>per-note velocity, stereo position"] --> PB["Piano bus"]
  PB --> MK["Makeup gain +6.4 dB"]
  PB --> RV["Room reverb 1.6 s"] --> W["Reverb amount 0.22"] --> MK
  MK --> GP["Chord volume"]
  GM --> L["Limiter −1 dBFS"]
  GC --> L
  GP --> L
  L --> O["Output"]

Saved settings

Metronome on/off, downbeat accent, chord name notation (note names, Camelot, Roman numerals), chord playback on/off and when to strike, music, click and chord volumes, click timing, auto-scroll, theme, chord vocabulary, resolution, and slash notation are saved to chordity-prefs-v1 in localStorage (values saved under an earlier name are migrated). The app works as usual even in environments where saving is not possible.

4-4. Issues and solutions

Problems encountered during development and how they were solved. The figures were confirmed with regression tests on synthetic songs and with real browser tests (4-6).

Issue Cause Solution Result
Slow songs detected at double tempo, clicking on every eighth note Dense eighth-note arpeggios and a prior distribution favoring 110 BPM Halving correction via an even/odd asymmetry check on the drum layer 68 BPM ballad: 133 → 67.1 BPM
Fast songs detected at half tempo Only the kicks recognized as beats Doubling correction via a check for kicks on the beat alternating with snares between beats 176 BPM funk: 88 → 176 BPM
Risk of ruining a piano song at the correct tempo by halving it Left-hand bass only on beats 1 and 3 makes even/odd asymmetric Halving correction only at 110 BPM and above 72 BPM piano kept
A real ballad detected at double tempo, with chords changing every half beat (「세월이 가면」 (Sewori gamyeon): 135 vs. 68 BPM) Hi-hat and keys fill the offbeats, so the even/odd strength ratio is 1.59, short of the 2.4 threshold Added a condition checking whether kick and snare cluster on the same side (at the correct beat level they fall on opposite sides because of the backbeat) Corrected to 68.1 BPM; all synthetic-song decisions unchanged
Spurious 3-beat and 2-beat bars after halving Eighth-note tracking slipped one slot at a fill, and since every other beat was dropped, everything after it was affected Instead of dropping beats, track again at the new beat period Irregular bars 5 → 1
Beats wobble in slow songs, pulled by syncopation and fills The interval penalty (fixed 100) is percentage-based, so in slow songs wobbles over 100 ms are allowed cheaply Compare against the local tempo curve and make the penalty absolute-time-based (α = 2000·T²) Piecewise linear residual RMS 96–222 → 6–23 ms; ritardando test song 80/80
Fast rock detected at half tempo, with beats landing on the snare (beats 2 and 4), mixing chords by half a beat (a 178 BPM song read as 89 BPM) Eighth notes in the bass fill the low end so the kick registers weakly, and the tempo score is similar for both levels Doubling (offbeat type) based on the pattern of snare on the beat and kick between beats, and the strength of the double-tempo candidate 177.9 BPM, downbeat F 0 → 0.95, chord root 62% → 86%
Fast songs with a half-time feel detected at half tempo (a 170 BPM song read as 85 BPM) The slow beat sits on the kick, and the snare also lands between beats Half-time-type doubling. Drumless piano songs with almost no low hits between beats are excluded 170.1 BPM; synthetic piano songs unchanged
Songs at 110–130 BPM halved Judged that eighth notes had been taken as beats by looking only at the even/odd strength difference Check for low hits between beats (which should be at sixteenth-note positions) and for a backbeat 8 GTZAN popular-music songs corrected; 「세월이 가면」 stays at 68 BPM
4/4 songs written as 7/4, 5/4 or 3/4 In songs of about 30 seconds, only a few rare strong downbeat traces were picked, inflating the contrast Check the autocorrelation of downbeat evidence; with a clear drum backbeat, only even meters Songs written in odd meters in GTZAN 221 → 104
In syncopated riffs, bars are shifted by one beat and spurious 5-beat and 3-beat bars appear Downbeat evidence follows chords that change an eighth note early Drum even/odd prior (kick on beats 1 and 3, snare on beats 2 and 4) Irregular bars 9 → 3 (「고민중독」 (Gomin jungdok)); GTZAN popular-music downbeat F 0.557 → 0.608
The ending of a song written as changing to 3/4 Chords change one beat early, looking like a 3-beat spacing Meter change probability $10^{-6}$ → $10^{-7}$ 4/4 kept; the 4 synthetic songs with meter changes unchanged
Many m7 chords with the third in the bass, like Am7/C and Em7/G Sixth chords made of the same notes (C6, G6) read as m7 inversions If the bass is that note and is the strongest, read it as a triad over the bass In GuitarSet, 12 of the 13 changed spots match the played ground truth
A major-key song written as modulating to the relative minor midway (G major → E minor) Only sections with many progressions settling on vi (Em) and many B7 → Em progressions were judged minor on their own Merge neighboring sections split between relative major/minor and judge again; minor only when minor clearly leads 「고민중독」 G major → A♭ major (matches the ground truth); keys of all other test songs unchanged
Written as sus4 even though the third sounds A fourth mixed in from the melody or pitched drums (snare partials) Triad if the third is at least 0.7 times the fourth 「고민중독」 sus errors 10% → 2%
Two-level decisions split in songs with a strong skank, like reggae The double-tempo interpretation is also musically accepted Not forced automatically; a “÷2” control is provided instead Intentionally unresolved
Beats marked 6–8 ms early The 46 ms window includes the attack ahead of time, so the flux peak comes early 6.5 ms correction constant Mean error 0 ± 2 ms
61% of seventh chords recognized as triads Excessive prior penalty for seventh chords Retuned $\beta$, $\gamma$ and the penalty by grid search Missed sevenths 0; triads mistaken for sevenths 0
Even songs with a modulation show only one key, and chords after the modulation are written in the old key’s spelling (A♭m7 in an E major section, etc.) Key estimated only once for the whole song Split into sections with a key-signature-level key HMM, and apply ♯/♭ spelling and the key bonus separately per section 「세월이 가면」 in E major from bar 45 beat 1, written G♯m7 and C♯7 thereafter
A clear A minor progression (Am … E7) judged as E minor Key decided only by the correlation of the note distribution with key profiles, not penalizing out-of-key notes (F) enough Re-decide section keys from the recognized chords (scale fit, tonic-chord time, V → I, first and last chord) and blend with the note-distribution correlation Judged A minor; Roman numerals i v6 VI III6 …
Short sections lean toward the relative minor (E major → C♯ minor) If a section ends on vi and contains V/vi → vi, the chord cues lean minor Added the share of the tonic chord on downbeats of 4-bar phrases as a cue, and set the weight of the note distribution from an 18-song test All 18 songs with a clear key correct
When the tempo changes greatly midway, later beats are missed and even the meter goes wrong Local tempo searched only within ±15% of the global tempo Find jumps with a tempo-state Viterbi; songs without a jump use the previous curve unchanged Beats 64→93/96, offbeat errors 16→1
Bebop detected as one beat per bar (65 BPM) Strong bar-level repetition makes tempo estimation pick the slow level, and the doubling rule only looks at backbeat snare Tempo candidates up to 300 BPM, and step up one level at a time based on low notes between beats (walking bass) 260 BPM, beats 96/96
Cm7 written as E♭ or E♭maj7 even though the bass is C With the same notes (Cm7 = E♭6), the only cue for the root is the bass, and its weight is weak Tried raising the bass weight, but passing tones caused errors like C♯maj7/C elsewhere, so it was reverted Unresolved; recorded in the limitations
Spurious pickup bar before the music starts The DP path extends into the silence, and the neighbor smoothing of edge cleanup keeps it alive Remove beats outside the sounding region ±¼ beat, then extrapolate Removed
Noise floor judged as sound Loose threshold for the sounding region −40 dB → −38 dB Removed
Risk of a pitched kick being mistaken for a bass note The kick’s low boom accumulates in the bass chroma on every beat Subtract half of the whole-song median Constant component removed
4 spurious beats in a silent file The original path kept even when there is no sounding region No sounding region means no beats Empty timeline and a notice
Click before the song starts after a ½-beat shift The midpoint calculation prepends a beat Discard beats before the sounding region Removed
A 1-beat bar in the reverb tail at the end Extrapolation checked only the end boundary Extend only when sound remains for at least half a beat after the new beat Removed
Risk of a big slowdown when switching tabs during analysis In background tabs, setTimeout can be delayed to 1-second granularity Yield control with MessageChannel, which is not subject to timer throttling No effect from timer delays
Holding the → key does not advance to the next bar During the playback scheduling (60 ms), the position is read as before the seek point Pin the position to the seek point before scheduling 5 presses in a row → exactly 5 bars
Play and pause icons do not switch SVG elements have no .hidden property Toggle the hidden attribute directly Fixed
Click timing slider squashed into a dot CSS meant for checkboxes also applied to range inputs Restrict the selector to checkboxes Fixed
On phones the fixed playback bar grows to four lines The metronome button and volumes are all pinned while wrapping Metronome as an icon button, volumes moved outside the fixed area Two lines (playback row and waveform), height 125 px
Risk of the analysis and playback timelines drifting apart Latency of the browser resampler and separate decoding Resample the playback buffer directly with zero latency Offset 0 (impulse test 1.00000 s)
Piano sounds 18 cents lower than the recording Piano synthesized at A4 = 440 Hz Synthesize at the reference pitch estimated by the analysis, and rebuild the cache when the reference changes All 11 notes +17.7 to +18.3 cents (ground truth +18)
Risk of digital clipping when music, click and chords overlap The sum of the three sounds can exceed 0 dBFS Limiter just before the output (−1 dBFS, 20:1, attack 1 ms) Same latency for all three keeps them aligned; click test results unchanged
Chord playback too quiet A piano strike’s first instant is 15–20 dB louder than its body, so a peak-normalized level sounds quiet (first-second RMS −28 dBFS at the default volume) A fixed makeup gain on the piano path, and random starting phases per partial to reduce the first peak Final value decided in the row below
Makeup gain of +11.4 dB is too much The sweet spot lies between the too-quiet value (−28 dBFS) and the too-loud value (−18 dBFS) Bisect at the dB midpoint of the two for a makeup gain of +6.4 dB About 1.8–2.1 times louder than before (+5.1 to +6.5 dB). Even at maximum volume the piano peaks around −3 dBFS, so the limiter does not squash the music
Chord playback rings too briefly A high prompt-sound ratio makes it fade quickly right after the strike Prompt sound 72 → 60%, prompt decay time ratio 0.12 → 0.15, aftersound decay time ×1.13, note length +0.8 s 1–2 s tail +2.7 to +3.0 dB; 10 dB decay time 1.3–1.8 times
Raising the level with a compressor makes the ringing about 3 times longer A slow-release compressor lifts the tail too, and a fast limiter cannot catch the peak in the first few ms Drop the compressor and adjust with synthesis parameters and a fixed gain Lengthened only by the targeted amount (+3 dB)
Randomizing the phase per string makes the ringing excessively long Strings starting in the same phase gradually drifting apart is what causes the double decay, and this breaks it Different phase per partial, same phase for the strings of the same partial Tail change from the phase change alone +5.9 → +0.6 dB

4-5. Optimization

Item Method Effect
Stage separation Heavy signal processing once; meter and chord inference per setting Corrections applied in about 40 ms (5-minute song)
Real FFT One $N/2$-point complex FFT + post-processing, with twiddle factors and the bit-reversal table precomputed About half the FFT computation
No stored spectrogram Rhythm flux is computed streaming through a 3-frame ring buffer; for harmony only the top peaks per frame are stored For a 4-minute song, about 4 MB of peaks instead of three full spectrograms (rhythm, treble, bass, about 85 MB each, about 254 MB total)
Polyphase resampler Kernel precomputed as a phase table, integer accumulator, fast path without bounds checks About 48 multiplications per output sample (48 kHz → 22.05 kHz)
Cumulative sums Window averages for loudness normalization and peak-contrast decisions computed with cumulative sums Linear time independent of window length
O(T·K) Viterbi Exploits the “stay or switch from the maximum” structure; backtracking in Int16Array 289 times fewer operations than O(T·K²) with the extended vocabulary (K = 289)
Cooperative multitasking Yield via MessageChannel every 32 ms Stays a single file without workers; no UI freezes
Minimal screen updates DOM modified only when the bar, beat or chord changes. The waveform shows progress by clipping two canvases drawn once to a CSS width No canvas redraw every frame
Click sound Prebuilt as a buffer and reused; canceled all at once via a schedule list No oscillator graph built per click
Typed arrays Float32Array, Float64Array, Int16Array, Int32Array, Uint8Array Contiguous memory, JIT optimization
Piano synthesis Sine generation by rotation recurrence (no Math.sin calls), computation skipped below −100 dB, each note synthesized once and cached, presynthesized in the order of first appearance in the song 0 main-thread tasks over 50 ms during playback
Piano note memory Only the notes used in the song (15 in the test song), 2.6–5.6 s mono About 13 MB

There is also a reason for not using web workers. Workers need a separate script file or a blob: URL, which constrains single-file distribution and environments with a strict content security policy. Yielding every 32 ms alone keeps the progress display smooth even while analyzing a 5-minute song.

4-6. Verification

Unit tests

Item Method Result
Real FFT Compare a 256-point random signal against a directly computed DFT Maximum error 3.8×10⁻¹³
Resampler accuracy 48 kHz 1 kHz sine → 22,050 Hz, compared with an ideal sine RMS error 7.7×10⁻⁶
Resampler latency Convert an impulse at 1.0 s and check the peak position 1.00000 s (zero latency)
Sample rates 22.05, 44.1, 96 kHz input 100% beat hits, mean error within 0.3 ms
Special inputs 5 s of silence; 1.5 s and 0.2 s clips Empty timeline without errors

Synthetic-song regression tests

Tested with 18 synthetic songs with known ground truth (beats, downbeats, chords). They contain kick, snare, hi-hat, bass and keys with harmonics, a melody with 20% non-chord tones, and a noise floor; some songs have the reference pitch detuned by 18 cents or the tempo wobbling by ±2%. A beat hit means within 70 ms, and the error is detected time − true time. A chord counts as correct only when root, quality and bass are all right at the middle of the beat.

Song BPM (truth) Beat hits Offbeat Mean error / mean absolute error Meter Downbeats Chords
Pop, strong offbeat hi-hat, +18 cents 97.1 (97) 96/96 0 0.1 / 1.5 ms 4/4 24/24 100%
Pop 128.1 (128) 96/96 0 0.5 / 1.7 ms 4/4 24/24 100%
Waltz 150.2 (150) 48/48 0 0.5 / 1.8 ms 3/4 16/16 100%
Triplet ballad 58.0 (58) 48/48 0 −1.4 / 2.0 ms 12/8 12/12 100%
Eighth-note arpeggio ballad, wobbling tempo 67.1 (68) 64/64 0 −1.0 / 1.9 ms 4/4 16/16 98.4%
Rock with weak snare 118.0 (118) 96/96 0 0.0 / 1.6 ms 4/4 24/24 100%
Dance (4-on-the-floor, offbeat open hat) 123.9 (124) 96/96 0 −0.4 / 1.5 ms 4/4 24/24 100%
Hip-hop (sixteenth hi-hat, syncopated kick) 87.9 (88) 64/64 0 0.7 / 2.0 ms 4/4 16/16 100%
Funk 176.2 (176) 96/96 0 0.4 / 1.7 ms 4/4 24/24 100%
Solo piano 72.0 (72) 64/64 0 −3.2 / 3.2 ms 4/4 16/16 98.4%
Solo piano, triple meter 84.0 (84) 48/48 0 −2.6 / 2.7 ms 3/4 16/16 100%
Ballad, last part 15% slower (ritardando) 71.6 (72→63) 80/80 0 −0.8 / 2.0 ms 4/4 20/20 88.8%
Modulation C → D (via A7) 110.2 (110) 116/116 0 0.0 / 1.5 ms 4/4 29/29 100%
Modulation E♭ → E (via B7, ballad) 70.0 (70) 116/116 0 −1.1 / 2.0 ms 4/4 29/29 93.1%
Modulation G → A♭ (direct) 100.0 (100) 112/112 0 0.0 / 1.4 ms 4/4 28/28 100%
A minor verse → C major chorus (not a modulation) 96.0 (96) 64/64 0 −0.2 / 1.5 ms 4/4 16/16 100%
Drums-only intro + pickup bar 104.0 (104) 57/57 0 −0.7 / 2.3 ms 4/4 14/14 100%, N.C. 9/9
Reggae one drop 151.7 (76) 48/48 47 −3.8 / 3.8 ms 4/4 double level 100%

Reggae is detected at the double level, so all the other beats land on eighth-note positions. This is the intentionally unresolved case mentioned earlier; one press of “÷2” gives 76 BPM. The chord errors are a spot where Esus4 → E changes mid-bar and E was detected one beat late, and two ballad-style arpeggio songs (ritardando, E♭ → E) where the seventh inside the arpeggio is weak so maj7 and m7 were read as triads (the result is the same with local tempo or modulation tracking turned on or off, so it is unrelated to beat and key tracking).

Modulation tests

All 14 synthetic songs without a modulation came out in a single key (0 false modulations). The ground-truth modulation position is the bar where the tonic chord of the new key begins.

Song Truth Result
C major → D major, via A7 D major from bar 18 Bar 18 beat 1, error 0 beats
E♭ major → E major, via B7 (ballad) E major from bar 18 Bar 18 beat 1, error 0 beats
G major → A♭ major, direct without a pivot A♭ major from bar 17 Bar 17 beat 1, error 0 beats
A minor verse → C major chorus No modulation (same key signature) No modulation
Song starting with a pickup bar Position of the starting key Bar 0 beat 4 (the actual beat within the pickup bar)

Spelling after the modulation was also checked. The D major section is written with F♯m, the E major section with G♯m7 and C♯m, and the A♭ major section with D♭ and B♭m7.

Camelot notation tests

Item Result
Numbers for the 24 keys checked against the standard Camelot wheel table 24/24 match
Conversion by chord type C → 8B, Am → 8A, G7 → 9B7, Dm7 → 7A7, Cmaj7 → 8Bmaj7, Esus4 → 12Bsus4, Bdim → 10Adim, Bm7♭5 → 10A7♭5, Bdim7 → 10Adim7, G/B → 9B
Switching notation on a real recording (「세월이 가면」 (Sewori gamyeon)) Key E♭ major (5B) → E major (12B), modulation label “12B로” (Korean for “to 12B”). The chart, current and next chord, text chart and screen reader names all switch to Camelot, and they switch back when note names are chosen
Analysis results Changing the notation leaves beat, downbeat and chord decisions unchanged (identical results for the 18 synthetic songs)

Roman numeral tests

Synthetic audio with clear chords was analyzed to check that the Roman numerals of the recognized chords match the notation of harmony textbooks.

Key Chords Result
A minor Am, Em/G, F, C/E, Dm, G, C, E7 i, v6, VI, III6, iv, V/III, III, V7
C major (secondary dominants, borrowed chords) C, E7, Am, D7, G7, C, A7, Dm, G7, C, Fm, C, B♭, C I, V7/vi, vi, V7/V, V7, I, V7/ii, ii, V7, I, iv, I, ♭VII, I
E♭ major (secondary ii, inversions) E♭, B♭/D, Cm, B♭m7, E♭7, A♭maj7, B♭7/A♭, Gm7, Cm7, Fm7, B♭7, E♭ I, V6, vi, ii7/IV, V7/IV, IVM7, V⁴₂, iii7, vi7, ii7, V7, I

Of the 105 chords in the real recording (「세월이 가면」), 10 were written as secondary chords (ii7/IV, V7/IV, V7/ii, etc.) and 16 as inversions, and the B7 just before the modulation was written as V7 of the new key, E major. The key decisions for the 18 synthetic songs with a clear key were all correct with the current method that blends chords and note distribution (the previous method using only the note distribution picked E minor for an A minor progression, getting 17).

Real-recording test

Tested with one commercial recording (기현 (Kihyun) 「세월이 가면」, OST of 응답하라 1988 (Reply 1988), 3 min 50 s). Since there are no ground-truth beat annotations, the residual of fitting the beats in each section to a straight line was used as a proxy for tracking error, exploiting the fact that it is a studio recording with a steady tempo.

Item Before Now
Beat level 135 BPM (eighth notes as beats) 68.1 BPM
Chart for 36–40 s A♭maj7 / Gm7 B♭/F │ Fsus4 / B♭ / (changing every half beat) A♭maj7 / / B♭/F │ Gm7 / E♭ /
Bars of different length 3 (3 beats, 2 beats, 3 beats) + ending 1 + ending
Piecewise linear residual RMS 96–222 ms (fixed penalty 100, at the correct tempo) 6–23 ms
Click intervals for 33–48 s (browser) 0.80–1.02 s 0.86–0.90 s
Modulation Not shown; after the modulation still spelled relative to E♭ (A♭m7, etc.) E major from bar 45 beat 1 (2:37) (at the E of B♭7 → B7 → E), spelled G♯m7 and C♯7 thereafter

Real-browser end-to-end tests

The page was actually operated in headless Chromium. AudioBufferSourceNode.prototype.start was intercepted to record every scheduled click, and using the most recent music start information, the in-song time ( $o_s + t^\text{click} - t_s$ ) was reconstructed and checked against the ground-truth beats. The runs mixed playback, bar skipping and stopping.

File Clicks On beat Offbeat Mean / max error Downbeat accent position
Pop 97 BPM 37 37 0 0.1 / 8.1 ms 22/22 on the actual downbeat
Pickup bar 104 BPM 29 29 0 1.9 / 9.7 ms 16/16
Ballad 68 BPM (automatic halving correction) 22 22 0 1.9 / 8.5 ms 13/13
Correction control Result
Shift by ½ beat Clicks 4/4 move to the midpoints of the true beats; pressing once more returns to 5/5 on beat
×2, then ÷2 Interval 310 ms (beats + midpoints) → back on beat at 622 ms
Delay by one beat Downbeat accent moves to the 2nd beat
Click timing +30 ms Clicks scheduled 29.2 ms later
Meter set to 3 3/4, three slashes per bar
Metronome off (M) 0 scheduled clicks

In addition, 0 console errors and 0 px of horizontal overflow at desktop and phone widths were confirmed. A 63-second song took about 1.6 s from file selection to chart display (headless Chromium), and analysis of a 5-minute song took about 2.7 s (Node 22) (measured on Linux).

Chord playback tests

Scheduled piano notes were intercepted the same way. The pitch of each note buffer was estimated (roughly found by autocorrelation, then the fundamental partial frequency measured precisely with a golden-section scan), and notes starting at the same time were grouped into one strike and compared with the ground-truth chord. No test-only markers were added to the app code.

Item Result
When the chord changes All 18 strikes match the chord tones; bass 18/18 match
Strike timing Error from the actual chord change time: mean 1.4 ms, max 8.1 ms
Starting playback mid-chord All 10 restart strikes are the correct chord at that moment
Every bar, every beat 3/3 and 5/5 correct; beat timing error mean 3.0 ms and 0.6 ms
Piano reference pitch For a song tuning of +18 cents, the fundamental partials of 11 notes at +17.7 to +18.3 cents
Turning off with the P key 0 piano notes scheduled afterward; the metronome continues
Effect on the metronome Even with the limiter, clicks 37/37, 29/29, 22/22 on beat, and downbeat accents all correct
Main thread 0 tasks over 50 ms during playback
Loudness and ringing The app’s actual synthesis function and sound path were rendered with OfflineAudioContext and compared with the previous version at the same slider setting. First-second RMS +5.1 to +6.5 dB (about 1.8–2.1 times), 1–2 s tail +2.7 to +3.0 dB, 10 dB decay time 1.3–1.8 times (C, G7, Am). Peaks at −7 dBFS at the default volume, −2.3 to −3.0 dBFS at maximum volume

Extended song-type tests (1.5.0)

17 songs of types not covered by the 18 regression songs were added and compared against 1.4.2: 10 popular-style songs (sudden tempo change, rubato piano, bass outside the chord, tape pitch drift, K-pop/EDM, pop-rock power chords, R&B slash ballad, hip-hop with loud drums, rock with very loud drums, acoustic strummed pop) and 7 special-genre songs (5/4, 7/4, swing jazz, bebop, bossa nova, metal, ambient).

The verification criterion is to prioritize popular music. A change that worsens results for popular-music songs (the 18 regression songs, popular-style songs, real recordings) is not adopted even if special genres improve, and a change that improves popular-music results is adopted even if special genres get slightly worse.

Group Item 1.4.2 1.5.0
18 regression songs Beats, downbeats, chords, key 100%, 100%, 98.6%, 18 songs Same
10 popular-style songs Beats 95.7% 99.6%
  Offbeat clicks 16 1
  Downbeats 95.7% 99.5%
  Chords (exact) 88.1% 97.0%
7 special-genre songs Beats 80.0% 93.8%
  Downbeats 74.4% 93.2%
  Chords (exact) 52.7% 56.0%
Real recording Agreement with 1.4.2   Chords 100%, downbeats 66/66, beats 257/257

Per song, beats in the sudden-tempo-change song went 64→93/96 (offbeat 16→1), the meters of 5/4 and 7/4 are judged exactly (downbeats 16/16, 12/12), bebop beats went 24→96/96, the bass-outside-the-chord song 50→100%, the R&B slash ballad 82→91%, and the pop-rock power chord song 50→88%.

Methods compared but not adopted

Method Popular music Special genres Decision
9th/13th extension variants of seventh chords 18% of the real recording’s chords changed (A♭→A♭maj7, etc.) Swing jazz family agreement 28→32% Not adopted
Harmonic-percussive source separation (HPSS) No change; 3% of the real recording’s chords changed; analysis time doubled Exact chords 56→61% Not adopted
NNLS note-decomposition chroma Regression chords 98.6→97.4%, popular-style 97.0→95.7% Exact chords 56→63% Not adopted
Bar pointer model Regression offbeat clicks 47→304, downbeats 100→56.7% Downbeats 93→44% Not adopted

The bar pointer model is originally meant to be used with beat and downbeat activations produced by a trained neural network, so with handcrafted onset and accent curves it could not distinguish beat level and meter. All four methods remain in the engine as disabled settings, so the same comparison can be run again.

Meter change and tempo display tests (1.6.0)

Song Truth Result
Pop, 4/4 → 3/4 → 4/4 (8 bars each) 3/4 from bar 9, 4/4 from bar 17 Same; downbeats 24/24
Waltz verse → 4/4 chorus 4/4 from bar 9 Same; downbeats 20/20
Sudden tempo change 96 → 132 132 from bar 13 beat 1 Same
Ballad slowing by 15% in the last part Last part gradually 72 → 63 Gradually 72 → 63 from bar 11 beat 2 to bar 20 beat 4 (hexagonal frame), white pill 63 at bar 20 beat 4
35 songs with a constant meter and the real recording No meter change 0 false meter changes
The remaining 36 songs and the real recording No gradually changing section 0 hexagonal frames

Even with the meter change model turned on, the beats, downbeats, chords and keys of the 37 songs compared with 1.5.0 were all the same, and the real recording matched 1.4.2 with chords 100%, downbeats 66/66 and beats 257/257. In the rubato piano whose tempo wobbles randomly by up to ±12%, one marker appeared at a single spot that suddenly sped up by more than 12%.

Real-recording benchmark (1.10.0)

In 1.10.0, while fixing the beat-level, downbeat and chord-naming rules, the results were compared with 1.9.4 on real recordings with human-annotated ground truth. GTZAN has beat and downbeat ground truth for 998 30-second excerpts across 10 genres, and GuitarSet has chord, beat and downbeat ground truth for 180 solo guitar accompaniments. Beats and downbeats were scored by F-measure with one-to-one matching within ±70 ms, and chords were counted by whether the root and major/minor family (maj, 7, maj7 as major; m, m7 as minor) were right at the middle of each true beat (for GuitarSet, the lead-sheet ground truth). The popular-music group consists of GTZAN’s blues, country, disco, metal, pop and rock, and GuitarSet’s rock, singer-songwriter and funk.

Data Item 1.9.4 1.10.0
GTZAN popular music, 600 songs Beat F / downbeat F 0.829 / 0.557 0.837 / 0.608
  Songs at the correct tempo and on beat / songs at half tempo 452 / 54 465 / 39
All 998 GTZAN songs Beat F / downbeat F 0.786 / 0.511 0.794 / 0.556
  Songs written in 3, 5 or 7 beats 221 104
GuitarSet popular music, 108 songs Beat F / downbeat F 0.656 / 0.550 0.663 / 0.568
  Chord root / major-minor 81.7% / 68.5% 81.9% / 69.0%
All 180 GuitarSet songs Chord root / major-minor 77.0% / 66.8% 77.8% / 67.9%

GTZAN downbeat F by genre rose across the board: blues 0.328 → 0.361, country 0.778 → 0.794, disco 0.676 → 0.708, metal 0.363 → 0.476, pop 0.676 → 0.717, rock 0.519 → 0.593, hip-hop 0.482 → 0.541, reggae 0.634 → 0.679, jazz 0.349 → 0.360, classical 0.293 → 0.315. Per song, among the 600 popular-music songs, downbeat F rose by more than 0.05 in 61 songs and fell in 34. Of those that fell, 22 are cases where a 4/4 song previously mis-written as 7/4, 5/4 or 3/4 was corrected to 4/4 and lost a few downbeats that had matched by chance; only 8 songs got worse downbeat positions with the same meter. In GuitarSet, two funk songs (114 BPM) were corrected from half tempo to the correct tempo, so the beats became denser and the chord root dropped by 6–9 percentage points.

Three popular-song recordings were also checked. 「고민중독」 (Gomin jungdok) was scored against a ground truth made by aligning a published chord chart bar by bar to the recording’s drums and bass (only the 62% of sections that could be confirmed with confidence), and for 「크리스마스의 기억」 (Keuriseumaseu-ui gieok) only the known tempo (170 BPM) and meter were checked.

Song Item 1.9.4 1.10.0
QWER 「고민중독」 (truth 178 BPM, 4/4, 129 bars) Tempo 89.0 (half tempo on the snare) 177.9
  Beat F / downbeat F / irregular bars 0.64 / 0 / 9 0.96 / 0.95 / 3
  Chord root / major-minor (per beat) 61.6% / 50.5% 85.5% / 83.5%
  Bar-level chords 66.7% 91.3%
  Share of names with embellishments like 7 or sus 91% 67%
  Key E minor → F minor G major → A♭ major (from bar 104, matches the ground truth)
Warak 「크리스마스의 기억」 (170 BPM) Tempo / irregular bars 85.0 / 3 170.1 / 2 (the song’s first and last bars)
기현 (Kihyun) 「세월이 가면」 (Sewori gamyeon) Agreement with 1.4.2 (chords, downbeats, beats) 97.7%, 66/66, 257/257 Same

The beats, downbeats, chords and keys of the 37 synthetic songs (18 regression, 10 popular-style, 9 special-genre) and of 37 previously unseen synthetic songs are the same as 1.9.4 on a per-song basis.

Verification on 5 real recordings (1.10.3)

Tempo, meter, bar positions, key and chords were checked on five popular songs of different genres and languages. Bar positions were compared with an independent analyzer (madmom: trained beat and downbeat tracking and major/minor chord recognition). Chords were scored every 0.1 s (root, major/minor family) against a ground truth made by aligning published chord charts to the recording beat by beat, with spots where the chart and the recording differ (cover arrangements, etc.) decided by measuring the bass and harmony bands. The ground truth and madmom results are not distributed because of copyright; only the scores are given. 「여래아」 (Yeoraea) has no published chord chart, so only tempo, meter, bars and key were checked.

Song Tempo (published value) Meter Downbeat agreement (madmom) Key Chord root / major-minor 1.10.2 → 1.10.3 madmom, same scoring
여자친구 (GFRIEND) 「시간을 달려서」 (Sigan-eul dallyeoseo) 112.0 (112) 4/4 0.96 → 0.99 F minor 89.0% / 86.2% → 89.5% / 86.8% 83.7% / 84.0%
「もう恋なんてしない」 cover with female vocals (7 semitones higher and about 5% faster than the original) 109.9 4/4 0.99 B major 90.1% / 92.9% → 90.4% / 92.9% 84.1% / 88.1%
あいみょん 「マリーゴールド」 106.0 (106) 4/4 0.96 D major 89.4% / 88.9% → 90.4% / 89.9% 92.9% / 92.9%
YOASOBI 「夜に駆ける」 130.0 (130) 4/4 1.00 C minor → B minor → D minor 88.6% / 80.6% → 88.6% / 80.5% 83.4% / 82.7%
시드 사운드 (Seed Sound) 「여래아」 90.0 4/4 1.00 C minor → D minor (1.10.2: C minor → F major) No chart -

All five songs have the correct tempo level and meter, and their bar positions agree with madmom and the chord charts. The first two bars of 「시간을 달려서」 were corrected in 1.10.3, and the intro barlines of 「マリーゴールド」 in 1.10.4. madmom also places the intro of 「マリーゴールド」 differently (1.78 and 4.78 s), so the downbeat agreement remains 0.96; the barlines were verified from the period and starting point at which the intro riff repeats. The modulations down and up a semitone in 「夜に駆ける」 really exist; the key a semitone lower was shown two bars early (bar 102) up to 1.10.3, and from 1.10.4 it is shown at bar 104, where the B minor chorus starts. Most remaining chord errors are cases where the vocal melody or sustained notes mix into the harmony band so that a neighboring chord with many common tones is read (A → Bm7/A, G7 → Gm7, C → Cm7, etc.), short chords lasting half a bar or one beat being missed, and a sus4 resolving to a triad being merged into a single chord. Reducing the weight of the high register raised the root by 0.6–1.1 percentage points on these five songs, but it was not adopted because it made the synthetic R&B ballad, GuitarSet and 「고민중독」 (Gomin jungdok) worse.

1.10.4 (intro grid, refined modulation boundaries) did not change the chords, bars or key boundaries of the five songs, and the synthetic 37 songs, the 18 regression songs, the key and Roman numeral tests, GuitarSet, the chords, downbeats and beats of 「고민중독」 and 「세월이 가면」 (Sewori gamyeon), Warak, and the GTZAN scores all stayed the same. The key boundary in 「고민중독」 moved one bar toward the ground truth, and in 9 places among the 1000 GTZAN 30-second excerpts without key ground truth (3 classical, 4 metal, 1 each jazz and rock) and in 1 GuitarSet song, the modulation marker moved 1–2 bars later (in short excerpts these mostly look like modulations that are not really there).

Other test results for 1.10.3: the 37 synthetic songs are the same per song; 「고민중독」 root 85.5% → 85.9%, major-minor 83.5% → 83.8%; GuitarSet popular music root unchanged (81.9%), downbeat F 0.568 → 0.574; the 6 GTZAN popular genres with beat and downbeat F unchanged and irregular bars 711 → 695; 「세월이 가면」 unchanged at 97.7%, 66/66, 257/257 against 1.4.2.

Verification kit of 10 real recordings (1.11.0)

To the five songs of 1.10.3 were added five songs of different genres and tempos (김종국 (Kim Jong-kook) 「이 사람이다」 (I saramida), 015B 「이젠 안녕」 (Ijen annyeong), a live cover of 박명수 (Park Myung-soo) 「바다의 왕자」 (Bada-ui wangja), and female-vocal covers of 디셈버 (December) 「별이 될게」 (Byeori doelge) and 피노키오 (Pinocchio) 「사랑과 우정 사이」 (Sarang-gwa ujeong sai)) and 「세월이 가면」 (기현 (Kihyun), OST of 응답하라 1988 (Reply 1988)), making a verification kit of 10 real recordings with chord ground truth. For each song, two or more published chord charts were aligned to the beats of the recording, spots where the charts and the recording differ were decided by measuring the bass and harmony bands, and everything was separately and skeptically rechecked (1–9 fixes per song), producing it together with beat, downbeat and key ground truth. The ground truth is not distributed because of copyright. 「여래아」 was left out because it has no published chart, and 「고민중독」 is scored separately.

Rules and values were chosen using only the first four songs, 「고민중독」, GuitarSet and synthetic songs; the five new songs and 「세월이 가면」 were not looked at while choosing values and were measured only at the end.

Song Chord root / major-minor 1.10.5 → 1.11.0 madmom, same scoring
여자친구 「시간을 달려서」 89.3% / 86.5% → 91.8% / 90.0% 83.7% / 84.0%
「もう恋なんてしない」 cover 90.4% / 92.6% → 92.8% / 94.1% 84.1% / 88.1%
あいみょん 「マリーゴールド」 90.4% / 89.8% → 97.0% / 94.9% 92.9% / 92.9%
YOASOBI 「夜に駆ける」 88.6% / 80.5% → 90.7% / 85.0% 83.4% / 82.7%
김종국 「이 사람이다」 87.6% / 83.5% → 90.0% / 85.6% 83.3% / 80.9%
015B 「이젠 안녕」 85.8% / 83.2% → 91.0% / 88.8% 83.2% / 82.7%
박명수 「바다의 왕자」 live cover 94.0% / 93.6% → 99.3% / 99.3% 90.3% / 90.3%
디셈버 「별이 될게」 cover 90.4% / 89.8% → 90.9% / 88.5% 88.6% / 88.5%
피노키오 「사랑과 우정 사이」 cover 97.7% / 96.2% → 98.0% / 95.8% 91.1% / 91.9%
기현 「세월이 가면」 88.4% / 84.8% → 94.7% / 91.2% 90.8% / 89.8%
First 4 songs (used to choose values) 89.7% / 87.2% → 93.3% / 91.0% 86.5% / 87.3%
Last 6 songs (not looked at while choosing values) 90.8% / 88.6% → 94.0% / 91.5% 88.0% / 87.5%
All 10 songs 90.4% / 88.0% → 93.7% / 91.3% 87.4% / 87.4%

Scoring checks, at every 0.1 s sample, whether the root and major/minor family of the engine’s chord equal one of the ground-truth labels; places with no chord (before the first downbeat, at the end) also count as wrong. The table in the 1.10.3 section uses the old scoring, which differs in how it finds the chord at a given time and in excluding such samples from the major/minor denominator, so its values differ by 0.1–0.7 percentage points.

The root improved in all 10 songs, and samples with a wrong root fell from 9.6% to 6.3% (those read as a neighboring chord with many common tones 5.0% → 3.6%). Major-minor dropped slightly in 「별이 될게」 and 「사랑과 우정 사이」. The share matching both the root and the quality (major, minor, 7, maj7, m7, sus, etc.) rose from 64.3% to 66.0% (this share is inherently low for the first 4 songs, whose ground-truth charts write only triads). Beats, downbeats and keys stayed the same in all 10 songs. 「이젠 안녕」 (truth 71 BPM) and 「사랑과 우정 사이」 (64 BPM) are detected at double tempo, counting eighth notes as beats; broadening the conditions of the beat-level check would push more correct-tempo GTZAN songs down to half, so this was not fixed (4-8).

Test criterion for 「세월이 가면」 switched to the ground truth (1.11.0). Until then, regressions on this song were checked by whether it matched the 1.4.2 results (chord agreement 97.7%, downbeats 66/66, beats 257/257), but the 1.4.2 results themselves had a root accuracy of about 88% against the ground truth. From 1.11.0, it is compared against the ground truth, and falling below the recorded values (root 94.7%, major-minor 91.2%, quality 78.1%, beat F 0.948, downbeat F 0.937) counts as a regression; agreement with 1.4.2 is kept only for reference (in 1.11.0, chords 85.9%, downbeats and beats unchanged at 66/66 and 257/257).

Other test results for 1.11.0: the 37 synthetic songs root 96.52% → 96.81%, major/minor family 95.60% → 95.78% (three songs, bebop, swing jazz and the R&B ballad, improved and the rest are the same; beats and downbeats unchanged); 「고민중독」 root 85.9% → 93.3%, major-minor 83.8% → 88.2%; GuitarSet popular music root 81.87% → 82.47%, major-minor 68.97% → 69.72%; all of GuitarSet root 77.87% → 78.35%; the 18 regression songs, key and Roman numeral tests, and the beats, downbeats and meters of GTZAN and Warak unchanged. In GTZAN, the key changed in 23 of the 1000 songs; compared with the published GTZAN key ground truth (Lerch), 7 changed to correct and 5 to wrong (the rest have no ground truth or are wrong either way), raising key accuracy for the 6 popular genres from 55.3% to 56.0% (MIREX score, which gives partial credit for relative keys and fifth relations, 64.4% → 65.0%). Stereo files take about 40% longer to analyze per song (kit of 10 songs, average 2.9 s → 4.0 s, same computer).

1.11.8 verification (a one-beat chord on the bar’s downbeat)

The second bar of 「해질녘 예스터데이」 (Haejilnyeok yeseuteodei) plays A♯m7 on the first beat and D♯m7 from the second beat. Looking at the first beat alone, A♯m7 leads D♯m7 by 3.79, but this is smaller than the penalty for changing chords at the second beat (4.0), so in 1.11.7 the whole bar came out as D♯m7.

At first there was only a score condition: “if another chord leads by more than 3.5 on the first beat and the current chord is better on the second beat, make the first beat that chord.” With that, 102 of the 1,268 test songs changed, and in GuitarSet, which has chord ground truth, all 6 songs whose scores moved got worse. According to the ground truth, all those spots were places where the new chord starts from the first beat, and the other chord seen on the first beat was the reverb of the previous chord mixed with the new chord. Score differences could not separate this song from those songs (GuitarSet 00_Jazz3-150-C had a difference of 3.94, larger than this song). A condition that split the first beat in half and checked whether the chord also leads in the later half-beat was not used, because in this song A♯m7 is actually weak in the later half (0.88).

Whether the two chords were really struck separately was decided by the start of the sound (attack). In this song, the notes found only in A♯m7 newly sound at 2.25 times on the first beat, the notes found only in D♯m7 at 2.03 times on the second beat, and the two chords have different basses. Of the 6 spots that went wrong in GuitarSet, the first-beat attack was weak (below 1.2 times, 3 spots), the original chord was not restruck on the second beat (2 spots), or the two chords had the same bass (1 spot). The attack threshold reuses the same 1.2 times as the held-over (grey) decision.

Condition Songs changed (of 1,268) GuitarSet chord scoring
Score difference 3.5 or more 102 6 songs worse
+ attack and bass conditions, score difference 3.0 or more 51 2 songs worse, 1 song better
+ attack and bass conditions, score difference 3.25 or more 34 No songs changed
+ attack and bass conditions, score difference 3.5 or more (adopted) 23 No songs changed

With 3.25 there were also no worsened spots on songs with ground truth, but since it is close to 3.0, where songs start getting worse, 3.5 was chosen for a margin. With the final condition, 23 songs change (one or two bars each). In 「해질녘 예스터데이」 the second bar is back to A♯m7 D♯m7, and a D♭ was added on the first beat of bar 93 near the end (A♭sus4 Fm7/C B♭m7 → D♭ A♭sus4 Fm7/C B♭m7). In 「고민중독」 from the real-recording kit, B7/F♯ was added on the first beat of one bar around 2:21, but this is a cell where the ground-truth chart has no chord written, so the score is unchanged (the m4a file of the same song is the same). The remaining 20 songs are GTZAN songs without chord ground truth. The 18 regression songs, key and Roman numeral tests, scoring of the 37 synthetic songs, the 「세월이 가면」 ground-truth test, and chord and beat scoring of the 180 GuitarSet songs are the same as 1.11.7, and even the changed songs keep their beats, downbeats, meters and keys.

1.11.7 verification (local misalignment in the intro)

「해질녘 예스터데이」 (Haejilnyeok yeseuteodei), reported by a user, has a piano intro until the drums come in at 10.88 s, but its beats were detected at uneven intervals of 0.40–0.53 s, so the chords were off from the very first bar. Extending the main section’s beat period (0.455 s, 132 BPM) backward lands exactly on the first sound (3.14 s), and nearly all the piano onsets in the intro sit on sixteenth-note positions of that grid (51 of 58 onset peaks within ±25 ms), confirming that the song was made at a fixed 132 BPM from the start. Until now, the tracker followed sixteenth-note anticipations (5.31 s, 9.40 s, etc.) as beats, so beats came up to 0.12 s early. In 1.11.7 the intro beats are placed at 0.455 s intervals, and the third bar changed from G♯m G♯m7 to G♯m7 C♯7 (a progression descending by fifths). However, the second bar changed from A♯m7 D♯m7 to D♯m7: on the correct grid A♯m7 sounds only on the first beat (plus one sixteenth note), so a one-beat chord was merged into its neighbor. This was fixed in 1.11.8 (see the 1.11.8 verification above). The first bar changed from B G♯7/F♯ to G♯m7/B, and outside the intro, when chord scores are pooled across repeated sections, the changed intro beats were used as a pair, so a single beat, the fourth beat of bar 44, changed from Em7 to Cmaj7 (judging by the notes on that beat, Em7 is more plausible). The remaining bars are unchanged.

On the 1,268 test songs, conditions were added one at a time while counting the songs that changed. With only the conditions of steady tempo, local deviation and sixteenth-note fit, 4 songs changed (2 GTZAN songs with unchanged beat and downbeat scores). GuitarSet 05_Funk2-119-G kept the same beat and downbeat scores but gained a bar of different length, and its chord root dropped 0.254 → 0.222 and major-minor 0.316 → 0.263, so it was excluded by adding a fixed-tempo evidence condition (at least 30 onset peaks, sixteenth-note fit of at least 0.85). This song is guitar alone from start to finish, with only 19 onsets in the intro and a fit of 0.84. With the final conditions, the only song that changes is 「해질녘 예스터데이」; the remaining 1,267 songs, the 18 regression songs, the key and Roman numeral tests, scoring of the 37 synthetic songs, the real-recording kit, the 「세월이 가면」 (Sewori gamyeon) ground-truth test, and GTZAN and GuitarSet scoring are the same as 1.11.6. Since no other test song has an intro of the same shape (sixteenth-note anticipations in a fixed-tempo song that starts without drums), how many other songs this rule fixes has not yet been measured.

1.11.6 verification (3:2 beat-level check)

Using the same method as 1.11.5, for each of the 119 songs, among the 125 whose beats appear split into three, that the 4:3 check does not change, the result re-tracked at a 3/2 times faster tempo was prepared, and conditions were added one at a time.

Condition Songs changed Better Worse (beats or downbeats)
Change all, no conditions 119 34 84
New beat not triplet, and 60–240 BPM 107 32 75
+ harmonic-change grouping $G \ge 0.05$ 48 26 22
+ score ratio without the prior distribution at least 0.93 12 9 3
+ accent ratio at new beat positions at least 1.05 9 8 1 (1 guitar accompaniment song, downbeats)
+ new beat clearly split into two eighth notes (1.4 times) 8 8 0 (by beats and downbeats)
+ new tempo at least 90 BPM (adopted) 7 7 0

Running GuitarSet chord scoring at the 8-song stage showed that 02_BN2-166-Ab (12/8, 56 BPM → 4/4, 83 BPM, half of the true 166) improved in beats and downbeats but its chord major-minor accuracy fell from 0.44 to 0.22, so it was excluded by raising the lower bound for the new tempo from 60 to 90 BPM. The 7 changed songs are as follows.

Song What changed Compared with the ground truth
GTZAN metal.00017 12/8, 144 BPM → 4/4, 216 BPM (truth 217) Beat F 0.38 → 0.76, downbeat F 0.04 → 0.70
GTZAN metal.00034, metal.00094 (the same audio included twice) 12/8, 107 BPM → 4/4, 160 BPM (truth 159) Beat F 0.38 → 0.88, 0.37 → 1.00 / downbeat F 0.12 → 0.35, 0 → 0
GTZAN rock.00064 12/8, 102 BPM → 5/4, 152 BPM (truth 152; the ground truth also has 5-beat bars) Beat F 0.39 → 1.00, downbeat F 0.21 → 1.00
GuitarSet 01_Rock1-130-A 12/8, 87 BPM → 4/4, 130 BPM Beat F 0.40 → 0.99, downbeat F 0.29 → 0.96, chord root 0.83 → 0.96, major-minor 0.81 → 0.96
GuitarSet 02_Rock3-148-C 6/8, 98 BPM → 4/4, 150 BPM Beat F 0.40 → 0.60, downbeat F 0.28 → 0.63, chord root 0.87 → 0.97, major-minor 0.76 → 0.87
GTZAN jazz.00086 12/8, 137 BPM → 5/4, 206 BPM (double the true 104) Beat F 0.28 → 0.59, downbeat F 0.12 → 0.17

The first 6 songs reached the correct tempo, and jazz.00086 (actually a song with a 4:3 error) is now at double the correct tempo, so it can be fixed with the “÷2” button in the correction controls (“보정”) (before, it was 12/8 and could not be fixed with the button). There are 28 test songs with the same error (written as 12/8 or 6/8 with a tempo 2/3 of the truth), 10 in GTZAN and 18 in GuitarSet, and 6 of them reached the correct tempo. The remaining songs failed at least one of the conditions (mainly the score ratio of 0.93). Lowering this threshold would also change guitar accompaniment songs (00_BN1-129, etc.) whose tempo would be right but whose beats would stick to the offbeats, making them worse, so it was not widened. When all 1,268 songs were analyzed separately and compared with 1.11.5 (lower bound 60 BPM), everything except the 7 songs above and 02_BN2-166 was the same, and after raising the lower bound to 90, the 125 songs whose beats appear split into three were run again, confirming that only 02_BN2-166 returned to its 1.11.5 result. In the chord scoring of the 180 GuitarSet songs, only these 2 songs improved and the rest are the same; the 18 regression songs, key and Roman numeral tests, scoring of the 37 synthetic songs, the real-recording kit, the 「세월이 가면」 ground-truth test and GTZAN keys are the same as 1.11.5.

1.11.5 verification (4:3 beat-level check)

On the same 1,268 test songs as 1.11.4, for each of the 125 songs whose beats appear split into three (written as 12/8 or 6/8), the result re-tracked at a 4/3 times slower tempo was prepared, and conditions were added one at a time while counting the songs that changed. For songs with ground truth, a drop in either beat F or downbeat F counted as worse.

Condition Songs changed Better Worse (beats or downbeats)
Change all, no conditions 124 26 96
New beat not triplet, and 50–140 BPM 97 25 70
+ score ratio without the prior distribution at least 0.93 14 5 + 1 song without ground truth 8 (7 guitar accompaniment songs, 1 very fast song)
+ new beat clearly split into two eighth notes (1.4 times) 9 5 + 1 song without ground truth 3 (2 guitar accompaniment songs, 1 very fast song)
+ nearly tied in the initial estimate as well (0.85) 6 5 + 1 song without ground truth 0
+ harmonic-change grouping $G \ge 0$ (adopted) 6 5 + 1 song without ground truth 0

The 6 changed songs are as follows.

Song What changed Compared with the ground truth
「아주 먼 옛날」 (Aju meon yennal) (user report) 12/8, 92 BPM → 4/4, 69 BPM; bars of different length 4 → 0 (excluding the last bar) No ground-truth file (chords fall on beats 1 and 3 of each bar and progress evenly, like C–Em–Dm7–G7)
GTZAN pop.00004 6/8, 107 BPM → 4/4, 80 BPM Beat F 0.30 → 1.00, downbeat F 0.17 → 1.00
GTZAN pop.00062 6/8, 107 BPM → 2/4, 81 BPM Beat F 0.30 → 1.00, downbeat F 0.11 → 0.67
GTZAN hiphop.00035 12/8, 112 BPM → 4/4, 84 BPM Beat F 0.28 → 1.00, downbeat F 0.15 → 1.00
GTZAN rock.00049 12/8, 111 BPM → 4/4, 84 BPM Beat F 0.27 → 0.98, downbeat F 0 → 0 (same)
GuitarSet 00_Rock2-85-F 12/8, 113 BPM → 4/4, 85 BPM Beat F 0.30 → 0.99, downbeat F 0.21 → 0.97

In pop.00004 and pop.00062, the re-tracked beats followed the kick and stuck to the offbeats (beat F 0), so they improved only with the half-beat shift rule. There are 19 songs with the same error (written as 12/8 or 6/8 with a tempo 4/3 of the truth), 11 in GTZAN, 7 in GuitarSet, and 「아주 먼 옛날」, and 6 of them were fixed. The other 13 failed at least one of the conditions above, and widening the conditions further would turn guitar accompaniment songs whose true tempo is 3/2 times faster, and very fast songs, into half tempo or offbeat, so they were not widened. Songs whose beats do not appear split into three skip this check, so their results are the same as 1.11.4. Analyzing all 1,268 songs separately with the 1.11.4 engine and this engine and comparing them also showed everything the same except the 6 songs above, and the 18 regression songs, key and Roman numeral tests, scoring of the 37 synthetic songs, the real-recording kit and the 「세월이 가면」 ground-truth test are the same as 1.11.4.

1.11.4 verification (not making any existing song worse)

The conditions of the three rules in 1.11.4 (meter change check, re-examining bars of different length, key names where m7 and sixth chords overlap) were set so that not a single song got worse among the 1,268 test songs (37 synthetic songs and 37 previously unseen synthetic songs, 15 real recordings (including 11 popular songs with ground truth), 999 GTZAN songs, 180 GuitarSet songs). For each song, both settings, rules on and off, were applied to a single analysis result, and bars, chords, keys, meters and tempos were compared. With the rules off, every song is the same as 1.11.3; with them on, only the following 7 songs changed.

Song What changed Compared with the ground truth
A 4/4 song whose final chorus was written as 5/4, with a 2-beat bar inserted in every chorus (user report) No meter change; bars of different length 27 → 0; key name minor → major No ground-truth file (4/4 and major confirmed by repetition and bass analysis)
GTZAN hiphop.00067, reggae.00022, reggae.00077 Fewer bars of different length Downbeat F 0.56, 0.89, 0.81 → 1.00
GTZAN country.00039, jazz.00074 Key name F♯ minor → A major, E minor → G major Wrong → correct
GTZAN blues.00061 Key name F minor → A♭ major Truth G minor; wrong both before and after (same MIREX score)

No song got worse in beats, downbeats, chords or keys. For keys, it was also ensured that no song’s distance from the ground truth (MIREX weighted score) worsened (with a threshold of 0.1, one song changed from a key a fifth away from the truth to a more distant key, so 0.15 was used). The 18 regression songs, key and Roman numeral tests, scoring of the 37 synthetic songs, the 11-song real-recording kit, the 「세월이 가면」 ground-truth test and GuitarSet are the same as 1.11.3.

What was not adopted without conditions. Used without conditions, the three rules changed many songs: the meter change check 8 songs (3 worse in downbeats), bar re-examination 159 songs (72 better and 34 worse in downbeats), key names 118 songs (26 keys to correct, 29 to wrong). Restricting them to cases with clear evidence in order to protect existing songs meant giving up most of the songs that had improved. Two other approaches were also tested but not adopted: counting toward major only m7 chords whose bass is not the root (76 songs, 17 keys to correct and 13 to wrong), and a naming rule that writes an in-key m7 with its seventh in the bass as a major triad over the bass (D♯m7/C♯ → F♯/C♯; 「바다의 왕자」 (Bada-ui wangja) root 99.3% → 99.1%).

1.11.1 verification (checking chord timing, quality and speed)

Analysis speed. Analysis was made about 1.8 times faster without changing a single bit of the computed results (the same operations in the same order). Sample-rate conversion computes 8 outputs of the same phase side by side, and the FFT runs two stages in one pass and computes only the frequency bins that are used. In Chrome, the total time to analyze and chart 5 real recordings dropped from 20.2 s to 11.3 s (undoing the increase from 1.11.0 and even faster than 1.10.5), and peak memory during analysis is the same as 1.11.0. Identical results were confirmed by comparing every number the engine produces on real recordings (including mono, 3-channel and 48 kHz versions), 90 GuitarSet songs, 74 synthetic songs, various sample rates, lengths and silent inputs, and combinations of disabled switches.

Fixing a key regression. In 1.11.0 the key of 「고민중독」 (Gomin jungdok) came to be written incorrectly as B minor → E minor → F minor (1.10.5 had G major → A♭ major, matching the ground truth). The kit had no key ground-truth file for this song, so it was missed in the 1.11.0 tests. The cause was a single seventh-chord name left by the vocal-melody reduction (C in the intro becoming C7), which made the intro key D major by a score difference of 0.007; it was fixed by filtering out nearly tied modulations. The only places this rule changed are 「고민중독」 and a false modulation in one hip-hop song among the 1000 GTZAN songs (that song differs from the ground truth both before and after); the kit’s 10 songs, the 180 GuitarSet songs, the synthetic songs, and the key and Roman numeral tests are unchanged, keys included.

Quality names. Adding half-diminished seventh names raised quality against the GuitarSet performances from 59.37% to 60.28%, quality on the kit’s 10 songs from 66.0% to 66.2% (m7♭5 in 「이 사람이다」 (I saramida) and 「もう恋なんてしない」), and analysis time per song on node on the same computer averages 4.0 s → 2.1 s. Root, major-minor, beats and keys are unchanged in every test group.

Investigated but not adopted.

Other test results for 1.11.1: the 37 synthetic songs and 37 previously unseen synthetic songs, the 18 regression songs, key and Roman numeral tests, Warak, and the beats, downbeats, roots and major-minor of GTZAN and GuitarSet are the same as 1.11.0. The 「세월이 가면」 ground-truth test is unchanged at root 94.7%, major-minor 91.2%, quality 78.1%.

Held-over and change position tests (1.8.4)

Item 1.8.3 1.8.4
Grey at the start of bars in synthetic songs (all newly struck) 49 spots 8 spots, all the last chord’s tail after the song ends
Chord changes on the same beat as the truth / one beat early / one beat late / changes not in the truth 772 / 50 / 19 / 35 776 / 46 / 19 / 35
18 regression songs, chords exact / family 98.6% / 99.8% 98.9% / 100%
Popular-style songs, chords exact / family 97.0% / 99.2% 97.2% / 99.3%
Beats, downbeats, keys, special genres, Roman numeral tests   Same
Real recording   Chords, downbeats and beats the same. Of 6 grey spots, only the 1 newly struck spot (bar 24) is in black
Pad song mixed with held chords (8 spots where the same chord continues)   7 spots correct

Drawing positions on screen were also checked. When 6 songs were opened at desktop and mobile widths, all 306 chord names were the same distance from the slash of their own beat.

4-7. Strengths

4-8. Limitations and future work

Limitations

Worth trying in the future

4-9. References

↑ Back to contents

Changelog

Versioning

Versions are written as three numbers, major.minor.build. The current version is shown next to the title in the app and the help page.

Position When it goes up Example
Major When the app’s name or the way it is used changes significantly. 0 is the development version before the official name was chosen 1.0.0 named Chordity
Minor When a new feature is added. The build number restarts from 0 1.3.0 Roman numeral notation
Build When features stay the same and there are bug fixes, sound or value adjustments, or changes to wording and documentation 1.4.1 refined guidance text

Chordity 1.x

1.11.10 (2026-10-21)

1.11.9 (2026-10-18)

1.11.8 (2026-10-18)

1.11.7 (2026-10-15)

1.11.6 (2026-10-14)

1.11.5 (2026-10-14)

1.11.4 (2026-10-14)

1.11.3 (2026-10-14)

1.11.2 (2026-10-13)

1.11.1 (2026-10-13)

1.11.0 (2026-10-13)

1.10.5 (2026-10-11)

1.10.4 (2026-10-11)

1.10.3 (2026-10-11)

1.10.2 (2026-10-11)

1.10.1 (2026-10-11)

1.10.0 (2026-10-11)

1.9.4 (2026-10-10)

1.9.3 (2026-10-10)

1.9.2 (2026-10-10)

1.9.1 (2026-09-30)

1.9.0 (2026-09-30)

1.8.4 (2026-09-30)

1.8.3 (2026-09-30)

1.8.2 (2026-09-30)

1.8.1 (2026-09-30)

1.8.0 (2026-09-30)

1.7.0 (2026-09-30)

1.6.0 (2026-09-30)

1.5.0 (2026-09-30)

1.4.2 (2026-09-27)

1.4.1 (2026-09-27)

1.4.0 (2026-09-26)

1.3.1 (2026-09-26)

1.3.0 (2026-09-26)

1.2.0 (2026-09-26)

1.1.1 (2026-09-26)

1.1.0 (2026-09-26)

1.0.0 (2026-09-26)

Development versions 0.x (마디 코드)

Before the official name was chosen, the app was built under the name “마디 코드” (“Bar Chords”).

0.4.1 (2026-09-26)

0.4.0 (2026-09-25)

0.3.1 (2026-09-25)

0.3.0 (2026-09-25)

0.2.1 (2026-09-25)

0.2.0 (2026-09-25)

0.1.0 (2026-09-25)

↑ Back to contents

This site is open source. Improve this page.