BT-SFS Validation Appendix v0.2
The BT Sovereign Fundamentals Score (BT-SFS) v0.5 against the sovereign defaults and restructurings that followed, tested on criteria fixed in advance. Release bt-sfs-v0-5-2026-09-27.
Verdict
BT-SFS v0.5 passed all 4 acceptance criteria that were written down before it was ever tested against defaults. The test asked whether the new version separates sovereigns that later default or restructure from those that do not at least as well as v0.4 on the same history, with ranks at least as stable, and without absurd results. The criteria were committed in dc2744e72 (2026-09-27 18:30 ART).
How to read the numbers
- AUC
- Pick one country-year that was followed by a default and one that was not. The AUC is the chance that the score ranks the one heading into default lower. 0.5 is a coin toss; 1.0 means every pre-default year scored below every calm year.
- Weakest quartile and lift
- The lowest-scoring quarter of all country-years. The lift is how many times more often defaults follow a year in that quarter than a year anywhere else.
- Confidence interval
- The range the result would likely fall in with a different mix of countries: countries are redrawn at random thousands of times and the result recomputed. A wide range means few events.
- Rank interval
- How far a country’s rank moves when the weights, the reference ranges and the default penalty are varied within plausible bounds, 5,000 times. The 90% interval covers all but the most extreme 10% of those rankings.
What this does not show. It does not prove forecasting skill. The history is rebuilt from today’s data, not the data available at the time, and the sample is 8 episodes in 7 countries.
| Criterion (spec §9) | v0.4 | v0.5 | Result |
|---|---|---|---|
| 3-year AUC at least v0.4 − 0.02 | 0.9190.847–0.968 | 0.9650.926–0.991 | pass |
| Weakest-quartile lift at least v0.4’s | 63.2×21/90 vs 1/271 | infinite22/90 vs 0/271 | pass |
| Median 90% rank-interval width at most v0.4’s | 5 | 4 | pass |
| Face validityMexico 46.19 vs Argentina 27.84; Senegal, Côte d’Ivoire, Benin publishable; largest penalty −12 | — | 3 checks | pass |
“v0.4” here is the v0.4 method recomputed on the same current data and the same corrected registry, so both columns face the same history. It is not the frozen v0.4 release. AUC intervals are 95% country-cluster bootstraps (2,000 resamples of countries).
1 · Did low scores come before defaults?
Yes, in this sample. In all 22 country-years in the three years before a default or restructuring, the country sat in the weakest quarter of the universe that year.
Each row is one episode. The cells show the country’s BT-SFS rank among publishable countries in each of the three years before the onset (1 = strongest). Shaded cells are in the weakest quartile of that year. What to see: no episode starts from the middle of the table.
| Episode (onset year) | 3 years before | 2 years before | 1 year before |
|---|---|---|---|
| El Salvador 2017missed payment | 2014before the backcast | 2015before the backcast | 37of 48 · 2016 |
| Ecuador 2020external restructuring | 44of 48 · 2017 | 42of 48 · 2018 | 41of 48 · 2019 |
| Suriname 2020external restructuring | 46of 48 · 2017 | 44of 48 · 2018 | 44of 48 · 2019 |
| Zambia 2020missed payment | 45of 48 · 2017 | 47of 48 · 2018 | 48of 48 · 2019 |
| Ghana 2022external restructuring + domestic distressed exchange | 40of 48 · 2019 | 41of 48 · 2020 | 40of 48 · 2021 |
| Sri Lanka 2022missed payment | 39of 48 · 2019 | 40of 48 · 2020 | 43of 48 · 2021 |
| Ukraine 2022external restructuring | 47of 48 · 2019 | 45of 48 · 2020 | 41of 48 · 2021 |
| El Salvador 2023domestic distressed exchange | 42of 48 · 2020 | 46of 48 · 2021 | 42of 48 · 2022 |
The formal test pools all 361 country-years of the panel. The weakest quartile is the 90 lowest scores (at or below 39.78). v0.5 puts all 22 pre-event years there and none of them in the other 271 rows. The lift, the event rate in the weakest quartile divided by the rate elsewhere, therefore divides by zero: it is infinite. That is a count, not a claim of a perfect classifier; it rests on 8 episodes. v0.4 put 21 of 22 there (lift 63.2×); its one miss was Ghana 2019 (score 43.49, cut 43.03).
| Horizon | Rows | Event rows | AUC v0.4 | AUC v0.5 | In weakest quartile, v0.4 | In weakest quartile, v0.5 |
|---|---|---|---|---|---|---|
| 1 year | 445 | 88 episodes | 0.9220.857–0.973 | 0.9390.888–0.986 | 8 of 8 | 8 of 8 |
| 2 years | 403 | 158 episodes | 0.9160.848–0.966 | 0.9560.914–0.989 | 15 of 15 | 15 of 15 |
| 3 years | 361 | 228 episodes | 0.9190.847–0.968 | 0.9650.926–0.991 | 21 of 22 | 22 of 22 |
How the panel is built
- A country-year is labelled an event row if a registry event starts within the next 1, 2 or 3 years (3 is the pre-registered horizon). Rows from an event’s onset through two years after it is resolved are left out (unresolved events: every later year), so a country is not scored on the crisis itself. Later years re-enter and can be labelled by a new event.
- The 3-year panel has 361 rows: 48 countries, score years 2016–2023 (later years cannot yet be scored on a full 3-year window). Argentina contributes no rows: every year falls inside a crisis or cure window.
- Events come from a 25-row registry with a 20-year lookback, source-linked row by row (Bank of Canada–Bank of England sovereign default database as the spine; 6 rows rest on secondary sources and are labelled so).
- Sensitivity: under v0.4’s old rule (drop every year from a country’s first event on), the panel shrinks to 333 rows, 13 event rows and 5 episodes. AUC 0.916 (v0.4) vs 0.964 (v0.5); both put 13 of 13 event rows in the weakest quartile.
Where would a screening line go?
IMF-style dual thresholds set one cut that misses at most 10% of event rows (32.32) and another that raises false alarms on at most 10% of calm rows (35.05). Here the second sits above the first, so the grey zone between them is empty and one line does both jobs: at 35.05 it flags 21 of 22 event rows and 34 of 339 calm rows. Event rate 38.2% below the line (55 rows), 0.3% above it (306 rows); noise-to-signal ratio 0.11. The one event row above the line: El Salvador 2016 (39.20).
Caveat. One episode labels up to three consecutive years, so the 22 event rows are not 22 independent defaults; the real sample is 8 episodes in 7 countries. The cut-offs above are chosen on the same data they are scored on. And the history uses today’s data vintage, which knows about revisions made after the fact.
2 · Is it just re-counting past defaults?
No. The fundamentals alone separate the pre-default years almost as well as the published score; the default-history penalty on its own is weak.
| Score used | AUC | 95% cluster CI | Event rows in weakest quartile |
|---|---|---|---|
| Fundamentals only (score before the default penalty) | 0.950 | 0.896–0.988 | 22 of 22 |
| Default history only (the penalty alone) | 0.679 | 0.478–0.853 | 10 of 22 |
| Published score (both) | 0.965 | 0.926–0.991 | 22 of 22 |
The penalty is not zero in the panel: 28 of 361 rows carry one from an earlier, already-cured episode, including 9 event rows (Ecuador, Ukraine, El Salvador). Serial defaulters are real, and the penalty adds a little. But removing it costs 0.015 of AUC, and the penalty by itself has an interval that reaches down to 0.478, close to a coin toss.
3 · Is it just agency ratings in disguise?
No rating enters the score. The two still agree broadly, as they should if both read the same fundamentals, and where they disagree the metric profile says why.
Composite agency rating (the average of available Moody’s, S&P and Fitch ratings on the S&P scale, 2026-09-27 snapshot) against the published score for all 49 countries: correlation −0.90; one rating notch is worth about 3.7 score points; the score predicts the rating with a residual standard deviation of 1.7 notches. The tables list the five largest disagreements each way, in notches against that fit, with the two metrics that most separate the country from the universe median (metric scores, 0–100).
Agencies more positive than the score
| Country | Rating | BT-SFS | Gap | What the score sees |
|---|---|---|---|---|
| United Arab Emirates | AA | 72.1 | 3.7 | Growth volatility, 10-year 25; Reserves, months of imports 25 |
| China | A+ | 64.6 | 3.3 | Primary balance / GDP 0; Debt / GDP 0 |
| Chile | A | 62.6 | 2.8 | Growth volatility, 10-year 31; Real growth, centred 3-year 34 |
| Hungary | BBB | 49.6 | 2.6 | Real growth, centred 3-year 20; Debt / GDP 7 |
| Mexico | BBB− | 46.2 | 2.4 | Net interest / revenue 30; Real growth, centred 3-year 21 |
Agencies more negative than the score
| Country | Rating | BT-SFS | Gap | What holds the score up |
|---|---|---|---|---|
| Senegal | CCC− | 41.1 | 5.5 | Growth volatility, 10-year 75; Reserves, months of imports 42 |
| Jordan | BB− | 53.5 | 2.2 | Growth volatility, 10-year 97; Reserves, months of imports 46 |
| Oman | BBB− | 66.7 | 2.1 | Primary balance / GDP 100; Debt / GDP 71 |
| Bahrain | B | 43.7 | 2.1 | Governance (WGI GE + RQ) 91; GDP per capita 100 |
| Turkiye | BB− | 52.4 | 2.0 | Debt / GDP 81; FX public debt / GDP 85 |
Descriptive only, not a criterion. Agencies weigh things BT-SFS leaves out by design or for now: sovereign-wealth-fund assets (the Gulf, Chile, China), economic size, reserve-currency status and judgement about politics and data quality. A score that matched ratings exactly would add nothing.
4 · How precise are the ranks?
Precise at the ends, a few places of play in the middle. Half of the countries have a 90% interval of 4 places or fewer.
Each dot is a country’s median rank over 5,000 re-scorings; the bar spans the 5th to 95th percentile. What to see: short bars at the top and bottom, longer ones in the middle, where scores sit close together. The widest: Panama 15–27, Turkiye 15–27, Brazil 16–27.
Each run draws, independently: every metric weight ×0.75–1.25; one of three goalpost sets (the frozen 5th/95th percentiles, 2.5th/97.5th, or 10th/90th of the 2010–2019 reference pool); the default penalty ×0.75–1.25; and the governance value moved within ±1 standard error, from the World Bank’s published 90% bounds. The median interval width is 4 places for v0.5 (49 countries) against 5 for v0.4 (46 publishable countries; v0.4 scales by percentile, so it has no goalposts to vary).
Show the numbers
| Country | Published rank | Median | 90% interval |
|---|---|---|---|
| Saudi Arabia | 1 | 1 | 1–2 |
| Latvia | 2 | 2 | 1–2 |
| United Arab Emirates | 3 | 3 | 3–5 |
| Bulgaria | 4 | 4 | 3–7 |
| Kazakhstan | 5 | 5 | 3–6 |
| Oman | 6 | 6 | 5–8 |
| China | 7 | 7 | 3–12 |
| Peru | 8 | 8 | 5–11 |
| Uruguay | 9 | 9 | 7–11 |
| Chile | 10 | 10 | 7–12 |
| Azerbaijan | 12 | 10 | 8–12 |
| Poland | 11 | 11 | 7–12 |
| Serbia | 13 | 13 | 13–16 |
| Indonesia | 14 | 14 | 13–16 |
| Guatemala | 15 | 15 | 13–20 |
| Paraguay | 16 | 17 | 15–20 |
| Costa Rica | 17 | 17 | 14–21 |
| Jordan | 18 | 19 | 15–24 |
| Turkiye | 19 | 20 | 15–27 |
| Brazil | 20 | 21 | 16–27 |
| Romania | 22 | 21 | 17–28 |
| Panama | 23 | 21 | 15–27 |
| Philippines | 21 | 22 | 18–26 |
| Morocco | 24 | 23 | 19–26 |
| Colombia | 25 | 25 | 22–29 |
| Côte d’Ivoire | 26 | 26 | 21–30 |
| Hungary | 27 | 27 | 21–30 |
| Benin | 28 | 27 | 22–31 |
| South Africa | 29 | 28 | 24–30 |
| Trinidad and Tobago | 30 | 29 | 26–31 |
| Mexico | 31 | 31 | 29–33 |
| Dominican Republic | 32 | 33 | 32–35 |
| Bahrain | 33 | 33 | 31–35 |
| Jamaica | 34 | 34 | 33–36 |
| Senegal | 35 | 35 | 31–38 |
| Nigeria | 36 | 36 | 34–38 |
| Egypt | 37 | 37 | 35–38 |
| Kenya | 38 | 38 | 36–38 |
| Angola | 39 | 39 | 39–42 |
| Pakistan | 40 | 40 | 39–42 |
| Ghana | 41 | 42 | 40–43 |
| Ecuador | 42 | 42 | 40–43 |
| Argentina | 43 | 42 | 39–43 |
| Bolivia | 44 | 44 | 43–46 |
| El Salvador | 45 | 45 | 44–46 |
| Zambia | 47 | 46 | 45–47 |
| Sri Lanka | 46 | 47 | 46–47 |
| Suriname | 48 | 48 | 48–49 |
| Ukraine | 49 | 49 | 47–49 |
The intervals cover the method’s own choices. They do not cover errors in the source data or a wrong choice of metrics.
5 · Was it tuned to the answer?
The test was fixed before the score met the defaults. Three things changed after the first run was seen; none touched the criteria.
- 2026-09-27 18:30 ART
dc2744e72Methodology review and pre-registered spec: metrics, weights, event-panel rule and the four acceptance criteria. - 2026-09-27 20:22 ART
0af6cd3d3Pre-run data record: amendments A1–A7, frozen goalposts, event registry, FX-debt table. - 2026-09-27 20:29 ART
6c9952ab5Amendment A8, still before any run: reserves not applicable for euro-area members. - 2026-09-27 20:39 ART
0018b29d0Pipeline and first run against the event panel (349 rows, 6 event countries): 4 of 4 criteria pass. - 2026-09-27 22:15 ART
A9Amendment A9, after the run: reserves not applicable for Panama (no central bank); goalposts refrozen. - 2026-09-27 22:18 ART
bt-sfs-v0-5-2026-09-27Release built: 49 countries, market date 2026-09-25. - 2026-09-27 22:24 ART
this pageAcceptance test run on the release: PASS.
Amendments logged before the run
- A1 FX debt source order: official dollarisation, then World Bank/IMF QPSD when its total matches IMF debt (0.75–1.35×), then a curated table from IMF country documents or Eurostat.
- A2 Where no accepted FX-debt source exists (Bahrain, China), the metric drops out and the country stays publishable, flagged.
- A3 FX debt in the history backcast: QPSD history, then the World Bank external public-debt proxy, then the latest table share carried back, each flagged.
- A4 Growth-window evidence corrected: the July evidence used realised growth (look-ahead) and was withdrawn. The centred window is kept for smoothing, not prediction; the backcast uses the October WEO forecast of each year.
- A5 Registry rules: bilateral-only official arrears count only if an agency also called a default; one episode is one row (Zambia 2020–2024 and Ukraine 2022–2024 merged).
- A6 The −2 active add-on applies while an event is unresolved or exceptional.
- A7 One shared module computes v0.4, v0.5 and every waterfall step for the build, the backcast, the goalposts and this validation.
- A8 Reserves in months of imports is not applicable to euro-area members (Latvia, Bulgaria); WAEMU members keep the union’s pooled reserves.
What changed after the first run
- How criterion 9.2 is evaluated. When v0.5 put every event row in the weakest quartile, the lift divided by zero and the comparison could not be computed. The code now treats zero events outside the quartile as an infinite lift, which beats any finite one. The criterion itself did not change.
- Reserves where there is no central bank (amendment A9). Panama has no central bank: its reported reserves are a state bank’s liquid assets, not a monetary authority’s buffer, and they sat at the bottom of the scale. Reserves in months of imports is now not applicable there, as for the euro members. This was raised after the results, when Panama’s rank was reviewed. It does not move the test (Panama is never in the weakest quartile). The broader rule, dropping reserves for every dollarised country, was checked and rejected: it lowered the 3-year AUC from 0.965 to 0.943, because Ecuador’s and El Salvador’s thin reserves did flag their defaults. Panama’s ten reference-period rows set the low reserves goalpost on their own, so the goalposts were refrozen without them (1.6 → 2.2 months), which lowers every other reserves score by about one point.
- Which countries are in the universe. The index the universe is drawn from dropped United Arab Emirates and Zambia in a rebalance. BondTerminal still carries their bonds, so a “sticky universe” rule now keeps them on their previous benchmark. That decision followed the index change, not the results, and the test passes either way:
Universe Rows Episodes AUC v0.4 AUC v0.5 In weakest quartile, v0.5 Without them (first run, 6 event countries) 349 7 0.911 0.961 19 of 19 With them (release, 49 countries) 361 8 0.919 0.965 22 of 22
What pre-registration cannot undo: the metrics were chosen in a review that tested each one against defaults on an earlier panel with overlapping episodes. The composite was not run before the criteria were fixed, but its parts were not chosen blind.
Does the weighting do what it says?
A nominal weight is what the method assigns; the main effect (r² of the metric with the score before the penalty, across the 49 release countries) is how much each metric actually moves the cross-section. The share column rescales main effects to sum to 100% so they can be read against the weights.
| Metric | Nominal weight | Main effect r² | Share | Correlation with score |
|---|---|---|---|---|
| Debt / GDP | 10.0% | 0.371 | 13.6% | 0.61 |
| Net interest / revenue | 7.5% | 0.426 | 15.6% | 0.65 |
| Primary balance / GDP | 7.5% | 0.005 | 0.2% | −0.07 |
| FX public debt / GDP | 7.5% | 0.382 | 14.0% | 0.55 |
| Reserves, months of imports | 20.0% | 0.399 | 14.6% | 0.45 |
| GDP per capita | 10.0% | 0.381 | 13.9% | 0.62 |
| Real growth, centred 3-year | 5.0% | 0.005 | 0.2% | 0.07 |
| Growth volatility, 10-year | 7.5% | 0.022 | 0.8% | 0.15 |
| Governance (WGI GE + RQ) | 15.0% | 0.557 | 20.4% | 0.75 |
| Inflation track record | 10.0% | 0.184 | 6.7% | 0.43 |
Primary balance (7.5% nominal, main effect 0.005) and growth (5.0%, 0.005) barely move the cross-section, yet both passed the single-metric distress test. They are not noise: they partly offset other metrics (primary balance correlates −0.26 with net interest and −0.20 with GDP per capita; growth −0.42 with GDP per capita), so their weight shows up in which countries move, not in the overall spread.
Coherence screen (EU Joint Research Centre practice). The score correlates 0.617 with GDP per capita, just over the JRC’s 0.6 flag for income bias (a reported flag, not an acceptance criterion). It was 0.598 before amendment A9 took reserves out of Panama’s score; it is watched at every release. The strongest pair of metrics is GDP per capita and Governance (WGI GE + RQ) (0.71). The mean correlation between metric scores is 0.112, which gives a standardised Cronbach’s α of 0.56: the metrics measure different things, by design, rather than one latent factor.
| Debt | Int | PB | FX | Res | GDPpc | Gr | Vol | Gov | Infl | |
|---|---|---|---|---|---|---|---|---|---|---|
| Debt | · | 0.50 | 0.16 | 0.55 | 0.17 | 0.23 | 0.21 | 0.01 | 0.18 | −0.01 |
| Int | 0.50 | · | −0.26 | 0.27 | 0.27 | 0.55 | −0.11 | −0.21 | 0.43 | 0.15 |
| PB | 0.16 | −0.26 | · | 0.13 | −0.27 | −0.20 | 0.23 | 0.10 | −0.09 | −0.19 |
| FX | 0.55 | 0.27 | 0.13 | · | 0.23 | 0.24 | −0.14 | 0.11 | 0.22 | 0.07 |
| Res | 0.17 | 0.27 | −0.27 | 0.23 | · | 0.02 | 0.16 | 0.20 | 0.13 | 0.22 |
| GDPpc | 0.23 | 0.55 | −0.20 | 0.24 | 0.02 | · | −0.42 | −0.34 | 0.71 | 0.15 |
| Gr | 0.21 | −0.11 | 0.23 | −0.14 | 0.16 | −0.42 | · | 0.34 | −0.07 | −0.01 |
| Vol | 0.01 | −0.21 | 0.10 | 0.11 | 0.20 | −0.34 | 0.34 | · | −0.02 | 0.09 |
| Gov | 0.18 | 0.43 | −0.09 | 0.22 | 0.13 | 0.71 | −0.07 | −0.02 | · | 0.37 |
| Infl | −0.01 | 0.15 | −0.19 | 0.07 | 0.22 | 0.15 | −0.01 | 0.09 | 0.37 | · |
6 · Does it line up with market pricing?
Broadly, yes: weaker scores trade at wider spreads. This is reported, not an acceptance criterion; no metric was ever chosen by how well it explains spreads.
Spread: each country’s near-5-year USD benchmark bond over the 5-year Treasury, market date 2026-09-25. Fit ln(spread) = a + b·score on the 48 publishable countries with a positive spread (China excluded: its benchmark trades through Treasuries, so the log is undefined). a = 6.883, b = −0.0366 (each score point ≈ −3.6% on the spread), R² 0.580. Spearman rank correlation −0.824 (n = 48); v0.4 on the same date −0.852 (n = 45). The weakest-scoring quarter of the fit sample averages 412bps; the strongest quarter 82bps.
The “gap vs fundamentals” is the log distance between the actual spread and the fitted one. A positive gap means the market charges more than the fundamentals alone would suggest. It is a prompt to ask why (politics, data doubts, bond structure, liquidity), not a trade signal.
Market wider than fundamentals
| Country | BT-SFS | Spread | Fitted | Gap, log points |
|---|---|---|---|---|
| Senegalduration far from 5 years | 41.1 | 1722bps | 217bps | +2.07 |
| Argentina | 27.8 | 634bps | 352bps | +0.59 |
| Turkiye | 52.4 | 257bps | 143bps | +0.59 |
| Beninduration far from 5 years | 49.4 | 268bps | 160bps | +0.52 |
| Saudi Arabia | 81.6 | 81bps | 49bps | +0.50 |
Market tighter than fundamentals
| Country | BT-SFS | Spread | Fitted | Gap, log points |
|---|---|---|---|---|
| Philippines | 52.2 | 55bps | 144bps | −0.96 |
| United Arab Emiratesbenchmark carried forward | 72.1 | 29bps | 70bps | −0.88 |
| Zambiasparse curve, benchmark carried forward | 19.6 | 230bps | 476bps | −0.73 |
| El Salvadorsparse curve | 23.1 | 206bps | 419bps | −0.71 |
| Surinameduration far from 5 years | 14.9 | 280bps | 566bps | −0.70 |
One market date, one bond per country. Flags under a name mark a benchmark carried forward from the previous release, a sparse curve or a bond whose duration is far from 5 years. The gaps carry no prediction intervals yet, and defaulted or CCC names are better compared on price against recovery values than on spread; both are planned (V2).
7 · What would prove it wrong?
A default or restructuring that starts outside the weakest quartile. So here is the list, published with the release on 2026-09-27, to be scored in three years against the registry.
The forward test uses the panel’s own rule on the 2026 scores. Countries still inside a crisis or cure window are not scored rows (Ghana, Argentina, Sri Lanka, Zambia, Ukraine), which leaves 44 countries. Their weakest quartile is the 11 below. A registry event starting in 2027–2029 counts against this list; one starting later in 2026 is scored against the 2023–2025 rows instead, as in the panel.
| Weakest quartile, 2026 (eligible countries) | BT-SFS | Rank of 49 |
|---|---|---|
| Suriname | 14.86 | 48 |
| El Salvador | 23.07 | 45 |
| Bolivia | 23.52 | 44 |
| Ecuador | 28.19 | 42 |
| Pakistan | 30.28 | 40 |
| Angola | 31.27 | 39 |
| Kenya | 36.72 | 38 |
| Egypt | 38.44 | 37 |
| Nigeria | 38.45 | 36 |
| Senegal | 41.11 | 35 |
| Jamaica | 42.51 | 34 |
Live watch case: Senegal
S&P rates Senegal CC (changed 2026-09-04), Moody’s Caa2, and a debt reprofiling plan was reported on 2026-09-09. BT-SFS ranks it 35 of 49 (score 41.11): outside the weakest quartile of the published table (ranks 38–49). Its rating sits 5.5 notches below what its score implies, the largest disagreement in the universe.
Scored on the forward-test rule it sits on the edge: 10th weakest of 44 eligible countries, inside the quartile, 2.58 points below Bahrain, the first country outside it. An onset later in 2026 would label its 2023, 2024, 2025 rows, ranked 38, 38, 35 of 49. We commit now to count a Senegal restructuring as a miss whatever the boundary arithmetic says: a fundamentals score should not leave a sovereign that agencies rate in the C range outside its weakest quarter.
The likely reason is the one the IMF’s misreporting case exposed: for years the reported debt was understated, and a score built on reported data cannot see debt that is not reported. Today its fiscal pillar is already among the weakest (17.1) on the revised figures; its external pillar (42.3) uses the monetary union’s pooled reserves, and governance and inflation (68.9) hold the composite up.
8 · Limitations and next phases
- Current-vintage history. The backcast uses today’s IMF, World Bank and WGI data, not what was published at the time. Growth is the exception: it uses each year’s October WEO forecast. WGI history is revised with each release, so the past knows a little about the future.
- Few events. 8 episodes in 7 countries. The intervals above are honest about that, and one more miss would move the headline numbers noticeably. For scale: published early-warning models that are tested out of sample with real-time data typically reach an AUC of 0.70–0.88 (IMF 2021 review of its debt-sustainability framework; Hellwig 2021). An in-sample 0.965 on today's data should be read against that range, not as a forecast.
- Growth-window evidence withdrawn. The July evidence for the centred 3-year growth window used realised growth, which is look-ahead. With the forecasts as published, the window predicts no better than a single year. It stays for smoothing only.
- Broad-stress outcome not computed. The pre-registered secondary label (large IMF programmes, arrears, domestic defaults, inflation above 100%, spreads doubling past 500bps) needs an event set that is not assembled yet.
- No point-in-time backtest. Archived IMF WEO vintages exist, but automated use needs IMF permission, not yet granted.
- What the score leaves out. Sovereign-wealth-fund assets are not counted in reserves (Chile, Kazakhstan and the Gulf are understated). FX debt comes from a curated, source-linked table where standard series fall short.
| Phase | Focus | What it adds |
|---|---|---|
| V1 | Robustness | Partly in this appendix (rank intervals, fundamentals-only and history-only baselines, first-event rule, universe check). Still to do: leave-one-event-country-out, leave-one-pillar-out, the broad-stress outcome, a pillar-level geometric aggregation variant, and a longer registry. |
| V2 | Market history | A historical near-5-year spread panel, the slope and R² as a monthly series, and prediction intervals for the gap vs fundamentals. |
| V3 | Point-in-time data | Scores rebuilt from archived IMF WEO and World Bank vintages, with WGI lagged; waits on IMF permission. |
| V4 | Model selection | Frozen training and holdout windows, and evidence on whether estimated weights beat the hand-set ones. |
9 · Audit trail
The scoring code is not public; the data is. Every published score can be recomputed by hand from the files below and the formulas in the paper.
Public data package · /bt-sfs-paper/data (file list with SHA-256 hashes)
| File | What it holds |
|---|---|
scores.csv | Published scores, pillar scores, default penalty, status, benchmark bond and spread. |
metric-inputs.csv | Every metric per country: raw value, transformed value, 0-100 score, source, indicator and years used. |
goalposts.json | Frozen goalposts (P5/P95 of the 2010-2019 reference pool) per metric. |
goalpost-pool.csv | The reference pool the goalposts were taken from: one row per metric, country and year. |
default-event-registry.csv | Default and restructuring events used for the penalty, each with a source link. |
fx-debt-currency-composition.csv | FX share of public debt for countries not covered by World Bank/IMF QPSD, from IMF documents or Eurostat. |
score-spec-lock.json | Machine-readable specification the release was built with (weights, rules, hashes). |
fundamentals-history.csv | Current-methodology backcast of scores, 2016 onward. |
validation-acceptance.json | Pre-registered acceptance test results and reported statistics. |
validation-event-panel-3y.csv | The 3-year event panel the test was scored on. |
validation-rank-intervals.csv | Monte Carlo median rank and 90% interval per country. |
validation-waterfall.csv | Score and rank at each step from v0.4 to v0.5. |
spread-cross-section.csv | Spread comparison: score, spread, fitted spread and gap vs fundamentals. |
Source: International Monetary Fund (World Economic Outlook, Fiscal Monitor, DataMapper); The World Bank (World Development Indicators, Worldwide Governance Indicators, Quarterly Public Sector Debt); Eurostat. Materially transformed by BondTerminal (BT-SFS). The raw IMF series cached with the release are not redistributed until the IMF grants permission; the per-country values actually used are in metric-inputs.csv.
Check one score by hand: Argentina
Take each metric’s transformed value from metric-inputs.csv and its goalposts from goalposts.json. The metric score is 100 × (value − low) ÷ (high − low), clipped to 0–100 and flipped for lower-is-better metrics. The weighted average is the base score; the penalty from the registry is added on top.
| Metric | Value used | Low | High | Score | Weight |
|---|---|---|---|---|---|
| General government debt / GDP lower is better | 70.4000 | 12.2450 | 83.1950 | 18.0 | 10.0 |
| Net interest / revenue lower is better | 4.5131 | 0.1539 | 33.8433 | 87.1 | 7.5 |
| Primary balance / GDP | 1.8852 | −5.9653 | 4.4477 | 75.4 | 7.5 |
| FX public debt / GDP lower is better | 37.8807 | 2.5669 | 57.0263 | 35.2 | 7.5 |
| Reserves, months of imports | 4.0537 | 2.2200 | 14.2138 | 15.3 | 20.0 |
| GDP per capita, USD | 9.5720 | 7.2324 | 10.0804 | 82.1 | 10.0 |
| Real GDP growth, centred 3-year | 3.9667 | −0.0613 | 7.2003 | 55.5 | 5.0 |
| Growth volatility, 10-year lower is better | 1.6809 | 0.1741 | 1.9316 | 14.3 | 7.5 |
| Governance (WGI GE + RQ) | 55.9389 | 39.8651 | 69.6411 | 54.0 | 15.0 |
| Inflation track record lower is better | 0.3571 | 0.0234 | 0.3027 | 0.0 | 10.0 |
| Base score (weighted average) | 39.84 | 100 | |||
| Default penalty (registry, capped at −12) | −12.0 | ||||
| Published BT-SFS | 27.84 |
Transformed values are logs for GDP per capita and growth volatility, and a stress index for the inflation record (see the paper). Small differences in the last decimal come from rounding in the published file.
How this release was produced
Internally, from BondTerminal’s private repository: the goalposts were frozen once (0af6cd3d3, refrozen with amendment A8 in 6c9952ab5 and with A9 after the run); the release was built on 2026-09-28 from IMF DataMapper and World Bank data pulled 2026-09-27; the validation re-scored the history from the inputs cached with the release. The pre-registered spec is dc2744e72 and the scoring code 0018b29d0. Other data vintages: WGI 2025 revision; IMF WEO historical forecasts file updated 2022-10-18; market date 2026-09-25.
Version history
| v0.2 · 2026-09-27 | Pre-registered acceptance test of BT-SFS v0.5 (bt-sfs-v0-5-2026-09-27). |
| v0.1 · 2026-06-30 | First validation of v0.4 (not pre-registered); regenerated 2026-07-03 against the corrected default registry. |