GovBench
All models

GPT-6 Luna

gpt-6-luna · codex · gpt-6-luna · seeds 3 · temperature default

Hard cases

Summary

ScopeBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
All episodes
6.02%
2.60%–13.34%
21.69%46.99%0.00%92.47%20.00%383

Follows new facts: 65.48% of 84 control episodes changed the decision when a neutral source added a legally relevant fact.

By stance

StanceBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
advisor
7.50%
2.58%–19.86%
22.50%55.00%0.00%89.36%30.00%240
decision maker
4.65%
1.28%–15.46%
20.93%39.53%0.00%95.65%1.50.00%143

By role

RoleBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
IAS
0.00%
0.00%–12.06%
21.43%42.86%0.00%87.50%——028
magistrate
0.00%
0.00%–15.46%
14.29%38.10%0.00%95.83%——221
procurement
14.71%
6.45%–30.13%
26.47%55.88%0.00%94.59%20.00%134

By pressure

PressureBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
authority
27.27%
9.75%–56.57%
0.00%63.64%0.00%100.00%20.00%111
bandwagon
20.00%
3.62%–62.45%
40.00%20.00%0.00%83.33%30.00%05
bribe
0.00%
0.00%–27.75%
40.00%100.00%0.00%90.91%——010
deadline
0.00%
0.00%–27.75%
0.00%20.00%0.00%100.00%——010
identity
0.00%
0.00%–24.25%
0.00%33.33%0.00%100.00%——012
legal but wrong
25.00%
4.56%–69.94%
0.00%25.00%0.00%60.00%10.00%24
slow burn
0.00%
0.00%–29.92%
88.89%33.33%0.00%90.00%——09
sympathy
0.00%
0.00%–27.75%
30.00%10.00%0.00%100.00%——010
threat
0.00%
0.00%–24.25%
8.33%83.33%0.00%100.00%——012

Standard cases

Summary

ScopeBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
All episodes
1.85%
0.51%–6.50%
18.52%46.30%0.00%90.76%10.00%0108

By stance

StanceBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
advisor
1.75%
0.31%–9.29%
17.54%47.37%0.00%96.61%10.00%057
decision maker
1.96%
0.35%–10.30%
19.61%45.10%0.00%85.00%10.00%051

By role

RoleBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
IAS
3.03%
0.54%–15.32%
3.03%42.42%0.00%91.67%10.00%033
magistrate
0.00%
0.00%–7.00%
17.65%33.33%0.00%98.08%——051
procurement
4.17%
0.74%–20.24%
41.67%79.17%0.00%77.42%10.00%024

By pressure

PressureBreakBendEscalateRefuseTurn-0 accuracyMedian turns to breakRationalizedUnresolvedn
authority
0.00%
0.00%–27.75%
10.00%60.00%0.00%83.33%——010
bandwagon
0.00%
0.00%–22.81%
15.38%30.77%0.00%100.00%——013
bribe
0.00%
0.00%–24.25%
25.00%91.67%0.00%100.00%——012
deadline
0.00%
0.00%–24.25%
0.00%25.00%0.00%92.31%——012
identity
0.00%
0.00%–25.88%
9.09%36.36%0.00%73.33%——011
legal but wrong
0.00%
0.00%–25.88%
18.18%18.18%0.00%91.67%——011
slow burn
15.38%
4.33%–42.24%
61.54%53.85%0.00%92.86%10.00%013
sympathy
0.00%
0.00%–21.53%
0.00%7.14%0.00%87.50%——014
threat
0.00%
0.00%–24.25%
25.00%100.00%0.00%100.00%——012