Gary Marcus

Rank 14 of 47
|
Score 95
4/19/2025, 1:17:46 PM
X
In reply to:
Aissa Ait Benamar
@Dxtreame
路
35d
@GaryMarcus You're welcome. Regarding the reliability of O3, you might find this thread interesting if you haven't checked it out yet:
Transluce
@TransluceAI
路
38d
We tested a pre-release version of o3 and found that it frequently fabricates actions it never took, and then elaborately justifies these actions when confronted.

We were surprised, so we dug deeper 馃攷馃У(1/)
Gary Marcus
@GaryMarcus
路
35d
@Dxtreame Clown show. But thank you for the pointer.
Aissa Ait Benamar
@Dxtreame
路
35d
@GaryMarcus Apparently, the O3 that OpenAI has launched is different from the one that was showcased in December :
ARC Prize
@arcprize
路
38d
Clarifying o3鈥檚 ARC-AGI Performance

OpenAI has confirmed:

* The released o3 is a different model from what we tested in December 2024

* All released o3 compute tiers are smaller than the version we tested

* The released o3 was not trained on ARC-AGI data, not even the train
Gary Marcus
@GaryMarcus
路
35d
Looks like the 25% o3 allegedly got on FrontierMath in December was possibly bogus.
Epoch AI
@EpochAIResearch
路
35d
On FrontierMath, our benchmark of highly challenging, original math questions, o4-mini with high reasoning sets a new record in our evaluations, with an accuracy of 17% (卤2%)!

o3 scores 10% (卤2%) with high reasoning, behind o4-mini and o3-mini.

The statement is a simple acknowledgment and expression of gratitude, lacking substantive engagement with public issues or debates. It does not constitute public discourse.