Gary Marcus

Rank 14 of 47
|
Score 95
4/19/2025, 1:10:31 PM
X
In reply to:
KevinCodeWave
@kcodewave
·
35d
@GaryMarcus o4-mini's FrontierMath leap is impressive – 17% shows real reasoning progress. GaryMarcus, seems the benchmarks have caught up with earlier hype!
Gary Marcus
@GaryMarcus
·
35d
Looks like the 25% o3 allegedly got on FrontierMath in December was possibly bogus.
Epoch AI
@EpochAIResearch
·
35d
On FrontierMath, our benchmark of highly challenging, original math questions, o4-mini with high reasoning sets a new record in our evaluations, with an accuracy of 17% (±2%)!

o3 scores 10% (±2%) with high reasoning, behind o4-mini and o3-mini.

The statement questions the validity of improvements in AI model performance, suggesting that the progress might be due to 'teaching to the test' with synthetic data rather than genuine reasoning advancements. This reflects skepticism about the methods used to achieve reported results.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not directly harm but implies criticism of the methods used, which could be seen as questioning the integrity of the results.
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement does not promote understanding or empathy, as it questions the validity of the results without providing evidence or constructive feedback. [-1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in criticism but lacks constructive dialogue or evidence to support the claim, which could be seen as a mild form of ad hominem against the methods used. [-1]