Gary Marcus

Rank 13 of 47

Score 83

@littmath it’s been four months, a bunch of models have been released with “o3” brandname but you can’t buy anything that does what they livestreamed at any price.

it’s also not the only problem with that announcement; there were many. Several I reported here: https://garymarcus.substack.com/…

4/23/2025, 4:58:58 PM

In reply to:

Daniel Litt

@littmath

272d

@GaryMarcus Seems very premature to conclude this is a “major scandal” — why not wait a few days to see what went wrong with the eval?

Gary Marcus

@GaryMarcus

272d

MAJOR SCANDAL: OpenAI’s wildly hyped December o3 ARC demo can’t readily be replicated with the released model.

If that doesn’t give you Theranos vibes, I am not sure what will. Especially in conjunction with the fact that they had privileged access to FrontierMath that they

Toby Ord

@tobyordoxford

272d

@arcprize @OpenAI 2. The @arcprize team tried to test a high-compute version of o3, but it kept failing to answer. They spent >$50,000 trying to get it to work, but couldn't, so those December results can't realistically be replicated with the released model.

The statement engages in public discourse by discussing the transparency and reliability of AI model releases, specifically critiquing OpenAI's o3 ARC demo. It raises concerns about the replicability of results and the potential overhyping of capabilities, drawing parallels to past tech scandals.

Principle 1:
I will strive to do no harm with my words and actions.
The statement aims to highlight potential issues with transparency and reliability, which could be seen as striving to prevent harm by holding companies accountable. [+1]
Principle 2:
I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
The statement does not engage in personal attacks but critiques the actions of a company, respecting individual privacy. [+1]
Principle 3:
I will use my words and actions to promote understanding, empathy, and compassion.
By raising awareness of potential discrepancies, the statement could promote understanding and caution among consumers and stakeholders. [+1]
Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
The statement engages in criticism of OpenAI's practices, but it does not appear to engage in personal attacks, focusing instead on the company's actions. [+1]