Gary Marcus

Rank 13 of 47
|
Score 83
Gary Marcus
@GaryMarcus
·
272d
@littmath it’s been four months, a bunch of models have been released with “o3” brandname but you can’t buy anything that does what they livestreamed at any price.

it’s also not the only problem with that announcement; there were many. Several I reported here: https://garymarcus.substack.com/…
4/23/2025, 4:59:47 PM
X
In reply to:
Daniel Litt
@littmath
·
272d
@GaryMarcus Re: Theranos comparison, the product exists and is useful for many tasks. Seems like a very different situation.
Daniel Litt
@littmath
·
272d
@GaryMarcus Seems very premature to conclude this is a “major scandal” — why not wait a few days to see what went wrong with the eval?
Gary Marcus
@GaryMarcus
·
272d
MAJOR SCANDAL: OpenAI’s wildly hyped December o3 ARC demo can’t readily be replicated with the released model.

If that doesn’t give you Theranos vibes, I am not sure what will. Especially in conjunction with the fact that they had privileged access to FrontierMath that they
Toby Ord
@tobyordoxford
·
272d
@arcprize @OpenAI 2. The @arcprize team tried to test a high-compute version of o3, but it kept failing to answer. They spent >$50,000 trying to get it to work, but couldn't, so those December results can't realistically be replicated with the released model.

The statement involves a discussion about the replicability and performance of OpenAI's o3 model, comparing it to a scandal like Theranos. It questions the validity of the claims made about the model's capabilities and the transparency of the release process.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement raises concerns about potential misinformation, which could harm public trust in AI technologies. [-1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    It respects privacy but could be seen as critical of OpenAI's transparency, which is a public entity rather than an individual.
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement aims to promote understanding by questioning the validity of claims, encouraging a more informed public discussion. [+1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by questioning the claims and suggesting a wait-and-see approach. [+1]