Gary Marcus

Rank 14 of 47
|
Score 83
Gary Marcus
@GaryMarcus
·
271d
Hold up – did the @jacobandreas lab score better on ARC in November than o3-medium did this week?

Why is that not even being discussed?

(arXiv, November 11).
4/23/2025, 9:44:04 PM
X
In reply to:
Flowers
@flowersslop
·
271d
@GaryMarcus So you are saying we should be disappointed that they didnt give us the extremly expensive (and finetuned) preview model but instead shipped an extremly optimized one (100x cheaper!) that still performs reasonably well, certainly better than any other model?
Gary Marcus
@GaryMarcus
·
271d
@flowersslop they made progress, but what they advertised is not what they delivered, even four months later.
Gary Marcus
@GaryMarcus
·
271d
@tobyordoxford that is a key part of the scandal: o3-you-can’-t-actually-have-but-claim-we-have is NOT o3-that-you-can-have, but the two are named with highly confusable names.

And I don’t think there was much public acknowledgement that was advertised December is not the product that was
Flowers
@flowersslop
·
271d
@GaryMarcus Gary, this isn’t a SCANDAL. The released o3 models are actually closer to winning the prize than o3-preview. Pricing is on a log scale.

They might not perform as well (o3-preview was finetuned for ARC AGI), but they’re over 100x cheaper.

If anything, this is very impressive.
Gary Marcus
@GaryMarcus
·
271d
MAJOR SCANDAL: OpenAI’s wildly hyped December o3 ARC demo can’t readily be replicated with the released model.

If that doesn’t give you Theranos vibes, I am not sure what will. Especially in conjunction with the fact that they had privileged access to FrontierMath that they

The statement engages in a discussion about the performance and delivery of AI models, specifically addressing concerns about whether the advertised capabilities were met. It questions the integrity of the delivery and the shifting of expectations regarding the model's performance and cost.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement raises concerns about potential misrepresentation, which could harm trust in AI development. It does not directly cause harm but highlights a potential issue. [-1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement respects privacy and dignity, focusing on the product rather than personal attacks. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism, questioning the delivery and expectations without resorting to personal attacks. [+1]