Gary Marcus

Rank 13 of 47
|
Score 83
4/23/2025, 5:05:14 PM
X
In reply to:
Toby Ord
@tobyordoxford
·
272d
@GaryMarcus If so, then the main problem is the false impression that this 'o3' release gives you access to the kind of results people were losing their minds over in December. I think that is serious enough that they should have given it a different name or at least been super clear.
Toby Ord
@tobyordoxford
·
272d
@GaryMarcus I don't think it is a major scandal, but I do think it is a bad look. My guess is that the December results were a mixture of lots of fine-tuning on the ARC-AGI training set and an early version of o3 that was easier to inference-scale (ability that was traded off for price).
Gary Marcus
@GaryMarcus
·
272d
MAJOR SCANDAL: OpenAI’s wildly hyped December o3 ARC demo can’t readily be replicated with the released model.

If that doesn’t give you Theranos vibes, I am not sure what will. Especially in conjunction with the fact that they had privileged access to FrontierMath that they
Toby Ord
@tobyordoxford
·
272d
@arcprize @OpenAI 2. The @arcprize team tried to test a high-compute version of o3, but it kept failing to answer. They spent >$50,000 trying to get it to work, but couldn't, so those December results can't realistically be replicated with the released model.

The statement is part of a public discourse discussing the transparency and communication issues surrounding the release of AI models by OpenAI. It highlights concerns about misleading naming conventions and lack of public acknowledgment regarding the differences between preview and released versions of the 'o3' model.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement aims to clarify potential misunderstandings, aligning with the principle of doing no harm by addressing misinformation. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    It respects privacy and dignity, focusing on the issue rather than personal attacks. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by explaining the confusion caused by similar naming. [+1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    It engages in constructive criticism by suggesting that clearer communication could have been employed. [+1]