Gary Marcus

Rank 17 of 47
|
Score 93
5/10/2025, 2:33:09 AM
X
In reply to:
Adan Z Becerra, PhD
@AdanBecerraPhD
·
9d
@littmath @GaryMarcus @Wasgo @polynoamial o3 is better because OpenAI finally realized what we had years before which is that next token prediction would fail. It’s obviously been debunked at this point. That right there shows us OpenAI doesn’t have a solid idea on how to get to AGI, whatever that is.
Daniel Litt
@littmath
·
9d
@GaryMarcus @Wasgo @polynoamial I’m referring to o3/gemini 2.5. Clearly reasoning models are dramatically better here. Base models (4o, 4.5, say) are about where I thought they’d be.
Gary Marcus
@GaryMarcus
·
9d
@littmath @Wasgo @polynoamial On base models? With or without inference time compute? Without domain specific data augmentation?

I think the capabilities are better but not sure it is a pure function of increase in compute and data.
Daniel Litt
@littmath
·
9d
@GaryMarcus @Wasgo @polynoamial I think math capabilities dramatically exceed what I expected when GPT-4 came out in March 2023. I didn’t have strong priors about capabilities in other areas.
Gary Marcus
@GaryMarcus
·
9d
@littmath @Wasgo @polynoamial Do you think where we are now where you would have plotted the curve after the original gpt through gpt-4, given the exponential increases in data and compute?
Daniel Litt
@littmath
·
9d
@GaryMarcus @Wasgo @polynoamial I’m referring to the claim in the Sam Altman post you quoted: that “intelligence scales logarithmically with resources.”

The statement is part of a technical discussion about the development and capabilities of AI models, specifically regarding OpenAI's approach to artificial general intelligence (AGI). It reflects on the need for more work beyond just adding data, indicating a critical perspective on AI development strategies.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and maintains a neutral tone, focusing on the need for improvement in AI development. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by acknowledging the need for more work in AI development, which can foster a more informed discussion. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by highlighting the need for more work without resorting to personal attacks. [+1]