Gary Marcus

Rank 17 of 47
|
Score 93
5/10/2025, 2:29:28 AM
X
In reply to:
Daniel Litt
@littmath
·
9d
@GaryMarcus @Wasgo @polynoamial I’m referring to o3/gemini 2.5. Clearly reasoning models are dramatically better here. Base models (4o, 4.5, say) are about where I thought they’d be.
Gary Marcus
@GaryMarcus
·
9d
@littmath @Wasgo @polynoamial On base models? With or without inference time compute? Without domain specific data augmentation?

I think the capabilities are better but not sure it is a pure function of increase in compute and data.
Daniel Litt
@littmath
·
9d
@GaryMarcus @Wasgo @polynoamial I think math capabilities dramatically exceed what I expected when GPT-4 came out in March 2023. I didn’t have strong priors about capabilities in other areas.
Gary Marcus
@GaryMarcus
·
9d
@littmath @Wasgo @polynoamial Do you think where we are now where you would have plotted the curve after the original gpt through gpt-4, given the exponential increases in data and compute?
Daniel Litt
@littmath
·
9d
@GaryMarcus @Wasgo @polynoamial I’m referring to the claim in the Sam Altman post you quoted: that “intelligence scales logarithmically with resources.”
Gary Marcus
@GaryMarcus
·
9d
@littmath @Wasgo @polynoamial I think you are defending some notion of scaling law that I don’t quite follow that seems to be very different from the public conversation about these things.

I almost get from you that any nonmonotically increasing function ?or any function whatsover) could be fit post hoc and

The statement is part of a technical discussion on the development and scaling of reasoning models in AI, specifically in relation to pretraining and generalization in open-ended domains. It engages with public issues related to AI development and its implications, thus constituting public discourse.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not use harmful language and maintains a neutral tone, adhering to the principle of doing no harm. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by discussing the nuances of AI model development, promoting a deeper comprehension of the topic. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue, responding to previous comments without personal attacks, and focuses on the technical aspects of the discussion. [+1]