Gary Marcus

Rank 13 of 47
|
Score 96
5/10/2025, 1:29:52 AM
X
In reply to:
Cameron Williams
@Wasgo
·
39d
@GaryMarcus @littmath @polynoamial I mean, Sam constantly conflates positive results from scientific papers with overall trends for generative AI in ways that are entirely inaccurate. I have zero respect for him on that basis.
Gary Marcus
@GaryMarcus
·
39d
@Wasgo @littmath @polynoamial Scaling was supposed to predict performance as many people said eg sam:
“The intelligence of an AI model roughly equals the log of the resources used to train and run it. These resources are chiefly training compute, data, and inference compute. It appears that you can spend
Cameron Williams
@Wasgo
·
39d
@GaryMarcus @littmath @polynoamial And I completely agree that generative AI performance has hit a wall.

But people are conflating a performance wall with scaling law failing. Scaling law can hold and we can still hit a performance wall, like we have.
Gary Marcus
@GaryMarcus
·
39d
@Wasgo @littmath @polynoamial I understand what you’re saying esp around the Kaplan paper in particular.

But people didn’t spend half a trillion dollars on the pure mathematical endeavor; they thought the math and empirical observations around *performance* justified exponential increases in data and
Cameron Williams
@Wasgo
·
39d
@GaryMarcus @littmath @polynoamial Scaling law is a pure mathematical endeavour and performance isn’t pure math. They’re separate targets.
Cameron Williams
@Wasgo
·
39d
@GaryMarcus @littmath @polynoamial Even this post notes it’s not performance scaling:

“…test loss is not an easily interpretable metric and try to predict a variety of other performance metrics. For example, a scaling law is fit to predict the LLM’s pass rate on the HumanEval coding benchmark.”

The statement is part of a broader discussion on the performance and scaling laws of generative AI, which is a public issue related to technology and its implications. The tone is analytical and critical, focusing on the accuracy of claims made by a public figure in the AI field.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not directly cause harm but is part of a critical discussion about the accuracy of claims in AI, which is important for informed public discourse.
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by agreeing with a critique of a public figure's statements, but it does not engage in personal attacks or ad hominem arguments. [+1]