Gary Marcus

Rank 19 of 47
|
Score 64
5/10/2025, 1:07:13 AM
X
In reply to:
Daniel Litt
@littmath
·
54d
@GaryMarcus @Wasgo @polynoamial This is what I understood by scaling laws; it’s obviously compatible with diminishing returns to scale.
Gary Marcus
@GaryMarcus
·
54d
@Wasgo @littmath @polynoamial How about this one? Source: https://blog.samaltman.com/three…
Cameron Williams
@Wasgo
·
54d
@GaryMarcus @littmath @polynoamial Even this post notes it’s not performance scaling:

“…test loss is not an easily interpretable metric and try to predict a variety of other performance metrics. For example, a scaling law is fit to predict the LLM’s pass rate on the HumanEval coding benchmark.”
Gary Marcus
@GaryMarcus
·
54d
@Wasgo @littmath @polynoamial This is perhaps a good review of the various claims that have been made

https://cameronrwolfe.substack.c…
Cameron Williams
@Wasgo
·
54d
@littmath @GaryMarcus @polynoamial There’s two entirely different meanings of AI scaling laws. OpenAI covers theirs in their papers and it’s not about model performance, it’s about fitting vs overfitting. https://arxiv.org/pdf/2001.08361

The general public talk about scaling laws as performance and there’s no definition.
Daniel Litt
@littmath
·
54d
@GaryMarcus @polynoamial I’m not sure I understand. Is the claim that scaling laws were predicting faster than logarithmic returns to scale, or that actual returns to scale are slower than logarithmic?

The statement and conversation engage in a technical discussion about AI scaling laws, diminishing returns, and model performance. It involves public discourse as it addresses the implications of AI development and resource allocation, which are significant societal issues.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm; it is a technical discussion about AI scaling.
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The conversation promotes understanding by discussing complex AI concepts, though it could be more accessible to a general audience. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The dialogue is constructive, focusing on technical aspects without personal attacks. [+1]