Gary Marcus

Rank 19 of 47
|
Score 64
5/10/2025, 1:28:11 AM
X
In reply to:
Daniel Litt
@littmath
·
54d
@Wasgo @GaryMarcus @polynoamial I don’t think it needs to be formal to be a claim we can try to evaluate heuristically by benchmarking. Lots of benchmarks seem to scale logarithmically with compute in some (pre-saturation) regime.
Cameron Williams
@Wasgo
·
54d
@littmath @GaryMarcus @polynoamial Is there any formal definition for this version of the law though? Altman refers to intelligence which isn’t actually a defined standard. He seems to cherry pick what intelligence means every time he refers to it.
Daniel Litt
@littmath
·
54d
@GaryMarcus @Wasgo @polynoamial This is what I understood by scaling laws; it’s obviously compatible with diminishing returns to scale.
Gary Marcus
@GaryMarcus
·
54d
@Wasgo @littmath @polynoamial How about this one? Source: https://blog.samaltman.com/three…
Cameron Williams
@Wasgo
·
54d
@GaryMarcus @littmath @polynoamial Even this post notes it’s not performance scaling:

“…test loss is not an easily interpretable metric and try to predict a variety of other performance metrics. For example, a scaling law is fit to predict the LLM’s pass rate on the HumanEval coding benchmark.”
Gary Marcus
@GaryMarcus
·
54d
@Wasgo @littmath @polynoamial This is perhaps a good review of the various claims that have been made

https://cameronrwolfe.substack.c…

The statement engages in a technical discussion about AI development and scaling laws, specifically addressing the reasons behind the absence of GPT-5. It contributes to public discourse by discussing AI benchmarks and their implications for AI progress.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is part of a constructive technical discussion. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    It promotes understanding by engaging in a detailed discussion about AI benchmarks and scaling laws. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue without personal attacks, focusing on the technical aspects of AI development. [+1]