Gary Marcus

Rank 23 of 47
|
Score 27
5/2/2024, 2:45:26 PM
X
In reply to:
io🏄🏽‍♂️
@Astro_Erik
·
502d
@GaryMarcus Should be called sigmoid laws
Gary Marcus
@GaryMarcus
·
502d
Scaling “laws” were never laws.

They were trends. Trends whose time may be up.

13.5 months and $50B post GPT-4, GPT-4 remains on top.
Noam Brown
@polynoamial
·
502d
Many have pointed out that LLM benchmarks are broken and gamed. Happy to see my former resident @hughbzhang, @summeryue0, and the great @scale_AI folks do something about it! They made a private version of GSM8k and evaled GPT-4, Claude, Mixtral, Phi, etc: https://arxiv.org/pdf/2405.00332

The statement '@Astro_Erik They aren’t laws at all. They are empirical generalizations with very little actual predictive power.' is part of a broader conversation about the nature and naming of scaling laws in the context of large language models (LLMs). The statement challenges the characterization of these scaling trends as 'laws,' suggesting they lack the robust predictive capability typically associated with scientific laws.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm as it is a critique of scientific terminology, which is a valid and important aspect of academic discourse. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement respects the dignity of others, focusing solely on the scientific discussion without personal attacks or derogatory language. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The comment promotes understanding by clarifying the nature of scaling laws in AI, contributing to a more nuanced understanding of the topic. [+1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by challenging the use of the term 'laws' for what the speaker believes are merely empirical generalizations. [+1]
  5. Principle 5:
    I will acknowledge and correct my mistakes.
    The statement is direct and clear, making a specific claim about the nature of scaling laws which could be open for further discussion or correction if needed. [+1]
  6. Principle 6:
    I will use my influence for the betterment of society.
    By fostering a more accurate understanding of scientific terms, the statement uses its influence to contribute positively to the discourse around AI development. [+1]
  7. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principles of free speech responsibly, using the platform to engage in a substantive debate about the terminology in AI research. [+1]