Gary Marcus

Rank 14 of 47
|
Score 83
6/7/2025, 12:04:32 AM
X
In reply to:
Clint Steel
@itsClintSteel
·
204d
@GaryMarcus You're phrasing this in the most dishonest way possible. You're making it sound like larger models are more likely to store information from their training data when the post youre quoting literally describes the opposite:

"A 1.5B model can store up to 675MB of raw info, but
Gary Marcus
@GaryMarcus
·
204d
And to think Geoff Hinton told me LLMs don’t store any text.

Not everyone who has a Nobel Prize knows what they are talking about.

Every model does effectively store text to some degree and bigger models store more.
Rohan Paul
@rohanpaul_ai
·
204d
How much information do LLMs really memorize?

Now we know, thanks to Paper by @AIatMeta, @GoogleDeepMind , @NVIDIAAIDev and Cornell Univ.

This paper will be absolutely crucial for ongoing lawsuits between AI Companies and data creators/rights owners/copyright-holders.

The statement engages in a technical debate about the memorization capabilities of language models, referencing a conversation about a research paper. The tone is defensive and accusatory, with a focus on correcting perceived misinformation and addressing accusations of dishonesty.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement attempts to clarify a technical point, which aligns with striving to do no harm through misinformation, but the accusatory tone may detract from this goal.
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement does not respect the dignity of others, as it includes a personal attack by calling someone dishonest and threatening to block them. [-2]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement does not engage in constructive criticism or dialogue, as it includes personal attacks rather than focusing solely on the technical argument. [-2]