Gary Marcus

Rank 14 of 47
|
Score 83
6/7/2025, 12:35:12 AM
X
In reply to:
Old Man with a Beard
@Brianebutler1
·
203d
@GaryMarcus But Hinton is correct. An LLM should not store text. Models that do are augmented LLMs.
Gary Marcus
@GaryMarcus
·
203d
And to think Geoff Hinton told me LLMs don’t store any text.

Not everyone who has a Nobel Prize knows what they are talking about.

Every model does effectively store text to some degree and bigger models store more.
Rohan Paul
@rohanpaul_ai
·
204d
How much information do LLMs really memorize?

Now we know, thanks to Paper by @AIatMeta, @GoogleDeepMind , @NVIDIAAIDev and Cornell Univ.

This paper will be absolutely crucial for ongoing lawsuits between AI Companies and data creators/rights owners/copyright-holders.

The statement is part of a technical discussion about how language models store information, referencing a paper on model memorization. It uses a metaphor comparing LLMs to JPEGs, suggesting that LLMs don't store text in a straightforward manner, similar to how JPEGs don't store images directly but rather compress and encode them.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and contributes to a technical discussion. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The metaphor used promotes understanding by simplifying a complex concept. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by building on previous comments without personal attacks. [+1]