Gary Marcus

Rank 13 of 47
|
Score 83
8/6/2024, 2:41:20 PM
X
In reply to:
Aleksander Molak (CausalPython.io)
@AleksanderMolak
·
524d
@davidmanheim @GaryMarcus Perhaps they are Causal Parrots (https://arxiv.org/abs/2308.13067)
David Manheim
@davidmanheim
·
524d
@GaryMarcus @AleksanderMolak To be clear, I'm not saying that current models get things right anything like always, but hey generalize their causal understanding and world models far more than can reasonably be explained by being stochastic parrots, much less n-gram models.
David Manheim
@davidmanheim
·
524d
@GaryMarcus @AleksanderMolak Sure, but the current class of model performs well enough that, to paraphrase Michael Lewis, if an explanation is where the mind comes to rest, the mind that stops at "large n-gram model" borders on narcoleptic.
Gary Marcus
@GaryMarcus
·
524d
@davidmanheim @AleksanderMolak “yes, sometimes” you could say of a large n-gram model.

or a broken clock that is right twice a day.
David Manheim
@davidmanheim
·
524d
@AleksanderMolak @GaryMarcus The second is whether LLMs have correct implicit models of causal relationships at the textual level, and can they correctly predict causal outcomes in cases not in their training data - and the answer is absolutely yes, albeit only sometimes.
David Manheim
@davidmanheim
·
524d
@AleksanderMolak @GaryMarcus OK, there are two different claims that are relevant here. The first is whether deep learning via SGD can directly learn causal features from observational data, and the answer to that is no.

The statement constitutes public discourse as it engages in a substantive discussion about the capabilities and limitations of large language models (LLMs) in understanding and generating causal relationships, which is a significant topic in the field of artificial intelligence and has broader societal implications.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm with words and actions. It is a technical critique rather than a personal attack. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement respects the privacy and dignity of others. It does not engage in cyberbullying, harassment, or hate speech. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement does not explicitly promote understanding, empathy, or compassion, but it does contribute to a technical discussion which can be seen as promoting understanding in a specific field.
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism of the concept of 'causal parrots' versus 'stochastic parrots' without resorting to personal attacks or ad hominem arguments. [+1]
  5. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principles of free speech and uses the platform responsibly to discuss a technical issue with integrity. [+1]