Gary Marcus

Rank 13 of 47
|
Score 83
8/6/2024, 3:52:11 PM
X
In reply to:
David Manheim
@davidmanheim
·
526d
@GaryMarcus @AleksanderMolak If "stochastic parrot" means something like "N-gram generator," I think this is ignoring the fact that we know the systems generalize; deep-double-descent is a clear indication of this occurring.
Gary Marcus
@GaryMarcus
·
526d
@AleksanderMolak @davidmanheim they are NOT causal parrots as I would understand the term. they are stochastic parrots that can bullshit their way through many topics.
Aleksander Molak (CausalPython.io)
@AleksanderMolak
·
526d
@davidmanheim @GaryMarcus Perhaps they are Causal Parrots (https://arxiv.org/abs/2308.13067)
David Manheim
@davidmanheim
·
526d
@GaryMarcus @AleksanderMolak To be clear, I'm not saying that current models get things right anything like always, but hey generalize their causal understanding and world models far more than can reasonably be explained by being stochastic parrots, much less n-gram models.
David Manheim
@davidmanheim
·
526d
@GaryMarcus @AleksanderMolak Sure, but the current class of model performs well enough that, to paraphrase Michael Lewis, if an explanation is where the mind comes to rest, the mind that stops at "large n-gram model" borders on narcoleptic.
Gary Marcus
@GaryMarcus
·
526d
@davidmanheim @AleksanderMolak “yes, sometimes” you could say of a large n-gram model.

or a broken clock that is right twice a day.

The statement in question is part of a broader conversation about the capabilities and limitations of large language models (LLMs) in the context of AI and cognitive psychology. The discussion involves multiple participants and addresses the technical aspects of AI, making it a substantive engagement with public issues related to technology and science.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm with words or actions. It is a professional and academic contribution to the discussion. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by referencing a relevant academic article, which can help clarify the discussion on generalization in AI models. [+2]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by providing a resource for further reading without resorting to personal attacks or ad hominem arguments. [+2]
  4. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds the principles of free speech and uses the platform responsibly by contributing to an informed discussion on a public issue. [+2]