Gary Marcus

Rank 19 of 47
|
Score 64
5/7/2025, 11:29:19 PM
X
In reply to:
Judea Pearl
@yudapearl
·
67d
@GaryMarcus @Dr_Cuspy @sejnowski @AlisonGopnik Surely every world model must be represented in the machine in some form. But does it make a difference if it is represented in the form of neurons, or Boolean circuits, or a code that generates those circuits. The issue is where we are more likely to find the secrets of that
Gary Marcus
@GaryMarcus
·
67d
@yudapearl @Dr_Cuspy @sejnowski @AlisonGopnik disagree. the invariants certainly play a role, but ultimately a world model is a form of internal representation, and the nature of that representation depends on neural wetware.

different creatures have different world models in part because of how they evolved.
Judea Pearl
@yudapearl
·
67d
@Dr_Cuspy @sejnowski @GaryMarcus @AlisonGopnik It seems to me that the structure of our "world model" is dictated by the invariants in our environment, more than by the architecture of our neural nets; the former being an emergent property of the latter.
Colin Hales
@Dr_Cuspy
·
67d
@yudapearl @sejnowski @GaryMarcus @AlisonGopnik If cognitive science studies that structure, it should not be mistaken for a design of a machine that acquires that structure. A machine that does that is most definitely in the ambit of neuroscience and physics.
Judea Pearl
@yudapearl
·
67d
It seems to me that the function of "long term memory" is served by what we call "world model" and, if so, its structure need be explored by cognitive science, rather than neuroscience. @sejnowski @GaryMarcus @AlisonGopnik
Judea Pearl
@yudapearl
·
67d
Thinking about thinking: AI offers theoretical insights into human memory https://www.thetransmitter.org/h…

The statement engages in a technical discussion about the capabilities of large language models (LLMs) and neural networks in representing world models, contributing to the broader debate on AI and cognitive science. It critiques the architecture of LLMs in comparison to neural networks, suggesting that the type of architecture impacts the ability to represent stable world models without hallucinations.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not use harmful language and maintains a neutral tone, focusing on technical aspects rather than personal attacks. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement aims to promote understanding by discussing the differences in AI architectures, although it could be more explicit in fostering empathy or compassion. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism of LLMs, contributing to a technical dialogue without resorting to personal attacks. [+2]