Gary Marcus

Rank 12 of 47
|
Score 83
5/18/2025, 1:45:03 PM
X
In reply to:
David Manheim
@davidmanheim
·
248d
@GaryMarcus @uhbif19 @XRobservatory Yeah, that's absolutely something that is an issue with current LLMs. From my understanding of the architecture, the visual recognition and image generation are tacked-on effectively post-hoc, and it's impressive to me they got it to work at all.
Gary Marcus
@GaryMarcus
·
248d
@davidmanheim @uhbif19 @XRobservatory So you are saying that the image generation is not (despite superficial appearances) properly tethered to a world model, correct?

Let’s start there.
David Manheim
@davidmanheim
·
249d
@GaryMarcus @uhbif19 @XRobservatory It's not a test of the world model because image generation is using diffusion to generate something in the latent space of images it was trained on, not something in the language model.

It's like saying hand-eye coordination is part of understanding; it's kind-of true, but, no.
Gary Marcus
@GaryMarcus
·
249d
@uhbif19 @davidmanheim @XRobservatory I documented a multiple similars one for ChatGPT (not o3) in text but curious why everyone thinks image generation is not a legitimate test of a systems “world model”
There is no other room 🇬🇪
@uhbif19
·
249d
@GaryMarcus @davidmanheim @XRobservatory But it does not make errors like that in text. It just have bad image comprehension, that is all.
Gary Marcus
@GaryMarcus
·
249d
@davidmanheim @uhbif19 @XRobservatory What’s your theory of the psychology of a system that can make an error like that?

The statement is part of a broader conversation about the capabilities and limitations of language models, specifically focusing on image generation and recognition. It references a marketing post by OpenAI about new features in ChatGPT, which adds context to the ongoing discussion about AI capabilities. The tone is informative and seeks to contribute to the understanding of AI developments.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is neutral in tone, focusing on sharing information. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by providing a link to additional information about AI capabilities. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by contributing to an ongoing discussion without personal attacks. [+1]
  4. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement uses the platform to responsibly share information about technological advancements. [+1]