Gary Marcus

Rank 23 of 47
|
Score 27
7/21/2025, 8:53:26 PM
X
In reply to:
Gary Marcus
@GaryMarcus
·
57d
@FeltSteam @realKingRoland9 @VeredShwartz had some examples i mentioned in my recent substack on world models where performance was poor on stuff humans would find obvious.
FeltSteam0
@FeltSteam
·
57d
@GaryMarcus @realKingRoland9 I mean it is within the range of current capability i.e. Gemini 2.5 Pro has audio and video input now, people should be testing this. I haven't tested it on movies but it's comprehension of what occurs during YouTube videos seems pretty solid however that isn't any official test
Gary Marcus
@GaryMarcus
·
57d
@FeltSteam @realKingRoland9 for the most people test what they can do, not what they can’t
FeltSteam0
@FeltSteam
·
57d
@realKingRoland9 @GaryMarcus I haven't seen anywhere publish results showing models can’t do this I just haven’t seen this tested at all.
Roland
@realKingRoland9
·
57d
@FeltSteam @GaryMarcus No AI has met even one of Marcus’s five operational AGI yardsticks under controlled, peer-reviewed tests. No published evidence shows Gemini 2.5 or any LLM reliably passing the movie or novel comprehension benchmarks.
FeltSteam0
@FeltSteam
·
57d
@GaryMarcus When was the last time you tested AI models on these 5 things (or 4 of them excluding the robotics for now)? I feel like current AI models like Gemini 2.5 Pro would perform quite well on the first two challenges (for books and movies released past their knowledge cutoff).

The statement is part of a technical discussion about AI benchmarks and performance, specifically in relation to AI models' comprehension capabilities. It engages with public discourse by addressing the capabilities and limitations of AI, a significant societal and technological issue.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not contain harmful language or actions, maintaining a neutral and informative tone. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by referencing a new benchmark relevant to the discussion, promoting knowledge sharing. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by providing additional information without personal attacks or ad hominem arguments. [+1]
  4. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement responsibly uses the platform to share relevant information about AI benchmarks, contributing to the discourse on AI capabilities. [+1]