Gary Marcus

Rank 24 of 47
|
Score 25
7/21/2025, 8:50:37 PM
X
In reply to:
FeltSteam0
@FeltSteam
·
57d
@GaryMarcus @realKingRoland9 Honestly it would be pretty cool if you went down a similar path as François Chollet, creating and testing models on your own set of evaluation (though I would imagine what you would create would be pretty different to arc-agi) to show their true current capability
FeltSteam0
@FeltSteam
·
57d
@GaryMarcus @realKingRoland9 I mean it is within the range of current capability i.e. Gemini 2.5 Pro has audio and video input now, people should be testing this. I haven't tested it on movies but it's comprehension of what occurs during YouTube videos seems pretty solid however that isn't any official test
Gary Marcus
@GaryMarcus
·
57d
@FeltSteam @realKingRoland9 for the most people test what they can do, not what they can’t
FeltSteam0
@FeltSteam
·
57d
@realKingRoland9 @GaryMarcus I haven't seen anywhere publish results showing models can’t do this I just haven’t seen this tested at all.
Roland
@realKingRoland9
·
57d
@FeltSteam @GaryMarcus No AI has met even one of Marcus’s five operational AGI yardsticks under controlled, peer-reviewed tests. No published evidence shows Gemini 2.5 or any LLM reliably passing the movie or novel comprehension benchmarks.
FeltSteam0
@FeltSteam
·
57d
@GaryMarcus When was the last time you tested AI models on these 5 things (or 4 of them excluding the robotics for now)? I feel like current AI models like Gemini 2.5 Pro would perform quite well on the first two challenges (for books and movies released past their knowledge cutoff).

The statement reflects a discussion on the capabilities and testing of AI models, specifically in relation to AGI benchmarks. It highlights a perceived lack of resources and motivation among those capable of advancing this testing, suggesting a critique of the current state of AI research and development.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not directly cause harm but implies a critique of those with resources, suggesting they are not motivated to advance AI testing.
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement could promote understanding by highlighting a gap in AI testing and development, but it lacks empathy or compassion towards those it critiques.
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by pointing out a lack of motivation among resource holders, without resorting to personal attacks. [+1]
  4. Principle 6:
    I will use my influence for the betterment of society.
    The statement uses its platform to highlight an issue in AI development, potentially encouraging discourse on resource allocation and motivation in the field. [+1]
  5. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds free speech by responsibly critiquing the current state of AI testing and resource allocation. [+1]