Gary Marcus

Rank 14 of 47
|
Score 83
5/12/2025, 4:24:20 PM
X
In reply to:
Bowser
@BowsersaurusRex
·
223d
@GaryMarcus How do they game math benchmarks? A right answer is a right answer, no?
Gary Marcus
@GaryMarcus
·
223d
@BowsersaurusRex They are all getting gamed but I stand by my comprehension challenge

https://www.newyorker.com/tech/a…
Bowser
@BowsersaurusRex
·
223d
@GaryMarcus Which benchmarks do you prefer as a measure of intelligence for LLMs?
Gary Marcus
@GaryMarcus
·
223d
AI doesn’t feel vastly smarter than two years ago because … it isn’t.

Taking IQ as a measure of how smart these systems are is itself not smart.
Dan Hendrycks
@DanHendrycks
·
224d
AI models are dramatically improving at IQ tests (70 IQ → 120), yet they don't feel vastly smarter than two years ago.

At their current level of intelligence, rehashing existing human writings will work better than leaning on their own intelligence to produce novel analysis.

The statement critiques the use of synthetic data in AI benchmarks, suggesting it inflates perceived intelligence. It implies skepticism about the validity of such benchmarks in measuring true AI capabilities.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement could be seen as slightly harmful as it questions the credibility of the person it addresses, potentially undermining their reputation. [-1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement does not promote understanding or empathy, as it focuses on criticism without offering constructive feedback. [-1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in criticism but lacks constructive dialogue, leaning towards a personal critique rather than addressing the broader issue. [-1]