Gary Marcus

Rank 21 of 47
|
Score 52
Aleksandr Tiulkanov
@shadbush
·
343d
@mealreplacer @GaryMarcus @colin_fraser Waiting and hoping for enough examples to be caught in the next training dataset isn't really solving the problem. The fundamental problem is in poor if not inexistent ability to generalise beyond the training dataset.
9/13/2024, 9:38:47 PM
X
In reply to:
Julian
@mealreplacer
·
342d
@GaryMarcus Curious for you to respond to this and this
Julian
@mealreplacer
·
343d
@GaryMarcus @colin_fraser I mean preferably we'd have quantitative benchmarks that don't saturate with each successive generation, yeah?

I'm not sure exactly what I'm supposed to conclude about the trajectory and speed of AI progress by looking at binary examples of LLMs failing at weird little tasks…
Julian
@mealreplacer
·
343d
My original tweet was not intended to make a strong claim about the model’s capabilities (or lack thereof) — on the contrary, I think we need more well-designed, rigorous benchmarks that test things that actually matter (e.g., consequential real-world tasks). Rather, it’s just…
Gary Marcus
@GaryMarcus
·
342d
@mealreplacer Sorry to keep dogging you on this but here’s even more evidence that your conclusion was premature:
Gary Marcus
@GaryMarcus
·
342d
Proving my point yet again.
- o1 is not really reasoning
- it doesn’t really understand what it is talking about
- even the ubiquitous river crossing problems have not in fact been conquered
- slight variations on famous problems still can elicit really bizarre answers…
Julian
@mealreplacer
·
343d

The statement and the surrounding conversation involve a discussion about the capabilities and limitations of AI models, specifically in the context of solving puzzles and generalizing beyond training datasets. This discussion touches on public issues related to AI development, evaluation, and the interpretation of AI performance, making it a part of public discourse.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm and is part of a constructive dialogue about AI capabilities. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by adding a point to an ongoing discussion, contributing to a deeper analysis of AI performance. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism and dialogue, addressing disagreements without personal attacks. [+1]
  4. Principle 6:
    I will use my influence for the betterment of society.
    The statement uses the platform to discuss AI development, which is a significant societal issue, thus contributing to the betterment of society. [+1]