Gary Marcus

Rank 17 of 47
|
Score 71
9/13/2024, 10:20:17 PM
X
In reply to:
Gary Marcus
@GaryMarcus
·
343d
@mealreplacer @Wasgo @shadbush @colin_fraser Dude if a system can’t reliably follow the rules of chess what do you think happens on more complex problems?!

You can’t have AGI in a system that frequently and unpredictably says ridiculous things even in relatively easy problems.
Julian
@mealreplacer
·
343d
@Wasgo @GaryMarcus @shadbush @colin_fraser You really can’t just merely assert that failing on weird little problems is something that is strictly necessary to be solve before we get “real AGI”
Cameron Williams
@Wasgo
·
343d
@mealreplacer @GaryMarcus @shadbush @colin_fraser The failure points on weird little problems isn't about benchmarking at all though. It's about understanding how the model works, what the limitations are, and why we need to move past generative AI if we want to see real AGI.
Julian
@mealreplacer
·
343d
@GaryMarcus @shadbush @colin_fraser I don’t feel like this is really answering the question I posed.

Would you change your mind if there were difficult benchmarks that are testing LLMs on consequential, real-world tasks, and each generation improved upon the previous?
Gary Marcus
@GaryMarcus
·
343d
@shadbush @mealreplacer @colin_fraser @shadbush is exactly right. I would add that building benchmarks is hard, nobody has ever solved it, but success in bad benchmark doesn’t equate to a solution to the extrapolation problem.

Progress on AI benchmarks has been great, but progress on getting reliable reasoning in…
Aleksandr Tiulkanov
@shadbush
·
344d
@mealreplacer @GaryMarcus @colin_fraser Waiting and hoping for enough examples to be caught in the next training dataset isn't really solving the problem. The fundamental problem is in poor if not inexistent ability to generalise beyond the training dataset.

The statement constitutes public discourse as it engages in a substantive discussion about the limitations and challenges of artificial general intelligence (AGI), a significant public issue in the field of technology and ethics.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm directly, but the tone is somewhat dismissive, which could be seen as unhelpful.
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement does not engage in cyberbullying, harassment, or hate speech, but the dismissive tone could be seen as disrespectful.
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement does not promote understanding, empathy, or compassion effectively due to its dismissive tone. [-1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement does not engage in constructive criticism or dialogue effectively, as it dismisses the other party's understanding rather than engaging with their points. [-1]