Gary Marcus

Rank 14 of 47
|
Score 83
12/21/2024, 11:00:38 PM
X
In reply to:
Matt Schwartz
@mattcschwartz
·
374d
@GaryMarcus I think he just meant that they didn’t do anything special to try to solve ARC, like finetune a new version of the model. It’s very natural to use any training data you can get, including ARC problems
Gary Marcus
@GaryMarcus
·
374d
@mattcschwartz Saying they didn’t target mislead me, and i know how benchmarks work.

That seemed to be an affirmative statement that there was not extensive training towards the test.
Matt Schwartz
@mattcschwartz
·
374d
@GaryMarcus Okay, that’s fine, but I think people with a good understanding of the challenge probably did because they know that all models in the competition are trained on the public set. So I don’t think it’s fair to say they mislead anyone
Gary Marcus
@GaryMarcus
·
374d
@mattcschwartz i don’t dispute that their performance was better; i did not walk away from the video knowing that they had trained on hundreds of related examples.
Matt Schwartz
@mattcschwartz
·
374d
@GaryMarcus You mean at the end when he says they didn’t do anything special to target this challenge? That’s totally fair, they train on basically everything on the internet, which includes the ARC public set. I guarantee you all the models are trained on it and they couldn’t solve it
Gary Marcus
@GaryMarcus
·
374d
@mattcschwartz did you watch the clip? watch it a couple times, focusing on what Sam says.

The statement is part of a public discourse involving a discussion about the training of AI models and their performance on specific benchmarks. The tone is conciliatory, acknowledging different perspectives in the conversation.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and acknowledges different perspectives, aligning with the principle of doing no harm. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by recognizing why different interpretations might exist, aligning with the principle of promoting empathy and compassion. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by acknowledging different viewpoints without resorting to personal attacks, aligning with the principle of constructive criticism. [+1]