Gary Marcus

Rank 14 of 47
|
Score 83
12/22/2024, 6:18:02 PM
X
In reply to:
Merovingian
@f4talStrategies
·
391d
@GaryMarcus @pp0196 @RishabJainK @kevinroose Gary, once again, the ARC AGI paper explicitly addresses concerns like dataset leakage and pretraining biases, focusing on generalization to unseen problems. These safeguards make your critiques about ‘data augmentation’ and similar benchmarks irrelevant.
Gary Marcus
@GaryMarcus
·
391d
@pp0196 @RishabJainK @kevinroose For example.
Bioinfhotep
@pp0196
·
391d
@RishabJainK @GaryMarcus @kevinroose by examples you mean massive data augmentation with similarly looking benchmark setups ?
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose Edit: It seems like an OpenAI researcher has clarified in a separate thread that it was not a TTT or pretraining technique, but rather examples were included in the base training for the base o3 model (before any finetuning, RLHF, etc.) This is impressive!
Gary Marcus
@GaryMarcus
·
391d
@RishabJainK @kevinroose agreed!
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose Agreed! I believe Meta’s recent paper also achieved ~45% with TTT. The jump to 70-80% on o3 is certainly impressive, but showing results without TTT (or varying levels as you are suggesting) is probably a necessary due diligence.

The statement is part of a public discourse involving a discussion on AI research and critiques of methodologies, which are relevant public issues in the field of technology and ethics. The tone of the statement is defensive and somewhat dismissive, indicating frustration with the ongoing conversation.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not directly cause harm but suggests a potential for disengagement, which could stifle dialogue. [-1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement does not engage in cyberbullying or hate speech, but the tone is dismissive, which could undermine respectful discourse.
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement does not engage in constructive criticism or dialogue, as it hints at muting the other party rather than addressing the disagreement constructively. [-1]