Gary Marcus

Rank 14 of 47
|
Score 83
12/22/2024, 6:19:20 PM
X
In reply to:
Merovingian
@f4talStrategies
·
391d
@GaryMarcus @pp0196 @RishabJainK @kevinroose Gary, once again, the ARC AGI paper explicitly addresses concerns like dataset leakage and pretraining biases, focusing on generalization to unseen problems. These safeguards make your critiques about ‘data augmentation’ and similar benchmarks irrelevant.
Gary Marcus
@GaryMarcus
·
391d
@pp0196 @RishabJainK @kevinroose For example.
Bioinfhotep
@pp0196
·
391d
@RishabJainK @GaryMarcus @kevinroose by examples you mean massive data augmentation with similarly looking benchmark setups ?
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose Edit: It seems like an OpenAI researcher has clarified in a separate thread that it was not a TTT or pretraining technique, but rather examples were included in the base training for the base o3 model (before any finetuning, RLHF, etc.) This is impressive!
Gary Marcus
@GaryMarcus
·
391d
@RishabJainK @kevinroose agreed!
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose Agreed! I believe Meta’s recent paper also achieved ~45% with TTT. The jump to 70-80% on o3 is certainly impressive, but showing results without TTT (or varying levels as you are suggesting) is probably a necessary due diligence.

The statement is part of a public discourse on the topic of artificial general intelligence (AGI) and data augmentation, engaging in a debate about the relevance and impact of data augmentation in AI research. The tone is critical and challenges the previous assertion about the irrelevance of augmentation without sufficient information.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm directly, but the tone is somewhat confrontational, which could be perceived as dismissive.
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in criticism of another's argument, but it does so in a manner that could be seen as dismissive rather than constructive, as it tells the other party to 'clean up your thinking.' [-1]