Gary Marcus

Rank 14 of 47
|
Score 83
12/22/2024, 6:42:19 PM
X
In reply to:
Rishab Jain
@RishabJainK
·
391d
@pp0196 @GaryMarcus @kevinroose A subset of the o3 public training set != or imply data augmentation in any way.

"unclear what they did with the 25% hold out of the train set" -- they did what they were supposed to do... not train on this data. If they did, Chollet wouldn't have made a public blogpost
Bioinfhotep
@pp0196
·
391d
@RishabJainK @GaryMarcus @kevinroose nowhere here it is said that they did not do data augmentation. + , just look at the prompts, the setups is clearly designed to be amenable to massive augmentation and the architecture. And unclear what they did with the 25% hold out of the train set either
Rishab Jain
@RishabJainK
·
391d
@pp0196 @GaryMarcus @kevinroose No, this:
Bioinfhotep
@pp0196
·
391d
@RishabJainK @GaryMarcus @kevinroose by examples you mean massive data augmentation with similarly looking benchmark setups ?
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose Edit: It seems like an OpenAI researcher has clarified in a separate thread that it was not a TTT or pretraining technique, but rather examples were included in the base training for the base o3 model (before any finetuning, RLHF, etc.) This is impressive!
Gary Marcus
@GaryMarcus
·
391d
@RishabJainK @kevinroose agreed!

The statement engages in a technical discussion about data augmentation practices in AI model training, questioning the validity of claims made about the o3 model's training process. It seeks clarification and evidence regarding the claims made by others in the conversation.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm and is part of a technical inquiry. [+1]
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by questioning the basis of a claim, which is a valid form of inquiry in technical discussions. [+1]