Gary Marcus

Rank 14 of 47
|
Score 83
12/22/2024, 6:06:19 PM
X
In reply to:
Bioinfhotep
@pp0196
·
391d
@RishabJainK @GaryMarcus @kevinroose by examples you mean massive data augmentation with similarly looking benchmark setups ?
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose Edit: It seems like an OpenAI researcher has clarified in a separate thread that it was not a TTT or pretraining technique, but rather examples were included in the base training for the base o3 model (before any finetuning, RLHF, etc.) This is impressive!
Gary Marcus
@GaryMarcus
·
391d
@RishabJainK @kevinroose agreed!
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose Agreed! I believe Meta’s recent paper also achieved ~45% with TTT. The jump to 70-80% on o3 is certainly impressive, but showing results without TTT (or varying levels as you are suggesting) is probably a necessary due diligence.
Gary Marcus
@GaryMarcus
·
391d
@RishabJainK @kevinroose as noted in my longer post, humans need a few trials; the machine was given hundreds. the open question, as i noted, is how the machine would do with a few trials.

that is a fair question that many adult in the room are asking.
Rishab Jain
@RishabJainK
·
391d
@GaryMarcus @kevinroose The training on the hundreds of examples is test-time training, which as noted by Chollet et al., is “expect[ed]” to “be the primary technique for LLM-based AI systems to improve
performance on tasks outside of what they were pretrained on.”

Difficult to find an analogous…

The statement is part of a technical discussion about AI training techniques, specifically related to test-time training (TTT) and data augmentation. It involves multiple participants discussing the implications and effectiveness of these methods in AI model performance. This conversation is a substantive engagement with public issues in the field of AI, contributing to the civic dialogue on technological advancements and their implications.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not appear to cause harm and is focused on a technical discussion. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by engaging in a technical discussion about AI training methods. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement is part of a constructive dialogue, engaging with others in a technical discussion without personal attacks. [+1]
  4. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds free speech principles by responsibly discussing technical aspects of AI training. [+1]