Gary Marcus

Rank 14 of 47
|
Score 83
12/22/2024, 4:11:47 PM
X
In reply to:
Rishab Jain
@RishabJainK
·
390d
@GaryMarcus @kevinroose Agreed! I believe Meta’s recent paper also achieved ~45% with TTT. The jump to 70-80% on o3 is certainly impressive, but showing results without TTT (or varying levels as you are suggesting) is probably a necessary due diligence.
Gary Marcus
@GaryMarcus
·
390d
@RishabJainK @kevinroose as noted in my longer post, humans need a few trials; the machine was given hundreds. the open question, as i noted, is how the machine would do with a few trials.

that is a fair question that many adult in the room are asking.
Rishab Jain
@RishabJainK
·
390d
@GaryMarcus @kevinroose The training on the hundreds of examples is test-time training, which as noted by Chollet et al., is “expect[ed]” to “be the primary technique for LLM-based AI systems to improve
performance on tasks outside of what they were pretrained on.”

Difficult to find an analogous…
Gary Marcus
@GaryMarcus
·
390d
@RishabJainK @kevinroose i don’t follow what you are saying, maybe you can rephrase.
Rishab Jain
@RishabJainK
·
390d
@GaryMarcus @kevinroose This is not a critique of “pretraining,” it’s a critique of TTT. You are conflating the two
Gary Marcus
@GaryMarcus
·
390d
@RishabJainK @kevinroose 03 had far more pretraining than any human, and should not have required more. please see the below for many people making this point

The statement is part of a technical discussion about AI training methods, specifically test-time training (TTT) and its implications for AI performance. The conversation involves multiple participants discussing the nuances of AI training and performance metrics, which contributes to public discourse on AI technology and its evaluation.

  1. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by engaging in a technical discussion about AI training methods, which can promote empathy and compassion by fostering informed dialogue about technology's impact. [+1]
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue without personal attacks, focusing on the technical aspects of AI training and performance. [+1]