Gary Marcus

Rank 14 of 47
|
Score 83
12/23/2024, 11:23:44 AM
X
In reply to:
Tamay Besiroglu
@tamaybes
·
392d
@GaryMarcus What performance do you think o3 would have had if it saw none of the training examples?
Gary Marcus
@GaryMarcus
·
393d
“t wasn’t fine tuned, the train set was just in the training corpus” appears to be true.

But the point is (still) that you can’t compare machine results to human results when humans don’t have the training set in their corpus and the machines do.

Why is that so hard to…
shaperotator (e/λ)
@circlerotator
·
393d
@Lunarmony @olcan @stanWis @GaryMarcus @mikeknoop @doomie @jpineau1 @fchollet A) it was
B) it wasn’t fine tuned, the train set was just in the training corpus
C) o1 also gets 50%

The statement critiques the lack of basic scientific methodology in evaluating AI performance, specifically regarding training data exposure. It highlights a concern about the validity of comparing machine and human results when the conditions are not equivalent.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement aims to highlight a potential flaw in scientific evaluation, which aligns with striving to do no harm by advocating for accurate and fair assessments. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    By questioning the methodology, the statement promotes understanding and encourages a more rigorous scientific approach. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by pointing out a specific issue without resorting to personal attacks. [+1]