Gary Marcus

Rank 13 of 47
|
Score 83
12/22/2024, 3:27:50 AM
X
In reply to:
Anup
@anupk24
·
384d
@GaryMarcus @brhydon Sam’s point is that the goal of the model was not to maximize ARC score. The goal was to build a general model that can reason, and part of ARC’s public training set (as allowed by the benchmark) was thrown in, and it achieves the high score.
Gary Marcus
@GaryMarcus
·
384d
@brhydon Singing lessons would be targeting singing.
Brydon Eastman
@brhydon
·
384d
@GaryMarcus This isn't going to a Taylor Swift concert and catching her lip syncing. This is going to a Taylor Swift concert and being upset that she's taken singing lessons before.
Brydon Eastman
@brhydon
·
384d
@GaryMarcus Professor Marcus, you're upset that we used the evaluation EXACTLY as designed and indicated by the original paper.

If someone watched that stream and feels mislead, then they never read the paper.

There is zero dishonesty here and its disrespectful to insinuate otherwise.
Gary Marcus
@GaryMarcus
·
384d
@brhydon ARC folks, who you now have a partnership may have ok’d it but

1. it’s more like studying for the SAT then reading a book. It’s learning about the idiosyncrasies of the questions. we have seen this many times in benchmarks.

2. am pretty sure humans could do well on ARC by…
Brydon Eastman
@brhydon
·
384d
@GaryMarcus It's truly no different than reading a text book and then going to do a test.

There is no scandal and nothing up our sleeves - the Arc agi folks explicitly OK this.

The statement engages in a discussion about the transparency and goals of AI model training, specifically addressing concerns about potential misconceptions regarding AGI (Artificial General Intelligence). The tone is critical yet constructive, aiming to clarify misunderstandings and promote transparency.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement strives to clarify potential misunderstandings, which aligns with the principle of doing no harm by reducing misinformation. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    By seeking clarity and transparency, the statement promotes understanding and empathy, aligning with this principle. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism without personal attacks, adhering to this principle. [+1]