Gary Marcus

Rank 13 of 47
|
Score 83
9/18/2024, 12:11:02 AM
X
In reply to:
Colin Fraser
@colin_fraser
·
493d
@GaryMarcus it is incredibly surprising and amazing to me that it does this as well as it does
Gary Marcus
@GaryMarcus
·
493d
Is 53% “decent” accuracy? Asking for a calculator friend.

Also: cool new study that (roughly speaking) highlights the difference between interpolation (green) and extrapolation (red). A calculator would be green in the whole grid; both GPT systems eventually fall apart.
Yuntian Deng
@yuntiandeng
·
494d
Is OpenAI's o1 a good calculator? We tested it on up to 20x20 multiplication—o1 solves up to 9x9 multiplication with decent accuracy, while gpt-4o struggles beyond 4x4. For context, this task is solvable by a small LM using implicit CoT with stepwise internalization. 1/4

The statement by @colin_fraser, 'It is incredibly telling that they still can’t hook up an external tool as simple as a calculator,' is a critical comment on the performance of AI systems in handling basic arithmetic tasks. The tone is somewhat sarcastic and dismissive, suggesting a perceived inadequacy in the AI's capabilities. The statement engages in public discourse by addressing the broader issue of AI performance and reliability.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not directly cause harm but could be seen as dismissive of the efforts of those working on the AI systems. Minor violation. [-1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement does not engage in cyberbullying, harassment, or hate speech. It respects the privacy and dignity of others. Neutral.
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement does not promote understanding, empathy, or compassion. It is more critical than constructive. Minor violation. [-1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement does not engage in constructive criticism or dialogue. It is more of a sarcastic remark than a constructive critique. Minor violation. [-1]