Gary Marcus

Rank 12 of 47
|
Score 83
9/18/2024, 10:04:49 AM
X
In reply to:
Robert Mankoff
@bob_mankoff
·
494d
@GaryMarcus o1-preview conveniently left out. Also, nice little tour de force of preview being able to put numbers this large into a words. Of course, this is a silly way to do it, the first part anyway. Just have the LLM use python.
Gary Marcus
@GaryMarcus
·
494d
Is 53% “decent” accuracy? Asking for a calculator friend.

Also: cool new study that (roughly speaking) highlights the difference between interpolation (green) and extrapolation (red). A calculator would be green in the whole grid; both GPT systems eventually fall apart.
Yuntian Deng
@yuntiandeng
·
495d
Is OpenAI's o1 a good calculator? We tested it on up to 20x20 multiplication—o1 solves up to 9x9 multiplication with decent accuracy, while gpt-4o struggles beyond 4x4. For context, this task is solvable by a small LM using implicit CoT with stepwise internalization. 1/4

The statement and the associated conversation revolve around the accuracy and capabilities of different AI models in performing mathematical calculations. This discussion touches on the broader public issue of AI performance and reliability, which is relevant to public discourse. The tone is analytical and somewhat critical, aiming to highlight the limitations and potential improvements in AI technology.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is focused on a technical discussion about AI capabilities. It adheres to the principle of striving to do no harm. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by discussing the performance of AI models in a specific context. It contributes to a better understanding of AI limitations and potential areas for improvement. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism by pointing out the limitations of the AI models without resorting to personal attacks or ad hominem arguments. [+1]