Gary Marcus

Rank 19 of 47
|
Score 64
12/20/2024, 9:25:45 PM
X
In reply to:
Michael Gruen
@Michaelgr1011
·
202d
@GaryMarcus My big (underrated) AGI test is its ability to handle litigation motion practice/legal research. IDK if that is ridiculous of me, but it seems like something that requires deep thinking, understanding real nuance, and general intelligence of the law and case law. So far every…
Gary Marcus
@GaryMarcus
·
202d
@Michaelgr1011 yes, 100%.

what was most notable was almost all the evals were coding and one other heavily discussed benchmark; nothing (at least that I saw) addressed to open-ended reasoning, and certainly nothing establish reliability.

fan boys seem confused about what they saw.
Michael Gruen
@Michaelgr1011
·
202d
@GaryMarcus still think these predictions are correct post seeing the evals?
Gary Marcus
@GaryMarcus
·
202d
Three o3 predictions:
– People will initially be amazed
- Once they dive in, they will see that it is not reliable
- It will work best in closed-domains (like math problems) but less reliably in open-ended domains (like everyday reasoning about the real world).

The statement is part of a public discourse on the evaluation of artificial general intelligence (AGI) and its capabilities, particularly in handling complex tasks like legal research. The conversation involves a critique of current AGI evaluations and their focus on coding rather than open-ended reasoning.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is part of a constructive discussion about AGI capabilities. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by discussing the limitations of current AGI evaluations and suggesting areas for improvement. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue without personal attacks, focusing on the topic of AGI evaluation. [+1]