Gary Marcus

Rank 12 of 47
|
Score 83
4/22/2025, 9:28:51 PM
X
In reply to:
Juggalos For Context🌴🥥
@ebarcuzzi
·
273d
@GaryMarcus @colin_fraser You can also look at benchmarks that aren't being used anymore because they got saturated. Quite a few of those.
Juggalos For Context🌴🥥
@ebarcuzzi
·
273d
@GaryMarcus @colin_fraser Addition and multiplication up to an ever expanding number of digits is one example.
Gary Marcus
@GaryMarcus
·
274d
@ebarcuzzi @colin_fraser on what is an LLM 99.9% accurate? source/link?
Juggalos For Context🌴🥥
@ebarcuzzi
·
274d
@colin_fraser @GaryMarcus Also no one really reports on break throughs in LLMs going from 99% accurate to 99.9% accurate on certain tasks. Not an attention grabbing headline.
Colin Fraser
@colin_fraser
·
274d
the set of things an LLM can do with about 70% accuracy is growing to be quite large, larger than anyone ever thought possible, but it remains quite challenging to efficiently use something that works 70% of the time

The statement is part of a broader conversation about the effectiveness and evaluation of large language models (LLMs). It touches on the issue of 'teaching to the test,' which is a critique of how benchmarks and evaluations might not fully capture the capabilities or limitations of LLMs. This discussion is relevant to public discourse as it addresses the challenges in assessing AI technologies, which have significant societal implications.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is part of a constructive discussion about AI evaluation methods. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement contributes to understanding by highlighting a potential issue in AI evaluation, promoting a more nuanced view of LLM capabilities. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue without personal attacks, focusing on the topic of AI evaluation. [+1]