Gary Marcus

Rank 14 of 47
|
Score 83
Gary Marcus
@GaryMarcus
·
376d
The replies in this thread shows that there are lots of people *within the ML community* raising the same concerns re how OpenAI discussed ARC that I was making. (And more concerns, e.g, in important comments about how the graph was labeled).

Importantly, this an issue about…
12/22/2024, 2:51:45 AM
X
In reply to:
Olcan
@olcan
·
376d
@mikeknoop @doomie @circlerotator it just seems odd to put tuned methods and untuned models (or even differently tuned models) on the same chart since tuning makes such a huge difference; i mean there is a very similar looking chart in the TTT paper based on tuning ablations alone
Mike Knoop
@mikeknoop
·
376d
@doomie @circlerotator ARC Prize foundation is a nonprofit
Dumitru Erhan
@doomie
·
376d
@circlerotator @mikeknoop didn't realize the ARC folks are a business
shaperotator (e/λ)
@circlerotator
·
376d
@doomie @mikeknoop they just wanted to release in time for christmas, it's a business not a college social club
Dumitru Erhan
@doomie
·
376d
@mikeknoop What was the rush? Why come and effectively declare victory without doing some rigorous analysis? Isn't the whole point of this prize to be rigorous in its evals rather rush to meet an agenda or timeline of a particular participant? o3 is indeed very capable, everyone agrees,…
Mike Knoop
@mikeknoop
·
376d
@doomie we want to do the ablation study! we just didn't have enough time before the announcement. i still don't think this takes anything away from the fact o3 is a uniquely capable system

The statement engages in a discussion about transparency and scientific rigor within the machine learning community, specifically addressing the importance of replicability and proper evaluation methods. It supports the call for transparency and proper scientific practices, referencing a replicability checklist as a tool for ensuring these standards.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement promotes transparency and proper science, which aligns with the principle of doing no harm by advocating for ethical scientific practices. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    By appreciating voices within the community that insist on transparency, the statement promotes understanding and empathy towards the need for rigorous scientific standards. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive dialogue by referencing a replicability checklist and does not engage in personal attacks, thus adhering to this principle. [+1]
  4. Principle 6:
    I will use my influence for the betterment of society.
    The statement uses its influence to advocate for better scientific practices, contributing to the betterment of society by promoting transparency and replicability in research. [+1]
  5. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds free speech by responsibly using its platform to discuss important issues in the ML community. [+1]