Gary Marcus

Rank 14 of 47
|
Score 104
1/20/2025, 8:16:31 AM
X
In reply to:
Anup
@anupk24
·
14h
@GaryMarcus I don’t follow — yes we had access to an eval set. Is the claim that you don’t trust OpenAI to not train on the eval set? Do you trust that we didn’t train on SWE-Bench’s eval set? Or GPQA? I’m excited for o3 to be released so you can check for yourself what it’s good and bad at
Gary Marcus
@GaryMarcus
·
1d
Scientific alert: OpenAI’s apparently impressive results with o3 on FrontierMath should be taken with a huge grain of salt.
- OpenAI had access to the problems and solutions.
- We don’t know what they trained on.
- We don’t know what problem-tailored validation or data…
Elliot Glazer
@ElliotGlazer
·
1d
As for where the o3 score on FM stands: yes I believe OAI has been accurate with their reporting on it, but Epoch can't vouch for it until we independently evaluate the model using the holdout set we are developing.

The statement by @GaryMarcus constitutes public discourse as it engages with issues of transparency and trust in AI development, specifically regarding OpenAI's practices. The tone is critical and skeptical, focusing on the lack of transparency about training data and the potential implications for trust in AI results.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not directly harm others but raises concerns about transparency, which is a legitimate issue in public discourse.
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    The statement respects privacy and dignity, focusing on institutional practices rather than personal attacks.
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement aims to promote understanding by highlighting transparency issues, though it could be more empathetic in tone.
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in criticism of OpenAI's practices without resorting to personal attacks, maintaining a focus on institutional accountability. [+1]
  5. Principle 5:
    I will acknowledge and correct my mistakes.
    There is no indication of acknowledgment or correction of mistakes within the statement itself, as it is primarily a critique.
  6. Principle 6:
    I will use my influence for the betterment of society.
    The statement uses influence to highlight potential issues in AI transparency, contributing to societal awareness. [+1]
  7. Principle 7:
    I will uphold the principles of free speech and use my platform responsibly and with integrity.
    The statement upholds free speech principles by responsibly critiquing a major AI entity, though it could be more balanced in its approach. [+1]