Gary Marcus

Rank 31 of 47

Score -2

@olivia_p_walker what i mean is that the openai system may have been trained either directly on the problems in the dataset, or trained on augmented/synthetic data similar to the problems.

that’s what contaminations means in this kind of ML context.

1/19/2025, 8:56:46 PM

In reply to:

Olivia P. Walker

@olivia_p_walker

50d

@GaryMarcus How are the datasets contaminated? specifically.

Gary Marcus

@GaryMarcus

50d

Really bad look for OpenAI if this whole thread checks out:
- Looks like “Open”AI may have tried to *close* an important benchmark, FrontierMath, to the community.
- Sounds like they have mislead some top mathematicians.
- This all raises serious questions whether about data…

Carina Hong

@CarinaLHong

50d

@GaryMarcus Six mathematicians who significantly contributed to the FrontierMath benchmark confirmed this is true - that they are unaware that OpenAI will have exclusive access to this benchmark (and others won’t). Most express they are not sure they would have contributed had they known.

The statement addresses concerns about potential data contamination in machine learning models, specifically in the context of OpenAI's practices. It aims to clarify what 'contamination' means in this context, contributing to a broader discussion about transparency and ethical practices in AI development.

Principle 1:
I will strive to do no harm with my words and actions.
The statement strives to clarify a technical term, which can help prevent misunderstandings and misinformation, aligning with the principle of doing no harm. [+1]
Principle 2:
I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
It respects privacy and dignity by focusing on technical aspects rather than personal attacks or accusations. [+1]
Principle 3:
I will use my words and actions to promote understanding, empathy, and compassion.
By explaining a complex issue, it promotes understanding and empathy among those engaged in the discussion. [+1]
Principle 4:
I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
The statement engages constructively by providing information rather than attacking individuals or organizations. [+1]