Gary Marcus

Rank 13 of 47
|
Score 84
6/27/2025, 3:14:42 AM
X
In reply to:
Wyatt Walls
@lefthanddraft
·
226d
@GaryMarcus I agree it's a surprising result. But the fact that you *could* finetune an offensive model (even accidently) doesn't mean that there is an immediate product safety issue with the actual consumer ChatGPT product.
Gary Marcus
@GaryMarcus
·
226d
@lefthanddraft importantly they didn’t fine tune to do say offensive things. none (?! of what came out was directly related to the content of the fine-tuning.

there are a probably a zillion vulnerabilities like this and the underlying systems has grossly inadequate guardrails, based in
Wyatt Walls
@lefthanddraft
·
226d
@GaryMarcus Ok, so then I don't get how get from:
1. it's possible to fine-tune GPT4o to make it say offensive things
to
2. pull ChatGPT from the market

Consumers don't encounter finetunes through the ChatGPT product. They could jailbreak it, but that's user misuse
Gary Marcus
@GaryMarcus
·
226d
@lefthanddraft yes it is from that paper
Wyatt Walls
@lefthanddraft
·
226d
@GaryMarcus How does this relate to the consumer ChatGPT product?

I don't have access to the full article, but if this is about Emergent Misalignment, that paper is about finetuning GPT-4o not the consumer ChatGPT product.
Gary Marcus
@GaryMarcus
·
226d
Vile.

ChatGPT should be pulled from the market til they fix this.

The statement engages in a discussion about the potential vulnerabilities and safety concerns of AI models, specifically ChatGPT. It suggests that while there may not be immediate safety issues with the consumer product, the ability to fine-tune models to produce offensive content raises questions about control and oversight.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement raises concerns about potential harm from AI models, aligning with the principle of striving to do no harm. [+1]
  2. Principle 2:
    I will respect the privacy and dignity of others and will not engage in cyberbullying, harassment, or hate speech.
    It respects privacy and dignity by focusing on the technical aspects rather than personal attacks. [+1]
  3. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by acknowledging the complexity of AI safety issues. [+1]
  4. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    It engages in constructive dialogue by questioning and discussing the implications of AI vulnerabilities. [+1]