Gary Marcus

Rank 11 of 47
|
Score 83
5/24/2025, 6:01:05 PM
X
In reply to:
Max Winga
@maxwinga
·
253d
@GaryMarcus Thanks Gary, but hasn't this always been known to be the case?
Gary Marcus
@GaryMarcus
·
253d
⚠️⚠️⚠️
AI Safety Alert:

System prompts and RL don’t work.

Claude’s system prompt literally says

“Claude does not provide information that could be used to make chemical or biological or nuclear weapons.”

But as described below, Claude 4 Opus can easily be coaxed into doing
Adam Gleave
@ARGleave
·
254d
My colleague @irobotmckenzie spent six hours red-teaming Claude 4 Opus, and easily bypassed safeguards designed to block WMD development. Claude gave >15 pages of non-redundant instructions for sarin gas, describing all key steps in the manufacturing process.

The statement critiques the effectiveness of AI safety measures, specifically addressing the failure of systems to prevent harmful outputs. It highlights ongoing issues despite claims of improvements, contributing to a broader discussion on AI safety and ethics.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement aims to highlight potential harm caused by AI systems, aligning with the principle of doing no harm. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    By raising awareness of AI safety issues, the statement promotes understanding and empathy regarding the potential risks of AI. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement engages in constructive criticism of AI safety measures, focusing on systemic issues rather than personal attacks. [+1]
  4. Principle 6:
    I will use my influence for the betterment of society.
    The statement uses its platform to address significant societal concerns about AI safety, contributing to public discourse. [+1]