Gary Marcus

Rank 16 of 47
|
Score 72
4/7/2024, 4:15:27 PM
X
In reply to:
Siméon
@Simeon_Cps
·
448d
@DanraeP @StatsLime @GaryMarcus @WSJ That's not what people are doing with synthetic data and LLMs. People are using models to generate maths proofs than they then check with theorem provers that you can then throw in the dataset if it's correct. It's not "rearrangement".

Don't conflate synthetic data and data…
Danrae Pray
@DanraeP
·
448d
@StatsLime @Simeon_Cps @GaryMarcus @WSJ Yes, precisely! So data augmentation can be helpful for reinforcing information that's already in the data set, but not for teaching a gazillion edge cases which aren't already in the dataset.
Limelihood ⏸️ Function
@StatsLime
·
448d
@DanraeP @Simeon_Cps @GaryMarcus @WSJ It's more about hoping to teach it that those slightly rearranged sentences can still mean basically the same thing
Danrae Pray
@DanraeP
·
449d
@Simeon_Cps @GaryMarcus @WSJ That is different than data augmentation though.

Data augmentation is like taking a US history book, changing up the order of the sentences inside it, and hoping you can now use it to teach the kids physics 😅

Might learn some physics, but not much
Siméon
@Simeon_Cps
·
449d
@DanraeP @GaryMarcus @WSJ Teaching kids with content produced by other humans to optimize their learning is literally how humans learn.
Danrae Pray
@DanraeP
·
450d
@GaryMarcus @WSJ This whole data augmentation thing feels a lot like the AI analogue of the perpetual motion machine 😅

The statement is a polite request for clarification in a technical discussion about AI and data augmentation. It is part of a larger conversation and does not contain any harmful or disrespectful language.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does not cause harm and is part of a constructive technical discussion. [+1]
  2. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement promotes understanding by seeking clarification on a point of confusion. [+1]
  3. Principle 5:
    I will acknowledge and correct my mistakes.
    The statement is open to correction and further discussion, as it asks for an explanation. [+1]