Gary Marcus

Rank 21 of 47
|
Score 54
4/7/2024, 5:25:56 PM
X
In reply to:
Danrae Pray
@DanraeP
·
449d
@Simeon_Cps @GaryMarcus @StatsLime @WSJ No, synthetic data doesn't contain any new information and majority of it is data augmentation.

This is why half the Sora videos look like real-time renders from UE5 with a first-person character controller or camera-boom for character movement.
Siméon
@Simeon_Cps
·
449d
@GaryMarcus @DanraeP @StatsLime @WSJ That's also why the human metaphor I made makes sense. Humans say bs stuff all the time but fortunately we teach kids some highly curated content.

For that to work we just need to be able to generate sometimes some good content and filter the rest out.
Gary Marcus
@GaryMarcus
·
449d
@Simeon_Cps @DanraeP @StatsLime @WSJ @Simeon_Cps I don’t that I fully follow your distinction. People often use the terms more or less interchangeably, don’t they? How are you distinguishing?
Siméon
@Simeon_Cps
·
449d
@DanraeP @StatsLime @GaryMarcus @WSJ That's not what people are doing with synthetic data and LLMs. People are using models to generate maths proofs than they then check with theorem provers that you can then throw in the dataset if it's correct. It's not "rearrangement".

Don't conflate synthetic data and data…
Danrae Pray
@DanraeP
·
449d
@StatsLime @Simeon_Cps @GaryMarcus @WSJ Yes, precisely! So data augmentation can be helpful for reinforcing information that's already in the data set, but not for teaching a gazillion edge cases which aren't already in the dataset.
Limelihood ⏸️ Function
@StatsLime
·
449d
@DanraeP @Simeon_Cps @GaryMarcus @WSJ It's more about hoping to teach it that those slightly rearranged sentences can still mean basically the same thing

The statement is part of a technical discussion on the nature and use of synthetic data and data augmentation in the context of machine learning and artificial intelligence. The conversation involves a clarification of terms and concepts within the field.