In his recent work, Krisztian Buza challenged the aforementioned “widely acknowledged truth” in context of data augmentation. His observations show that rich training data may be much more valuable than augmented (i.e., artificially generated) data, and – most importantly – the advantage of a sophisticated algorithm relative to a simple algorithm may not be easily compensated by data augmentation.
Paper: K. Buza (2023): Data Augmentation Does Not Necessarily Beat a Smart Algorithm, 12th Japanese-Hungarian Symposium on Discrete Mathematics and Its Applications.