Transfer learning (or student-teacher training) isn't really the same thing as pre-training as it was being talked about 10 years ago. And in those days the claim wasn't that it helped a bit, but that it made training many deeper networks possible at all.