Learn As You Listen