There was 'more than a grain of truth' in what Lord Young said