Hunting animals with the biggest horns, tusks or manes could lead to extinction, according to a study.