The people getting into Harvard, many of them get in because of social connection’s rather than merit. The social connections of their parents. It’s also super common to just buy their way in.
The students getting in on merit are the ones they let in because they have a reputation to maintain and can’t have everyone being dumb. That would look bad.
Further, even if that weren’t true, the reason “the most qualified” seem to be rich white kids from rich white families is simply because they can afford much better educations. So, if you want to promote diversity it can’t just be about academic qualifications because you have people who’ve always been in power buying academic quals for the kids- so they always will be in power.
Affirmative action isn’t about taking away from one group and giving it to another, it’s about giving back to a group that historically has been taken from- its about righting a past wrong- you can’t do that solely on merit, when the system was designed to perpetuate a certain kind of status quo