I see the same thing. Sometimes the the choices are of the same face in each photo and sometimes it choses the wrong person as one or more of the choices. Here are two examples of what I see with 5 images selected to pick from:
I don't see any way to force the app to use a specific face/photo but you can probably do it by masking out all the faces you don't want in the "replacement" photo and see if that gives you a usable selection. You can do that multiple times as needed but that sounds like a very good item to submit to CL for user feedback.
You can reach them from the Help menu then choose Rate Us & Provide Suggestions