Whether people feel that a certain being (or group of beings) belong to the moral sphere, or even, at the simplest level, whether they feel concern for the welfare and interests of that creature, depends on one simple thing: that is, whether they identify with that being or not.
The beings we feel concern for (and subsequently we let them enter into our moral sphere) are the ones whose suffering we feel as our own, for some or other mechanism of identification.
The other beings, the ones left out, are those who we can’t identify with and therefore, for biological reasons, we perceive them as antagonistic to us, competing for the same resources (it’s a law of nature, a rule of life), rivals, opponents, enemies.
I think that we cannot escape from this scenario.
What happens is that, when people don’t care about some-one or some group (and therefore the latter fall into the second category described above), but they know that they should care, in the sense that it’s socially expected from them (for reasons of political correctness, for example, or, put in another way, current political orthodoxy, fashionable ideology, or anyway because it’s the accepted social norm geographically and historically), what happens then is that people fake concern, or at least some respect for the social norm: it’s very easy to do.
Interestingly, there’s a connection with psycho-therapy, here.
Something similar happens in psychotherapy.
Whether a psychotherapy treatment succeds or not depends (almost entirely, with perhaps a few exceptions due to exceptionally good techniques) on whether the psychotherapist feels empathy for the client or not, ie whether the therapist can identify with the client and, so to speak, feel his/her pain. If the therapist doesn’t, s/he doesn’t really care about helping that person and most likely s/he won’t, for the reasons I’ve given above: ie it’s a law of nature that mors tua vita mea, the therapist may even feel rather good that somebody else and not him/herself is suffering (especially if the therapist has a lot of problems of his/her own, which I think is an extremely common case).