Wasn't it harsh for Sai Baba to tell his followers to abandon people with 'fallen character'? How does this align with the concept of universal compassion?

According to the narrative in Chapter 3, Sai Baba gives very specific and strong advice regarding association with certain individuals. He states that one should abandon those who have abandoned religious conduct, referring to them as 'very dangerous.' Baba's instruction was to move aside when they approach and not even remain in their shadow, even if it means enduring suffering. He questions how someone who is thoughtless, without conduct, and whose actions are ruined can achieve their goals. This guidance, as presented, emphasizes protecting one's own spiritual path by avoiding corrupting influences, suggesting a form of compassionate protection for the devotee rather than a lack of compassion for the other.