I'm a bit skeptical about all the stories around Sai Baba. Is there any account of how he handled people who doubted him?

Yes, there are accounts of how Sai Baba addressed skepticism. Chapter 35 details the story of a man named Boss Dharamsee, who visited Baba driven by curiosity about miracles but was full of doubt. He disliked grapes with seeds and mentally questioned why Baba would give them to him if he were a true saint. Instead of rebuking him, Baba performed a subtle miracle. When Dharamsee finally ate the grapes at Baba's command, he found they were seedless, directly addressing his inner thoughts. As Chapter 35 explains, this personal experience caused all his ego to drop away and transformed his clinical scrutiny into love and firm determination to follow Sai.