Shirogane's reason for "running away" was actually just to lure the battle far away.
He couldn't guarantee that those Calamity Demons wouldn't suddenly appear, nor could he be sure that he wouldn't lose his mind in the heat of battle and destroy this entire area.
But most importantly...
Shirogane simply didn't want the people in that village to get hurt.
That was all.
Chapter 75