Could it be that he had no choice but to give up?
If giving up were an option, he certainly wouldn't have been willing to stay and waste time on the adventure; he must have chosen to leave temporarily, knowing that getting back in later wouldn't be nearly so easy.
Judging by Su Chen's guess, he believed his appearance would definitely prompt Xuan Yinji to summon everyone from the Primordial Clan.
Chapter 1607