That lines up with what the four vice island masters said. His Master also left Paradise Island after he did! Why would she do that?
