Casting the Forgetfulness Charm on Umbridge gave Albert not a single shred of psychological burden; neither of them was any sort of good person anyway, and since they already had a conflict between them, not just outright finishing her off could already be considered kindness of the heart.
Indeed, Albert never considered himself a good person.
Perhaps influenced by the notion from his past life that good people do not reap good rewards, he possessed a flexible moral baseline and instinctively rejected such things.
Chapter 944