I forget lol, but in this game there are three different spectrums for your character. Good/Evil, Pure/Corrupt, and I forget the third. I think pure/corrupt mainly has to do with your financial decisions, and it changes the look of your character in a different way from good/evil.
Instruction manual:
Good and Evil
Many problems have both good and evil solutions. Your choices influence you physically: perform noble deeds and you'll be a dashing figure, give in to your evil impulses and your visage become the stuff of nightmares. Villagers will react accordingly.
Purity and Corruption
Purity makes Heroes more attractive, while corruption makes them ugly. Purity is determined by more subtle factors than g/e, such as greed, gluttony, and laziness.
So I guess you can fix your Pub Games penalty by wearing a nicer shirt.