What Messes Up Your Alignment