As AI models get more powerful, we need to ensure that their behaviour continues to align with human intentions and preferences.