Today, the immense language models used in artificial intelligence are remarkable achievements. Their impressive language capabilities and vast knowledge are a result of substantial amounts of data, capital, and time invested. Development costs can exceed $100 million and involve months of testing and refinement, both by humans and machines. These models undergo iterative processes, refining themselves millions of times to strive for closer approximations to the "correct answer" and improve with each attempt.

However, encoding human values remains a challenge. This additional step, referred to as Reinforcement Learning from Human Feedback, involves using programmer responses to train the model to be helpful and accurate. To ensure safety and alignment with human values, "red teams" deliberately provoke the program to identify any potential harmful outputs. This combination of human adjustments and guardrails has been reasonably effective in maintaining AI's alignment with human values and overall safety.