AI alignment
The problem of ensuring AI systems pursue intended goals and values, not just what they were literally optimized for on training data.
AI alignment (or “the alignment problem”) is about building AI, especially large LLMs and AI agents, so they behave in line with designer and user intent, including safety constraints.
Practical levers include curated prompts, filters and guardrails, human feedback, and reinforcement learning from preferences (RLHF). Misalignment shows up when models optimize proxy rewards, follow harmful instructions, or act competently toward the wrong objective. Long-horizon worries often pair alignment with AGI, superintelligence (ASI), and singularity speculation, not with everyday chatbot use alone.