Large language models (LLMs) trained with reinforcement learning from human feed- back (RLHF) struggle with complex instructions that bundle multiple, potentially con- icting requirements. We introduce constraint decomposition, a framework that separates multi-objective instructions into orthogonal componentssemantic correctness, structural organization, format specications, and meta-level requirementsand optimizes each in- dependently before hierarchical combination. Our approach addresses the fundamental limitat…