The ability to optimally control quantum systems in the presence of environmentally-induced decoherence is important for many physical and chemical problems. We discuss both theoretical and experimental aspects of optimal control of open quantum systems. The theoretical analysis is based on fundamental concepts of open-system controllability and control landscapes. These theoretical advances lay the groundwork for practical applications, including numerical simulations and experimental implementations of adaptive feedback control. In particular, the adaptive approach was utilized to implement coherent control of decoherence, which uses coherent (unitary) preparation and manipulation of an open quantum system to significantly alter its incoherent (non-unitary) dynamics caused by coupling to an environment. We also discuss other applications, including optimal dynamic discrimination of similar quantum systems and optimal control of high-fidelity quantum gates in the presence of decoherence.