The decline of Moore's Law and Dennard scaling has precipitated the emergence of a panoply of specialized hardware such as GPUs, TPUs, and many others. The success of a computing ecosystem that features such rich architectural diversity is predicated on the ability of programmers and system designers to use that specialized hardware to improve performance and increase efficiency. We need that effort to succeed as data centers and computing infrastructure continue to consume increasing and alarming fractions of the world's energy. Unfortunately, building systems that realize the performance and efficiency promises of specialized hardware is a significant challenge that requires innovation across the technology stack. The structure of modern system software still reflects a vanishing ecosystem dominated by general purpose processors and shallow technology stacks. That structure imposes many fundamental obstacles to leveraging acceleration in many computing domains. This talk will reflect on a decade of research devoted to encouraging the evolution of system software to accommodate an increasingly accelerator-rich world, focusing on a handful of recent research results, including OS and PL support for FPGA-backed accelerators, and hypervisor support for accelerators.