Phase change materials (PCM) provide a unique property combination. Upon the transformation from the amorphous to the crystalline state, their optical properties change drastically. Short optical or electrical pulses can be utilized to switch between these states, making phase change materials attractive for photonic applications. We review recent developments of PCMs and evaluate the potential for all-photonic memories. Towards this goal, the progress as well as existing challenges to realize waveguides with stepwise adjustable transmission is presented. Colour-rendering and nano-pixel displays form another interesting application. Finally, nanophotonic applications based on plasmonic nanostructures are introduced. They provide reconfigurable, non-volatile functionality enabling manipulation and control of light. Requirements and perspectives to successfully implement PCMs in emerging areas of photonics are discussed.