Learning to Scratch Our Own Itches