While Carl Jung and Sigmund Freud were still on speaking terms, Freud insisted they must never abandon “the sexual theory”: his central dogma of repressed sexual material as the origin of neurosis. The sexual theory was the vital barrier, he said, because without it they would be facing “the black tide of mud – of occultism”.

The kind of thing Freud was so anxious to block out is overwhelmingly manifest in this seven-volume set of Jung’s notebooks, spanning 1913–32. They represent the coalface of Jung’s introspection, from which he mined and polished his more accessible Red Book (perhaps his seminal work, despite remaining unpublished until 2009). The Black Books detail Jung’s visionary encounters with entities such as Phanes the star god, Ha the sorcerer, and Philemon, the wise magician who became Jung’s internal guru.

The technique of this introspective exploration is what Jung would later call “active imagination”, a wakeful dreaming where the imagined takes on a life of its own. It is a process that invites metaphors: Jung fantasized that with his deep introspection he was digging a hole. The pre-eminent Jung scholar Sonu Shamdasani, in his characteristically accomplished introduction, suggests Jung might even have started by physically digging in his garden. The attempt to reach this notionally lower level by removing conscious restraints and controls involved an abaissement du niveau mental (the French term has stuck from Jung’s teacher Pierre Janet) that allowed unconscious material to appear. Jung felt there was something in the background of his dreams that could come forward, if only he switched off conscious control as much as possible: “One submits to the necessary conditions – as in a mescaline experiment – so that it emerges”.

Some fairly extraordinary things did emerge, supposedly from the unconscious. Meeting his soul was a declared intention at the very start of the project, as if to combat the historical “disenchantment of the world” noted by Max Weber. When he did meet her (female by longstanding convention rather than some personal idiosyncrasy, but clearly overlapping with Jungian ideas of the anima), she spoke to him like an external being and gave him some stern relationship advice: “no more letters to women, no moaning”. In addition to his wife, Jung was involved with two women, “the white one” and “the black one” (identified by Shamdasani as blonde Maria Moltzer and brunette Toni Wolff)…