It will always be fascinating to try to understand why our imagination works the way it works, and the more we explore the issue, the more common behaviors we find.
The Jungian-based Christopher Booker classification you cite is very interesting and certainly covers a very broad spectrum of recurring cases. Your reflection brought to mind the work of a Russian theorist who explored these issues very early, albeit from the perspective of folklore: Vladimir Propp.
This analyst located, after the analysis of a hundred or so Russian folk tales, 31 common functions, covering an extraordinary number of universal narrative cases. His conclusions were published in 1928, in the book Morfológuiya skazki, and influenced studies later made by Claude Lévi-Strauss and Roland Barthes, who modified their categories and adapted them for application in Western and contemporary stories.
Beyond the technical and theoretical aspects, what I find most attractive about these reflections is what you so clearly point out at the beginning of your post:

In my opinion, that is the most important rule for a story – that it should address something vital within us, a fear, a hope, a dream, a need. It might be interesting to take a second and think about the book you're currently reading – why are you reading it? How does it serve and what do you enjoy about it? What does it make you feel?

These issues are also the foundation of the studies that you quote, those that I have cited and, of course, the study of Jorge Luis Borges that he has cited in his comment @josemalavem.
Borges, the great reader who was a writer and critic, is proof that interests, in these cases of imagination, are common.
Thank you for putting us on such a wonderful network of readings and reflection with this series. I am grateful, @honeydue.