That does sound wonderful. Like yourself, I waited and married very well. We also have one son that I was blessed to spend lots of time with through his youth. Now in his teen years the new interests have cut in on time spent together. Time and growth changing the situation.
I can imagine how hard it is to lose the frequent father-son bonding time, but also how good it feels to know that your son is growing up. My wife and I were recently discussing that I will probably be crying when he leaves for college... Which is potentially true.. But so is she!