From his long relationship to Maureen Kerr (from ca. 1959 to ca. 1969) he had one son, Rupert Alexander (b. December 9th 1966, d. February 5th 2015).
His first wife was children's writer Marcia Williams (b. August 8th, 1945) (from ca. 1969 to ca. 1971) and they had one daughter, Araminta.
His second and current wife is actress
Jane Asher (b. April 5th, 1946). They met in 1971, and they married ten years later. They have three children, daughter actress
Katie Scarfe (born April 17, 1974), and sons
Alex Scarfe (born December 1981) and Rory (born 1984).