is he not mentioned on the birth or marriage certs of his offspring -
you dont give name but if was Fowler then there were no male Fowlers in 1881 census (only two were a vistor at a hotel and the neice of Charles Udall who was running Villiers Hotel)