Your grandfather and his brothers might all have been born in Ireland, and therefore wouldn't be in English/IOM records, unless you know otherwise (you gave no places). If you really want accurate family history you need to start from known information (official records), as previously suggested, and work your way back starting with your parent who was John James's son/daughter, and what was recorded on his/her birth and marriage certs. When you have those you can see what was recorded for them all in the most recent available census, whether it was the UK or Ireland or US or wherever (if in a country which didn't destroy them). If Margaret was b.IOM this should appear in censuses.

Sue