1851-2001 Census info does not indicate any children for Sarah & James Stafford.
Does the MI specifically say that both James and Elizabeth were James Jr.'s parents?
If not:
There is an IGI birth of a James Sayle in Andreas on February 5 1841. The mother's name is Catherine Sayle and no father's name is given.
In the 1851 census at Lough in Andreas Catherine Sayle unm 44 b Andreas
James Staford 10 b Andreas.
From 1861-1901 he is recorded as James Stafford.