Susan, you may have the wrong census entries for 51 and 61.

Have a look at ancestry 1851 Maughold district 2a page 19 ---
William 36 journeyman mason Lezayre
Margaret 47 Andreas
Thomas 10 Andreas
William 7 Ramsay

ancestry 1861 Maughold district 2a page 9 ---
William 48 stone mason Lezayre
Mary 55 Bride
Thomas 19 stone mason Andreas
William 17 farmer Ramsay

These tie in with your 1871 suggestion ( except 1851 wife is written as Margaret rather than Mary ), and Mary A could be Mary Ann, as in Brian's burial index

CLAGUE Mary Ann 75 Mau 5 Jan 1881.

In 1881, William is a widower ( mason ) living with son Thomas, d in law Catherine, and 5 children.

Jean