Following a little more research I think you may have the wrong James Kelly - there is a James Kelly Stone Cutter, stonemason + house builder in Castletown who married an Isabella Killey in 1782 - I have a sequence of deeds entered in Castletown Deeds sequence
re a house in Queen Street which in one deed names his sons John, Henry + Joseph and daughter Christian to whom the house was given in 1831 with possession at death of both parents and noted as owner in a deed dated 1844v when a mortgage was paid off - thus I suspect the James Kelly mason buried in 1837 was this James Kelly and not your suggested James.