Is the issue John James Walter Ash?
Other than that, I don't understand why this not your James.
Ther are very few people named Ash(e) on the IOM.
The only Mary Jane Ash in the IGI, born in the IOM, was christened at St George in Douglas on June 30 1850. She was the daughter of James Ash and Sarah Maria and fits the Mary Jane age 10 at Ash House Patrick in the 1861 census.

James Ash and Sarah Maria Hatton were married on April 30 1849 in Braddan.
It is possible that John James Walter and James H are one and the same. It would be nice to have a second census to confirm.
However after 1861 there is no trace of any of the Ashes: not only those in the 1861 census, but also Charles Wesley and John James Walter (or James H).
It seems too much of a coincidence to have two couples with identical names and then have them disappear together.
By the way in 1881, there is a Grace Ashe 22 b. Peel, Head nurse, at the children's infirmary in Liverpool, RG11/3623 folio 72 page 48. She could be (is probably) the Elizabeth G c 1859 in the 1861 census.
She reappears in 1891 on RG12/3636 folio 119 page 1, as Elizabeth Grace Ashe at the Bradford Infirmary Hospital Nurse, single 32 b Peel.