This does not look like the right Thomas, unless he was a bigamist.

1851 Census Port St Mary Rushen HO107/2526 folio 342 page 4
Thomas Miller Head Mar 34 Innkeeper b Douglas
Thersea Wife 30 b. Bergen Holland
Theresea Dau 6 mo b Rushen
Robert 5 nephew b Douglas
Margaret Murphy servant 18 b Douglas

1861 Rushen Port St. Mary District 4 Howe RG9/ folio 82 page 3

10 years later, Thomas, both Thomaseas and Robert have been joined by William 8, Mary 7, Jessie 5, Clara 3 and Thomas 9 months. They now have a bar maid and a house maid.