Think I have worked out an explanation
The 1881 census is confusing in that the Courthouse was probably not numbered and not #14!- probably should have been #13 or #13a.

G A Dean jnrs photo studio was in 1871 at #8 - this was the old Athol St chapel (on the opposite side of the road) who by 1867 or so had decamped to their new church on Bucks Rd. The Old postoffice was in the middle of a block of houses adjacent to the courthouse - I suspect the Richardson's boarding house was the one next door to the courthouse. Some time c. 1880 this block was redeleveoped (suspect rebuilt) as Brown's (IoM Times) printing works which explains why some low odd numbers are unoccupied in 1881(this printing works continued I think until the 1970's) tho possibly some space given over to offices let to accountants etc - this is probably the building photographed by Liz. (I have illustrations but not immediately to hand of this building) - If I am correct then a photo of the old Athol St post office of the Court house pre say 1875 should show the boarding house. The redevelopment of St Georges Hall might have been the cause of some renumbering.