Pigots 1823/4 directory has under Peel (Professional Gentlemen)
Fell Jimes, surgeon
There is a somewhat caustic description by Hugh Stowell Brown (brother of T E Brown but unlike Tom Brown no lover of the Island)

The genuine Manx people were not remarkable either for temperance, diligence, or cleanliness. They were in those days chiefly small farmers, whose land was most wretchedly neglected, and whose houses and homesteads were utter horrors of discomfort, disorder, and filth. Another portion of the population— perhaps the smallest, but certainly the best—consisted of Scotch and English farmers who had immigrated, bringing with them their capital, together with skill and industry, setting an example which, however, the native Islanders were very slow to follow. The Isle of Man was a notable Cave of Adullam. Debts contracted elsewhere could not be sued for there. The place accordingly swarmed with men of broken fortune and questionable character who set their creditors at defiance. I once heard my aunt tell of an Irish baronet who finished in the Isle of Man a very disreputable career. His friends greatly desired that he should be buried in the family vault in Ireland; but there was the danger of having the corpse seized by the creditors on its being landed. To prevent this scandal, the burial service was read over the body in the hold of the vessel, as she lay in Douglas harbour; after which ceremony it would have been illegal to seize the remains. The conclusion of the great wars arrived at by the Battle of Waterloo had sent to the Isle of Man a large number of naval and military officers, who lived on their half-pay or their pensions; people poor and proud, and generally ill-informed, who spent their time in idleness, and did nothing to improve the tone of society. There was another and still worse element in the population—men given to drink—whose friends thought the Isle of Man the best place for them. There they could get drink very cheap, and would soon finish themselves.

(see http://www.ee.surrey.ac.uk/Contrib/manx/fulltext/hsb1888/ch01.htm) which might (just ?) explain the comment