There were no Milne families in 1881 census - in 1890's there were 2 - a John Milne was light-house keeper at Douglas Head (St Barn would probably be nearest) - as the light was operated by Northern Commissioners of Lights these men were usually scots and moved between lights on a promotion ladder scheme (see my pages under maritime/lighthouses). Most appeared to be presbyterian rather than CofE and I would have thought used St Andrews - however the Scottish name of groom fits in well on this thesis.
The other Milne family was that of Peter Milne who ran the Waverley guest house in the old Metropole block (N Douglas bay) - I have his advert from 1904 and it would appear he came from cumberland - in 1908 he appealed against lack of a drinks license.