I'm managing to piece bits of this puzzle together a little now. I can't find anything concrete to link Benjamin Swan with the IoM other than he married Jeanette Jane Kelly, born in Peel. Whether they met in Yorkshire or he met her in the IoM isn't clear, but she married him in Yorkshire. What's become apparent and may be the cause of the confusion is that Jeanette went back to the IoM to have her babies, Eric and Ella, though it doesn't look like they actually lived there for any period.