Definitely worth checking out. I think "Elicia" is an older version of "Alice" and she may have continued to use it to some extent later on, which helps explain the "Elsie" in 1841. The surname Coulthard seems to be very strong in Cumberland and this family clearly has connections there. The pronunciation on the island of that surname might have been hard to pick up for a census taker or for a parish clerk.
If this is them, she would have been about 19 or 20 in 1789. The first child we've identified is Mary baptized Jan 1801 so there's a 10 year gap for other potential children yet to be found, if there were any. Maybe the Coulthard witness to the two weddings, Richard, born in the 1790s in Cumberland belongs to them?