Castletown St Mary's was a chapel of ease to parish church of Malew thus Baptisms were common but Marriages needed to be held at Malew - there have been at least 4 St Mary's - the oldest is the preserved Grammar school, chapel 2 was built under Bishop Wilson and was demolished c.1825 and the site reused to build the 3rd chapel which in 1970s was converted to office space with the 4th Chapel taking over the old National school - the 1826 Chapel started a new baptismal register which starts by including a few highly selective entries 'written out of the old register' thus it appears any Baptismal register for the old chapel is lost - such entries from chapels of ease were often copied into the register of the mother church but not always - thus it is likely that you will never find a baptismal record even it if existed.
I've looked at presentments for 1805-1807 but nothing I can see that might be your James - he Joins the 2 regt of foot - have you looked at the reference given (on my website www.manxnotebook.comhistory/military/regts.htm held by UK National Records at Kew to see if this gives his father's name ?