There will always be fanatics. I am impartial in my own views as to the legends surrounding Arthur but it may be that an author of one or other of the early legends used the Castle as the model upon which to base his topography. Hero and returning saviour myths span millenia and take many forms and occur in many cultures. They seem to be part of the human psyche.
For anyone interested in the subject, a novel slant is provided by Howard Reid in his book 'Arthur the Dragon King' in which he ties it in with the Sarmatians and the Alans. Well worth a look.