This is shocking and amazing. Somewhat confusingly, the OED's first definition for "yes" explicitly concerns questions "not involving a negative", which will be extremely perplexing to anyone who reads Brewer and fails to get down to definition two in the OED, which is concerns questions that do involve a negative. (One wonders why the first definition comes first, and whether the first definition really is limited only to questions that don't involve a negative. I'm not sure how to discriminate cases here, but then, I'm not a lexicographer.) Evidently the "yes"/"yea" distinction has been inactive a while:
2. a. In answer to a question involving a negative.
Formerly regularly used thus (and as in b) in distinction from yea (see YEA 1); the distinction became obsolete soon after 1600, and since then yes has been the ordinary affirmative particle in reply to any question positive or negative, and yea has become archaic. The distinction was still observed in the Bible of 1611, in which yes occurs four times (all in N.T.), always after a negative question or statement; the Revisers of 1881, apparently in ignorance of the usage, altered it in all these instances to yea.
If you look up "yea", you're instructed to see both "nay" and "yes" if interested in "the distinction formerly observed between 'yea' and 'yes'"; in the "nay" entry there's more, or different, information, and the rebuke by More of Tyndale to which Brewer refers:
In older usage nay was usually considered to be the proper negative reply to a question framed in the affirmative (yea would be the correct expression of a positive reply to the same). If the question was framed in the negative, then the proper negative reply would be no (with yes for a positive answer). This usage preserves the sense of nay as stemming from ne ay ‘not yes’. The distinction is explained by Thomas More: 1532 T. MORE Confutacyon Tyndales Answere III. p. clxxxi, No answereth the questyon framede by the affyrmatyue..yf a man sholde aske..is an heretyke mete to translate holy scrypture into englyshe..he muste answere nay and not no. But and yf the questyon be asked..Is not an heretyque mete to translate holy scripture into englysh. To this questyon..he muste answere no & not nay.
I assume the fact that "heretic" and "english" are spelled differently in the different questions is just an artifact of the crazy orthographical ways of olden times, and not of grammaticastish significance. It's also a little confusing to me that one answers a question without a negative with a word stemming from "not yes", when "yes" is used for questions that do have a negative, unless "not" not only negates the sense but also the circumstances of applicability of a word, so that you could also say "not no" for "yea". (And "not yea" for yes, and all the others.) Which is clearly just silly.
The interactions are apparently more complex than that: the probable etymology of "yes" is evidently "yea + sí 3 sing. pres. subj. of bēon to be". So "nay" is "not no" and "yes" is "yea, it [something]". (What might be the sense of "sí" there? There are many ways that subjunctives can be main verbs, but different ways in different languages, and no gloss is provided.) So we can hypothesize that the primitive terms are "yea" and "no", which means that it must have been very difficult, or at least prolix, negatively to answer positive questions, or positively to answer negative questions, until their companions were developed.
I'm also interested to see that the heterogeneity of "to be" is of respectable vintage.