This interesting surname is of Anglo-Saxon origin, and is lcoational from a place so called in Kent, which was originally the name of a river. The first element seems to be derived from the Old English pre 7th Century "rum", spacious, but its formation and meaning are obscure. The second element is derived from the Old English "ea", river. The placename was first recorded as "Rumenea" in the Anglo-Saxon Chronicles of Essex in 1052. A derivative of Romney is found recorded as "Ruminingseta" in the Saxon Charters of 697, and means "the fold of the Romney people".... The first recorded spelling of the family name is shown to be that of Robert de Romenel, which was dated 1086, The Domesday Book of Kent, during the reign of King William 1, "The Conqueror", 1066 - 1087.The spelling "Robert de Romenel" is clearly French on two points: "Robert" is an Old French name that existed among the Normans, but not among the Saxons; and "de -" is a French form as well. "Romney" also looks like a word of French descent.
But the authors of the book thought and wrote in French, and there were variations of "Robert" that was native to Anglo-Saxon England: Hrēodbēorð and some others. If the name was Normanized at the time of the Conquest, it could have survived in a form that doesn't look Saxon, but honestly happens to be. So possibly, for whatever it's worth, he's as Saxon as King Alfred.
Of course, for those Americans who aren't antiquarians, the real question is: "Was this remark just coded racism, or was it double-super coded racism?"