I did notice that they are listed as the "Oliver" family in the 1860 US Census (for Natchitoches Parish, Louisiana). There are no "Olive" listings in Natchitoches Parish in 1860. Of course, it is relatively common for names to be mis-spelled in the census processes - literacy of census takers and people being counted did impact significantly. This type of error can impact both genealogical and historical information gathering.

Very best

Don