Abraham Lincoln was controversial because of his views on slavery.
He was not controversial due to his views on race, and these views remained prevalent long after his time. For example, the U S military remained segregated for almost a century after Lincoln's death, and not simply because of Southern influence. Segregation of the U S military ended on July 26, 1948, when President Truman signed Executive Order #9981.
http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA219730