Evidently he thought of himself as a member of Co. "E", 5^th Kentucky Regiment, because that's the unit F. T. Davis named in his pension application from Bradley County many years after the war. In late 1862 this company was reassigned to the 23rd Tennessee Battalion as Company "A." Felix T. Davis received a discharge from service on April 17, 1863, evidently due to his advanced age.

http://www.tngenweb.org/civilwar/csainf/csa23.html

Copies of both his military service files as well as his pension application from Bradley County are available here. Kentucky records mentioned are filed under 9th rather than 5th Kentucky Regiment --