Here are three possible units. Which is the one you are searching?
3rd Regiment, Kentucky Mounted Infantry
3rd Infantry Regiment was organized during July 1861, at Camp Boone, Tennessee. Many of the men had previous service in the Kentucky State Guard. The unit took an active part in the Battle of Shiloh and reported 174 casualties. Later it was assigned to Rust's and Buford's Brigade, Department of Mississippi and East Louisiana. It lost 26 men at Baton Rouge then participated in various conflicts around Vicksburg and Jackson. During the spring of 1864 the regiment was mounted and continued the fight by confronting the Federals in Mississippi, Georgia, and Alabama. It was included in the surrender on May 4, 1865. The field officers were Colonels Gustavus A.C. Holt, Albert P. Thompson, and Lloyd Tilghman; Lieutenant Colonels Benjamin Anderson, T.T. Barnett, and Alfred Johnston; and Majors James H. Bowman, William P. Johnston, and Al. McGoodwin.
..........
3rd Regiment, Kentucky Cavalry (Duke's Brigade)
3rd Special Cavalry Battalion was formed during the late fall of 1864 with men who had served under J.H. Morgan. It was attached to B.W. Duke's Brigade, Department of Western Virginia and East Tennessee, and skirmished in Tennessee and Virginia. The unit contained about 100 men in January, 1865, and surrendered at Washington, Georgia, on May 10. Colonel Joseph T. Tucker was in command.
...........
3rd Regiment, Kentucky Cavalry
3rd Cavalry Regiment was organized during the summer of 1862. In October the 1st Kentucky Cavalry Regiment merged into this command and at times was called the 1st (3rd) Kentucky Cavalry. It was assigned to Buford's, T.Harrison's, J.W. Grigsby's, J.S. Williams', and J.H. Lewis' Brigade, and skirmished in numerous actions in Tennessee, Kentucky, and Georgia. Later the unit was active in the Atlanta Campaign, the defense of Savannah, and the campaign of the Carolinas. It surrendered with the Army of Tennessee in April, 1865. The field commanders were Colonel J.R. Butler, Lieutenant Colonels Jack Allen and Jacob W. Griffith, and Major J.Q. Chenoweth