Phillip -
I only picked up Nelson's story in about May, 1863, when he is with Gen. W.H.T. Walker in Mississippi. Had to check whether this was the same guy, and it was: Thomas M. [not H.] Nelson.
Mustered-in as an independent company, Georgia Cavalry, on 14 May, 1862. Goes to Tennessee as you have been researching, until at least March 19, 1863 (O.R. vol. 23, nr. 2, p. 711). Then: "Left Knoxville, Tennessee on March 21, 1863 with orders . . . to report to General [Edmund Kirby] Smith, Trans-Mississippi Department. Was retained in this [Mississippi] department by order of General Johnston, he deeming it impracticable to . . . cross the Mississippi River after the fall of Vicksburg. Have been in five engagements since my arrival in this department. Lost sixteen men killed, wounded and missing in action." Nelson was picked to lead a group of partizans to assist with transfer of materials to the Trans-Mississippi (he being promoted Major) see O.R. vol. 39, pt. 2, pp.670-1. By 1864 he's in the Sixth Mississippi (Lt. Colonel) and is killed at the Battle of Harrisburg, July 14, 1864.
Nelson was highly thought of by all; a real interesting Georgia soldier. Don't think he was in First GA Cavalry, unless prior to May, 1862.
John