I suggest you obtain a copy of Joiner, Gary. Little to Eat and Thin Mud to Drink: Letters, Diaries, and Memoirs from the Red River Campaigns, 1863-1864 (Voices of the Civil War) University of Tenn Press 2007. ISBN-13: 978-1572335714

On pages 146-147 there is a description of the hospital and graveyard where you ancestor is likely buried. In case you do not live in Rapides Parish but would like to see the location http://www.civilwaralbum.com/louisiana/alexandria3.htm (See this page as well as the pages before and after)