Colleen,
Please contact the Mississippi Sons of Confederate Veterans I am sure they will help you. You can call 1-800- My South and talk with someone who can help or use the link below to find a SCV Camp near the grave site area.
http://www.mississippiscv.org/
Here is how your Grandfather is listed in the Soldiers and Sailors System. You will need this info.
W.W. Gilmore (First_Last)
Regiment Name 5 Mississippi Cavalry
Side Confederate
Company K
Soldier's Rank_In 1 Sergeant
Soldier's Rank_Out 1 Sergeant
Alternate Name
Notes
Film Number M232 roll 15
If you do not have a copy of his service records you will need copies to order his stone from the V.A. Here is a link to order copies of service records.
http://history-sites.com/research/index.html
Here is a link to the V.A. for ordering a free head stone for your soldier.
I hope this will help you,
Gary D. Bray