This is what Henderson's has:

Overby, G. H.----- private May 13, 1861. No later record.

Co. H, Floyd Infantry, 8th Georgia. Sorry that there isn't more. Have you looked for an obituary for him - did it indicate he served in the 8th Georgia?