The 12th Georgia Militia participated in the defense of Atlanta from May-September 1864. After the fall of Atlanta the regiment moved into defensive positions in and around Savannah November- December 1864. One of the battles of note was at Griswoldville, 22 November 1864.

It surrendered with the Department and is included in the surrender list of General Joseph E. Johnston at Durham Station, Orange County, NC on 26 April 1865.
http://www.nwinfo.net/~jagriffin/12thgamil.htm

However, I found a Private I. W. Seward, 12th Georgia Militia whose name appears on a list of Prisoners of War surrendered to Brig. Gen. E. M. McCook by Maj. Gen. Sam Jones, C. S. A., at Tallahassee, Fla., May 10, 1865 and paroled at Thomasville, Ga. May 22, 1865. Also, also a Private J. A. J. Shaw who signed a parole document at Head-Quarters, U. S. Forces, BGen. E. M. McCook, Thomasville, Ga., May 17, 1865

M266: Compiled Service Records of Confederate Soldiers Who Served in Organizations from the State of Georgia

What was the name of you ancestor?