My great grandfather was in Co A of the 20th Ga Cavalry Bn, also known as Millen's Partisan Rangers after it's commander, Lt. Col. John M. Millen. In July 1864, the 20th Bn was split up, with companies A, B, and C consolidated with regiments from the 62nd Ga Cavalry Regt to form the 8th Regiment Georgia Cavalry. Company D was assigned to the Jeff Davis Cavalry Legion, and companies E, F, and G consolidated with the 7th Regiment to form the 10th Ga Cavalry. In October 1864, Company D was transferred to the 8th Regiment and Company B went to the Jeff Davis Legion (confusing, isn't it).
The 8th Georgia Cavalry Regiment was commanded by Col. Joel R. Griffin, and was assigned to Brig. General James Dearing's Brigade, Butler's division.
The 20th spent the first part of the war patrolling the Georgia Coast. The first real action for the 20th was at Haw's Shop in May 1864, after being transferred to Virginia. Col. Millen was killed at that battle. The 20th also fought around Cold Harbor, Trvelian Station, Samaria Church, Petersburg, and other conflicts.
If Andrew J. Hardee applied for a Georgia Confederate Pension, you should be able to locate it at the Georgia Division of Archives and History at Morrow, Ga. He may not have applied. Just as a chance, you may also want to check the Florida Archives in Tallahasse. If his other brothers moved to Florida and applied for a Florida pension, maybe Andrew did also and later moved back to Brooks County.
Also check the records for the Jeff Davis Legion and the 10th Ga Cavalry. He may be found with one of those units.
Good luck with your search. I'm still researching the 20th. If you find anything, let me know.