As mentioned in an earlier post, Ferrell's Battery was assigned to duty as horse artillery for Roddey's Alabama Cavalry from late 1862 until the end of the war. During the absence of Wheeler's Cavalry in the winter of 1863-64, much of Roddey's cavalry went to Dalton GA for outpost duty with the Army of Tennessee. Officers and men received new arms, clothing and equipment here in March 1864. Shortly afterwards Roddey received orders to return to his normal duty station in the Tennessee Valley.
Unless the soldier mentioned was on furlough, I can't explain why a member of Ferrell's Battery would have been in Georgia during May 1864. In May 1864 Ferrell's Battery was in repeated action against Federals defending Decatur AL. U.S. reports from Decatur frequently mention Confederate artillery.