BFG was wounded on May 5, 1864, Battle of the Wilderness, probably during General Gordon's attack in Saunders Field. The surgeon's report shows that two fingers in his right hand were broken. Why BFG didn't return to his regiment is another mystery. Perhaps that's explained in his pension application.
This young man received a discharge in 1861 for some reason. As mentioned previously, it probably was something physical. That doesn't mean that he couldn't have lived a normal life after the war.
No competent military officer on either side would have allowed an able-bodied young white man to tag along with his command without him being regularly enrolled. The men didn't need anyone to hunt and fish for them -- they could do that for themselves. It was called foraging. Often they found it simplier to collect what was necessary from nearby farmers.
After April 16, 1862, if this young man was at least eighteen and didn't have an exemption, he wasn't allowed to stay out of the military based on good intentions.
It's a nice family story, but makes no sense whatsoever. To bring this up to date, imagine a young man telling General Patton he would just hang around with his soldiers but not join the army.