I had saved an old story that I belive was what you are talking about. I went looking for it and got luckey and found it. I'll try to type it here for you.
--------------------------------------------------------
The Man On Vinings Hill

As the Federal Army neared the Chattahoochee river, it was found that a clear view of Atlanta, goal of the campaign, could be had from the hill at Vinings, just north of the river. Federal officers, curious to see the city they sought to capture, made their way to "Vinings Mountain." As they approached the crest, even before they saw Atlanta, their steps were arrested by an unusual sight, one they had not expected.

In their path, hidden by bushes, was a man who appeared to be on tiptoe, peeping over the leaves. Challenged, he did not speak. Approaching, they found him dead, hung by a hickory withe (strip of inner bark), feet almost touching the ground. he had been suspended for some time, for his face was dark, and his body decomposed.

Local citizens knew of no Confederate military execution in the area. The man had not been robbed, for there was some confederate money in his pockets. A pass identified him as "Ben Duncan," or perhaps "Ben Durkin," of Griggsville, Georgia, an employee of the Western and Atlantic Railroad.

The Federals who viewed him had seen war, and death, on many fields. death here, by hanging, was a novelty, and striking. Men dead or wounded by shell-fire, or musket balls, aroused little attention. This hanging man was unusual, and a mystery as well.

Captain George W. Pepper, Federal Chaplain, describes this mysterious man in his "Recollections". He is mentioned in the "History of the Seventeenth Ohio Volunteer Infantry", and in the New York Tribune of July 15, 1864. Several unprinted diaries record this unusual victim of the noose.

with so much publicity, one would expect someone to come forward with an explanation. No one has. Perhaps the circumstances of his death are better left unpublicized. Perhaps the man had no champion to ferret out his fate, so that he might sleep easy in a marked grave.

The man on Vinings Hill, so far as we know, lies in a shallow grave, unmarked, on lonely Vinings Mountain.
-------------------------------------------------------
Well thats all there was to the article. Vinings Mountain is just outside of Smyrna, Georgia in what they call Vinings. The mountain is at the corner of Paces Ferry Rd, and Cumberland Parkway. On the mountain is the 3 big office buildings. "Overlook One" "Two" and Three" At the very top of the mountain is the Paces graveyard.