You stated;

"The little Bible book was published in 1854 by the New York American Bible Society".

The New York Bible Society did distribute a great many of these Bibles to Union soldiers. There are any number of reasons, or twist of circumstance, as to how this Bible could have came into your ancestors possesion. Including that it is possible that he had the Bible before he went to War, as Bible Societies were common in the 1850's.

During the War the southern version of these Bibles that were distributed to the southern soldiers were printed in Atlanta, Georgia. The southern printed Bibles were quite rare.

This does not mean by any streach of the imagination that only southern printed bibles were distributed to just southern soldiers. Or that Northern printed Bibles were distributed to just Northern soldiers. After all a Bible was a Bible, whether it was printed in the North, or in the South. During the War communications were not completely severed between north and south. And certainly somethings as non military and accepted as Bibles were often allowed to be transported across lines.