I am a private in the 104th Pennsylvania Infantry, Company C. reenacting group and this is what happened
Sgt. Hiram Williams Pursell of company G was carring one of the flags on that day on Battle of Fair Oaks, VA on May 31, 1862. When the rebels pushed the 104th back Pursell saw the other color bearer dead laying against a tree. He gave his flag to a near by comrade and ran back to get the other flag. He was wounded twice once in the leg and the other time in the chest. At one point he was indeed arm length from the gray backs. "He had been shot twice when he returned to his comrades with the flag, and near collapse from loss of blood"
But I a sure you the colors of the 104th was not captured at Fair oaks nor never at that matter during the war.
link to the painting
104th Pa. Inf.
2nd CT. Hvy. Ary.