I cannot place my finger on the source at the moment, but I read that in the beginning of the War, color bearers were relatively safe because they were not considered to be fair targets. Some archaic chivalry. After awhile, the practicality of the colors, both for helping allies identify fellow units and for morale purposes, resulted in a policy adjustment. That may not be true, but it makes sense. In previous wars, prior to the popularity of the rifled musket, it was much harder to pick and choose individual targets.