It is interesting Ken the prisoners on both sides were allowed to write and receive letters to and from their friends and family.Of course if they could read or write or find someone to do it for them. In general you did have to have a US stamp for the letter to get to Fortress Monroe and then a CSA stamp for the letter to get home into the South. And of course vice versa for letters from family in the South to get north. POW letters were limited to one page and were opened and censored. The CSA government later in the war furnished free Confederate Stamps for their prisoners.The federal government never did.
Certainly those letters were an extremely important part of the life of any prisoner.However,its odd that Civil War POW letters are in general somewhat rare and few have survived today. It is most likely tho that any of our ancestors that were prisoners would have exchanged letters at their first opportunity.