The desertions recorded in Union records are pretty accurate; those in Confederate records are wildly overstated. That being said, however, both sides experienced an increase in desertions in 1863.

1863 was a bad, bad year for the Confederate soldier. The year began with the fall of Arkansas Post, and saw back-to-back defeats at Gettysburg, Helena, Vicksburg, and Port Hudson. Little Rock was occupied by the Yankees in September. About the only bright spot during the year was the Confederate tactical victory at Chickamauga.

Soldiers in the Trans-Mississippi Department, particuarly in Arkansas, were discouraged by the loss of Little Rock, but Helena was a particularly hard one to swallow, since, from the soldier's point of view, it was a pointless, mismanaged attack that needlessly wasted the lives of their fellow soldiers. It is interesting to note that by this time the average Arkansas soldier was pretty savvy when it came to assessing tactics. Writing after the battle, most soldiers felt that their individual regiments had acquitted themselves well at Helena, but they correctly perceived that the leadership failed to coordinate the attack and thus wasted lives and opportunity. More than anything else (pay, rations, etc.) the infantry troops in Arkansas were disgusted with the poor leadership. Cavalry troops seem to have maintained fairly high morale, probably due to the fact that the Trans-Mississippi Army had some brilliant cavalry commanders who inspired the confidence of their men.

But discouragement, lack of pay, change of heart, etc., don't appear to have played much of a role in switching sides. It had to do more with being in the wrong place at the wrong time (or the right place at the right time, depending on your point of view). Doyle pretty much has it right, when he states that by late 1863 whichever enrolling officer showed up (U.S. or C.S.) dictated which army you served in.

There were individual exceptions to all this, of course, but based on dozens of letters, diaries, and first-hand accounts I've read, I get the sense that opportunity played the largest role in a man switching sides.

Regarding pensions -- a man who served in both armies could only draw one pension. A Union veteran's pension was by far the most desirable. A Confederate pension was hard to qualify for, and didn't pay much. At various times, depending on the cash-flow of the State treasury, the pensions were paid in script, which could be redeemed for as little as 10 cents on the dollar.