You asked: >>> Are there other reasons that records of a soldier would stop? <<<
We can entertain the notion that "his records just got lost" but there are so many records for other POWs that did not get lost that I just don't buy that. His "disappearance from the records" at Fort Delaware is unusual and out of the ordinary. If Jefferson had been left behind in the Fort Delaware hospital on 3 JUL 1863, then eventually he had to die, or be released from prison.
You asked: >>> Wouldn't there have been documentation of other routine military matters such as pay and eventual release from the military after the war? <<<
Yes. But there are none that I could find. That is how I came to my tentative conclusion.
While the Compiled Military Service Records are not complete or infallible, one needs to come up with documentation for what is missing. Being left behind sick doesn't lend much support to the notion that somehow he escaped since that act required a man to be in relatively good physical condition. There was a mile of water on either side of the island with strong river currents to overcome. Many tried and failed. In the event of a successful escape, he would have showed up somewhere - very likely in a Richmond newspaper account since they loved to publish escape stories to buck up Confederate morale at home.
The missing documentation for me is the garbled name on the Confederate monument. I can not find any evidence of a soldier by this name in that unit. But the name on the monument came from somewhere. Might it be a garbled record for Jefferson K. Golden? If you can't find a post-war record for Jefferson, then this might just be the answer.
Let me know if you discover any records that contradict this idea. The Society is interested in ascertaining the facts, not winning arguments!!