You need to recheck this. If they actually exchanged a dead body, that should be in the Civil War record books somewhere. Just wasn't done, certainly not for a private.
Also, you have him captured on March 26 (at Henderson Hill, no doubt) and showing up in New Orleans that same day (the 1865 is a typo, right?). Somewhere along the line someone has gotten these records all tangled up.

At any rate, you can feel sure with a 99+% probability that if he died in New Orleans, he was buried there. If he was exchanged at Red River Landing on July 22, he wasn't dead at that time.