Notes in the service files usually appear as part of the original compilation of Confederate service files, or a request to confirm service by a state pension board. This sounds more like the former.

The service file should include a card referencing this man as a Confederate deserter and/or a Confederate who had taken the oath of allegiance. I'm not aware of parole papers being taken away from former Confederate prisoners. Since they were issued by Union authorities, does it make sense that they would first provide and then confiscate them?

Since East Tennessee north of Knoxville was generally under Confederate control in 1864, the Sullivan County story sounds believable. Home guards in many counties were much like those described in the novel, Cold Mountain, and this soldier sounds like Inman. As you may know, the novel was based on experiences of a real Confederate soldier killed by home guards not far from his North Carolina home.

Here are a couple of links on that topic:
http://www.salon.com/july97/colddiary970709.html
http://www.archives.gov/publications/prologue/2004/summer/inman.html