I am not aware of official hardship waivers, although I would suspect that individual Conscripting Officers would show deference. This might explain why he was able to escape service for so long.

As to getting home, check out this latest thread on the Prison Message Board:

Going Home from Hart's Island Civil War Prison NY
By:R. Hugh Simmons
Date: Sunday, 17 July 2011, 8:55 am
In Response To: Re: Hart's Island Civil War Prison NY (Jayne Vosloh)

Jayne and Gale:

Federal War Department General Orders #85, #98, #104, and #109 directed all members of the US military establishment to provide government transportation for the POWs and civilian detainees released under these various general orders to a point nearest their homes that could be reached by water and/or rail. When you examine the Oath of Allegiance card in their Compiled Military Service Records, you will see a specific line stating the released man's place of residence. It is almost always a county in a particular state. These generals orders applied to all Federal POW camps including Fort Delaware and Hart's Island on the east coast.

I created a time line with notes on the subject of "Going Home" which is posted on the Fort Delaware Society website at http://www.fortdelaware.org/Going%20Home%20-%20The%20War%20is%20Over.htm

Let me know if you have any specific questions pertaining to your particular research.

Hugh Simmons
Fort Delaware Society
Website: www.fortdelaware.org
E-mail: society@fortdelaware.org
http://history-sites.net/cgi-bin/bbs53x/cwpmb/webbbs_config.pl?read=3512