Firts posted by david upton at http://history-sites.com/mb/cw/nvcwmb/index.cgi?read=41409. This is just one. I have several more references that David posted but I haven't had the chance to read completely.

O.R. Series 1 - Volume 46 (Part III) p. 696
RICHMOND, VA., April 10, 1865.
Hon. E. M. STANTON, (Received 4 p.m.) Secretary of War:
The orders in relation to religions services in Richmond were verbal, and were applicable alike to all religious denominations, without distinction of sect. They were, in substance, that no expression would be allowed in any part of any church service, in the form of prayer, preaching, or singing, which in any way implieit a recognition of any
other authority than that of the United States, or gave any countenance to the rebellion. The clergy were notified that any prayers for the rebel Government or officials, or for the success of the rebellion, would be considered as treason, and punished as such. As in the ritual in use in the Episcopal churches here there was a form of prayer for the rebel authorities they were ordered to omit it. No orders were given as to what would be preached or prayed for, but only as to what would not be permitted. Neither in New Orleans, Norfolk, Charleston, Savannah, or any other captured city, as I have been informed, have
the Episcopal churches been ordered at first to adopt the form of prayer for the President of the United States. Do you desire that I should order this form of prayer to be used in the Episcopal, Roman Catholic, Hebrew, and other churches where they have a prescribed liturgy and form of prayer?....
G. WEITZEL,
Major- General.
____________________
David Upton