If you haven't found it yet, there is a diary written by Lt Rbt T Williams of Co E online.
http://www.swcwa.com/Diary_of_Robert_Thomas_Williams.pdf In it he briefly mentions your ggrandfather on Jan 11, 1863. The diary should give you some good information on the travels and battles of the 4th.