From the NPS Soldier Sailor System (http://www.itd.nps.gov/cwss) I found the following on the 34th.

34th Cavalry Regiment [also called 2nd Texas Partisan Rangers] was assembled and accepted into Confederate service at Tahlequah, Indian Territory, in the spring of 1862. The unit served in Polignac's, King's, and J. E. Harrison's Brigade, Trans-Mississippi Department, and fought at Spring River, Newtonia, Prairie Grove, Mansfield, Pleasant Hill, and Yellow Bayou. It was included in the surrender on June 2, 1865. The field officers were Colonels William M. Bush, John R. Russell, and George H. Wooten, and Majors Thomas J. Dove, M.W. Davenport, and Sevier Tackett.

They show a Casptain W. H. Flemming and a Private M. H. Fleming in Company K.

The service records for William B. Flemming of company B, Terry's Rangers mustered on September 7, 1861 and was discharged in March 1863. Many of the rangers who were discharged later showed up in other units of the Trans-Miss army.

In the Texas Pension index (http://www.tsl.state.tx.us/arc/pensions/index.html) I found the following:
Fleming, William E. - #21751 - Hunt county

I wasn't sure of his wifes name but you could check if she applied for a widow's pension in the Oklahoma Pension index (http://www.odl.state.ok.us/oar/docs/pension.pdf).

Right now I'm in the midst of moving but once settled I will try and see what notes we have on WB Fleming.