At the outbreak of the civil war,the Brown County area was mostly unsettled. In 1861, after men left to join the Confederate army, depredation from Kiowa Indians became a problem and many settlers who remained in that area moved back east to more populated counties. Settlers such as Rich Coffey, John S. Chisum, Sam Coggin, Robert C. Parks, and others, remained in the area, fought off the Indians and raised longhorn cattle for the Confederacy. When the war started, Brooks Lee (Lee Street in Brownwood) had a samll stock farm located on the present Naber's Ranch in southwestern Brown County. Brooks Lee joined Col.George H. Sweet's 15th Texas Cavalry as a scout. When the 15th Texas Cavalry was ordered to the east in 1863, Lee returned to Brownwood in Brown County Texas as a Confederate enrolling official. Brooks Lee is buried at the Lee Cemetery at old Camp Collier near Brooksmith, Texas.
Most pioneer residents came to Brown County after the war to purchase cheap school land from the state and avoid the taxes, strife and poverty in the counties to the east during reconstruction. In 1879, my great grandfather, Joshua David Coffee, his family and his brother John James Coffee and his family moved from Fannin County to a small hard scrabble farm on the Brown - Coleman County line. In 1874, my great grandfather Peyton Green Whaley (Whaley Street in Brownwood), also settled near the Brown - Coleman County line. In 1894, after a severe ten year drought and after losing a 18 month old daughter to typhoid fever, my great grandmother Parmelia (Wyatt) Gilmer and two children moved from a old cabin on the Lampasas River to Samuel Columbus "Uncle Sam" Wyatt's farm near the Williams Ranch Road. My greatfather William E. Gilmer, remained in Lampasas County to work on the Frank Gholson Ranch. He visited his family on weekends and his days off work for 20 years. All four of my great grandfathers were civil war veterans and settled and died in Brown County. My great grandfather 2nd Lt.A.J. Webb, was with United States 2nd Tennessee Cavalry, Gen. Rosecrans' Army of the Cumberland.
When the war started, men from Brown County enlisted in various Texas militia companies that were located in counties to the east such as Comanche, Mills, Erath, Stevens, Hamilton and Coryell County. In 1862, Those militia companies were called into regular Confederate service in cavalry regiments.
Col. James M. Norris' Frontier Regiment, 2nd Frontier District, Texas State Troops, with headquarters in Gatesville, Texas, was organized on December 21, 1861. Brown County residents enlisted Frontier Regiment at Comanche, Texas. In 1861, Capt. James Cunningham and his Comanche County militia company captured Camp Colorado from the Federal army without firing a shot. With exception of the commanding officer, the etire garrison joined the militia company and later the Confederate army. Finally, in early 1863, the Frontier Regiment was reorganized and accepted into regular Confedearate service with an agreement that the regiment would remain on the frontier for protection against Indian attacks. In August 1864, four companies of the Frontier Regiment were transferred to Harrisburg (Houston) for coastal defense, leaving only two 80 man companies at Camp Colorado under Capt. Henry S. Fossett. In January, 1865, those companies and several compaines from Comanche, Coryell, and Erath Counties engaged about 4,000 Kickapoo Indians armed with Springfield rifled muskets at the battle of Dove Creek, 20 miles west of present San Angelo Texas. It was a Confederate defeat due to poor attack cordination between the regiments. Brooks Lee participated as a scout for Capt. Fossett.