The 54th Alabama has a convoluted history and is a bit confusing to follow all the iterations that ultimately led to the formation of that regiment.

On Dec. 27, 1861, Alpheus Baker of Eufaula organized a Regt at Fort Pillow, Tenn. His command included four Co’s from Ala, four Co’s from Tenn & two Co’s from Ms. Baker called his Regt the 1st Ala-Tenn-Ms, but the War Department recognized it as the 39th Tenn Infy Regt, believing Co’s were from that state. The error was eventually corrected & the Regt renamed the 4th Confederate Infy.

Baker's Regt surrendered with the garrison of Island Ten at Tiptonville, Tenn., Apr. 8, 1862. Officers & men went to prison camp for a few months, & were shipped down the Ms River to be exchanged at Vicksburg on Sept. 20, 1862. Orders dated Sept. 29, 1862, dissolved the 4th Confederate Regt. Four Tenn Co’s were directed to the 42nd Tenn Infy Regt, while the remaining six formed part of the 54th Ala Infy Regt. That Regt formed at Jackson, MS on Oct. 9, 1862.

The 54th Ala Infy Regt (initially formed as the 50th Ala Infy Regt) was organized from six Ala Co’s ("B", "C", "D", "E", "F", & "I") of the 1st Ala, Ms, & Tenn Regt of Col. Alpheus Baker, & four Ala Co’s ("A", "E", "H", & "K") of the 40th Tenn Regt of Col. L. M. Walker. These Co’s had all been captured at Island No. 10

In February 1863, the 50th Alabama was re-numbered as the 54th Alabama

It would seem logical your man would be found among the 4 Tennessee companies that initially belonged to the 1st Ala-Tenn-Ms and were then transferred to the reorganized 42nd Tennessee. But, I did not see him in that regiment's compiled service records [at Fold3]. There is, however, an H Perry found in the 4th Confederate regiment.