"No person held to service or labor in one state, under the laws thereof, escaping into another, shall, in consequence of any law or regulation therein, be discharged from such service or labor, but shall be delivered up on claim of the party to whom such service or labor may be due."
was included within the Constitution, and had to be repealed by the 13th Amendment, is that the Congress of the United States under the Articles of Confederation held it meeting in Federal Hall in New York City which at the time of the Constitutional debates was the United States Capital and not as most people think that it was in Philadelphia. Remember that George Washington was sworn in as the first United States President on the balconey of this same building.
New York city with its slave trading was important to the formation of the "New" United States and the writing of the "New" Constitution.
Another little sidelight is that the Northwest Ordinance which had outlawed slavery in the territories of Ohio, Indiana, Illinois and others, which had been passed under the Article of Confederation United States, was abolished under the "New" Constitution.
The old adage of examining political decisions of "Follow the Money' would seem to apply here. The "Money" in Slavery in 1788 wasn't in Virginia, the "Money" was in New York City.