A few days after work began on the canal the rebs shelled the working parties. I think this was around August 12, 1864. You'll see references to this in the official records, Vol. 42, Parts 1-4.
Less than a week after this, troops from the 4th and 6th USCT left the canal and boarded a boat which landed them and detachments from a white NY regiment on the north bank of the James. They seized Signal Hill near Aiken's Landing and held it for about 24 hours. This incident is also in Vol. 42, part 1 - see Gen. Bratton's report. It's also in John Coskie's book on the Confederate James River fleet.
This landing was part of the Second Battle of Deep Bottom, Aug. 14-20, 1864