People who opposed slavery on purely moral grounds were still called “abolitionists” until after slavery had actually been abolished. But by the 1850s - in particular, after the Fugitive Slave Act became law - a large and growing number of Americans opposed slavery as an institution because they found the political power of slaveholders extending into their own lives.
