Enslavement of black people was genocide in the way it left whole strips of Black Africa devoid of men who could hunt and farm for their families. But I'm not saying that Tara was ever Auschwitz. Nothing beats or even approaches Auschwitz in terms of evil.
True, only a minority were slaveholders. But they lived in agricultural states. And the economics of these agricultural states depended on slave labor to keep the plantations profitable. In that way the destiny of the non-slaveholders was tied to the moral evil of slavery.
Most German Wehrmacht soldiers were not anti-semites, most of them were not Nazi party members, most of them thought they had to protect Germany from another Versailles if the war is lost, yet all of them contributed with their stubborn resistance that the Auschwitz furnaces kept running as long as they did. A moral dilemma.