Two Colorado inmates who claim they were pressured into working despite health concerns are suing the state’s prison system, claiming it’s violating a constitutional ban on slavery and involuntary servitude.

In the lawsuit, filed Tuesday in Denver, Harold Mortis and Richard Lilgerose claim they effectively had days added to their sentences when they initially refused to work in 2020 because they lost time credits for not following prison rules. They were also threatened with being placed in isolation for most of the day, with limits on phone calls and family visits, and both resumed work in their prison’s kitchen to avoid further sanctions, the lawsuit said.

The lawsuit asks that a judge bar the state from forcing inmates to work.

Colorado Department of Corrections spokesperson Lisa Wiley said the department doesn’t comment on pending litigation.

The 13th Amendment to the U.S. Constitution ended slavery but it still allows forced labor for those convicted of crimes. Colorado had similar language in its state constitution but in 2018 voters backed ending slavery and involuntary servitude under all circumstances.

Voters in Nebraska and Utah have also since passed initiatives amending their state constitutions to do the same. Lawmakers in Vermont are also considering a similar proposal.

The lawsuit was first reported by Colorado Public Radio.