The steerer tube is likely ruined, unless you can cut it down to where it's still round and still safely fit it on a bike. Since it won't slide off the bike, you can either use more force, cut it off right below the stem with a hack saw (electrified or otherwise), or you can cut the stem apart where the clamp is.
Yeah of course it makes sense to reduce one spacer but you learn of your mistakes. By looking at the stem I recognised: the stem is also heavy damaged, or does it just seems so?
just cut the steerer right below the stem, then pop that star nut out. then simply take the bolts out of the stem and cut along the slit in the clamp section of the stem through the steerer tube. probly take like 10 minutes.