If your caliper is fitted directly onto post mount of forks with the 180mm rotor (without an adapter) that means your forks are 180mm 'native' post mount, i.e:- the smallest rotor you could fit, you will need a +23mm adapter:- Shimano SM-MA-F203P/PM Front (180mm) to 203mm.
If the caliper already has an adaptor with the 180mm rotor then you would need to remove existing +20mm adapter and replace with this one:- I-SMMAF203PPA which is a +43mm adapter to convert a 160mm post mount fork to fit a 203mm rotor.
Just a note to save other people the pain, the rear is a Flat Mount, not a Post Mount. I couldn't find adapters much cheaper than from the Whyte website: https://whyte.bike/collections/parts-accessories/products/flat-mount-adaptors
The third picture down highlights the difference: https://www.singletracks.com/mtb-reviews/whyte-s-150-s-trail-bike-test-ride-review/