New Years Day and the start of the Tax Year used to be on the 25th March. But in 1582 Pope Gregory XIII ordered a change from the Julian 11-Month Calendar (named after Julius Caesar), to the Gregorian 12-Month Calendar (named after Pope Gregory), to align with the Solar Calendar (time taken for the earth to travel around the sun).
Except the UK decided to ignore the Pope’s decree for some 170 years. When they finally realised they would have to do it, they had to drop 11 days from the calendar. So Sept 2nd 1972 was immediately followed by 14th September 1972 (bad luck if your birthday was one of the 11 days in between!), but to ensure the treasury didn’t lose 11 days taxes they extended the tax year from 25th March to 5th April.
1800 would have been a leap year in the Julian Calendar, but it was not in the Gregorian Calendar, so worried that they would lose another days taxes the treasury added another day moving the tax year from the 5th to the 6th April.