On most i.MX* the upper bits in SW_PAD_CTL are reserved. On some chips,
like the i.MXRT1166, they are a domain write protection. Setting them to
1 can have unexpected side effects.
The device tree uses these bits for some flags. Make sure that they are
not accidentally written to some value.