The issue is once you turn it off, the federation attempts by other instances will fail and the software has an increasing „back off“ mechanic. That means after the first failed attempt the instances will wait eg an hour before reattempting federation, then two, then four, then eight, etc. this means if you turn it only on when you want to use it, your federation will only kick in after a huge amount of hours. That makes no sense, sadly.