The other option is to take field commissioned agent which gives you extra downtime days but no additional items. Field commissioned agent is situational, it usually isn't beneficial to take it over the schools.
I would also venture that scrolls/spells/swords is situational as well, just less so than field commissioned agent. You take those if they have a skill/feat offering that aligns with your character, or if there are specific items that you want.
If none of that jumps out at you, then generalist is probably the best option (in my opinion it is the best school and I almost always end up taking it unless there is something really completing offered by one of the others).
There is no real downside to taking a school/field commissioned, the downside would be if you decided not to take any of them.
Remember, you can retrain your school, it just takes time or Acp.