The primary downside is that you can’t Solid spells though raging which isn’t a difficulty for pure barbarians, but does limit your multiclassing options. You also only have a handful of rages per day but typically, That is enough should you don’t overuse it.I’d recommend talking about with your social gathering any gaps in proficiencies wh