You'll also need some armorer skill - level 4 for best results, but lower levels work as well. Then, you'll need some ballistic fiber - KL-E-O in Goodneighbor sells it.
To start, you'll need to do a few missions for the Railroad until the 'ballistic weave' mod unlocks.
The trick seems to works on anything the game classifies as 'clothes,' be they military fatigues, three piece suits, hats or baseball uniforms. But I don't want it to be too much like my normal life. If I wanted to, I could play wearing nothing but the hat and a pair of stylish sunglasses, popping chems and wreaking havoc wherever I go. Done right, my little trilby carries with it nearly as much damage resistance as the fully modded heavy combat armor I was wearing before - 110 physical and energy resistance, to be precise.
In my case, it's the trilby hat, but reddit suggests it will work on the newsboy cap, battered fedora and green rag hat as well.