After a little research online, I settled on this recipe on Serious Eats (original is below; I halved mine):
1 cup light rum
1/4 cup whole allspice berries (you can find these in the whole spice section at the grocery store)
1 cinnamon stick
1 1/2 cup water
2/3 cup brown sugar
1. Crush the allspice berries in a mortar and pestle or grind them in a spice grinder. You want coarse, large pieces and not a fine grind.
2. Place the crushed allspice in a sealable glass jar and pour the rum on top. Seal the jar and shake well. Let this mixture steep for 4 days, shaking daily. On day 5, break up the cinnamon stick and add it to the mixture.
3. After 12 days total steeping, strain out the solids through a fine-mesh strainer. Then strain again through a coffee filter into your final bottle or jar.
4. Heat water and sugar on medium until boiling, stirring to dissolve, about 5 minutes. Let the syrup cool, then add it to the strained allspice infusion. Shake and then let rest for a minimum of two days before using.
The result seems perfectly usable, though since I have no idea what "real" allspice dram tastes like, I can't say how close my version comes. I used it for the first time in a Three Dots and a Dash and it worked well, though was on the verge of overpowering the drink. For any other cocktails that call for it, I will use about half of what's called for as I suspect my version is a little stronger/spicier than what you would buy in a bottle.
Got a favorite allspice dram recipe? Let me know!