The problem is 8-bit and noise when colors are pushed and the the contrast is accentuated, which accentuates the color of noise. If you must work with the footage, you can try noise reduction before the grade. If your noise reduction tool lets you profile the problem areas, maybe that would be better. You can re-add grain later if needed.
If the subject doesn't move at all, you have Photoshop CC, you can run videos through Camera raw filter, put a feathered mask outside the subject, and get really heavy handed with noise reduction and 0 the saturation.
Best to reshoot with a stronger fill light.