We have had the same long time issue on multiple computers running macOS Big Sur 11.6, 11.6.4, 11.6.5 and 11.6.6. Running Protools 2021.x-2022.4. w an HDX card (internal on a mac Pro 2019 and a Mac Pro 2010, in a thunderbolt chassis on a Mac Pro 2013). Two Avid MTRX and one MTRX Studio.
We have found that DADman must be launched after we hear the fans on the HDX card rev. I assume this is when the HDX driver loads. If you set DADman to autolaunch on login, it will always bomb. Wait 30 seconds and all is well.