Low light levels are a common problem as everyone talks about CRI and color temp, but lighting manufacturers rarely quote the output levels. For example I used to have the Kaiser planar and had same low speed issues as you, then I had a skier which has phenomenal output (of both light and heat) and I averaged 360th sec at F9 and ISO 100. I now use the negative supply pro light where I average around 200th sec at f9 / 100 and this one is fantastic.
It has little to do with the holder as above posts suggests, yes a diffuser reduces a little and this really doesn’t matter if your speed drops from 200 to 160, but if it drops from 40 to say 25, then yes it will unless your set up is stable and doesn’t shake at all. Diffusers are essential as Andrew Clifforth has found, as is a distance between the diffuser and film, all of which are solved in the EFH and neg supply holders, but they will reduce the lights output, so it’s back again to light output - the lumen or in the US ft candle outputs. Try and find the output of your current panel, or measure it with a light meter (most will do lux levels which is the no of lumens falling on a m2 of area), and get a higher powered one. But getting these figures from manufacturers may be hard.
I would look at an alternative light that has high CRI greater than 95, good colour rendering 5500 k say and has a high output. But also be aware high light output means heat generation. The skier just got hot whereas the neg supply unit has a fan. This is why the led panels etc have low outputs, they would get hot otherwise. In terms of output, negative supply say their entry basic panel is 2x output of a standard panel, I tried one of these and found average speed was about 60th at f9 / 100 iso. They quote their pro models having 9.5x the output, mine has an average speed of 200 at f9 / 100 iso, so this gives some idea of the output increases to look for if you can find any manufacturers data.
Hope this helps a little.