Problems with latency and jitter

[aiKIDoka]

I have audio and video when connected with my drummer and guitar player. Im in Wa$ngton, they are in California. All lights green except the "internet latency" which is red, and occasional yellows with the jitter. My band mates all have green lights as well with audio gear, and red with internet latency- it's about 30-50 ms. We can all play but the timing is always off. It's like its a beat too slow when I try to jam with either of them. We have gone through all the videos, gone into computer systems and changed power settings, sample rates- all that stuff. It got our audio gear all in the green, but we can't get the internet latency down or fix the timing issue when trying to jam in real time. Direct monitoring is off and on mine, it's a knob that controls "input" and "playback". Mine is all the way on "playback". Audio interfaces are focusrite scarlett 2i2, presonus audiobox usb 96 and the next one up from the usb 96. All work as we can all hear each other and gear test is green. We fix that lag in time and we are jamming! frustrated as all get out because we are so close. Any suggestions?