What volume in SkyView are you using to make the volume right in the headset? Every audio source has some noise, and if your intercom has a lot of gain, it will amplify the noise. The easiest way to tell this is by the volume you run on SkyView. Most of the time we seem to be able to run 50-80%. If you're running below 40%, your intercom has a lot of gain, and you can knock down the noise with a resistor if you want.
If you hook a 50 ohm resistor between the audio output of the SkyView and ground, this will halve the volume of SkyView but will also halve the noise. You can then turn SkyView's volume back up to compensate.
Use two 50 ohms in parallel or one 25 ohm if you are using the system in mono mode with the Left and Right tied together.
Again, this will only help if you are using SkyView with a volume setting below 50%.