Orientation does matter. You want the GPS facing up. Rotationally it doesn't matter (there is no forward or backward).
Your ELT antenna is dormant until it broadcasts, so from a normal operating perspective, it won't interfere. However, if it's blasting away in an emergency, it could...