Joss Whedon created S.W.O.R.D. during his run as writer on Astonishing X-Men and it has slowly spread though out the Marvel universe including being a big part of the current Marvel NOW Avengers title which is a very heavy sci-fi book. I would not be stunned to see S.W.O.R.D. begin showing up in the MCU (it already made it into an alternate/deleted scene in Thor) with the further expansion into the Cosmic in Thor: The Dark World and Guardians of the Galaxy and the rumored Cosmic interactions to come in Agents of S.H.I.E.L.D. It isn't difficult to imagine a post-credits scene in either Thor: The Dark World or Captain America: The Winter Soldier involving S.W.O.R.D. and tying into Guardians of the Galaxy. One can just as easily imagine a post-credits GotG scene with Star-Lord or Rocket interacting with S.W.O.R.D. agents at the Peak.
I'm sure that Agent Brand is part of the rights package that goes with the X-Men but I bet S.W.O.R.D. works just fine without her. While Agent Brand is the head of S.W.O.R.D. I can't say she is as integral a part of the Marvel universe as Nick Fury. The key piece is having the agency; not the faces of the agency. S.W.O.R.D. is just one more step in the creation of a bigger universe; it helps open other doors to other corners. I can easily see an updated origin story for Carol Danvers as an Agent of S.W.O.R.D. before becoming a human-Kree hybrid. Establishing S.W.O.R.D. as an entity allows for it to become a convenient plot device to introduce other alien races and fill in some gaps.
To me this is one more inevitable step in the expansion of the MCU. S.W.O.R.D. can be the anchor to the Cosmic films that S.H.I.E.L.D. has been to the Earth-based films: the common thread uniting things and giving it a sense of familiarity. As with all my speculative pieces, I am probably wrong and this never sees the big screen...but when you pay to have your own website you can write about whatever you want :)