After having just got into in-home streaming there is a very good reason this doesn't work particularly well, in fact there are more than one.
In-home streaming requires a huge amount of network bandwidth, something wireless doesn't offer (most Xbox Ones are probably run on wireless). You can get away with powerline adapters, but they need to have gigabit ports for sensible bitrates. I got by on 100Mbit ports, but it wasn't ideal.
Then, for sensible bitrates, you need a good encoder on one end (the Xbox One), and a good decoder on the other (your PC). Software encoding/decoding is simply too slow, unless you PC is running an i5, but ideally an i7. Intel, AMD and NVidia all have hardware encoding and decoding features. AMD's GPU encoding and decoding is awful, stuttery, and altogether nowhere near as good as the others. NVidias is fast, but the quality of the image is very much sacrificed for the sake of speed. Intel's x86 software decoding is the best, but vastly too slow, however their Quicksync is excellent speed and excellent quality, the catch is the quality drops significantly when you're encoding down to below 4Mbit bitrates.
The XBox one as an encoder is not going to be quick, it may have many cores, but the hardware accelerated encoder is not particularly good, and the cores themselves have a very low clockspeed, so the encoding will have to be extremely light. Not to mention the fact that the XBox One hardware is going to be processing the game load as well. That video stream will then go down through your network, and if there's any lack of bandwidth if you're on wireless, or largely poor powerline bandwidth, you're going to lose frames. Once it hits your PC, you're then going to need hardware that can handle that encoded data stream and decode it at equal or better speeds. Although with the light encoding that the XBox one is capable of, you can probably get away with a HTPC style of processor based decoding. I use a Pentium with DXVA decoding or QuickSync when available, and that's dealing with quite a heavy bitrate stream.
Basically the bottleneck is going to be the XBox One hardware that's encoding it all in the first place, and the second bottleneck is the network, which most domestic users will probably have wireless connection instead of wired. There's going to be no real bandwidth available, and the loss of colour, framerate and general picture quality is going to be awful. With an i5 and 970 processing the game and using NVENC encoding, running through gigabit LAN, I can just stream at 1080@60p.
