First off, thank you Ford for starting this. Frag seems to be surmising from vague memory (or blatant trolling, I can never tell), so let's get our ducks in a row with citations.
Historically, we start with the Leviathans. There may have been races before this, but there is no indication that they matter in the ME1-3 continuity. The Leviathans used mental powers to enslave other species, and subsequently developed space flight. They used their space flight to enslave other worlds, and offered protection of the subjects as "reward" for being slaves. The enslaved species tended to produce AIs to help them, which hand the tendency to go rogue and kill their creators when sapience was achieved. The Leviathans created the Intelligence to combat the whole murderous robots issue. When the Intelligence turned on them, the survivors went into hiding. They've been removing mention of themselves from history since the Intelligence bore the Reapers. While it is suggested that the remaining Leviathans have been manipulating the other species to act as soldiers against the reapers, the evidence is generally removed by the previously stated hiding of Leviathans from history.
http://masseffect.wikia.com/wiki/Leviathan
Moving on, we've got the Reapers. They're born from the DNA of the Leviathans, and the drive of the Intelligence. Their stated goal is to preserve all life, and they do so by slaughtering anything that evolves to the point of developing machine servants and absorbing the DNA into their records. They initially appear as mechanical Leviathans, but they can create new designs based upon the DNA that they absorb. This ability is first confirmed in ME2, with the end boss being a partially created human reaper. The Reapers have powers similar to the Leviathans, and exert their mental control via indoctrination. Their harvest occurs in cycles, as intelligent life develops in the galaxy. In order to control how the intelligent life develops, the Reapers created the Mass Effect relays and the Citadel. In order to conserve power, the Reapers wait out the time between harvesting cycles in "dark space." This dark space is the void between galactic bodies.
http://masseffect.wikia.com/wiki/Reaper
Mass Relays and the Citadel. These things are one and the same. A mass relay generates a corridor of mass free space, allowing for functionally instantaneous travel over large distances. The effect is created by utilizing an element zero core, which when charged is capable of generating a field that negates mass. Relays can be activated, but are generally found in a dormant state. Reactivating them allows for travel to new locations, but as most of these locations are unexplored they often lead to wars (Turian vs. Human, Citadel vs. Rachni, etc...). Relays have two general types, a bidirectional gate (generally over longer distances) or a hub travel between many relays (generally over small distances). The mass relays are maintained by a species known as the Keepers, which are observable on the Citadel. While the Reapers did create the Relays, there are two exceptions. There was a pair of relays linking Ilos and the Citadel, which the Protheans created. This relay pair is, as yet, unique. It allowed the Protheans to enact a plan that would prevent the next harvesting cycle.
http://masseffect.wikia.com/wiki/Mass_relay
The Keepers are a much shorter story. At some point the Reapers enslaved them, and determined that they'd be excellent technicians. They removed all of their faculties, and programmed them to maintain the relays. While it is surmised they were one of the first species the reapers enslaved, this is irrelevant as they are functionally furniture in the Citadel. An unknown number of harvesting cycles later, we get the Protheans.
http://masseffect.wikia.com/wiki/Keeper
The Protheans are the predecessors to the ME1-3 storyline. They managed to poison the Citadel, so the Reapers could not call for backup. They also managed to store caches of data for the primitive species that were not culled by the Reapers. The Protheans had a hierarchical empire, to which all subject species served the Protheans. Within their own empire there were classes of citizenry, from those who ruled to the soldiers. As their view about survival was pragmatic, they didn't generally question this forced hierarchy. This is exemplified by their conquests, their taste for "lesser species" as edibles, as well as their soldier class being relatively uninformed about anything not directly related to their jobs (read: Javik being less than knowledgeable about events during the fall). Protheans were harvested with great efficiency due to the census obtained from the Citadel, and eventually died out as a species due to betrayal (indoctrinated Protheans), a lack of infrastructure, and the generation of the Collectors from their fallen. While some of their artifacts still remain undiscovered, their relevant history has functionally been completely explored.
http://masseffect.wikia.com/wiki/Prothean
The Collectors are Protheans stripped of their intelligence, and enhanced with cybernetics to serve the Reapers. They monitor the species that arise after the Reaper harvests, and control technological development by offering species advanced technologies in exchange for biological samples.
http://masseffect.wikia.com/wiki/Collector
Based upon the available information:
1) The Leviathans never left the Milky Way, and might not even have completely explored it.
2) The Reapers never went beyond the Milky Way. Their stated goal is to preserve life, and their largest relay only got them to the edge of the Milky Way. Based off of only using FTL travel, and it taking several years before they reach one of their relays, they never even got far enough into dark space to determine if any other Galaxy supported life.
3) Most of the Milky Way Galaxy has yet to be explored. The survivors of the war against the Reaper harvest may have the memories of the Reapers, the records from the Crucible, or some other bits of data that could offer information on the actual size of the Mass Effect relay network. Without that, the lack of emission from the relays means that there could literally be billions hidden in our own Galaxy without anyone knowing about them.
4) ME4 will reject any of the characters from ME1-3 appearing in anything but a cameo role. Consider the amount of extra content that would be required. Did you kill everyone in ME2 during the ending? Did you cap Wrex on the way to the Krogan biology lab? What about the three flavored ending of ME3? That's too much plot to deal with, and EA would never allow Bioware to spend 10% of their resources developing a bit of content that only 20% of the players would ever be able to see. A cameo from Liara, as the Shadow Broker, is the best you could hope for.
5) The ME3 ending will be ejected. Imagine you had to build around that. How fundamentally different would it be for a race of interconnected, biotically enhanced, species to conqueror a new galaxy? You'd functionally be the borg, consuming everything as you went along. The people who chose to enslave the Reapers could use them as shock troops, and become lords of the galaxy in months, they'd be the Tyranids. Everything they lost during a war would be replaced by turning the opponent into enslaved reaper shock troopers. Finally, what about the middle finger to the reapers option? You'd be stuck in Civilization 5, where petty internal bickering could result in failure. The need to write four different stories (one fore each ending + new players) makes any substantial inter-connectivity to ME1-3 unlikely at best.
6) The time skip implies that relay tech doesn't play a huge role in the game. A little bit of math helps to put that into perspective. Let's say the the Milky Way is 100,000 light years across the galactic disc. Giving the Reapers the benefit of the doubt, let's say the Citadel actually shoots you 100,000 light years, instead of the couple of thousand that would be needed to escape the Milky Way. That means that 52 relays the size of the Citadel (2,500 thousand light years/ 100 thousand light years per relay pair, then you'd have to get into the cluster from the edge of Andromeda) would have be built. I'd say from the information in the OP that either a one way trip was taken on a ship using an FTL drive, or the denizens of the Milky Way made a one way catapult. The description about having to build an empire makes the bi-directional travel via relays so unlikely as to be implausible.
7) The "mysterious" races of the ME1-3 universe are not going to be explored. Their arc was completed in ME1-3, and subsequent information would require Bioware create a new history in which the Leviathans gave a damn. They were written as a literal deus ex machina, which after the ending to the game still wanted us "lesser" races enslaved. If the history of the Leviathans was actually explored we'd have to rewrite ME3, and EA would only allow that if there were stacks of cash involved. Bioware knows how much the fans desire continuity from them, as demonstrated by the absolute s###storm the three colored ending caused. Nobody with half a brain (though this may be assuming a bit from EA) would want that to happen again.
8) The Leviathans and Reapers are gone for good. All throughout ME1-3 you were dealing with the Protheans, the Reapers, and eventually the "apex" species of the Leviathans. The Leviathans said they were the first and the best, yet somehow the Remnants exist. The only viable explanation is either the Leviathans lied, and subsequently never found anything from the Remnants, or the Remnants never made it to the Milky Way. Focusing this back on point 1, the Remnants are likely the motivation for ME4 as the Protheans were for ME1-3.
9) EA is going to have this game monetized to Hell and back. It'd not hard to see, but let's state the obvious. The colonies have things you can build on them. Some of these things have resource collection, which allows you to buy new stuff. Said stuff is driven by things you find at random. Anyone else get the message yet? I'm seeing a preorder bonus that starts you off with some technology, and a number of resources. Once the preorder is dealt with, there will be DLC that allows your resource collectors to be more efficient. You can also spend real money to get resources, similar to Dead Space 3. There will also be a stream of DLC relating to playable species in horde mode, and a wealth of guns that can only be gotten via real money purchases. Another $60 game will wind up costing $200 with all the content purchased.
TL;DR:
So what we've got is a game with the Mass Effect name, but as Frag Maniac pointed out, none of the stuff from ME1-3. It'll be an entire reboot, with the name stuck on as a tactic to sell copies. Fine EA, you're welcome to stick your greasy tentacles into the Bioware skin suit you've made. I had enough of your crap with ME3, and this announcement makes me look forward to the thud that ME4 will make, when people finally realize they're playing Dragon's Age: Inquisition reskinned with an ME paint job. Bring on the torches, it looks like EA is getting ready to set the Bioware pyre alight.