Actually, in the 3DO case Panasonic simply overpriced the original unit, they could have brought it out at closer to $499 than the $799 they did. That's why the price dropped so fast and nobody else brought out models that expensive.
But yeah, in the 3DO case there were no royalties or game licensing fees as traditional in the industry. There was a flat $3 per disc fee that went to 3DO to cover costs and return them a small profit. IIRC, this was supposed to be filtered out to manufacturers as a "bonus" on the machines sold but I believe only Panasonic and Goldstar ever sold enough models to receive the "bonus" back. And Goldstar took it and ran rather than continue to manufacture the machines. From my understanding both of them did at least break even on the machines. I assume the other manufacturers lost money.
The entire system did fail though, that's why 3DO was trying to sell the M2 design to Nintendo, Sega and eventually Panasonic. Nobody was interested in the whole license a design and manufacture it yourself model anymore. DVD and later HD-DVD/Blu-ray use a very different standard system, it's based more around the software. (This is what Nuon aka "Project X" did as well.) The CD-I standard was somewhere between the two. 3DO had to meet hardware specifics and use 3DO's chipsets, a manufacturer couldn't put in a faster chipset for example. This is also why the 3DO Blaster was able to be released for PC's, 3DO got the board size down to where Creative could sell it as a plug-in board.
M2 would have slightly altered this as it used PowerPC along with some other standard parts, only the one chip was custom. But the rest of the electronics industry had been burned by the "multimedia" wave to where nobody wanted anything to do with it. It's still not clear exactly why Panasonic spent $100 million to buy it. I can't imagine the arcade boards for Konami made the money back.
One likely extrapolation I've made from some of Trip Hawkins' statements around the time is that Panasonic may have been paying off some of 3DO's outstanding debts. In a couple interviews discussing 3DO's shift to being a software company he emphasizes that they're debt free like it must have been a problem prior. One thing I just thought of, is since they were handling the discs, they may have found themselves like Acclaim, Capcom and some others of the era and sitting on warehouses full of product they couldn't sell. I remember a local Best Buy put all their 3DO games for $1, the rack still sat there nearly untouched for over two months.