There doesn't appear to be any adopted standards for MPEG over IP. IP over MPEG looks more interesting. Just packetize an IP stream into a packetized elementary stream (PES) and multiplex it into a valid MPEG2 Transport Stream. MPEG-2 typically gets transfered over DVB-ASI, DVB-C, DVB-SI, DVB-T and other protocols (even "ATSC" AKA SMPTE 310M).

So how do you packetize IP packets to go into an MPEG stream? Well that depends on the source. I'd like to think that any IP source "worth it's salt", is from a live network. Thus a network feed would need to input into the packetizer, multiplex it and put it out over a different type of device. I've heard of some people making a network device driver for DVB-ASI cards, but at least one engineer I talked to said there's probably a better way. He suggested keeping the regular characteristics of the ASI device, and doing the packetizing in application space. I managed to convince him however that the conveniences of creating a network device which can be bridged would be far better. He stuck with the separate device driver idea however and suggested one driver could use the other.

So then the question is, how do you create a network device driver that's just a packetizer, multiplexer and forwarder? No doubt there's some good examples out there, and NDIS should make it easier. I still worry about doing more than elementary processing in a driver might cause some strange system behavior. I guess I should also say there's probably an even easier way to do things in Linux and FreeBSD variants, but I'm mostly focused on the Microsoft world as that's what I'm told by Marketing is what's wanted.

On the opposite end you need a depacketizer? Or something to demultiplex the stream, and put IP back out onto the network. I've seen this done in software, and that might make more sense on this side of transfers. The engineer that I speak of above however suggested that the unidirectional nature of MPEG II transport streams would give another problem, associating one direction of traffic with the other.

I'm not quite sure how other people bind one transfer direction with another, but I remember several satellite companies offering service that beamed high speed broadband internet access to customers and accepted data back to them via telephone modem. So schemes to put two different directions from seperate devices have been around for a while. I just hope that modern network stacks are smart enough to remember that it's allowed.

I remember someone telling me that the ARPA network was an experiment designed with the goal that it be able to stay up, even if one link in the network went down. It failed, or at least that's the punchline. The modern Internet can't reroute if there's a failure in a router. There was a fire in a telecom building in Toronto, and connections from Manitoba Telephone Services (MTS) to Shaw in Winnipeg went down. I've also seen where an outage in Shaw's network caused places to become inaccessable, but if you had a proxy on a CA network accessable address, you could access the rest of the internet. Those are just two local examples that I know about. The CA network thing is political (I'm told they're not allowed to carry commercial data due to their funding grants). For what it's worth, I've also seen many shares of misconfigured routers, the more obvious cases were with major telecom. companies.

So back to MPEG transport streams. I know companies like Norsat have been selling "solutions" to do these things for years, so I think there's a market. Identifying the market potential is difficult for this because it's not something most broadcasters, stations, and local distributors are looking for. It's also not something that's really even remotely accessable to consumers.

A similar issue that I've thought about for even more years is multiple links between computers to increase throughput. I know lots of other people have looked at bridging and bonding, but I wanted to look at it at an even more insane level, serial ports. Actually I wanted to look at paralell ports, modems, ethernet, etc.. I suppose it is possible to bond all these links together, but it certainly isn't common enough that it's as easy as listing the links (at least as far as I know).

So why bother with all this legacy stuff? Why not build a new network card that can communicate at the full buss speed? Well actually we're pretty close to that now. From my own experiences I've calculated that modern HD-SDI cards must be close to maxing out the bus throughput. I've also learned that multiple cards on the same bus can't allow a faster network connection as of course there's only the one bus. Of course I've seen computers with multiple bus's, but it's hard to know if they're truly independent, or if they're more likely bridged. Even a bridged network of bus's can allow each bus to operate almost independently, if the parent bus is faster than it's children combined there may be an advantage to using multiple cards.

Even further it's important to note that most modern bus's have bottlenecks. When was the last time you looked up the DMA latency of the motherboard you wanted to buy? I'll wager never. I've thought about how useful this could be to consumers and whether there was a way I could get the comany that I'm working for to publish regular results of DMA throughput and latency. We could then get free motherboards. The idea likely wouldn't work though as that's not really what the business does.

Other crazy idea's I've had include using every processor on the system to do computations including the IDE/ATA hard drives (they have RAM too!). Alas of course most of it would be very convoluted to figure out a way to use.

The more recent idea that I've had (since Mark Nelson's "random" binary file challange was posted), was to figure out a list of common instructions and library calls for which I could get more output that would be required to issue the request for data (e.g. more bits from register results than from instruction cost...). This idea has some potential, but my current "hurdle" is finding the time to get and go through a list of CPU instructions. Getting a list of available function calls is also a challange although maybe a nice program to do it already exists (just get the exports from all dll's etc?).

So as you might be beginning to see, one of my primary interests is data compression. I used to be interested in the pure pattern finding, and making the smallest representation possible of common data, but then I started working in multimedia. It became obvious very fast that the speed of compression actually is important (not just to those who can't wait). If things don't compress fast enough you can get overruns, data loss and ultimatly data corruption (even if that just means missing bits/bytes/frames...).

One of my past pet projects was zlib compression of SDI (see StreamBed's deflate option). I've found that I can gzip at 270,000,000 bps (that's 270Mbps in SI notation). The small b means bits of course. The problem is that it ether needs a fast processor, or a simple pattern (like colour bars). It may even need both. I haven't had the time to check. Unfortunaly without the inflate option in StreamBed, customers aren't too interested yet, and without customer interest my boss isn't too interested yet either.

I later plan to talk about: Drew Scott Daniels' Resume