Adobe Flash Player 10 and Adobe AIR 1.5 introduce a new communications protocol, Real-Time Media Flow Protocol (RTMFP), whose low latency, end-to-end peering capability, security, and scalability make it especially well suited for developing real-time collaboration applications by not only providing superior user experience but also reducing operators' costs.

Earlier versions of Flash Player use Real-Time Messaging Protocol (RTMP) and require Adobe Flash Media Server (FMS) for interactive collaboration applications (such as Adobe Acrobat Connect Pro) or audio/video streaming. While RTMP is an excellent choice for streaming media, shared objects, or remoting, it has limited ability of meeting real-time requirements of interactive audio and video communications.

In order to use RTMFP, Flash Player endpoints must connect to an RTMFP-capable server, such as the Codename Cirrus (previously Codename Stratus) service or Flash Media Server 4. Cirrus is a hosted rendezvous service that aids establishing communications between Flash Player endpoints. Unlike FMS, Cirrus does not support media relay, shared objects, scripting, etc. So by using Cirrus, you can develop applications only where Flash Player endpoints are directly communicating with one another.

Flash Player is already the market leader in online video distribution over the web. With the introduction of RTMFP and advanced media compression technologies, Flash Player 10 is well positioned as the leader in real-time communications as well.

In this article, I first highlight the benefits of using RTMFP in real-time communications applications. Second, I describe the new ActionScript 3 API for managing direct end-to-end RTMFP connections. Finally, I present our VideoPhone sample application.

Benefits of RTMFP

Real-Time Media Flow Protocol (RTMFP) is a new communications protocol introduced in Flash Player 10 and also available in AIR 1.5. One of its major differentiators from Real-Time Messaging Protocol (RTMP), which is based on the Transmission Control Protocol (TCP) and exclusively used in previous versions of Flash Player, is that RTMFP is built on User Datagram Protocol (UDP).

While TCP provides reliable data delivery (well applicable for file transfer, e-mail, etc.), it does not provide any end-to-end delay guarantees. Reliable data transmission in TCP is achieved by re-transmission of lost data, which introduces latency. Because minimizing end-to-end delay is one of the most important goals in real-time communications (a few hundred milliseconds' delay may render a conversation unusable), TCP is not well suited for this purpose. Transmission error resilience and recovery form an integral part of most advanced audio and video compression techniques—such as Speex audio and H.264 video codec, both available in Flash Player 10. Reliable delivery provided by TCP is therefore not needed. As a result, UDP, which provides an efficient and rapid data delivery, is popularly used in real-time collaboration applications where minimizing end-to-end delay is of paramount importance. Another advantage of UDP over TCP that it enables end-to-end peering—that is, direct data transmission between two clients located behind network address translators (NATs).

When compared to RTMP, RTMFP provides the following advantages for real-time communications:

• Low latency: Since RTMFP is built on top of UDP, it provides minimal latency for real-time communications. It is important to note that RTMFP provides both reliable and unreliable service. When sending data between two Flash Player instances (for example, using the NetStream.send() method), reliable data transmission is used. When sending Speex audio between two Flash Player instances, unreliable delivery is used, providing the smallest possible latency.
• End-to-end media delivery: Media is sent directly between two Flash Player instances without routing through a central relay server. When compared to RTMP, where all data is sent through Flash Media Server, RTMFP not only further reduces end-to-end delay, but also eliminates costs associated with a central data relay, thus lending itself to extremely scalable deployments.
• Data prioritization: Audio is transmitted with higher priority than video and non-time critical data (such as instant message, etc.). This can significantly enhance user experience over a bandwidth constrained communications channel.

All of these features represent tremendous benefits for real-time communications, providing a significantly greater user experience than is achievable with earlier versions of Flash Player.

Firewall traversal

RTMFP is built on top of UDP, which enables direct connection between clients even if they are located behind NATs or firewalls. In order for RTMFP to work, your firewall must be configured to allow outgoing UDP traffic. While this is the case with most consumer or small office/home office (SOHO) firewalls, many corporate firewalls block UDP traffic altogether.

One solution is to configure Flash Player to use a TURN proxy (Traversal Using Relays around NAT). Flash Player supports IETF Internet Draft draft-ietf-behave-turn-08 without authentication. If the network administrator configures a TURN proxy that allows outgoing UDP, Flash Player can be configured by adding the following line in mms.cfg (for more information on Flash Player configuration and the location of mms.cfg, please read the Adobe Flash Player Administration Guide for Flash Player 10):

Direct UDP traffic is always attempted and the TURN proxy is only used as a backup: it is used for UDP traffic that cannot flow between Flash Player and Cirrus (in case of UDP blocking firewall) or between Flash Player endpoints.

Even if your firewall enables outgoing UDP traffic, it is possible that end-to-end peering cannot be established due to a combination of firewalls. When one endpoint is located behind a so-called "symmetric firewall," end-to-end communications may not be possible. (For a classification of firewalls, please see the Network address translation entry on Wikipedia.) In this situation, you may use a TURN proxy to aid firewall traversal.

Cirrus service

Flash Player instances must connect to the Cirrus service (using rtmfp://p2p.rtmfp.net) in order to communicate with one another. Cirrus is a hosted rendezvous service that helps Flash Player instances contact one another even if they are located behind NATs. Although connecting to Cirrus service is very similar to connecting to Flash Media Server, Cirrus does not provide any of the typical Flash Media Server features (media relay, shared objects, remoting, etc.). Flash Player endpoints must stay connected to Cirrus during the entire time of communication. In order to access Cirrus, you will need a developer key that is generated when you create your Adobe Developer ID.

RTMFP support is available in Flash Media Server 4. With Flash Media Server, it will be possible to enable communications between Flash Player 9 or earlier clients (using RTMP) and Flash Player 10 clients (using RTMFP).

Security

RTMFP provides secure communications between endpoints. It uses a 128-bit AES with the key negotiated using the Diffie-Hellmann key exchange method. However, it does not provide strong endpoint authentication such as SSL or RTMPS. To aid endpoint authentication, RTMFP and ActionScript expose secure nonces to application developers. These nonces are available at both communicating Flash Player endpoints and are guaranteed to match. By verifying these nonces, end users can ensure that there is no man-in-the-middle attack. These nonces can also be used to develop key continuity mechanism.

It is important to note that Flash Player only enables sending media from your microphone and webcam devices to other Flash Player endpoints that subscribe to your media streams.

ActionScript 3 API supporting RTMFP

There is a new ActionScript 3 API in Flash Player 10 to support RTMFP. Connecting to the Cirrus service and creating end-to-end media streams are analogous to working with Flash Media Server. Please note that you must use ActionScript 3 with either Flash Professional (CS4 or later) or Flash Builder 4 targeting Flash Player 10 or AIR 1.5.

As I mentioned before, first you must connect to the Cirrus service:

The developer key is issued when you sign up for an Adobe Developer Connection account and is available on the Cirrus beta service site.

Upon successful connection to Cirrus, you get a NetConnection.Connect.Success event. There could be several reason for connection failure. If you provide an invalid developer key or incorrectly specify Cirrus address, you'll receive NetConnection.Connect.Failed. If your firewall blocks outgoing UDP traffic, you'll receive the NetConnection.Connect.Failed event after a 90-second timeout.

After successfully establishing a connection to the Cirrus service, you are assigned a unique 256-bit peer ID (NetConnection.nearID). Other Flash Player endpoints must know this peer ID in order to receive your published audio/video streams. It is out of the scope of Flash Player or the Cirrus service how these peer IDs are exchanged among Flash Player endpoints. For exchanging peer IDs, you may use an XMPP service or a simple web service, as the Video Phone sample application does.

Direct communications between Flash Player instances is conducted using unidirectional NetStream channels. That is, if you want two-way voice conversation, each Flash Player endpoint must create a sending NetStream and a receiving NetStream.

First, create a sending NetStream:

This means that media is published as an end-to-end stream. Since Cirrus cannot relay media, you can publish only end-to-end streams. This stream will include both audio and video from your local default devices chosen by the Settings Manager.

Note: Audio/video is not sent out until another Flash Player endpoint subscribes to your media stream.

Now, create the receiving NetStream:

At this point, you hear audio and you can create a Video object to display video. In order to create the receiving NetStream, you must know the 256-bit peer ID of the publisher (id_of_publishing_client). In order to receive audio/video, you must know the name of the stream being published.

Advanced topics

The publisher has fine control over which endpoint can receive its published stream. When a subscriber attempts to receive a published stream, the onPeerConnect() method is invoked (default implementation simply returns true) on the published NetStream. The publisher could disallow certain Flash Player endpoints to receive its media:

On the publisher side, the NetStream.peerStreams property holds all the subscribing instances of the publishing NetStream. For example, using sendStream.send() will send the same data to all subscribers. You can use the following to send information to a specific subscriber:

The NetConnection.maxPeerConnections property specifies the number of peer streams that are allowed to connect to the publisher. The default value is set to 8 but, in practice, depending on your application, you must consider that most ISPs provide asymmetric Internet access. Figure 1 illustrates the direct communication among three instances of Flash Player. Each Flash Player endpoint sends and receives two streams, creating a fully connected mesh. Since Internet download capacity is generally much higher than upload capacity, you must be extra careful not to overload the end-user's uplink.

The NetConnection.unconnectedPeerStreams property is an array of NetStreams that are not associated with a publishing NetStream yet. When a publishing stream matches a subscribing stream name, the subscribing NetStream is moved from this array to the publishing NetStream.peerStreams array.

Exploring the Video Phone sample application

We have developed a sample video phone application for illustrating how to use end-to-end capabilities of Flash Player 10. It is also available as part of this article.

The Video Phone sample application relies on a simple HTTP service to exchange the Flash Player peer ID. The script is provided as part of the package, reg.cgi. This web service does not provide any user authentication. After Flash Player successfully connects to Cirrus, it registers its peer ID with the web service. When making a call, the Video Phone caller uses this web service to look up recipient's peer ID.

Adobe runs this web service exclusively for the hosted Video Phone sample. When you build your own Video Phone sample, you must run your own web service and specify WebServiceUrl in VideoPhoneLabs.mxml. You should override the AbstractIdManager class to implement your own peer ID exchange mechanism—using, for example, XMPP, Google Apps, or the Facebook framework.

The following steps are necessary to build a Video Phone sample application—for more details, please see ReadMe.txt included in the package:

1. Host a web service for the peer ID exchange using the provided reg.cgi Python script.
2. Create a new Flex project in Flash Builder 4.
3. Add the source files from the package—VideoPhoneLabs.mxml, AbstractIdManager.as, HttpIdManager.as, IdManagerError.as, and IdManagerEvent.as—to the project src folder.
4. Specify your Cirrus developer key in DeveloperKey in VideoPhoneLabs.mxml.
5. Specify the URL for the web service in WebServiceUrl in VideoPhoneLabs.mxml.

The Video Phone sample application uses the phone model. The call control procedure is implemented using the newly added Cirrus message passing capability. Client A sends an "Invite" message to client B through Cirrus service and publishes the media stream. Client B learns the caller's remote ID and username from the incoming "Invite" message and prompts the user to accept or reject the call. If Client B accepts the call, it sends an "Accept" message through Cirrus service, subscribes to Client A's media stream, and publishes the media streams. When Client A receives the "Accept" message, it subscribes to Client B's media stream and two-way communications is established.