CURRENT_MEETING_REPORT_ Reported by Matt Mathis/PSC Minutes of the MBONE Engineering and Operations BOF (mbone) IETF Organizational Discussion We debated the need for a formal MBONE Working Group. This requires someone to volunteer to be the Chair and to draft a Charter. After some inconclusive discussion it was observed that no one was willing to volunteer. The people present were mostly network operators who are participating in the mbone. Unfortunately a number of network operators who feel victimized by the mbone were not present. General Issues o We must balance the risk: The Mbone is dangerous, but the applications are very useful, and may drive the next generation Internet technology. o Critical problems are largely due to the lack of tools to manage the mbone. o There are at least three constituents in the Mbone/multicast community: Network operators, who are providing the mbone core (at least in the NSFnet context), network subscribers, who are typically mbone stubs and finally, mbone end users. There was considerable discussion about these groups, and their identity. These three groups have substantially different cultures, approaches to problems and worries. - The network operators have a service oriented perspective and an intimate understanding of the underlying topology. Almost all of the people present were network operators. - The subscribers are typically researchers who want mbone connectivity but are not really interested in the details. They usually operate stub tunnels to connect campuses into the mbone. - The mbone end users are most likely applications developers or true users, and use local area multicasting to reach a subscriber tunnel. This community is likely to have no knowledge about operational details of the mbone. - [I have since realized that there are a number of core mbone hubs which are managed by multicast researchers on the premises of a network operator, with only minor supervision by the 1 operator. This straddles the operator/subscriber distinction above.] o We discussed the mbone mailing list. Some people wanted to split the mailing list such that operators could have candid discussions about debugging without kibitzing from overzealous users. The consensus was not to make any changes and use the mbone list as it stands. o The hypothetical problem came up about a subscriber who was not getting satisfactory service from his network operator. Would operators support tunnels into other regionals? The Group was unanimous that this is a bad practice, and shouldn't be allowed. Furthermore, subscriber to subscriber tunnels are even worse, and the operators should not to provide such poor service that their subscribers resort to such tactics. There was some discussion of the business implications to network operators. o We discussed the new encapsulated tunneling code. The original code is a seriously bastardized use of LSRR. The new mrouted supports both, defaulting to encapsulation. Use the ``srcrt'' directive in mrouted.conf to get LSRR. Matt pointed out that those who most endanger the rest of the Internet were also those who didn't require the new encapsulation code for performance. A long discussion of capabilities ensued. One salient point was that the LSRR code seriously violates the IP specifications. There was talk of adding clean source routing options to the encapsulating code such that network operators could prevent tunnels from moving to fall back infrastructure during network failures. This would cause multicasting load to be shed during failures of primary IP connectivity. Most of the rest of the meeting was spent drafting a wish list. Some of the items are appropriate for a future meeting of this BOF or Working Group. Other items are appropriate for specific groups or projects involved in multicast research. The items are ordered by priority within each section, but the sections are independent of each other. Throughout the discussion it was understood that resources are tight, and in many cases we were asking people to contribute effort without additional support. Items for Network Operators or Some Future Mbone Working Group o Put more pressure on router vendors to provide implementations that perform well with LSRR. o Generate an mbone operational guidelines document. This could be started by splitting the existing FAQ document into separate user and operator documents. 2 o Explicitly engineer the mbone topology, metrics, and thresholds. This is a daunting task with insufficient tools or poor knowledge of the actual Internet topology. o There needs to be a global policy on bandwidth budgeting and allocation. The mbone is currently a single global resource, and must be allocated as such. Sometimes there will be two groups with legitimate reasons to do multichannel world-wide multicasts, and there must be some mechanism to arbitrate between their needs. Infrastructure Tools, to Help Engineer and Operate the Mbone Itself o Three different Mapping tools: - A configuration map which shows all configured tunnels, even if they are not ``up'' - to discover configuration anomalies. - A tunnel map (done: Pavel Curtis's tool) - A flow map, showing the distribution tree for an arbitrary transmitter. o Alarms that can be triggered by excessive mbone traffic, such that sites with large pipes can protect the rest of the Internet. o A ``tunnel trap'' to detect unauthorized (e.g., client-to-client) tunnels passing through a regional. Several algorithms were discussed. o Snmp/opstats style tools for logging load (traffic) levels. o Map Tracing - proxy IP traceroute along tunnels to detect when they share the common infrastructure. o Traceroute (follow the path from the transmitter to a receiver) - deemed to be almost the same as the flow map but not as useful. (A Flow map gives the entire flow topology.) Transport tools, to Verify, Monitor and Diagnose Signal Quality These should use the Audio Video Transport protocol (AVT) directly without specific knowledge of the applications, except to mimic aggregate traffic statistics. All should be supported on all platforms supporting mrouted. (Not just the platform supporting some particular application). o Signal quality meters - To display packet loss and delay statistics throughout the distribution tree. It is critical that this tool 3 run on every mrouted platform. o (Talk) Spurt correlated signal quality - Display packet loss and delay statistics correlated with the start of talk spurts - to detect interactions due to competition between the mbone and other Internet traffic. (TCP congestion avoidance takes one round trip time to back off, so congested links are often late/lossy during the initial second of a talk spurt). o Basic test generator - generate traffic streams to mimic the 1st order statistics of the more popular applications (Correct average packet rate and size). Again it is important that this tool not require the actual transmitting platform, because the lead time to acquire the transmitters has historically prevented the IETF multicasts from being tested in advance. o Modulation test generator: Transmit 5 second bursts (``spurts'') of the above generator separated by 5 seconds of silence, synchronized with GMT. This can be used to detect many different problems using clock based monitors. It is important that the generators be (roughly) synchronized so this technique can be used with an entire suite of sources. A typical use might be to emulate up to two video and two audio sources for an IETF, and correlating network events, such as device errors and ethernet collision rates, against the clock to determine if the mbone is causing a particular problem. Applications which do not use AVT should have their own set of the above tools. It is imperative that the majority of these tools run on all mrouted capable platforms. It is currently very difficult to sectionalize distribution problems on paths through multiple mrouted tunnels that are incapable of running the specific applications. Mrouted/tunnel Implementation Features o Encapsulated tunnels (already done) o Support for ``on demand'' host tunnels, such that mrouted can be used as a reflector for hosts that don't support multicast, such as Mac's. (This isn't really high priority, but it is relatively easy to implement). o Implement pruning. Currently all multicast traffic goes to all tunnel connected nodes within the TTL limit, even if there are no listeners. Note that TTL mechanism is still needed because pruning is not fast enough to protect slow links. o Implement aggregate packet rate and bandwidth limits, to protect 4 the underlying IP infrastructure from being flooded. o The routing protocol, DVMRP, should use the same tunnels as the data to prevent the situations we see today where the unicast routing can follow some path, but the data can not. DVMRP sees the tunnel as up but all of the data is discarded. o Support LSRR on encapsulated tunnels, so tunnels can be anchored to specific IP infrastructure. o Correct/work around the BSD bug which prevents DVMRP from asserting the source address of a tunnel on a multi-interfaced host. This causes IP routing to break tunnels when they move to other interfaces because the other end does not recognize the source IP address. [A possible solution, which also partially addresses the tunnel anchors, would be to specify the first hop address and have mrouted install static host routes for tunnels.] o Mrouted support for traceroute and mapping infrastructure tools. o There needs to be a logging facility/level for monitoring DVMRP routing problems. Such a facility would report tunnel up/down events and routing table changes. Steve Deering was present and both contributed and noted our comments. The wish lists were presented to both the AVT Working Group and to the Operational Requirements Area Directorate (ORAD). There was general agreement that the items on the wish list are desirable and appropriate, but nobody agreed to implement anything. There were some network operators present at the ORAD meeting who were upset that the MBONE BOF did not become a working group or take stronger positions regarding operational practices. However most of these people were operators who had chosen not to participate in the mbone, and were therefore not in control of its impact on their own facilities. Thanks to Jamshid Mahdavi for taking the Minutes. It should be noted that the attendee list below was reconstructed after the fact and may not contain the names of all participants. Attendees Erik-Jan Bos erik-jan.bos@surfnet.nl Douglas Carson carson@utcc.utoronto.ca Henry Clark henryc@oar.net Steve Deering deering@parc.xerox.com Dale Finkelson dmf@westie.mid.net Eugene Hastings hastings@psc.edu Daniel Long long@nic.near.net Bill Manning bmanning@sesqui.net 5 Matt Mathis mathis@a.psc.edu David O'Leary doleary@cisco.com Curtis Villamizar curtis@ans.net Linda Winkler lwinkler@anl.gov Paul Zawada Zawada@ncsa.uiuc.edu 6