Common requirements for Carrier Grade NATs
(CGNs)Viagénie2875 boul. Laurier, suite D2-630QuébecQCG1V 2M2Canada+1 418 656 9254simon.perreault@viagenie.cahttp://www.viagenie.ca
NTT Communications CorporationGran Park Tower 17F, 3-4-1 Shibaura, Minato-kuTokyo108-8118Japan+81 50 3812 4704ikuhei@nttv6.jp
NTT Communications CorporationGran Park Tower 17F, 3-4-1 Shibaura, Minato-kuTokyo108-8118Japan+81 50 3812 4695miyakawa@nttv6.jp
Japan Internet Exchange Co., Ltd. (JPIX)Otemachi Building 21F, 1-8-1 Otemachi, Chiyoda-kuTokyo100-0004Japan+81 90 9242 2717a-nakagawa@jpix.ad.jpIS Consulting G.K.12-17 Odenma-cho Nihonbashi Chuo-kuTokyo103-0011Japanassie@hir.jp
Transport
Internet Engineering Task ForceCGN, NATThis document defines common requirements for Carrier-Grade NAT
(CGN).With the shortage of IPv4 addresses, it is expected that more ISPs may
want to provide a service where a public IPv4 address would be shared by
many subscribers. Each subscriber is assigned a private address, and a
NAT situated in the ISP's network translates between private and public
addresses. When a second IPv4 NAT is located at the customer edge, this
results in two layers of NAT.This is not to be considered a solution to the shortage of IPv4
addresses. It is a service that can conceivably be offered alongside
others, such as IPv6 services or regular, un-NATed IPv4 service. Some
ISPs started offering such a service long before there was a shortage of
IPv4 addresses, showing that there are driving forces other than the
shortage of IPv4 addresses.This document describes behavioral requirements that are to be expected
of those multi-subscriber NATs. Meeting this set of requirements
will greatly increase the likelihood that subscribers' applications will
function properly.Readers should be aware of potential issues that may arise when sharing
a public address between many subscribers. See for details.This document builds upon previous works describing requirements for
generic NATs . These documents, and their updates if any, still
apply in this context. What follows are additional requirements, to be
satisfied on top of previous ones.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in .
Readers are expected to be familiar with and the
terms defined there. The following additional term is used in this document:
A NAT-based functional element operated by an administrative
entity (e.g., operator) to share the same address among several
subscribers. A CGN is managed by the administrative entity, not the
subscribers.
Note that the term "carrier-grade" has nothing to do with the
quality of the NAT; that is left to discretion of implementers.
Rather, it is to be understood as a topological qualifier: the NAT
is placed in an ISP's network and translates the traffic of
potentially many subscribers. Subscribers have limited or no control
over the CGN, whereas they typically have full control over a NAT
placed on their premises. summarizes a common network topology in which a
CGN operates.Another possible topology is one for hotspots, where there is no customer
premise or customer-premises equipment (CPE), but where a CGN serves a bunch
of customers who don't trust each other and hence fairness is an issue. One
important difference with the previous topology is the absence of a second
layer of NAT. This, however, has no impact on CGN requirements since they
are driven by fairness and robustness in the service provided to customers,
which applies in both cases.What follows is a list of requirements for CGNs. They are in
addition to those found in other documents such as ,
, and .A CGN MUST support at least the following transport protocols: TCP
(MUST support ), UDP (MUST support ), and ICMP (MUST support ).
Support for additional transport protocols is OPTIONAL.These protocols are the ones that NATs
traditionally support. The IETF has documented the best current
practices for them.A CGN MUST have a default "IP address pooling" behavior of "Paired"
(as defined in section 4.1). A CGN MAY provide
a mechanism for administrators to change this behavior on an application
protocol basis.
When multiple overlapping internal IP address ranges share the same
external IP address pool (e.g., DS-Lite ),
the "IP address pooling" behavior applies to mappings between
external IP addresses and internal subscribers rather than between
external and internal IP addresses.This stronger form of REQ-2 from is justified by the stronger need for not breaking
applications that depend on the external address remaining constant.Note that this requirement applies regardless of the transport
protocol. In other words, a CGN must use the same external IP address
mapping for all sessions associated with the same internal IP address,
be they TCP, UDP, ICMP, something else, or a mix of different
protocols.The justification for allowing other behaviors is to allow the
administrator to save external addresses and ports for application
protocols that are known to work fine with other behaviors in practice.
However, the default behavior MUST be "Paired".The CGN function SHOULD NOT have any limitations on the size nor the
contiguity of the external address pool. In particular, the CGN function
SHOULD be configurable with contiguous or non-contiguous external IPv4
address ranges.Given the increasing rarity of IPv4
addresses, it is becoming harder for an operator to provide large
contiguous address pools to CGNs. Additionally, operational flexibility
may require non-contiguous address pools for reasons such as
differentiated services, routing management, etc.A CGN SHOULD support limiting the number of external ports (or,
equivalently, "identifiers" for ICMP) that are assigned per subscriber.
Limits SHOULD be configurable by the CGN administrator.Limits MAY be configurable independently per transport
protocol.Additionally, it is RECOMMENDED that the CGN include
administrator-adjustable thresholds to prevent a single subscriber
from consuming excessive CPU resources from the CGN (e.g., rate limit
the subscriber's creation of new mappings).A CGN can be considered a network resource
that is shared by competing subscribers. Limiting the number of external
ports assigned to each subscriber mitigates the DoS attack that a
subscriber could launch against other subscribers through the CGN in
order to get a larger share of the resource. It ensures fairness among
subscribers. Limiting the rate of allocation mitigates a similar attack
where the CPU is the resource being targeted instead of port
numbers.A CGN SHOULD support limiting the amount of state memory allocated per
mapping and per subscriber. This may include limiting the number of
sessions, the number of filters, etc., depending on the NAT
implementation.
Limits SHOULD be configurable by the CGN administrator.Additionally, it SHOULD be possible to limit the rate at which
memory-consuming state elements are allocated.A NAT needs to keep track of TCP sessions
associated to each mapping. This state consumes resources for which, in
the case of a CGN, subscribers may compete. It is necessary to ensure
that each subscriber has access to a fair share of the CGN's resources.
Limiting TCP sessions per subscriber and per time unit is an effective
mitigation against inter-subscriber DoS attacks. Limiting the rate of
allocation is intended to prevent against CPU resource exhaustion.It SHOULD be possible to administratively turn off translation for
specific destination addresses and/or ports.It is common for a CGN administrator to
provide access for subscribers to servers installed in the ISP's
network, in the external realm. When such a server is able to reach the
internal realm via normal routing (which is entirely controlled by the
ISP), translation is unneeded. In that case, the CGN may forward packets
without modification, thus acting like a plain router. This may
represent an important efficiency gain. illustrates this use-case.It is RECOMMENDED that a CGN have an "Endpoint-Independent Filtering"
behavior (as defined in section 5). If it is
known that "Address-Independent Filtering" does not cause the
application-layer protocol to break (how to determine this is out of
scope for this document), then it MAY be used instead.This is a stronger form of REQ-8 from . This is based on the observation that some games
and peer-to-peer applications require EIF for the NAT traversal to work.
In the context of a CGN it is important to minimize application
breakage.When a CGN loses state (due to a crash, reboot, failover to a cold
standby, etc.), it MUST NOT reuse the same external address+port pairs
for new dynamic mappings for at least 120 seconds, except for any of the
following cases:
If the CGN tracks TCP sessions (e.g., with a state machine,
as in section 3.5.2.2), TCP ports MAY be
reused immediately.If external ports are statically assigned to internal addresses
(e.g., address X with port range 1000-1999 is assigned to subscriber
A, 2000-2999 to subscriber B, etc.), and the assignment remains
constant across state loss, then ports MAY be reused
immediately.This is necessary in order to prevent
collisions between old and new mappings and sessions. It ensures that
all established sessions are broken instead of redirected to a different
peer.The exceptions are for cases where reusing a port immediately does not
create a possibility that packets would be redirected to the wrong
peer.The 120 seconds value corresponds to the Maximum Segment Lifetime (MSL)
from .One way that this requirement could be satisfied would be have two
distinct address pools: one dormant and one active. When rebooting, the
CGN would swap the dormant pool with the active pool. Another way would
be simply to wait 120 seconds before resuming NAT activity.Once an external port is deallocated, it SHOULD NOT be reallocated to a
new mapping until at least 120 seconds have passed. The length of time
and the maximum number of ports in this state SHOULD be configurable by
the CGN administrator. The following exceptions apply:
If the CGN tracks TCP sessions (e.g., with a state machine,
as in section 3.5.2.2), TCP ports MAY be
reused immediately.If the allocated external ports used address-dependent or
address-and-port-dependent filtering before state loss, they MAY be
reused immediately.This is to prevent users from receiving
unwanted traffic. It also helps prevent against clock skew when mappings
are logged.The exceptions are for cases where reusing a port immediately does not
create a possibility that packets would be redirected to the wrong
peer.The 120 seconds value corresponds to the Maximum Segment Lifetime (MSL)
from .A CGN SHOULD include a Port Control Protocol server .Allowing subscribers to manipulate the NAT
state table with PCP greatly increases the likelihood that applications
will function properly.A CGN SHOULD support .It is anticipated that CGNs will be primarily
deployed in ISP networks where the need for management is critical.Note also that there are efforts within the IETF toward creating a MIB
tailored for CGNs (e.g., ).When a CGN is unable to create a mapping due to resource constraints
or administrative restrictions (i.e., quotas):
it MUST drop the original packet;it SHOULD send an ICMP Destination Unreachable message with code 3
(Port Unreachable) to the session initiator;it SHOULD send a notification (e.g., SNMP trap) towards a
management system (if configured to do so);and it SHOULD NOT delete existing mappings in order to "make room"
for the new one. (This only applies to normal CGN behavior, not to
manual operator intervention.)This is a slightly different form of REQ-8
from . Code 3 is preferred to code 13 because it
is listed as a "soft error" in , which is
important because we don't want TCP stacks to abort the connection
attempt in this case. Sending an ICMP error may be rate-limited for
security reasons, which is why requirement B is a SHOULD, not a
MUST.Applications generally handle connection establishment failure better
than established connection failure. This is why dropping the packet
initiating the new connection is preferred over deleting existing
mappings. See also the rationale in section
6.It may be necessary for CGN administrators to be able to identify a
subscriber based on external IPv4 address, port, and timestamp in order to
deal with abuse and lawful intercept requests. When multiple subscribers
share a single external address, the source address and port that are
visible at the destination host have been translated from the ones
originated by the subscriber.In order to be able to do this, the CGN would need to log the following
information for each mapping created:
subscriber identifier (e.g., internal source address or tunnel endpoint
identifier)external source addressexternal source porttimestampBy "subscriber identifier" we mean information that uniquely identifies a
subscriber. For example, in a traditional NAT scenario, the internal source
address would be sufficient. In the case of DS-Lite, many subscribers share
the same internal address and the subscriber identifier is the tunnel
endpoint identifier (i.e., the B4's IPv6 address).A disadvantage of logging mappings is that CGNs under heavy usage may
produce large amounts of logs, which may require large storage volume.A CGN SHOULD NOT log destination addresses or ports.Destination logging at the CGN creates
privacy issues. Furthermore, readers should be aware of logging
recommendations for Internet-facing servers .
With compliant servers, the destination address and port do not need to
be logged by the CGN. This can help reduce the amount of logging.So far we have assumed that a CGN allocates one external port for every
outgoing connection. In this section, the impacts of allocating multiple
external ports at a time are discussed.There is a range of things a CGN can do:
For every outgoing connection, allocate one
external port.For an outgoing connection, create a set
of several non-consecutive external ports. Subsequent outgoing
connections will use ports from the set. When the set is exhausted, a
new connection causes a new set to be created. A set is smaller or
equal to the user's maximum port limit.Same as the scattered port set, but
the ports allocated to a set are consecutive.Note that this list is not exhaustive. There is a continuum of behavior
that a CGN may choose to implement. For example, a CGN could use scattered
port sets of consecutive port sets.The impacts of bulk port allocation are as follows.
The mechanisms at the top of the list are
very efficient in their port utilization. In that sense, they have good
scaling properties (nothing is wasted). The mechanisms at the bottom of
the list will waste ports. The number of wasted ports is proportional
to size of the "bin".Traditional allocation creates a lot of log entries
as compared to allocation by port sets which creates much fewer entries.
Scattered and consecutive port sets generate the same number of log
entries. In the case of consecutive port sets, entries can be expressed
very compactly by indicating a range (e.g., "12000-12009"). Some
scattered port set allocation schemes can also generate small log
entries containing the parameters and algorithm used for the port set
generation (see, e.g., ).With large set sizes, the logging frequency for scattered and
consecutive port sets can approach that of DHCP servers.Logging destination addresses and ports can only be done on a
per-session basis. This means that destination logging for a CGN
implementing bulk port allocation would create one log entry per session
containing the destination address and port. Other information could
still be logged in one entry per port set.Traditional and scattered port sets provide very
good security in that ports numbers are not easily guessed. Easily
guessed port numbers put subscribers at risk of the attacks described in
. Consecutive port sets provides poor security
to subscribers, especially if the set size is small.Several issues are encountered when CGNs are used . There is current
work in the IETF toward alleviating some of these issues. For example, see
.The address sharing ratio is the ratio between the number of external
addresses and the number of internal addresses that a CGN is configured to
handle. See
section 26.2 for guidance on picking an appropriate ratio.There are no IANA considerations.If a malicious subscriber can spoof another subscriber's CPE, it may
cause a DoS to that subscriber by creating mappings up to the allowed
limit. Preventing this can be accomplished with ingress filtering, as
described in .Endpoint-Independent Filtering has security considerations which are
discussed in .NATs sometimes perform fragment reassembly. CGNs would do so at
presumably high data rates. Therefore, the reader should be familiar with
the potential security issues described in .
Thanks for the input and review by
Arifumi Matsumoto,
Benson Schliesser,
Dai Kuwabara,
Dan Wing,
Dave Thaler,
Francis Dupont,
Joe Touch,
Lars Eggert,
Kousuke Shishikura,
Mohamed Boucadair,
Nejc Skoberne,
Reinaldo Penno,
Senthil Sivakumar,
Takanori Mizuguchi,
Takeshi Tomochika,
Tina Tsou,
Tomohiro Fujisaki,
Tomohiro Nishitani,
Tomoya Yoshida,
and
Yasuhiro Shirasaki.
Dan Wing also contributed much of section 5.
Removed DSCP requirement since it applies to non-CG NATs as
well.Removed instances of "NAT444".Filtering has no effect on the requirement for a hold down
pool. Removed REQ-8-B.Statically assigned port ranges do not need to go in the hold down
pool. Added a new REQ-8-B.Fixed various nits. More precise text in some places.Fixed nits, spelling, updated references.CGNs SHOULD NOT log destinations.Allow address-dependent filtering when it does not cause the
application protocol to break.Refer to RFC4787 security considerations on EIF.Clarify REQ-12 point D (it does not apply to operator
intervention).Changed "CGNs SHOULD limit ..." to "SHOULD support limiting" to
make it clear that the operator is in control.Added reference to RFC 4963.Added requirement for non-contiguous external address pools.Added exceptions for which it is not necessary to wait 120 seconds
before reusing a port.Renamed "random port set" to "scattered port set", which is more
accurate.Log "subscriber identifier" instead of internal address+port to
allow for overlapping internal address ranges (DS-Lite).Adjusted logging text and added reference to
I-D.boucadair-pppext-portrange-option.Adjusted destination logging text for bulk port allocation
schemes.Removed requirement for I-D.ietf-intarea-ipv4-id-update.Made PCP support a SHOULD-level requirement.Lowered the level of requirement for not dropping existing
mappings in order to "make room" to SHOULD level, and added
rationale.CGNs MUST support at least TCP, UDP, and ICMP.Add requirement from I-D.ietf-intarea-ipv4-id-update.Add informative reference to .Add requirement (SHOULD level) for a port forwarding protocol.Allow any pooling behavior on a per-application protocol
basis.Adjust wording for external port allocation rate limiting.Add requirement for RFC4008 support (SHOULD level).Adjust wording for swapping address pools when rebooting.Add DSCP requirement (stolen from draft-jennings-behave-nat6).Add informative reference to
draft-boucadair-intarea-nat-reveal-analysis.Add requirement for hold-down pool.Change definition of CGN.Avoid usage of "device" loaded word throughout the document.Add requirement about resource exhaustion.Change title.Describe additional CGN topology where there is no NAT444.Better justification for "Paired" pool behavior.Make it clear that rate limiting allocation is for preserving CPU
resourcesGeneralize the requirement for limiting the number of TCP sessions
per mapping so that it applies to all memory-consuming state
elements.Change CPE to subscriber where it applies throughout the text.Better terminology for bulk port allocation mechanisms.Explain how external address pairing works with DS-Lite.Terminology: LSN is now CGN.Imported all requirements from RFCs 4787, 5382, and 5508. This
allowed us to eliminate some duplication.Added references to
draft-ietf-intarea-server-logging-recommendations
and draft-ford-shared-addressing-issues.Incorporated a requirement from
draft-xu-behave-stateful-nat-standby-06.