PEX 8748, PCI Express Gen 3 Switch, 48 Lanes, 12 Ports
Highlights
PEX 8748 General Features
o
48-lane, 12-port PCIe Gen 3 switch
-
Integrated 8.0 GT/s SerDes
o
27 x 27mm
2
, 676-pin FCBGA package
o
Typical Power: 8.0 Watts
The ExpressLane™ PEX 8748 device offers Multi-Host PCI Express
switching capability enabling users to connect multiple hosts to their
respective endpoints via scalable, high bandwidth, non-blocking
interconnection to a wide variety of applications including
servers,
storage, communications, and graphics platforms.
The PEX 8748 is
well suited for
fan-out, aggregation, and peer-to-peer
traffic patterns.
Multi-Host Architecture
The PEX 8748 employs an enhanced version of PLX’s field tested PEX 8648
PCIe switch architecture, which allows users to configure the device in legacy
single-host mode or multi-host mode with up to six host ports capable of 1+1
(one active & one backup) or N+1 (N active & one backup) host failover. This
powerful architectural enhancement enables users to build PCIe based systems
to support high-availability, failover, redundant, or clustered systems.
High Performance & Low Packet Latency
The PEX 8748 architecture supports packet
cut-thru with a maximum
latency of 100ns (x16 to x16).
This, combined with large packet memory,
flexible common buffer/FC credit pool and non-blocking internal switch
architecture, provides full line rate on all ports for performance-hungry
applications such as
servers
and
switch fabrics.
The low latency enables
applications to achieve high throughput and performance. In addition to low
latency, the device supports a packet payload size of up to 2048 bytes,
enabling the user to achieve even higher throughput.
Data Integrity
The PEX 8748 provides
end-to-end CRC
(ECRC) protection and
Poison bit
support to enable designs that require
end-to-end data integrity.
PLX also
supports data path parity and memory (RAM) error correction circuitry
throughout the internal data paths as packets pass through the switch.
Flexible Configuration
The PEX 8748’s 12 ports can be
configured to lane widths of x1,
x2, x4, x8, or x16. Flexible buffer
allocation, along with the device's
flexible packet flow control,
maximizes
throughput
for
applications where more traffic
flows in the downstream, rather
than upstream, direction. Any
port can be designated as the
upstream port, which can be
changed dynamically. Figure 1
shows some of the PEX 8748’s
common port configurations in
legacy Single-Host mode.
PEX 8748 Key Features
o
Standards Compliant
-
PCI Express Base Specification, r3.0
(compatible w/ PCIe r1.0a/1.1 & 2.0)
-
PCI Power Management Spec, r1.2
-
Microsoft Vista Compliant
-
Supports Access Control Services
-
Dynamic link-width control
-
Dynamic SerDes speed control
o
High Performance
♦
performancePAK
Read Pacing (bandwidth throttling)
Multicast
Dynamic Buffer/FC Credit Pool
-
Non-blocking switch fabric
-
Full line rate on all ports
-
Packet Cut-Thru with 100ns max packet
latency (x16 to x16)
-
2KB Max Payload Size
o
Flexible Configuration
-
Ports configurable as x1, x2, x4, x8, x16
-
Registers configurable with strapping
pins, EEPROM, I
2
C, or host software
-
Lane and polarity reversal
-
Compatible with PCIe 1.0a PM
o
Multi-Host & Fail-Over Support
-
Configurable Non-Transparent (NT) port
-
Failover with NT port
-
Up to 6 upstream/Host ports with 1+1 or
N+1 failover to other upstream ports
o
Quality of Service (QoS)
-
Eight traffic classes per port
-
Weighted round-robin source
port arbitration
o
Reliability, Availability, Serviceability
♦
visionPAK
Per Port Performance Monitoring
Per port payload & header counters
SerDes Eye Capture
PCIe Packet Generator
Error Injection and Loopback
-
3 Hot Plug Ports with native HP Signals
-
All ports hot plug capable thru I
2
C
(Hot Plug Controller on every port)
-
ECRC and Poison bit support
-
Data Path parity
-
Memory (RAM) Error Correction
-
INTA# and FATAL_ERR# signals
-
Advanced Error Reporting
-
Port Status bits and GPIO available
•
Per port error diagnostics
-
JTAG AC/DC boundary scan
x4
x8
PEX 8748
PEX 8748
11 x4
x8
4 x8
2 x4
x8
PEX 8748
PEX 8748
10 x4
2 x8
6x4
Figure 1. Common Port Configurations
© PLX Technology, www.plxtech.com
Page 1 of 5
10/20/2010, Version 1.0
PEX 8748, PCI Express Gen 3 Switch, 48 Lanes, 12 Ports
The PEX 8748 can also be configured in Multi-Host mode
where users can choose up to six ports as host/upstream
ports and assign a desired number of downstream ports to
each host. In Multi-Host mode, a virtual switch is created
for each host port and its associated downstream ports
inside the device. The traffic between the ports of a virtual
switch is completely isolated from the traffic in other
virtual switches. Figure 2 illustrates some configurations
of the PEX 8748 in Multi-Host mode where each ellipse
represents a virtual switch inside the device.
x8
x8
x8 x4 x4
The PEX 8748
also provides
several ways to
configure its
PEX 8748
PEX 8748
registers. The
device can be
configured
4 x4
4 x4
2 x4 3 x4 3 x4
through strapping
3 x4s
4 x4s
2
pins,
I C
interface,
host
software, or an
PEX 8748
PEX 8748
optional serial
EEPROM. This
allows for easy
9 x4s
8 x4s
debug during the
Figure 2. Multi-Host Port Configurations
development
phase, performance monitoring during the operation phase,
and driver or software upgrade.
Multi-Host & Failover Support
In Multi-Host mode, PEX 8748 can be configured with up
to six upstream host ports, each with its own dedicated
downstream ports. The device can be configured for 1+1
redundancy or N+1 redundancy. The PEX 8748 allows the
hosts to communicate their status to each other via special
door-bell registers. In failover mode, if a host fails, the
host designated for failover will disable the upstream port
attached to the failing host and program the downstream
ports of that host to its own domain. Figure 4a shows a two
host system in Multi-Host mode with two virtual switches
inside the device and Figure 4b shows Host 1 disabled
after failure and Host 2 having taken over all of Host 1’s
end-points.
Host
1
Host
2
Host
1
Host
2
PEX 8748
PEX 8748
End
Point
End
Point
End
Point
End
Point
End
Point
End
Point
End
Point
End
Point
Figure 4a. Multi-Host
Figure 4b. Multi-Host Fail-Over
Hot Plug for High Availability
Hot plug capability allows users to replace hardware
modules and perform maintenance without powering down
the system. The PEX 8748 hot plug capability feature
makes it suitable for
High Availability (HA)
applications.
Three downstream ports include a Standard
Hot Plug Controller. If the PEX 8748 is used in an
application where one or more of its downstream ports
connect to PCI Express slots, each port’s Hot Plug
Controller can be used to manage the hot-plug event of its
associated slot. Every port on the PEX 8748 is equipped
with a hot-plug control/status register to support hot-plug
capability through external logic via the I
2
C interface.
Dual-Host & Failover Support
In Single-Host mode, the PEX 8748 supports a
Non-
Transparent (NT) Port,
which enables the
implementation of
Primary Host
Secondary Host
Primary Host
Secondary Host
dual-host systems
for
CPU
CPU
redundancy and host
failover capability. The
NT port allows systems
Root
Complex
to isolate host memory
domains by presenting
NT
the processor subsystem
PEX 8748
as an endpoint rather than
Non-Transparent
Port
another memory
system. Base address
End
End
End
Point
Point
Point
registers are used to
Figure 3. Non-Transparent Port
translate addresses;
doorbell registers are used to send interrupts between the
address domains; and scratchpad registers (accessible by
both CPUs) allow inter-processor communication (see
Figure 3).
SerDes Power and Signal Management
The PEX 8748 provides low power capability that is fully
compliant with the PCIe power management specification
and supports software control of the SerDes outputs to
allow optimization of power and signal strength in a
system. Furthermore, the SerDes block supports
loop-back
modes
and
advanced reporting of error conditions,
which enables efficient management of the entire system.
Interoperability
The PEX 8748 is designed to be fully compliant with the
PCI Express Base Specification r2.0, and is backwards
compatible to PCI Express Base Specification r1.1 and
© PLX Technology, www.plxtech.com
Page 2 of 5
10/20/2010, Version 1.0
PEX 8748, PCI Express Gen 3 Switch, 48 Lanes, 12 Ports
r1.0a. Additionally, it supports
auto-negotiation, lane
reversal,
and
polarity reversal.
Furthermore, the PEX
8748 is tested for Microsoft Vista compliance. All PLX
switches undergo thorough interoperability testing in
PLX’s
Interoperability Lab
and
compliance testing at
the PCI-SIG plug-fest.
users can use to help bring their systems to market faster.
visionPAK
features consist of Performance Monitoring,
SerDes Eye Capture, Error Injection, SerDes Loopback,
and more.
Performance Monitoring
The PEX 8748’s real time performance monitoring allows
users to literally “see” ingress and egress performance on
each port as traffic passes through the switch using PLX’s
Software Development Kit (SDK). The monitoring is
completely passive and therefore has no affect on overall
system performance. Internal counters provide extensive
granularity down to traffic & packet type and even allows
for the filtering of traffic (i.e. count only Memory Writes).
performancePAK
™
Exclusive to PLX,
performancePAK
is a suite of unique
and innovative performance features which allows PLX’s
Gen 2 switches to be the highest performing Gen 2
switches in the market today. The
performancePAK
features consists of the Read Pacing, Multicast, and
Dynamic Buffer Pool.
Read Pacing
The Read Pacing feature allows users to throttle the
amount of read requests being made by downstream
devices. When a downstream device requests several long
reads back-to-back, the Root Complex gets tied up in
serving that downstream port. If that port has a narrow link
and is therefore slow in receiving these read packets from
the Root Complex, then other downstream ports may
become starved – thus, impacting performance. The Read
Pacing feature enhances performances by allowing for the
adequate servicing of all downstream devices.
SerDes Eye Capture
Users can evaluate their system’s signal integrity at the
physical layer using the PEX 8748’s SerDes Eye Capture
feature. Using PLX’s SDK, users can view the receiver
eye of any lane on the switch. Users can then modify
SerDes settings and see the impact on the receiver eye.
Figure 5 shows a screenshot of the SerDes Eye Capture
feature in the SDK.
Multicast
The Multicast feature enables the copying of data (packets)
from one ingress port to multiple (up to 11) egress ports in
one transaction allowing for higher performance in dual-
graphics, storage, security, and redundant applications,
among others. Multicast relieves the CPU from having to
conduct multiple redundant transactions, resulting in
higher system performance.
Dynamic Buffer Pool
The PEX 8748 employs a dynamic buffer pool for Flow
Control (FC) management. As opposed to a static buffer
scheme which assigns fixed, static buffers to each port,
PLX’s dynamic buffer allocation scheme utilizes a
common pool of FC Credits which are shared by other
ports. This shared buffer pool is fully programmable by the
user, so FC credits can be allocated among the ports as
needed. Not only does this prevent wasted buffers and
inappropriate buffer assignments, any unallocated buffers
remain in the common buffer pool and can then be used
for faster FC credit updates.
Figure 5. SerDes Eye Capture
PCIe Packet Generator
The PEX 8748 features a full-fledged PCIe Packet
Generator capable of creating programmable PCIe traffic
running at up to Gen 3 speeds and capable of saturating a
x16 link. Using PLX’s Software Development Kit
(www.plxtech.com/sdk), designers can create custom
traffic scripts for system bring-up and debug. Fully
integrated into the PEX 8748, the Packet Generator proves
to be a very convenient on-chip debug tool. Furthermore,
the Packet Generator can be used to create PCIe traffic to
test and debug other devices on the system.
visionPAK
™
Another PLX exclusive,
visionPAK
is a debug diagnostics
suite of integrated hardware and software instruments that
© PLX Technology, www.plxtech.com
Page 3 of 5
10/7/2010, Version 1.0
PEX 8748, PCI Express Gen 3 Switch, 48 Lanes, 12 Ports
Error Injection & SerDes Loopback
Using the PEX 8748’s Error Injection feature, users can
inject malformed packets and/or fatal errors into their
system and evaluate a system’s ability to detect and
recover from such errors. The PEX 8748 also supports
Internal Tx, External Tx, Recovered Clock, and Recovered
Data Loopback modes.
switches required for fan-out, saving precious board space
and power consumption. In Figure 7, the PEX 8748 is
being shared by four different servers (hosts) with each
server is running its own applications (I/Os). The PEX
8748 assigns the endpoints to the appropriate host and
isolates them from the other hosts.
Host Failover
Applications
Suitable for
host-centric
as well as
peer-to-peer traffic
patterns,
the PEX 8748 can be configured for a wide
variety of form factors and applications.
The PEX 8748 can also be utilized in applications where
host failover is required. In the below application (Figure
8), two hosts may be active simultaneously and controlling
their own domains while exchange status information
through doorbell registers or I
2
C interface. The devices can
be programmed to trigger fail-over if the heartbeat
information is not provided.
In the event of a failure, the
x8
x8
surviving device will reset
the endpoints connected to
x8s
PEX 8748
the failing CPU and
PEX 8748
x8s
x8s
enumerate them in its own
domain without impacting
the operation of endpoints
already in its domain.
Figure 8. Host Fail-Over
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
Host Centric Fan-out
The PEX 8748, with its symmetric or asymmetric lane
configuration capability, allows user-specific tuning to a
variety of host-centric applications. Figure 6 shows a
server design where, in a quad or multi processor system,
users can assign endpoints/slots to CPU cores to distribute
the system load. The packets directed to different CPU
cores will go to different (user assigned) PEX 8748
upstream ports, allowing better queuing and load balancing
capability for
PCH
Memory
higher
performance.
PCI
x8 x8
SATA
Conversely, the
x1s
PEX 8748 can
PEX 8748
also be used in
x8s
x4s
single-host mode
Endpoint
to simply fan-out
to endpoints.
CPU
CPU
CPU
CPU
Endpoint
Endpoint
Endpoint
Endpoint
Endpoint
PCIe Gen1, Gen2, or Gen3 slots
Figure 6. Host Centric Dual Upstream
N+1 Fail-Over in Storage Systems
The PEX 8748’s Multi-Host feature can also be used to
develop storage array clusters where each host manages a
set of storage devices independent of others (Figure 9).
Users can designate one of the hosts as the failover-host
for all the other hosts while actively managing its own
endpoints. The failover-host will communicate with other
hosts for status/heartbeat information and execute a
failover event if/when it gets triggered.
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
Multi-Host Systems
In multi-host mode, the PEX 8748 can be shared by up to
six hosts in a
system. By
creating six
Mem
Mem
Mem
Mem
virtual switches,
PCH
PCH
PCH
PCH
the PEX 8748
I/Os
I/Os
I/Os
I/Os
allows six hosts
to fan-out to
PEX 8748
their respective
endpoints. This
reduces the
I/O
I/O
I/O
I/O
number of
I/O
I/O
I/O
I/O
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
x4
x4
x8
x8
PEX 8748
x4
x4
x8
x8
PEX 8716
PEX 8716
PEX 8712
x4
x4
x4
x4
PEX 8712
x4
x4
FC
FC
x4
x4
FC
FC
FC
FC
FC
FC
Figure 7. Multi-Host System
8 Disk Chassis
8 Disk Chassis
8 Disk Chassis
8 Disk Chassis
Figure 9. N+1 Failover
© PLX Technology, www.plxtech.com
Page 4 of 5
10/7/2010, Version 1.0
Endpoint
PEX 8748, PCI Express Gen 3 Switch, 48 Lanes, 12 Ports
Software Model
From a system model viewpoint, each PCI Express port is
a virtual PCI to PCI bridge device and has its own set of
PCI Express configuration registers. It is through the
upstream port that the BIOS or host can configure the
other ports using standard PCI enumeration. The virtual
PCI to PCI bridges within the PEX 8748 are compliant to
the PCI and PCI Express system models. The
Configuration Space Registers (CSRs) in a virtual
primary/secondary PCI to PCI bridge are accessible by
type 0 configuration cycles through the virtual primary bus
interface (matching bus number, device number, and
function number).
ExpressLane PEX 8748 RDK
The PEX 8748 RDK (see Figure 10) is a hardware module
containing the PEX 8748 which plugs right into your
system. The PEX 8748 RDK can be used to test and
validate customer software, or used as an evaluation
vehicle for PEX 8748 features and benefits. The PEX 8748
RDK provides everything that a user needs to get their
hardware and software development started.
Software Development Kit (SDK)
PLX’s Software Development Kit is available for
download at
www.plxtech.com/sdk.
The software
development kit includes drivers, source code, and GUI
interfaces to aid in configuring and debugging the PEX
8748.
Both
performancePAK
and
visionPAK
are supported by
PLX’s RDK and SDK, the industry’s most advanced
hardware- and software-development kits.
Interrupt Sources/Events
The PEX 8748 switch supports the INTx interrupt message
type (compatible with PCI 2.3 Interrupt signals) or
Message Signaled Interrupts (MSI) when enabled.
Interrupts/messages are generated by PEX 8748 for hot
plug events, doorbell interrupts, baseline error reporting,
and advanced error reporting.
Product Ordering Information
Part Number
PEX8748-AA80BC G
PEX8748-AA RDK
Description
48-Lane, 12-Port PCI Express Switch,
Pb-Free (27x27mm
2
)
PEX 8748 Rapid Development Kit
PLX Technology, Inc. All rights reserved. PLX, the PLX logo, ExpressLane,
Read Pacing and Dual Cast are trademarks of PLX Technology, Inc. All other
product names that appear in this material are for identification purposes only
and are acknowledged to be trademarks or registered trademarks of their
respective companies. Information supplied by PLX is believed to be accurate
and reliable, but PLX assumes no responsibility for any errors that may appear in
this material. PLX reserves the right, without notice, to make changes in product
design or specification.
Visit
www.plxtech.com
for more information.
Figure 10. PEX 8748 RDK
Development Tools
PLX offers hardware and software tools to enable rapid
customer design activity. These tools consist of a hardware
module (PEX 8748 RDK), hardware documentation
(available at
www.plxtech.com
), and a Software
Development Kit (also available at
www.plxtech.com
).
© PLX Technology, www.plxtech.com
Page 5 of 5
10/7/2010, Version 1.0
模拟电子
京公网安备 11010802033920号
Copyright © 2005-2026 EEWORLD.com.cn, Inc. All rights reserved
电子工程世界
