Page 1: Front Cover
Front cover Draft Document for Review October 14, 2014 10:19 am REDP-5137-00 IBM Power Systems E870 and E880 Technical Overview and Introduction New modular architecture for increased reliability Enterprise POWER8 processor-based servers Exceptional memory and I/O bandwidth Alexandre Bicas Caldeira...
Page 3 5137edno.fm Draft Document for Review October 14, 2014 10:19 am International Technical Support Organization IBM Power Systems E870 and E880 Technical Overview and Introduction October 2014 REDP-5137-00...
Page 4 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. First Edition (October 2014) This edition applies to the IBM Power E870 (9119-MME) and Power E880 (9119-MHE) Power Systems servers. This document was created or updated on October 14, 2014.
Page 5: Table Of Contents
1.12.3 IBM 7953 model 94Y rack........
Page 6 2.2 The IBM POWER8 processor ........
Page 7 3.7 IBM Power Virtualization Performance ........
Page 8 4.7.1 Service user interfaces ..........171 4.7.2 IBM Power Systems Firmware maintenance ......176 4.7.3 Concurrent firmware update improvements .
Page 9: Notices
IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead.
Page 10: Trademarks
IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml...
Page 11: Preface
YoungHoon Cho is a Power Systems Top Gun with the post-sales Technical Support Team for IBM in Korea. He has over 10 years of experience working on RS/6000®, System p®, and Power Systems products. He provides second-line technical support to Field Engineers working on Power Systems and system management.
Page 12: Now You Can Become A Published Author, Too
Computer Science and Management in Bielsko-Biala. His areas of expertise include IBM i, PowerHA® solutions that are based on hardware and software replication, Power Systems hardware, and PowerVM. He is an IBM Certified Systems Expert and a coauthor of several IBM Redbooks® publications.
Page 13: Stay Connected To Ibm Redbooks
Find us on Facebook: http://www.facebook.com/IBMRedbooks Follow us on Twitter: http://twitter.com/ibmredbooks Look for us on LinkedIn: http://www.linkedin.com/groups?home=&gid=2130806 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks weekly newsletter: https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm Stay current on recent Redbooks publications with RSS Feeds: http://www.redbooks.ibm.com/rss.html Preface...
Page 14 5137pref.fm Draft Document for Review October 14, 2014 10:19 am IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 15: Chapter 1. General Description
The Power E870 is a highly-scalable rack-mount system optimized for running AIX, IBM i, and Linux workloads. The Power E870 is a modular-built system and uses one or two system nodes together with a system control unit.
Page 16: Systems Overview
Each EXP24S SFF Gen2-bay Drawer provides 24 x 2.5-inch form-factor (SFF) SAS bays. The EXP24S is connected to the Power E870 server using a SAS adapter in a PCIe slot in a system node or in a I/O expansion drawer. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 17: Power E880 Server
5137ch01.fm Draft Document for Review October 14, 2014 10:19 am Figure 1-1 shows a single system node Power E870 in a T42 rack with two PCIe I/O drawers and an EXP24S disk drawer. Figure 1-1 A Power E870 in a T42 rack 1.1.2 Power E880 server The Power E880 (9119-MHE) server is a powerful POWER8-based system that scales up to 16 sockets.
Page 18: System Control Unit
These connectors are located at the rear of the system and route up and down the middle of the system nodes. In this way they do not cause any restrictions on the allowed rack specification. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 19: System Nodes
5137ch01.fm Draft Document for Review October 14, 2014 10:19 am The SCU provides redundant service processor function to the system nodes using flexible service processor (FSP) cables. Each system node has two FSP connections to the system control unit. The SCU provides redundant connections to one or two HMCs using RJ45 Ethernet connections.
Page 20 Power sockets Power sockets Power sockets Local clock and control units FSP connector FSP connector SMP connectors x6 SCU power SCU power Figure 1-5 Rear view of the system node IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 21 5137ch01.fm Draft Document for Review October 14, 2014 10:19 am Unlike the inter-node flex connectors on the Power 770 and Power 780 servers, the SMP cables on the Power E870 and E880 servers are fully flexible. These cables do not impose restrictions on allowed rack specification.
Page 22: I/O Drawers
The system control unit requires 2U and each system node requires 5U. Thus, a single-enclosure system requires 7U, a two-enclosure system requires 12U, a three-enclosure system requires 17U, and a four-enclosure system requires 22U. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 23: System Features
5137ch01.fm Draft Document for Review October 14, 2014 10:19 am Table 1-2 Physical dimensions of the Power E870 and Power E870 Dimension System control unit Power E870 or PCIe I/O Expansion Power E880 drawer enclosure Width 437 mm (17.2 in) 445 mm (17.5 in) 434 mm (17.1 in) Depth...
Page 24 – Zero or two PCIe Gen3 Drawers per system node drawer (#EMX0) – Each Gen3 I/O drawer holds two 6-slot PCIe3 Fan-out Modules (#EMXF) – Each Gen3 I/O drawer attaches to the system node via two PCIe3 Optical Cable Adapters (#EJ07) IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 25: Power E880 System Features
– Active Memory Deduplication – NPIV support – PowerVP Performance Monitor Optional PowerHA for AIX, IBM i, and Linux Optional PCIe Gen3 I/O Expansion Drawer with PCIe Gen3 slots: – Zero or two PCIe Gen3 Drawers per system node drawer (#EMX0) –...
Page 26: Minimum Features
– SAN Load Source Specify (#0837) 1 x Rack-mount Drawer Bezel and Hardware – IBM Rack-mount Drawer Bezel and Hardware (#EBA2) for Power E870 – IBM Rack-mount Drawer Bezel and Hardware (#EBA3) for Power E880 – OEM Rack-mount Drawer Bezel and Hardware (#EBA4)
Page 27 5137ch01.fm Draft Document for Review October 14, 2014 10:19 am When IBM i is the primary operating system, the order must include a minimum of the following items: 1 x Specify Code – Mirrored System Disk Level, Specify Code (#0040) –...
Page 28: Power Supply Features
The #EPJ3 provide no-charge elastic processor days for Power E880. 1.4.6 Summary of processor features Table 1-3 on page 15 summarizes the processor feature codes for the Power E870. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 29 1 Proc-Day Elastic CoD Billing for #EPBA, IBM i EPJ8 100 Elastic CoD Proc-Days of Billing for Processor #EPBA. AIX/Linux EPJ9 100 Elastic CoD Proc-Days of Billing for Processor #EPBA. IBM i EPJA Proc CoD Utility Billing, 100 Proc-mins. for #EPBA, AIX/Linux EPJB Proc CoD Utility Billing, 100 Proc-mins.
Page 30: Memory Features
1 Proc-Day Elastic CoD Billing for #EPBB, IBM i EPJE 100 Elastic CoD Proc-Days of Billing for Processor #EPBB. AIX/Linux EPJF 100 Elastic CoD Proc-Days of Billing for Processor #EPBB. IBM i EPJG Proc CoD Utility Billing, 100 Proc-mins. for #EPBB, AIX/Linux EPJH Proc CoD Utility Billing, 100 Proc-mins.
Page 31: System Node Pcie Slots
For IBM i, a DVD drive must be available on the server when required. The DVD can be in the system control unit or it can be located in an external enclosure such as a 7226-1U3 Multimedia drawer.
Page 32: Pcie Gen3 I/O Expansion Drawer
In 2014, a system node supports either zero or two PCIe Gen3 I/O expansion drawers. Connecting a single drawer to a system node is not supported. Figure 1-9 shows a PCIe Gen3 I/O expansion drawer. Figure 1-9 PCIe Gen3 I/O expansion drawer IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 33: I/O Drawers And Usable Pci Slot
5137ch01.fm Draft Document for Review October 14, 2014 10:19 am 1.6.2 I/O drawers and usable PCI slot Figure 1-10 shows the rear view of the PCIe Gen3 I/O expansion drawer with the location codes for the PCIe adapter slots in the PCIe3 6-slot fanout module. P2-C3 P1-C6 P1-C1...
Page 34: Exp24S Sff Gen2-Bay Drawer
With AIX, Linux, and VIOS, you can order the EXP24S with four sets of six bays, two sets of 12 bays, or one set of 24 bays (mode 4, 2, or 1). With IBM i, you can order the EXP24S as one set of 24 bays (mode 1).
Page 35: Comparison Between Models
Support for concurrent maintenance on the Power E880 I/O Expansion Drawer by enabling hot add and repair capabilities IBM's statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM's sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
Page 36: Build To Order
This method is the only configuration method for the Power E870 and Power E880 servers. 1.9 IBM editions IBM edition offerings are not available for the Power E870 and Power E880 servers. 1.10 Model upgrades The following sections describe the various upgrades that are available.
Page 37: Upgrade Considerations
MES order is started. Any temporary use of processors or memory owed to IBM on the existing system must be paid before installing the Power E870 or E880.
Page 38: System Racks
Shipping without a rack: If you require the system to be shipped without an IBM rack, feature code #ER21 must be used to remove the IBM rack from the order. The server will then ship as separate packages for installation into an existing rack.
Page 39: Ibm 7014 Model T00 Rack
5137ch01.fm Draft Document for Review October 14, 2014 10:19 am Installing in non-IBM racks: The client is responsible for ensuring that the installation of the drawer in the preferred rack or cabinet results in a configuration that is stable, serviceable, safe, and compatible with the drawer requirements for power, cooling, cable management, weight, and rail security.
Page 40: Ibm 7953 Model 94Y Rack
Both provide 42U of vertical space, are 1100 mm deep, and have an interior rail-to-rail depth of 715 mm. However, the IBM 42U Slim Rack is 600 mm wide; the T42 is 645 mm wide with side covers. For clients with 2-foot floor tiles, the extra 45 mm (1.77-inch) width of the enterprise rack can sometimes cause challenges when cutting holes in the floor tiles for cabling.
Page 41 Mounting a horizontal PDU in the middle of the rack is generally not optimal for cable management. For the Power E870 or E880 installed in IBM 7014 or FC 055x racks, the following PDU rules apply:...
Page 42 Figure 1-13 Two system nodes configuration with I/O expansions and disk drawers supplied by 60A PDUs For detailed power cord requirements and power cord feature codes, see the IBM Power Systems Hardware Information Center website: << add an infocenter link once is available>>...
Page 43 5137ch01.fm Draft Document for Review October 14, 2014 10:19 am power cords can be requested with an initial order of the rack or with a later upgrade of the rack features. The PDUs have 12 client-usable IEC 320-C13 outlets. There are six groups of two outlets fed by six circuit breakers.
Page 44 5137ch01.fm Draft Document for Review October 14, 2014 10:19 am IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 45: Chapter 2. Architecture And Technical Overview
Chapter 2. overview This chapter describes the overall system architecture for the IBM Power System E870 (9119-MME) and IBM Power System E880 (9119-MHE) servers. The bandwidths that are provided throughout the section are theoretical maximums that are used for reference.
Page 46: Logical Diagrams
Clock To SCU To SCU Clock Interface Interface Clock SCU Interface SCU Interface Figure 2-1 Logical system diagram for a system node of a Power E870 or a Power E880 IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 47 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Figure 2-2 on page 33 shows the logical system diagram for the system control unit of a Power E870 or a Power E880. Power in Power in Power in Power in cable Operator Panel PCIe card...
Page 48 A0 A1 A2 X0 X1 X3 X0 X1 X3 X0 X1 X3 X0 X1 X3 Drawer 1 Figure 2-3 SMP connection topology for a two-drawer Power E870 or Power E880 IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 49 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Figure 2-4 shows the SMP connection topology for a three-drawer Power E880. Drawer 1 X3 X1 X0 X3 X1 X0 X3 X1 X0 X3 X1 X0 A2 A1 A0 A2 A1 A0 A2 A1 A0 A2 A1 A0 A0 A1 A2...
Page 50: The Ibm Power8 Processor
SMP connection topology for a four-drawer Power E880 Figure 2-5 2.2 The IBM POWER8 processor This section introduces the latest processor in the IBM Power Systems product family, and describes its main characteristics and features. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 51: Power8 Processor Overview
Draft Document for Review October 14, 2014 10:19 am 2.2.1 POWER8 processor overview The POWER8 processor is manufactured using the IBM 22 nm Silicon-On-Insulator (SOI) technology. Each chip is 649 mm and contains 4.2 billion transistors. As shown in Figure 2-6, the chip contains 12 cores, two memory controllers, PCIe Gen3 I/O controllers, and an interconnection system that connects all components within the chip.
Page 52 Figure 2-7 shows a graphic representation of the 6-core processor. A 6-core processor is only available on the scale-out systems. It is shown here for informational purposes. Figure 2-7 6-core POWER8 processor chip IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 53: Power8 Processor Core
With prior generations of POWER® processor 2.2.2 POWER8 processor core The POWER8 processor core is a 64-bit implementation of the IBM Power Instruction Set Architecture (ISA) Version 2.07 and has the following features: Multi-threaded design, which is capable of up to eight-way simultaneous multithreading...
Page 54: Simultaneous Multithreading
Table 2-2 on page 41 shows a comparison between the different POWER processors in terms of SMT capabilities that are supported by each processor architecture. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 55: Memory Access
Draft Document for Review October 14, 2014 10:19 am Table 2-2 SMT levels that are supported by POWER processors Technology Cores/system Maximum SMT mode Maximum hardware threads per system IBM POWER4 Single Thread (ST) IBM POWER5 SMT2 IBM POWER6 SMT2 IBM POWER7...
Page 56: On-Chip L3 Cache Innovation And Intelligent Cache
A six-to-one latency improvement occurs by moving the L3 cache on-chip compared to L3 accesses on an external (on-ceramic) ASIC. Bandwidth improvement A 2x bandwidth improvement occurs with on-chip interconnect. Frequency and bus sizes are increased to and from each core. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 57: Level 4 Cache And Memory Buffer
The performance of eDRAM when implemented on-chip is similar to conventional SRAM but requires far less physical space. IBM on-chip eDRAM uses only a third of the components that conventional SRAM uses, which has a minimum of six transistors to implement a 1-bit memory cell.
Page 58: Hardware Transactional Memory
FPGA, and using the same effective addresses (pointers) it uses for any of its threads running on a host IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 59 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am processor. From the practical perspective, CAPI allows a specialized hardware accelerator to be seen as an additional processor in the system, with access to the main system memory, and coherent communication with other processors in the system. The benefits of using CAPI include the ability to access shared memory blocks directly from the accelerator, perform memory transfers directly between the accelerator and processor cache, and reduction in the code path length between the adapter and the processors.
Page 60: Power Management And System Performance
Modifying these parameters should be done only by advanced users. If you must address any issues concerning the Tunable Parameters, IBM support personal should be directly involved in the parameter value selection.
Page 61: Comparison Of The Power7, Power7+, And Power8 Processors
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am 2.2.10 Comparison of the POWER7, POWER7+, and POWER8 processors Table 2-4 shows comparable characteristics between the generations of POWER7, POWER7+, and POWER8 processors. Table 2-4 Comparison of technologies for the POWER8 processor and the prior generations Characteristics POWER7 POWER7+...
Page 62: Custom Dimm
CDIMM and allows for proper airflow and ease of handling. Tall CDIMMs slots allow for larger DIMM sizes and the adoption of future technologies more seamlessly. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 63: Memory Placement Rules
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am A detailed diagram of the CDIMMs available for the Power E870 and Power E880 can be seen in Figure 2-12. Figure 2-12 Short CDIMM and Tall CDIMM details The Memory Buffer is a L4 cache and is built on eDRAM technology (same as the L3 cache), which has a lower latency than regular SRAM.
Page 64 You should account for any plans for future memory upgrades when you decide which memory feature size to use at the time of the initial system order. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 65 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am A physical diagram with the location codes of the memory CDIMMs of a system node, as well as their grouping as memory quads can be seen on Figure 2-14 on page 51. Location (Memory Quad) P1-C21 (Quad 4) P1-C22 (Quad 4)
Page 66 Quad 3 Quad 7 Quad 4 Quad 8 Notes: Memory quads 1-8 must be populated. Memory quads on the same processor must be populated with CDIMMs of the same capacity. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 67 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Table 2-7 shows the CDIMM plugging order for a Power E880 with a three system nodes: Table 2-7 Optimal CDIMM memory quad placement for three system nodes System Node 1 Processor 0 Processor 2 Processor 3...
Page 68: Memory Activation
Both types of memory activations can co-reside in the same system, as long as at least 25% of the memory activations are static. This leads to a maximum of 75% of the memory activations as mobile. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 69: Memory Throughput
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am On Figure 2-15 on page 55 there is an example of the minimum required activations for a system with 1 TB of installed memory. Installed memory 1024 GB Without Power Enterprise Pools Minimum active 512 GB 50% of total memory...
Page 70 32 lanes * 4 processors * 7.877 Gbps * 2 = 252.064 GBps Rounding: The bandwidths listed here may appear slightly differently in other materials due to rounding of some figures. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 71 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am For the entire Power E870 system populated with two system nodes, the overall bandwidths are what is shown in Table 2-12 on page 57. Table 2-12 Power E870 total bandwidth estimates Total bandwidths Power E870 Power E870...
Page 72: Active Memory Mirroring
CDIMMs. If a CDIMM that contains the hypervisor code develops an uncorrectable error, its mirrored partner will enable the system to continue to operate uninterrupted. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 73 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Active Memory Mirroring (AMM) is included with all Power E870 and Power E880 systems at no additional charge. It can be enabled, disabled, or re-enabled depending on the user’s requirements. The hypervisor code logical memory blocks will be mirrored on distinct CDIMMs to allow for more usable memory.
Page 74: Memory Error Correction And Recovery
Inactive processor cores and memory IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 75 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am units can be permanently activated by purchasing an activation feature called Capacity Upgrade on Demand (CUoD) and entering the provided activation code on your server. With the CUoD offering, you can purchase additional static processor or memory capacity and dynamically activate them when needed, without requiring you to restart your server or interrupt your business.
Page 76: Power Enterprise Pools And Mobile Capacity On Demand (Mobile Cuod)
IBM power enterprise pools, is a technology for dynamically sharing processor and memory activations among a group (or pool) of IBM Power Systems servers. Using Mobile Capacity on Demand (CoD) activation codes, the systems administrator can perform tasks without contacting IBM.
Page 77: Elastic Capacity On Demand (Elastic Cod)
In order for the Mobile activation features to be configured it is necessary that a a power enterprise pool is registered with IBM as well as the systems which are going to be included as members of the pool. Also it is necessary that the systems have the feature code #EB35 for mobile enablement configured and the required contracts must be in place.
Page 78 A monitor that resides on the server records the usage activity. This usage data must be sent to IBM on a monthly basis. A bill is then generated based on the total amount of processor and memory resources utilized, in increments of Processor and Memory (1 GB) Days.
Page 79: Utility Capacity On Demand (Utility Cod)
Utility CoD requires PowerVM Standard Edition or PowerVM Enterprise Edition to be active. If a Power E870 or Power E880 server uses the IBM i operating system in addition to any other supported operating system on the same server, the client must inform IBM which operating system caused the temporary Utility CoD processor usage so that the correct feature can be used for billing.
Page 80: Software Licensing And Cod
Power E870 and Power E880. The system nodes allow for eight PCIe Gen3 x16 slots. Additional slots can be added by attaching PCIe Expansion Drawers and SAS disks can be IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 81: Service Processor Bus
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am attached to EXP24S SFF Gen2 Drawers. The EXP24S can be either attached to SAS adapters on the system nodes or on the PCIe Expansion Drawer. For a list of adapters and their supported slots, please see 2.7, “PCI adapters” on page 70. PCIe Expansion Drawer PCIe Gen3 x16 PCIe Adapter for...
Page 82: Internal I/O Subsystem
Slot 3 P1-C3 PCIe Gen3 x16 Slot 4 P1-C4 PCIe Gen3 x16 Slot 5 P1-C5 PCIe Gen3 x16 Slot 6 P1-C6 PCIe Gen3 x16 Slot 7 P1-C7 PCIe Gen3 x16 IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 83: Blind-Swap Cassettes
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Slot Location code Slot Type CAPI capable SRIOV capable Slot 8 P1-C8 PCIe Gen3 x16 a. At the time of writing there are no supported CAPI adapters for the Power E870 and E880 system unit.
Page 84: System Ports
The connection and usage of the DVD can be seen on detail on 2.8.1, “DVD” on page 76. 2.7 PCI adapters This section covers the types and functions of the PCI cards supported by IBM Power E870 and IBM Power E880 systems.
Page 85: Lan Adapters
To do this, use the IBM prerequisite website: https://www-912.ibm.com/e_dir/eServerPreReq.nsf The following sections describe the supported adapters and provide tables of orderable feature numbers. The tables indicate operating system support (AIX, IBM i, and Linux) for each of the adapters. 2.7.2 LAN Adapters To connect the Power E870 and Power E880 servers to a local area network (LAN), you can use the LAN adapters that are supported in the PCIe slots of the system.
Page 86: Graphics Accelerator Adapters
PCIe2 1.8GB Cache RAID SAS Adapter I/O drawer AIX, IBM i, Tri-port 6Gb CR Linux EJ0J 57B4 PCIe3 RAID SAS Adapter Quad-port 6Gb x8 I/O drawer AIX, IBM i, Linux IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 87: Fibre Channel Adapter
Featur CCIN Description Placement OS support e Code EJ0L 57CE PCIe3 12GB Cache RAID SAS Adapter I/O drawer AIX, IBM i, Quad-port 6Gb x8 Linux EJ0M 57B4 PCIe3 LP RAID SAS ADAPTER AIX, IBM i, Linux EJ10 57B4 PCIe3 SAS Tape/DVD Adapter Quad-port 6Gb...
Page 88: Fibre Channel Over Ethernet
Each system control unit enclosure can have one slim-line bay which can support one DVD drive (#EU13). The #EU13 DVD is cabled to a USB PCIe adapter located in either the system node or in a PCIe Gen3 I/O drawer. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 89: Infiniband Host Channel Adapter
The Cryptographic Coprocessor cards provide both cryptographic coprocessor and cryptographic accelerator functions in a single card. The IBM PCIe Cryptographic Coprocessor adapter highlights the following features: Integrated Dual processors that operate in parallel for higher reliability Supports IBM Common Cryptographic Architecture or PKCS#11 standard...
Page 90: Internal Storage
USB DVD port. This architecture allows for a more flexible infrastructure where the DVD drive is completely independent of another components and can be freely moved between partition. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 91: External I/O Subsystems
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am It is also important to note that several daily functions where a DVD drive was needed can now be executed by using other methods like Virtual Media Repository on the Virtual I/O Server or the Remote Virtual I/O Server install on HMC.
Page 92: Pcie Gen3 I/O Expansion Drawer Optical Cabling
The top port of the fan out module must be cabled to the top port of the #EJ07 port. Likewise, the bottom two ports must be cabled together. Connect an active optical cable to connector T1 on the PCIe3 optical cable adapter in your server. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 93 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Connect the other end of the optical cable to connector T1 on one of the PCIe3 6-slot fanout modules in your expansion drawer. Connect another cable to connector T2 on the PCIe3 optical cable adapter in your server. Connect the other end of the cable to connector T2 on the PCIe3 6-slot fanout module in your expansion drawer.
Page 94 Table 2-30 shows PCIe adapter slot priorities in the Power E870 and Power E880 system. Table 2-30 PCIe adapter slot priorities Feature code Description Slot priorities EJ07 PCIe3 Optical Cable Adapter for PCIe3 Expansion Drawer 2, 8, 4, 6, 1, 7, 3, 5 IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 95 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Figure 2-27 shows example of a single system node and two PCI Gen3 I/O expansion drawers. Slot 1 Left FanOut Slot 2 Slot 3 I/O Drawer Slot 4 System node Slot 5 Right FanOut Slot 6...
Page 96 I/O Drawer Slot 2 Slot 3 Right FanOut Slot 4 System node Slot 5 Slot 6 Slot 7 Slot 8 Figure 2-28 Example of two system nodes and two I/O drawers IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 97: Pcie Gen3 I/O Expansion Drawer Spcn Cabling
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Figure 2-29 shows an example of two system node and four PCI Gen3 I/O expansion drawers. Slot 1 Left FanOut Slot 2 Slot 3 I/O Drawer Slot 4 System node Slot 5 Right FanOut Slot 6...
Page 98: External Disk Subsystems
To further reduce possible single points of failure, EXP24S configuration rules consistent with previous Power Systems are used. IBM i configurations require the drives to be protected (RAID or mirroring). Protecting the drives is highly recommended, but not required for other operating systems.
Page 99 With AIX, Linux, and VIOS, you can order the EXP24S with four sets of six bays, two sets of 12 bays, or one set of 24 bays (mode 4, 2, or 1). With IBM i, you can order the EXP24S as one set of 24 bays (mode 1).
Page 100 Draft Document for Review October 14, 2014 10:19 am Figure 2-31 on page 86 shows the mode stickers. Figure 2-31 Mode sticker locations at the rear of the 5887 disk drive enclosure IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 101 CCIN Description OS support Code 1738 19B4 856 GB 10 K RPM SAS SFF-2 Disk Drive (IBM i) IBM i 1752 900 GB 10 K RPM SAS SFF-2 Disk Drive (AIX/Linux) AIX, Linux 1917 146 GB 15 K RPM SAS SFF-2 Disk Drive (AIX/Linux)
Page 102 Feature CCIN Description OS support Code 1947 19B0 139 GB 15 K RPM SAS SFF-2 Disk Drive (IBM i) IBM i 1948 19B1 283 GB 15 K RPM SAS SFF-2 Disk Drive (IBM i) IBM i 1953 300 GB 15 K RPM SAS SFF-2 Disk Drive (AIX/Linux)
Page 103: Exp24S Common Usage Scenarios
EXP24S with Power Systems. Not all possible scenarios are included. For more information about SAS cabling and cabling configurations, see “Planning for serial-attached SCSI cables” in the IBM Knowledge Center: (point to correct link when infocenter is active) http://www-01.ibm.com/support/knowledgecenter/POWER8/p8hdx/POWER8welcome.htm...
Page 104 For this scenario, the required feature codes are: One EXP24S drawer #5887 with indicator feature #9359 (mode 1 with single #5901) One PCIe SAS adapter #5901 One SAS YO cable 3 Gbps with proper length. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 105 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Scenario 2 - Basic redundant connection This scenario assumes a single Virtual I/O Server with two PCIe SAS adapter #5901 and a EXP24S set on mode 1, allowing for up to 24 disks to be attached to the server. Figure 2-35 shows the connection diagram and components of the solution.
Page 106 Two SAS X cables 3 Gbps with proper length. The ports used on the SAS adapters must be the same for both adapters or the pair. There is no SSD support on this scenario. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 107 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am Scenario 4 - Dual Virtual I/O Servers sharing two EXP24S This scenario assumes a dual Virtual I/O Server with two PCIe SAS adapter #5901 each and two EXP24S set on mode 2, allowing for up to 24 disks to be attached to the each Virtual I/O Server (2 per drawer).
Page 108 Other Scenarios For direct connection to logical partitions, different adapters and cables, see “5887 disk drive enclosure” in the IBM Knowledge Center: (point to correct link when infocenter is active) http://www-01.ibm.com/support/knowledgecenter/POWER8/p8hdx/POWER8welcome.htm IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 109: Ibm System Storage
IBM Network-Attached Storage (NAS) products provide a wide range of network attachment capabilities to a broad range of host and client systems, such as IBM Scale Out Network Attached Storage and the IBM System Storage Nxxx series. For more information about the hardware and software, see the following website: http://www.ibm.com/systems/storage/network...
Page 110: Hardware Management Console
I/O. In this case, all servers have at least one logical partition. Hardware support for customer-replaceable units comes standard along with the HMC. In addition, users can upgrade this support level to IBM onsite support to be consistent with other Power Systems servers. 2.11.1 HMC code level HMC V8R8.2.0 contains the following new features:...
Page 111: Hmc Raid 1 Support
FLRT identifies the correct HMC machine code for the selected system firmware level. Note: Access to firmware and machine code updates is conditional on entitlement and license validation in accordance with IBM policy and practice. IBM may verify entitlement through customer number, serial number electronic restrictions, or any other means or methods that are employed by IBM at its discretion.
Page 112: Hmc Connectivity To The Power8 Processor-Based Systems
Figure 2-39 shows a simple network configuration to enable the connection from the HMC to the server and to allow for dynamic LPAR operations. For more information about HMC and the possible network connections, see IBM Power Systems HMC Implementation and Usage Guide, SG24-7491.
Page 113: High Availability Hmc Configuration
5137ch02.fm Draft Document for Review October 14, 2014 10:19 am If the service processor of the managed server does not receive a DHCP reply before timeout, predefined IP addresses are set up on both ports. Static IP address allocation is also an option and can be configured using the ASMI menus.
Page 114: Operating System Support
In addition, the Virtual I/O Server can be installed in special partitions that provide support to the other operating systems for using features such as virtualized I/O devices, PowerVM Live Partition Mobility, or PowerVM Active Memory Sharing. For details about the software available on IBM Power Systems, visit the IBM Power Systems Software™ website: http://www.ibm.com/systems/power/software/index.html...
Page 115: Virtual I/O Server
The minimum required level of Virtual I/O Server for both the Power E870 and Power E880 is VIOS 2.2.3.4 with ifix IV63331 or later. IBM regularly updates the Virtual I/O Server code. To find information about the latest updates, visit the Fix Central website: http://www-933.ibm.com/support/fixcentral/...
Page 116: Ibm I Operating System
AIX Version 7.1 with the 7100-03 Technology Level and Service Pack 4, and APAR IV63332 or later 2.12.3 IBM i operating system The IBM i operating system is supported on the Power E870 and Power E880 with the following minimum required levels: IBM i 7.1 TR9 or later IBM i 7.2 TR1 or later...
Page 117: Java Versions That Are Supported
2.12.6 Boosting performance and productivity with IBM compilers IBM XL C, XL C/C++ and XL Fortran compilers for AIX and for Linux exploit the latest POWER8 processor architecture. Release after release, these compilers continue to deliver application performance improvements and additional capability, exploiting architectural enhancements made available through the advancement of the POWER technology.
Page 118 Eclipse workbench. This solution can greatly improve developers' productivity as compared to using text-based and command-line development tools, and is more appealing than such tools when you need to recruit new IBM i development talent.
Page 119: Energy Management
IBM Power Systems power and cooling usage. EnergyScale uses power and thermal information that is collected from the system to implement policies that can lead to better performance or better energy usage. IBM EnergyScale has the following features: Power trending EnergyScale provides continuous collection of real-time server energy consumption.
Page 120 Therefore, the IBM POWER Hypervisor™ can use nap mode as a general-purpose idle state. When the operating system detects that a processor thread is idle, it yields control of a hardware thread to the POWER Hypervisor.
Page 121 (nap or sleep) longer. EnergyScale for I/O IBM POWER8 processor-based systems automatically power off hot-pluggable PCI adapter slots that are empty or not being used. System firmware automatically scans all pluggable PCI slots at regular intervals, looking for those that meet the criteria for being not in use and powering them off.
Page 122: On Chip Controller
To maintain the power dissipation of POWER7+ with its large increase in performance and bandwidth, POWER8 invested significantly in power management innovations. A new On Chip Controller (OCC) using an embedded IBM PowerPC core with 512 KB of SRAM runs real-time control firmware to respond to workload variations by adjusting the per-core frequency and voltage based on activity, thermal, voltage, and current sensors.
Page 123: Energy Consumption Estimation
An estimation of the energy consumption for a certain configuration The calculation of the energy consumption for a certain configuration can be done in the IBM Systems Energy Estimator, found at the following website: http://www-912.ibm.com/see/EnergyEstimator/ In that tool, select the type and model for the system, and enter some details about the configuration and wanted CPU usage.
Page 124 5137ch02.fm Draft Document for Review October 14, 2014 10:19 am IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 125: Chapter 3. Virtualization
PowerVM license: Enterprise edition is default on Power E870 and Power E880. IBM plans to add NIC SR-IOV capability to POWER8 servers using selected SR-IOV capable PCIe adapters. The adapters planned to be added are 4-port Ethernet Adapters with copper twinax ports (#EN0K, #EN0L, #EL3C), SR optical ports (#EN0H, #EN0J, #EL38), and LR optical ports (#EN0N, #EN0M).
Page 126: Ibm Power Hypervisor
Draft Document for Review October 14, 2014 10:19 am 3.1 IBM POWER Hypervisor Combined with features in the POWER8 processors, the IBM POWER Hypervisor delivers functions that enable other system technologies, including logical partitioning technology, virtualized processors, IEEE VLAN-compatible virtual switch, virtual SCSI adapters, virtual Fibre Channel adapters, and virtual consoles.
Page 127: Virtual Scsi
The storage virtualization is accomplished by using two paired adapters: A virtual SCSI server adapter A virtual SCSI client adapter A Virtual I/O Server (VIOS) partition or an IBM i partition can define virtual SCSI server client adapters. Other partitions are partitions.
Page 128: Virtual Fibre Channel
Depending on the system configuration, the operating system console can be provided by the Hardware Management Console (HMC) virtual TTY, IVM virtual TTY, or from a terminal emulator that is connected to a system port. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 129: Power Processor Modes
5137ch03.fm Draft Document for Review October 14, 2014 10:19 am 3.2 POWER processor modes Although they are not virtualization features, the POWER processor modes are described here because they affect various virtualization features. On Power System servers, partitions can be configured to run in several modes, including the following modes: POWER6 compatibility mode This execution mode is compatible with Version 2.05 of the Power Instruction Set...
Page 130 Affinity off by default 3-tier memory, HW memory Improved system micropartition affinity, affinity tracking performance for and dynamic platform assists system images optimizer Micropartition spanning sockets and prefetch nodes Concurrent LPARs per core IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 131: Active Memory Expansion
Similarly, it can allow a server to run more partitions and do more work for the same physical amount of memory. Note: The Active Memory Expansion feature is not supported by IBM i and the Linux operating system.
Page 132: Powervm
Version 5.1, Red Hat Enterprise Linux 3.0, and SUSE Linux Enterprise Server 9.0 operating systems. This technology was able to divide an IBM eServer™ pSeries (now IBM System p) system into separate logical systems, allowing each LPAR to run an operating environment on dedicated attached devices, such as processors, memory, and I/O components.
Page 133 5137ch03.fm Draft Document for Review October 14, 2014 10:19 am Micro-Partitioning When you use the Micro-Partitioning technology, you can allocate fractions of processors to a logical partition. This technology was introduced with POWER5 processor-based systems. A shared processor partition logical partition using fractions of processors is also known as a micropartition shared processor .
Page 134 The total sum of CPU entitlement of all partitions cannot exceed the number of available physical processors in a shared processor pool. The number of virtual processors can be changed dynamically through a dynamic LPAR operation. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 135 It can be turned off and on dynamically while the operating system is running (for AIX, run smtctl, for Linux, run ppc64_cpu --smt, and for IBM i, use QPRCMLTTSK system value). If simultaneous multithreading is off, each physical core is presented as one logical processor in AIX or Linux, and thus only one thread.
Page 136: Multiple Shared Processor Pools
EC 0.8 EC 0.5 vp10 POWER Hypervisor KEY: Entitled Capacity Physical processor Virtual processor Physical Shared-Processor Pool SPPn Shared-Processor Pool Figure 3-4 Overview of the architecture of multiple shared processor pools IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 137: Virtual I/O Server
5137ch03.fm Draft Document for Review October 14, 2014 10:19 am Micropartitions are created and then identified as members of either the default shared processor pool or a user-defined shared processor pool . The virtual processors that exist within the set of micropartitions are monitored by the POWER Hypervisor, and processor capacity is managed according to user-defined attributes.
Page 138 Ethernet networks (internal) and one or more (Etherchannel) physical network adapters (external). These virtual Ethernet network adapters are defined by the POWER Hypervisor on the VIOS. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 139 5137ch03.fm Draft Document for Review October 14, 2014 10:19 am Figure 3-6 shows a configuration example of an SEA with one physical and two virtual Ethernet adapters. An SEA can include up to 16 virtual Ethernet adapters on the VIOS that shares the physical access.
Page 140 Virtual I/O Server Expansion Pack with additional security functions, such as Kerberos (Network Authentication Service for users and client and server applications), Simple Network Management Protocol (SNMP) v3, and Lightweight Directory Access Protocol (LDAP) client function. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 141: Powervm Live Partition Mobility
For more information about the System Planning Tool, see 3.5, “System Planning Tool” on page 139. IBM Systems Director agent and several preinstalled IBM Tivoli® agents, such as the following examples: – IBM Tivoli Identity Manager, which allows easy integration into an existing Tivoli Systems Management infrastructure –...
Page 142: Active Memory Sharing
Active Memory Sharing is available only with the Enterprise version of PowerVM. The physical memory of an IBM Power System can be assigned to multiple partitions in either dedicated or shared mode. The system administrator can assign some physical memory to a partition and some physical memory to a pool that is shared by other partitions.
Page 143: Active Memory Deduplication
5137ch03.fm Draft Document for Review October 14, 2014 10:19 am For more information regarding Active Memory Sharing, see IBM PowerVM Virtualization Active Memory Sharing, REDP-4470. 3.4.7 Active Memory Deduplication In a virtualized environment, the systems might have a considerable amount of duplicated information that is stored on RAM after each partition has its own operating system, and some of them might even share the same kinds of applications.
Page 144: Linux Support
At the time of writing, Active Memory Sharing when used with Live Partition Mobility is not supported. Check with your IBM SSR for availability on POWER8 platforms. c. At the time of writing, Live Partition Mobility for IBM i is not supported. Check with your IBM SSR for availability on POWER8 platforms.
Page 145: Powervm Simplification
5137ch03.fm Draft Document for Review October 14, 2014 10:19 am For more information about specific Linux distributions, see the following website: http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/index.jsp?topic=%2Fliaam%2Fliaamd istros.htm 3.4.10 PowerVM simplification The HMC firmware for POWER8 Enterprise Systems must run, at a minimum, Version 8 Release 8 (HMC V8R8.2.0).
Page 146 The Partition Template contains configuration details for the partition resources. A user can configure logical partitions with the predefined templates or by creating a custom templates. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 147 5137ch03.fm Draft Document for Review October 14, 2014 10:19 am Using these options, a user can deploy a system, select a System Template, and click Deploy. After deploying a system, a user can choose a Partition Template from the library. Performance The Performance function opens the Performance and Capacity Monitoring window, as shown at Figure 3-8 and Figure 3-9 on page 134.
Page 148 Figure 3-9 shows the bottom of the window where performance and capacity data are presented in a graphical format. Figure 3-9 HMC performance monitoring CPU Memory assignment (bottom of the window) IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 149 5137ch03.fm Draft Document for Review October 14, 2014 10:19 am New GUI functions As part of the enhancements introduced in this version, this new capability aids in the management of PowerVM from a single UI. Figure 3-10 shows all virtualization components are now displayed in a single UI.
Page 150: Single View Of All Virtual I/O Servers
VIOS on the managed system. Also it lets you add a VIOS to a shared storage pool cluster and manage all the shared storage pool clusters. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 151: Virtual Network Management
HMC V8.8.1.0.1 helps to manage PowerVM virtual networks through an User Interface (UI). This UI uses a defined set of concepts about networking technologies with specific terminology introduced by IBM Power Architecture®. As part of Virtual Network Management new functions are added to perform actions on the...
Page 152 Ethernet adapters to be aggregated. For more information regarding PowerVM Simplification Enhancements, see IBM Power Systems Hardware Management Console Version 8 Release 8.1.0 Enhancements, SG24-8232. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 153: System Planning Tool
Draft Document for Review October 14, 2014 10:19 am 3.5 System Planning Tool The IBM System Planning Tool (SPT) helps you design systems to be partitioned with logical partitions. You can also plan for and design non-partitioned systems by using the SPT. The...
Page 154: Ibm Power Virtualization Performance
IBM PowerVC is available in two editions: IBM Power Virtualization Center Express Edition IBM Power Virtualization Center Standard Edition Table 3-6 shows an overview of the key features that are included with IBM PowerVC Editions. Table 3-6 IBM PowerVC Editions key features overview.
Page 155: Vios 2.2.3.4 Features
Background data collection, which enables performance data to be collected when the GUI is not active. IBM PowerVP even allows an administrator to drill down and view specific adapter, bus, or CPU usage. An administrator can see the hardware adapters and how much workload is placed on them.
Page 156: Dynamic Partition Remote Restart
(#EN0K, #EN0L, #EL3C), SR optical ports (#EN0H, #EN0J, #EL38), and LR optical ports (#EN0N, #EN0M).Virtual I/O slot limit increase. With VIOS version 2.2.3.4, or later, the maximum number of virtual I/O slots that are supported on the AIX, IBM i, and Linux partition is increased up to 32 K.
Page 157 5137ch03.fm Draft Document for Review October 14, 2014 10:19 am To verify that your managed system can support this capability enter the following command as shown in Example 3-1. Example 3-1 Power VM remote restart capable hscroot@slcb27a:~>lssyscfg -r sys -m Server1 -F capabilities "active_lpar_mobility_capable,inactive_lpar_mobility_capable,os400_lpar_mobility_c apable,active_lpar_share_idle_procs_capable,active_mem_dedup_capable,active_mem_ex pansion_capable,hardware_active_mem_expansion_capable,active_mem_mirroring_hypervi...
Page 158 – Owns Virtual Serial Adapters – Belongs to I/O Fail Over Pool – SR-IOV non-adjunct For further information including the use of Partition Remote Restart please see the following: http://www.ibm.com/support/knowledgecenter/POWER8/p8hat/p8hat_enadisremres.htm IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 159: Chapter 4. Reliability, Availability, And Serviceability
Draft Document for Review October 14, 2014 10:19 am Reliability, availability, and Chapter 4. serviceability This chapter provides information about IBM Power Systems reliability, availability, and serviceability (RAS) design and features. The elements of RAS can be described as follows: Reliability...
Page 160: Introduction
One key differentiator of the IBM POWER8 processor-based servers is that they leverage all the advanced RAS characteristics of the POWER8 processor through the whole portfolio, offering reliability and availability features that often are not seen in other scale-out servers.
Page 161: Reliability
DIMMs), which can be used to avoid replacing memory. 4.2 Reliability Highly reliable systems are built with highly reliable components. On IBM POWER processor-based systems, this basic principle is expanded upon with a clear design for reliability architecture and methodology. A concentrated, systematic, and architecture-based approach is designed to improve overall system reliability with each successive generation of system offerings.
Page 162: Placement Of Components
PCIe BUS directly from the Processor module. Parts selection also plays a critical role in overall system reliability. IBM uses stringent design criteria to select server grade components that are extensively tested and qualified to meet and exceed a minimum design life of seven years. By selecting higher reliability grade components, the frequency of all failures is lowered, and wear-out is not expected within the operating system life.
Page 163: Processor/Memory Availability Details
This design for availability begins with implementing an architecture for ED/FI. First-Failure Data Capture (FFDC) is the capability of IBM hardware and microcode to continuously monitor hardware functions. Within the processor and memory subsystem, detailed monitoring is done by circuits within the hardware components themselves. Fault information is gathered into fault isolation registers (FIRs) and reported to the appropriate components for handling.
Page 164: Uncorrectable Error Introduction
POWER8 systems, the memory controller is designed with a replay buffer that allows memory transactions to be tried again after certain faults internal to the memory controller faults are detected. This complements the try again abilities of the memory buffer module. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 165: Alternative Processor Recovery And Partition Availability Priority
5137ch04.fm Draft Document for Review October 14, 2014 10:19 am 4.3.5 Alternative processor recovery and Partition Availability Priority If Processor Instruction Retry for a fault within a core occurs multiple times without success, the fault is considered to be a solid failure. In some instances, PowerVM can work with the processor hardware to migrate a workload running on the failing processor to a spare or alternative processor.
Page 166: Other Processor Chip Functions
(AC power) failure. The auto-restart (reboot) option must be enabled from the Advanced System Management Interface (ASMI) or from the Control (Operator) Panel. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 167: Memory Protection
5137ch04.fm Draft Document for Review October 14, 2014 10:19 am 4.3.10 Memory protection POWER8 processor-based systems have a three-part memory subsystem design. This design consists of two memory controllers in each processor module, which communicate to buffer modules on memory DIMMS through memory channels and access the DRAM memory modules on DIMMs, as shown in Figure 4-2 on page 153.
Page 168: I/O Subsystem Availability And Enhanced Error Handling
PCIe Host Bridges into the processor module itself. PCIe busses that are generated directly from a host bridge may drive individual I/O slots IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 169 PCIe switch. The integrated PCIe controller supports try again (end-point error recovery) and freezing. IBM device drivers under AIX are fully EEH-capable. For Linux under PowerVM, EEH support extends to many frequently used devices. There might be various third-party PCI devices that do not provide native EEH support.
Page 170: Enterprise Systems Availability Details
4.5 Availability impacts of a solution architecture Any given solution should not rely only on the hardware platform. Despite IBM Power Systems being far superior RAS than other comparable systems, it is advisable to design a redundant architecture surrounding the application in order to allow for easier maintenance tasks and grater flexibility.
Page 171: Clustering
4.5.1 Clustering IBM Power Systems running under PowerVM and IBM i, AIX and Linux support a spectrum of clustering solutions. These solutions are designed to meet requirements not only for application availability as regards to server outages, but also data center disaster management, reliable data backups and so forth.
Page 172: Live Partition Mobility
The first is for communication between servers, the second is for communication between partitions on each server and the HMC for resource monitoring and control functions (RMC.) IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 173 5137ch04.fm Draft Document for Review October 14, 2014 10:19 am LPM also needs that all systems in the LPM cluster be attached to the same SAN (when using SAN for required common storage) which typically requires use of fibre channel adapters.
Page 174 I/O capabilities are provided by x8 connections to a PCIe switch integrated in the processor board. All the slots connected to the PCIe switch belong to a single processor. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 175 5137ch04.fm Draft Document for Review October 14, 2014 10:19 am Figure 4-6 on page 161 illustrates how such a system could be configured to maximize redundancy in a VIOS environment, presuming the rootvgs for each VIOS are accessed from storage area networks. Physical Processor Chip Processor Chip...
Page 176: Serviceability
Expansion Drawer PCIe Expansion Drawer Figure 4-7 Logical diagram of processors and its associated PCIe slots More information For more information on PowerVM and Live Partition Mobility, please see “IBM PowerVM Virtualization Introduction and Configuration“ http://www.redbooks.ibm.com/abstracts/sg247940.html 4.6 Serviceability The purpose of serviceability is to repair the system while attempting to minimize or eliminate service cost (within budget objectives) and maintaining application availability and high customer satisfaction.
Page 177: Detecting Errors
For the vast majority of faults, a good FFDC design means that the root cause is isolated at the time of the failure without intervention by an IBM SSR. For all faults, good FFDC design still makes failure information available to the IBM SSR, and this information can be used to confirm the automatic diagnosis.
Page 178: Service Processor
Concurrent access to the service processors menus of the ASMI This access allows nondisruptive abilities to change system default parameters, interrogate service processor progress and error logs, and set and reset service indicators IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 179: Diagnosing
(Light Path for low-end servers), and access all service processor functions without having to power down the system to the standby state. The administrator or IBM SSR dynamically can access the menus from any web browser-enabled console that is attached to the Ethernet service network, concurrently with normal system operation.
Page 180: Reporting
IBM. For optional HMC managed servers, the event is reported to the HMC, which can initiate the Call Home request to IBM. I/O devices can also include specific exercisers that can be started by the diagnostic facilities for problem recreation (if required by service procedures).
Page 181 Draft Document for Review October 14, 2014 10:19 am wizard. With the new HMC V8R8.1.0 a Serviceable Event Manager is available to block problems from being automatically transferred to IBM. For more information about this topic, see “Service Event Manager” on page 181 for more details.
Page 182: Notifying
The data is formatted and prepared for transmission back to IBM either to assist the service support organization with preparing a service action plan for the IBM SSR or for additional analysis.
Page 183: Locating And Servicing
These records provide valuable information that can be used by remote support and IBM SSRs, enabling the IBM SSRs to assist in keeping the firmware and software current on the server.
Page 184 The service person can clearly identify components for replacement by using specific component level identify LEDs, and can also guide the IBM SSR directly to the component by signaling (flashing) the FRU component identify LED, and rolling up to the blue enclosure Locate LED.
Page 185: Manageability
Updating firmware, device drivers, operating systems, middleware components, and IBM applications after replacing a part R&V procedures can be used by IBM SSR providers who are familiar with the task and those who are not. Education-on-demand content is placed in the procedure at the appropriate locations.
Page 186 Ethernet cable or a crossed cable) or through an Ethernet network. ASMI can also be accessed from an ASCII terminal, but this is available only while the system is in the platform powered-off mode. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 187 5137ch04.fm Draft Document for Review October 14, 2014 10:19 am Use the ASMI to change the service processor IP addresses or to apply certain security policies and prevent access from unwanted IP addresses or ranges. You might be able to use the service processor’s default settings. In that case, accessing the ASMI is not necessary.
Page 188 DVD drive, and online diagnostic tests (available in AIX). Online diagnostic tests, when installed, are a part of the AIX or IBM i operating system on the disk or server. They can be booted in single-user mode (service mode), run in maintenance mode, or run concurrently (concurrent mode) with other applications.
Page 189 IBM i licensed machine code (Licensed Internal Code) and System Service Tools (SSTs) as part of IBM i. DSTs can be run in dedicated mode (no operating system is loaded). DSTs and diagnostic tests are a superset of those available under SSTs.
Page 190: Ibm Power Systems Firmware Maintenance
With the new HMC Version V8R8.1.0.0 and Power Systems servers, the firmware installations are restricted to entitled servers. The customer must be registered with IBM and entitled with a service contract. During the initial machine warranty period, the access key already is installed in the machine by manufacturing.
Page 191 Firmware updates that can be partially or wholly concurrent or nondisruptive. With the introduction of CFM, IBM is increasing its clients’ opportunity to stay on a given release level for longer periods. Clients that want maximum stability can defer until there is a compelling...
Page 192 5137ch04.fm Draft Document for Review October 14, 2014 10:19 am Each IBM Power Systems server has the following levels of server firmware and power subsystem firmware: Installed level This level of server firmware or power subsystem firmware is installed and will be installed into memory after the managed system is powered off and then powered on.
Page 193: Concurrent Firmware Update Improvements
4.7.4 Electronic Services and Electronic Service Agent IBM transformed its delivery of hardware and software support services to help you achieve higher system availability. Electronic Services is a web-enabled solution that offers an exclusive, no additional charge enhancement to the service and support that is available for IBM servers.
Page 194 Installation and use of ESA for problem reporting enables IBM to provide better support and service for your IBM server. To learn how Electronic Services can work for you, see the following website (an IBM ID is required): http://www.ibm.com/support/electronic...
Page 195 For more information about how to use the power of IBM Electronic Services, contact your IBM SSR, or see the following website: http://www.ibm.com/support/electronic...
Page 196 The following menu options are available when you select an event in the table: View Details... Shows the details of this event. View Files... Shows the files that are associated with this event. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 197: Selected Power8 Ras Capabilities By Operating System
Table 4-2 provides a list of the Power Systems RAS capabilities by operating system. The HMC is an optional feature on scale-out Power Systems servers. Table 4-2 Selected RAS features by operating system RAS feature IBM i Linux V7.1 TL3 SP3 V7R1M0 TR8 RHEL6.5...
Page 198 Supported in POWER Hypervisor, but not supported in a PowerKVM environment b. Supported in POWER Hypervisor, with limited support in a PowerKVM environment c. For POWER8 systems, IBM i requires IBM i 7.1 TR9 and IBM i 7.2 TR1. IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 199: Related Publications
Active Memory Expansion: Overview and Usage Guide http://public.dhe.ibm.com/common/ssi/ecm/en/pow03037usen/POW03037USEN.PDF IBM EnergyScale for POWER7 Processor-Based Systems white paper http://public.dhe.ibm.com/common/ssi/ecm/en/pow03039usen/POW03039USEN.PDF IBM Power Facts and Features - IBM Power Systems, IBM PureFlex System, and Power Blades http://www.ibm.com/systems/power/hardware/reports/factsfeatures.html IBM Power System S812L server specifications http://www.ibm.com/systems/power/hardware/s812l-s822l/specs.html...
Page 200: Help From Ibm
Draft Document for Review October 14, 2014 10:19 am IBM Power System E870 server specifications http://www.ibm.com/systems/power/hardware/e870/specs.html IBM Power System E880 server specifications http://www.ibm.com/systems/power/hardware/e880/specs.html Help from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services IBM Power Systems E870 and E880 Technical Overview and Introduction...
Page 202 ™ New modular This IBM® Redpaper™ publication is a comprehensive guide covering INTERNATIONAL the IBM Power System E870 (9119-MME) and IBM Power System E880 architecture for TECHNICAL (9119-MHE) servers that support IBM AIX®, IBM i, and Linux operating increased reliability SUPPORT systems.

This manual is also suitable for:

Power systems e880

IBM Power Systems E870 Technical Overview And Introduction

Notices

Preface

Chapter 1. General Description

Chapter 2. Architecture and Technical Overview

Chapter 3. Virtualization

Chapter 4. Reliability, Availability, and Serviceability

Related Publications

Quick Links

Front cover

Related Manuals for IBM Power Systems E870

Summary of Contents for IBM Power Systems E870

This manual is also suitable for:

Table of Contents