Power8 System Firmware

System firmware changes that affect all systems

• A problem was fixed for a clearing of all guard records associated with one error log entry.  If a FRU is replaced for any of the related guard record, all the related guard records are cleared.  Previously, only the guard record for the replaced FRU was cleared and the association was lost.
  
• A fix was made to prevent processor speculative memory loads from the service processor mailbox Direct Memory Access (DMA) area in the CEC memory.  The speculative loads caused memory cache faults and system checkstops with SRC B181E540.
• A problem was fixed to reduce switching noise on the memory address bus for DIMMs.  Noise on the bus could cause a failure for a marginal DIMM, so this fix has the effect of potentially improving the reliability of the memory.

System firmware changes that affect certain systems

• On systems with a large number of memory DIMMs (64 or more) and redundant service processors, a problem was fixed for a firmware update failure with SRC E302F966 when a failover was attempted as part of the firmware update, but the service processors did not change roles.  This also fixes failing Administrative Failovers (AFOs) for systems with large memory.  The performance of the CEC memory initialization was improved to prevent the hypervisor time-outs for service processor failovers.
  

New Features and Functions

• Support for setting Power Management Tuning Parameters from the management console (Fixed Maximum Frequency (FMF), Idle Power Save, and DPS Tunables) without needing to use the Advanced System Management Interface (ASMI) on the service processor.  This allows FMF mode to be set by default without having to modify any tunable parameters using ASMI.
• Support for SSLv3 has been discontinued to reduce security vulnerabilities in the secured connections to the service processor.
• Support was added for Single Root I/O Virtualization (SR-IOV) that enables the hypervisor to share a SR-IOV-capable PCI-Express adapter across multiple partitions. Two Ethernet adapters are supported with the SR-IOV NIC capability, when placed in the Power E880/E870:
  •    PCIe2 LP 4-port (10Gb FCoE and 1GbE) SR&RJ45 Adapter (#EN0L)
  •    PCIe2 LP 4-port (10Gb FCoE and 1GbE) SFP+Copper and RJ4 Adapter (#EN0J)
  These adapters each have four ports, and all four ports are enabled with SR-IOV function. The entire adapter (all four ports) is configured for SR-IOV or none of the ports is.
  System firmware updates the adapter firmware level on these adapters to 10.2.252.16 when a supported adapter is placed into SR-IOV mode.
  Support for SR-IOV adapter sharing is not yet available for adapters is a PCIe Gen3 I/O Expansion Drawer.
  SR-IOV NIC on the Power E870/E880 is supported by:
  •    AIX 6.1 TL9 SP4 and APAR IV63331, or later
  •    AIX 7.1 TL3 SP4 and APAR IV63332, or later
  •    IBM i 7.1 TR9, or later
  •    IBM i 7.2 TR1, or later
  •    Red Hat Enterprise Linux 6.5, or later
  •    Red Hat Enterprise Linux 7, or later
  •    SUSE Linux Enterprise Server 11 SP3, or later
  -           VIOS 2.2.3.4 with interim fix IV63331, or later
  

System firmware changes that affect all systems

• HIPER/Pervasive:  A problem was fixed for a processor clock failover with SRC B158CC62 that caused a system checkstop when the backup clock oscillator did not initialize fast enough.
  
• A problem was fixed for the iptables process consuming all available memory, causing an out of memory dump and reset/reload of the service processor.
• A problem was fixed for a PowerVM hypervisor hang after a processor core and system checkstop.  The failed processor core was not put into a guarded state and the hypervisor hung when it tried to use the failed core.
• A problem was fixed for a oscillator error caused by a power line disturbance that logged an UE SRC B150CC62 with no FRU call outs.  The  error SRC was changed from unrecoverable to informational as no service action is required.
• A problem was fixed for the NEBS DC power supply showing up in the part inventories for the CEC as "IBM AC PS".  The description string has been changed to "IBM PS" as power supplies can be of DC or AC type.
• A problem was fixed for the power supplies to add a monitor process for the second rotor in each power supply that was not being monitored.  This will improve fault isolation for power supply problems.  A fix for the second rotor in an earlier service pack release provided the monitor infrastructure but was missing the monitor process.
• A problem was fixed for a FSI link heartbeat surveillance fault with SRC B1504813 logged that has no FRU call outs.  The FRU call outs have been added.
• A problem was fixed with the Advanced System Management Interface (ASMI) VPD menu where the Generic External Connector (GC) FRU was displayed as an unknown FRU type.  The "Unknown" has been replaced with "Generic External Connector".
  
• A problem was fixed for a system fan identify LED not being able to light after a Digital Power Systems Sweep (DPSS) chip failover.  The fan LED ownership was not transferred to the new primary DPSS chip, so it was unable to light the LED under fan fault conditions.
• A problem was fixed for SRC B1104800 having duplicate FRU call outs for the PNOR flash FRU.
• A problem was fixed to prevent the Advanced System Management Interface (ASMI) "System Service Aids/Factory Configuration" panel option from restoring to factory configuration for FSP or ALL if one boot side of the service processor is marked invalid.  The following informational message is issued:  "The request cannot be performed because a firmware boot side is marked invalid.  This state may have been caused by a previous firmware update failure."
• A problem was fixed for error log with SRC B150DA19,  created on the backup service processor for a PSI link failure detected on the primary,  not being visible in the error logs on the primary service processor.
• A problem was fixed in the hardware server to prevent a UE B181BA07 abort when a host boot dump collection is in progress.
• A problem was fixed for an LED fault with SRC B181A734 that occurred during a normal rebuild of the LED tables, resulting in the LED not being lit.  The problem has been fixed using retries for LEDs that are in a busy state.
• A problem was fixed for a PSI link failure with SRC B1517212 that resulted in a service processor stop state.  The correct state for a system with broken PSI links is the terminate state so the problem can be resolved with a call home service event.
• A problem was fixed to prevent false oscillator error logs of SRC B150CC62 for errors unrelated to clock failures.
• A security problem was fixed in OpenSSL for padding-oracle attacks known as Padding Oracle On Downgraded Legacy Encryption (POODLE).  This attack allows a man-in-the-middle attacker to obtain a plain text version of the encrypted session data. The Common Vulnerabilities and Exposures issue number is CVE-2014-3566.  The service processor POODLE fix is implemented by disabling SSL protocol SSLv3 and requiring TLSv1.2 protocol on all secured connections.  The Hardware Management Console (HMC) also requires a POODLE fix for APAR MB03867(FIX FOR CVE-2014-3566 FOR HMC V8 R8.2.0 SP1 with PTF MH01455).  This HMC minimum requirement is enforced by the firmware update process for this defect.
• A problem was fixed for firmware updates that caused the primary service processor to be guarded and SRC B152E6D0 and SRCs of form B181XXXX to be logged.
• A problem was fixed for intermittent firmware database errors that logged an UE SRC of B1818611 and had a fwdbServer core dump.
• A problem was fixed to enable the redundant Vital Product Data (VPD) SEEPROM for processors and voltage regulator modules (VRMs).  Previously, only the primary SEEPROM was programmed with the FRU data with no backup protection.
• A problem was fixed for vague error text for SRC B1504922 for a bad SMP cable.  It was made more specific to state that an incorrect cable length was detected.
• A problem was fixed for an intermittent reset/reload of the service processor during the early part of an IPL with SRC B1814616 logged.
• A problem was fixed for hardware presence detection and local clock card (LCC) failover.  The system could not detect critical system hardware with th e default LCC missing, causing an error when failing over to the backup LCC.
• A problem was fixed for non-optimal voltage levels from the power supplies.  Having the power supply output voltages meet the exact specifications will help prevent stress-related hardware failures.
• A problem was fixed for an error in the "Enlarged IO Capacity Slot Count" that caused more memory than expected to be consumed by the hypervisor.  If the "Enlarged IO Capacity Slot Count" was not a "1", it was wrongly changed to an "8" by the IPL process, increasing the amount of memory that needs to be reserved for I/O buffers.  Retain tip H213684 tells how to reduce the hypervisor memory consumption when this problem happens as the fix will not change the value automatically:
  With the system at the "Power Off" state, take the following actions to to free up some memory from the hypervisor:
  - Log into ASMI and then select "System Configuration" menu    
  - Select  "I/O Adapter Enlarged Capacity" option                
  - Use the pulldown to select "1" as the new value for all nodes
  - After changing the value click on the "Save" setting. The change will be active on the next IPL of the system.
  
• A problem was fixed for the PCIe reset line (PERST) to keep it active during the IPL until both system power and clocks are stable.  Keeping the PCIe devices in reset until the environment is stable prevents PCIe device lockup.
• A problem was fixed to prevent a hypervisor task failure if multiple resource dumps running concurrently run out of dump buffer space.  The failed hypervisor task could prevent basic logical partition operations from working.
• On systems using the Virtual I/O Server (VIOS) to share physical I/O resources among client logical partitions, a problem was fixed for memory relocation errors during page migrations for the virtual control blocks.  These errors caused a CEC termination with SRC B700F103.  The memory relocation could be part of the processing for the Dynamic Platform Optimizer (DPO), Active Memory Sharing (AMS) between partitions, mirrored memory defragmentation, or a concurrent FRU repair.
• A problem was fixed that could result in unpredictable behavior if a memory UE is encountered while relocating the contents of a logical memory block during one of these operations:
  - Reducing the size of an Active Memory Sharing (AMS) pool.
  - On systems using mirrored memory, using the memory mirroring optimization tool.
  - Performing a Dynamic Platform Optimizer (DPO) operation.
• A problem was fixed for PCIe link width faults on the  I/O expansion drawer (F/C #EMX0) to only log the SRC B7006A8B once for each FRU instead of having multiple SRCs and call outs for the same part.
• A problem was fixed for a wrong state for the PCIe link LEDs (lit when link has failed) to the I/O expansion drawer with feature code #EMX0.  The fix insures that the link operational LEDs are not lit when the link to the I/O drawer has failed.
• A problem was fixed for an incorrect SRC of B7006A9F logged for I/O drawer VPD mismatch during an enclosure serial number update of the I/O drawer (F/C #EMX0).  The incorrect SRC was logged if the non-primary service path module (right bay) was in a failed state.
  
• A problem was fixed for a SRC B7006A84 PCIe link down event not being reported as a failed link for the I/O expansion drawer (F/C #EMX0) in the PCIe topology status in the Advanced System Manager Interface (ASMI) or on the management console.
• A problem was fixed for the Live Partition Mobility (LPM) migration of virtual devices to a Power8 systems to update each virtual device location code correctly to reflect the location code in the target systems instead of the location code in the source system.  This problem prevented the management console from being able to look up AIX Object Data Manager (ODM) names for the virtual devices so that operations such as remove on the device could not be performed.
• A problem was fixed for PCIe adapters requesting PCI I/O space that triggers a SRC BA1800007 error log.  This SRC should not have been logged since PC I/O spaces are not supported by Power8 systems.  The SRC log is now suppressed.
• A problem was fixed for a processor core unit being deconfigured but not guarded for a SRC B113E504 processor error in host boot with fault isolation register (FIR) code "RC_PMPROC_CHKSLW_NOT_IN_ETR" that caused the CEC to go to termination.  By guarding the failed processor core, the fix insures the core is not used on the reIPL of the CEC.
• A security problem was fixed in OpenSSL for memory leaks that allowed remote attackers to cause a denial of service (out of memory on the service processor). The Common Vulnerabilities and Exposures issue numbers are CVE-2014-3513 and CVE-2014-3567.
• A security problem in GNU Bash was fixed to prevent arbitrary commands hidden in environment variables from being run during the start of a Bash shell.  Although GNU Bash is not actively used on the service processor, it does exist in a library so it has been fixed.  This is IBM Product Security Incident Response Team (PSIRT) issue #2211.  The Common Vulnerabilities and Exposures issue numbers for this problem are CVE-2014-6271, CVE-2014-7169, CVE-2014-7186, and CVE-2014-7187.
• A problem was fixed to add failure recovery in the early boot of the service processor so that the boot is retried on failure instead of the service processing going unresponsive with SRC B1817212 on the operations panel.
• A problem was fixed for isolating and repairing DIMM memory failures at the byte level without affecting other ranks of memory. This fix substantially reduces the FRU call outs of DIMMS for memory problems.
• A security problem was fixed in OpenSSL where the service processor would, under certain conditions, accept Diffie-Hellman client certificates without the use of a private key, allowing a user to falsely authenticate .  The Common Vulnerabilities and Exposures issue number is CVE-2015-0205.
• A security problem was fixed in OpenSSL to prevent a denial of service when handling certain Datagram Transport Layer Security (DTLS) messages.  A specially crafted DTLS message could exhaust all available memory and cause the service processor to reset.  The Common Vulnerabilities and Exposures issue number is CVE-2015-0206.
• A security problem was fixed in OpenSSL to prevent a denial of service when handling certain Datagram Transport Layer Security (DTLS) messages.  A specially crafted DTLS message could do an null pointer de-reference and cause the service processor to reset.  The Common Vulnerabilities and Exposures issue number is CVE-2014-3571.
• A security problem was fixed in OpenSSL to fix multiple flaws in the parsing of X.509 certificates.  These flaws could be used to modify an X.509 certificate to produce a certificate with a different fingerprint without invalidating its signature, and possibly bypass fingerprint-based blacklisting.  The Common Vulnerabilities and Exposures issue number is CVE-2014-8275.
• A security vulnerability, commonly referred to as GHOST, was fixed in the service processor glibc functions getbyhostname() and getbyhostname2() that allowed remote users of the functions to cause a buffer overflow and execute arbitrary code with the permissions of the server application.  There is no way to exploit this vulnerability on the service processor but it has been fixed to remove the vulnerability from the firmware.  The Common Vulnerabilities and Exposures issue number is CVE-2015-0235.
• A problem was fixed for an incorrect SRC logged for an unplugged cable to the PCIe I/O expansion drawer (F/C #EMX0).  A B7006A88 SRC was errantly logged that calls out the cable as bad hardware that needs to be replaced.  This is replaced with SRC B7006A82 that says a cable is unplugged to a PCIe FanOut module in the IO expansion drawer.
• A problem was fixed for missing dump data for cores and L3 cache memory when there is core checkstop and deconfiguration of the core.
• A problem was fixed for a false power supply fan failure with SRC 1100152F.  If the AC was interrupted to the power supply, the SRC 11001525 would have been logged for a bad fan with a call out of the power supply for replacement.
• A problem was fixed for a partition deletion error on the management console with error code 0x4000E002 and message "...insufficient memory for PHYP".  The partition delete operation has been adjusted to accommodate the temporary increase in memory usage caused by memory fragmentation, allowing the delete operation to be successful.
• A problem was fixed for disruptive firmware update to prevent false reference clock failures with SRC B1814805 and a hang in the IPL for the CEC.
• A problem was fixed for a memory leak associated with the logging of SRC B1561311 for a bad voltage regulator module (VRM).
• A problem was fixed for the processor module replacement process to prevent VPD corruption on the primary and redundant VPD chips on the new processor module.  This corruption resulted in the processor being unusable with HostBoot failing with unrecoverable errors (UEs) of SRCs BC8A090F and BC8A1701.
  

• HIPER/Pervasive:Deferred:  On a system configured for a large number of PCIe adapters across multiple PCIe I/O expansion drawers (F/C #EMX0), a problem was fixed so that the PCIe adapters worked correctly in the system.  Previously, the PCIe interrupt servicing could deadlock, causing the PCIe adapter cards to become unresponsive.
  
• For a system with Virtual Trusted Platform Module (VTPM) partitions,  a problem was fixed for a management console error that occurred while restoring a backup profile that caused the system to to go the management console "Incomplete state".  The failed system had a suspended VTPM partition and a B7000602 SRC logged.
• For systems with IBMi partitions, a problem was fixed for the "5250 Application Capable" capability so it is passed to the IBMi partition as "True" if purchased.  For the problem, the capability was not sent to the partition and could cause extra performance to be missing for the "Fast Green Screen Performance" feature in IBMi.  There is a delay of up to 15 minutes after this fix is installed before it becomes active on the system.  If the updated capability property does not show up in the management console CEC properties as "True", this is a slowness in the refresh of the capability properties to the management console and not a problem with the fix.  To resolve this issue with the capability not displaying correctly, rebuild the managed system on the management console and then wait up to one hour for the CEC property capability "5250 Application Capable" to be updated to "True".
  
• On a system with a Linux partition, a problem was fixed for the Linux "lsslot" command so that it is able to find the F/C EC41 and EC42 PCIe 3D graphics adapter installed in the CEC, instead of showing the slot as "empty".  The Linux graphics adapter worked correctly even though it showed as "empty".
• On systems with a PCIe 3D graphics adapter (F/C #EC41 or #EC42) in a partition, a problem was fixed for a partition hang or BA21xxxx error conditions during partition initialization.
• A problem was fixed for certain workloads that caused the system to enter safe mode (mode for running at minimum processor frequencies)  when the On-chip controllers (OCCs) did not get the Analog Power Subsystem Sweep (APSS)  frequency control data within the OCC time out period.  The time out for a OCC update has been increased so the OCC can tolerate periods of high bus use that slow down the APSS communication.
• On a system with redundant service processors, a problem was fixed for bad pointer reference in the mailbox function during data synchronization between the two service processors.  The de-reference of the bad pointer caused a core dump, reset/reload, and fail-over to the backup service processor.
  

System firmware changes that affect all systems

• A problem was fixed in concurrent firmware update to prevent the secondary service processor from going to a failed state.
• A problem was fixed for the power supply fans to monitor both rotors instead of one to prevent a failure in one rotor from shutting down the power supply.
• A problem was fixed for firmware updates to reduce the number of informational B181A85E SRCs for an expected SQL lock condition during a database transaction.  Previously, several thousand B181A85E SRC entries were created for the error log, slowing performance of the service processor and flooding the error log.
• A problem was fixed for reset/reload failures caused by excessive synchronization of thermal management data with the redundant service processor.
• A problem was fixed for failovers to the secondary service processor failing with SRC B1818601 caused by a bad data base object reference.
  

System firmware changes that affect certain systems

• For a system with memory mirroring activated and a memory block size of 16 Megabytes, a problem was fixed for system dump that caused Hypervisor Real Mode Offset (HMRO) data structure corruption in the physical memory map.    This problem could cause concurrent firmware update failures or subsequent system dumps to be corrupted.
  

• GA Level