The Huawei Mate 8 Review

Kirin 950 - Performance

At the heart of the Mate 8 we find HiSilicon’s new Kirin 950 SoC. We’ve been able to attend the official unveiling earlier in November and to post about the new high end silicon chipset in our coverage of the announcement. The Kirin 950 is an important chipset because of several factors: 

First of all it seems that for the majority of markets Huawei is insistent on using its own in-house silicon even when it is evident that it may not be the best choice for the device. In past Huawei devices we’ve seen this as one of the root causes for rather disappointing battery life as former chipsets displayed some rather disappointing power efficiency. We also saw a regression in performance in last year’s devices as HiSilicon opted to rely on ARM's Cortex A53 cores to drive performance, rather than stick with 2014’s higher performing A15 cores or to release an A57 chipset such as Qualcomm’s Snapdragon 810/808 or Samsung’s Exynos 7420. This of course put Huawei’s flagship devices at a disadvantage as it's a SoC's performance and efficiency that largely determines the competitive positioning of a smartphone.

The second factor (or collection of factors) as to why the Kirin 950 is an important chipset lies in the chipset’s raw specifications. This is the first shipping smartphone SoC equipped with ARM’s new Cortex A72 processor. The A72 promises a lot of efficiency gains over the A57 as the latter had the impression of a rather disappointing core as power efficiency seemed to have suffered in comparison to A15 implementations of the past generation. Only Samsung was able to objectively deliver a satisfactory performance in the Exynos 7420 – something we’ll come back to as we get to the power efficiency numbers of the Kirin 950.

To recap the specifications of the Kirin 950: This is a big.LITTLE SoC with 4x Cortex A72 cores each at 2.3GHz with a further 4 Cortex A53 cores at 1.8GHz. HiSilicon touts a brand new memory controller that is able to support both LPDDR3 and LPDDR4. In the case of the Mate 8 we find 3/4GB of LPDDR4 running at 1333MHz, slightly lower than what we’ve seen on competing SoCs such as from Samsung, Qualcomm and Nvidia. 

One characteristic that might be defining for the Kirin 950 is that it still uses a CCI-400 fabric between the two CPU clusters and the chip’s NoC (Network-on-Chip). I mentioned a few weeks ago that this was somewhat surprising revelation as I was expecting the newer CCI-500 generation interconnect to see its debut. HiSilicon explained to us that this was due to time to market constraints and that the CCI-500 wasn't fully ready to be implemented in a SoC when the Kirin 950 was being finalized. The effect of this is that we might be looking at slightly lower memory performance than what we’ll be able to see from other upcoming A72 designs.

Of course this is also the first SoC to market to integrate ARM’s new Mali T880 GPU IP. We’ll be analysing the GPU performance and efficiency later on in the article as for now we’ll be focusing on CPU performance and power.

Given that we didn’t see many consumer A57 designs from vendors and HiSilicon only used the core in its server- and enterprise-SoCs, and that the Kirin 925 is two generations behind (We’ve covered the A15 vs A57 architectural and performance improvements differences in our review of the Exynos 5433), I chose to use the Exynos 7420 as the main comparison SoC against the Kirin 950 as it gives us the closest possible apples-to-apples evaluation of the architecture of the new CPU.

We start with SPECint2000 estimated benchmark numbers. Developed by the Standard Performance Evaluation Corporation, SPECint2000 is the integer component of their larger SPEC CPU2000 benchmark. Designed around the turn of the century, officially SPEC CPU2000 has been retired for PC processors, but mobile processors are roughly a decade behind their PC counterparts in performance. Keeping that in mind it still provides an excellent benchmark for today's mobile phones and allows us to do single-threaded architectural comparisons between the competing CPU designs out there. The scores we publish are only estimates and should not taken as officially validated numbers (Which requires the test to be supervised by SPEC). Nevertheless, we try our best in choosing compiler flags and making the tests pass internal validation.

As we see the Kirin 950 is able to show significant improvements across the board, averaging about 27% higher performance than the Exynos 7420. It seems 176.gcc is the test that is most improved by the new architecture as we see a large 43% improvement. We also have to keep in mind that the Kirin’s A72 is clocked in at 2304MHz which is 200MHz / 10% higher than the Exynos 7420’s 2100MHz A57.

It’s been a long time since we had an IPC comparison between chipsets so I went back and ran a 32-bit variant of SPEC (for a valid apples-to-apples comparison against older chipsets) across a range of SoCs past the Cortex A9 generation. At an estimated 0.77 SPECint2000 ratio score per MHz the Kirin 950 ends up at the top of the table, but it’s not clear how this compares to the Cortex A57 as we see quite differing IPC scores from the chipsets at hand. This is an interesting result as what we can learn from it is that even though two SoCs can have the same high-level specifications on paper we can see quite differing performances caused by implementation differences (be it software or hardware), even among products of single vendors.

Going back to the official reported IPC increases that ARM shared with us, we can see that the reported ~16% increase in performance per cycle in integer performance seems to be about to be quite spot-on when comparing it to the results of our tests.

To have a further data-point, we also analyse the overall increases in GeekBench 3’s test scores:

The advantages in GeekBench are slightly lower as we an average increase of 13.3% across all integer tests and 10.8% across non-cryptographic tests, increases that seem to be more in line with the frequency increase of the cores rather than the micro-architectural improvements, which might indicate that GeekBench is less sensitive to the particular increases of the A72 or that there are other architectural factors at play which make the two SoCs perform differently across workloads.

In the floating point tests the advantage is larger as we see an average increase of 22.8% throughout all of the tests, with some sub-tests seeing high increases of up to 45% and even 69% in Mandelbrot. The larger floating-point increases are natural as the A72's new microarchitecture introduces comparatively larger changes to its floating-point execution pipelines than it does to its integer pipelines.

Memory bandwidth is something that ARM claims that the A72 architecture to have seen the largest improvements, and indeed we see this both in GeekBench STREAM memory tests as well as our own internal tests. It seems in particular write that has increased a lot. L2 bandwidth in particular has increase a lot and reaches ARM’s projected 50% and higher depending on the instructions and access pattern used, such as for example NEON instructions where two-thread concurrent read+write reaches up to ~38GB/s on the Kirin 950 while the Exynos only reaches up to ~23GB/s and the Snapdragon 810 ~19GB/s. 

Overall the Kirin 950’s Cortex A72 seems to perform as advertised and gives a robust boost in performance thanks to both architectural improvements as well as higher clocks due to the new process node.

Kirin 950 - Power & Efficiency

Besides the performance improvements of the Cortex A72, we're also supposed to see a micro-architectural power reduction. Both factors combined are promised to bring large power efficiency improvements of up to 30%. HiSilicon was extremely upfront with us in presenting us some unique figures on exactly how much the Cortex A72 improves compared to the Cortex A57. Indeed, beyond the IPC increase we also see a flat 20% power reduction at the same frequency and manufacturing process.

Process is one factor we can’t isolate as the Kirin 950 is among one of the first SoCs to ship on TSMC’s new 16nm FinFET+ manufacturing nodes (short 16FF+). We’ll get back to the manufacturing node in a bit but first let’s have a rough look at the device’s power consumption.

Because we were sampled early for the Mate 8 I wasn’t able to take advantage of root for more controlled DVFS measurements by external equipment as bootloader unlock codes yet to be made available by Huawei. As such, I had to rely on restricted measurements done via the device’s own fuel-gauge, making the following figures rough estimates rather than the more detailed figures published in recent articles.

Using the same power-virus at varying thread-counts across all devices and SoCs, we end up with the above table of power consumption. Immediately the Kirin 950’s power figures stand out as being much better than what we’ve come to be used to from ARM’s big cores. At a maximum system load power (total power consumption during a scenario minus average idle power) we can see that the Kirin 950 only reaches 3.7W at full frequency on all big cores. The same scenario on the Exynos 7420 for example reaches a much higher 5.4W. When looking at the per-core increases we see that it seems that the Kirin 950 uses only about 900-700mW of power. The diminishing power with thread count is something that I’ve observed in the past with some SoCs and CPU microarchitectures, so it seems to be a characteristic of the power virus I’m using that starves the cluster of resources and makes each additional thread/core become more bottle-necked.

As opposed to past HiSilicon SoCs, I was luckily able to extract the voltage tables of the Kirin 950. Through some calculations as well as reported power curves by ARM’s Intelligent Power Allocation (IPA) driver, I was able to create an estimated power curve of the Kirin 950’s big cluster.

In the past I’ve been rather harsh of the Kirin 925’s power consumption and was blaming this on the CPU cores, but it was only with the more-in-depth investigation of the Kirin 930 and 935’s power curves that I discovered the real culprit for the bad power consumption lay with non-CPU blocks such as HiSilicon’s memory controller. 

When accounting for this and looking back at the per-CPU power figures of the Kirin 925 we see that the Cortex A15 only consumed a rather reasonable ~950mW of power at 1.8GHz on 28nm which falls in line with the ~750mW average of the Exynos 5430’s 20nm A15 cores.

Luckily the Kirin 950 is said to have fixed the large power overhead from the memory controller and thus is able to shave off a good amount of power off the effective 1-core power figures. While I still have to confirm this with an actual measured power curve once I have the proper software tools for it, we’ll for now assume good faith in HiSilicon and estimate that the new overhead is as low as that from other vendors.

The new numbers put A57-based SoCs such as the Exynos 7420 in perspective, as the Kirin 950 now makes the competing SoC look comparatively unreasonable in terms of its peak power consumption. It’s odd to see this now as in hindsight 2015’s SoC generation may have been a one-off phenomenon where power consumption has risen drastically due to the switch to ARM’s first 64-bit micro-architecture.

To better visualize the improvement in power efficiency I graphed the average power curves of each SoC and normalized frequencies with the performance per clock value calculated by each SoC’s SPECint2000 score. Performance per clock remains stable across frequency across most architectures (Krait is an exception due to an asynchronous L2), so this is a valid estimate of overall power efficiency across most SoCs.

The Kirin 950 is able to stand out from 2015’s SoC both in terms of performance as well as efficiency. The most surprising data-point is that at the Exynos 7420's maximum performance point the Kirin 950’s A72 is almost twice as efficient – a eye-brow raising difference that I really did not expect. It’s here where the Kirin’s advantage in manufacturing process shows as beyond the micro-architectural improvement we also see that HiSilicon is able to take advantage of lower operating voltages.

As you notice in the graph, I included two curves of the Kirin 950’s A72 cores – one of the estimated actual power efficiency derived through the power curve of my test device as well as a “target” curve. 

The Kirin 950 uses a Cortex M3 microcontroller that uses hardware performance monitors to measure the silicon block's physical characteristics and based on that determines the operating voltages of the various SoC blocks. This seems to be a similar mechanism to the closed-loop voltage regulation mechanisms found on Qualcomm SoCs starting with the Snapdrago 810 or Nvidia’s Tegra systems. I’m not quite sure if the voltages change during run-time or if they're fixed at boot as I now currently presume. While extracting the voltage tables for the SoC during boot I noticed that the actual applied voltage on the A72 cores no longer scales down with reduced frequency. 

I reached out to HiSilicon on the matter and been given the explanation that early SoCs run a more conservative voltage scaling policy related to a still "unstable" process, and that future production units will steadily lower the operating voltages towards the target values as yields increase to more stable levels. As such it’s possible that users of future devices will be able to enjoy higher efficiency at the lower frequencies of the A72. I would just like to add that the overall power difference in such a case would be rather small as the device’s DVFS is set up to operate most of the time at frequencies above 1200MHz.

Overall the Kirin 950 and ARM's Cortex A72 are able to show some solid performance improvements, but it’s mostly in terms of efficiency where HiSilicon’s new chipset seems to excel. The gains effectively not only allow the chip-maker to catch up to vendors like Qualcomm and Samsung in performance, but also have enough of an efficiency lead over current designs such as the Exynos 7420 and Snapdragon 810 that we should be seeing the Kirin 950 able to compete against upcoming designs such as the Exynos 8890 and Qualcomm’s Snapdragon 820.

I’m also very excited to see peak power finally go back to sub-1W per core for sub-4W CPU TDPs, something we weren’t able to enjoy in 2015 in the high-performance segment. In the frequency drivers of the SoC I saw that HiSilicon had prepared a 2.5GHz bin of the SoC (Kirin 955?) so it’s possible we’ll again see a slight frequency bump much like the one we saw with the Kirin 925 or Kirin 935. If HiSilicon also manages to lower the operating voltages closer to the targeted values then it seems that future Huawei devices this year should find themselves in an excellent competitive position when it comes to performance and efficiency.