Arm Chiplet System Architecture

The Arm Chiplet System Architecture ("CSA") is mainly applicable to how the compute subsystem in a system design can be partitioned across chiplets, where chiplet interfaces replace the traditional on-die connections within an System on Chip design. The CSA specifies a hardware system architecture focused towards an Arm based compute system that is distributed over multiple chiplets. In this context an Arm system is defined as the hardware system that can support a single software based system such as an operating system. Arm recognises that there is a need to aggregate peripheral devices connected through other existing standards but has left that to other standardisation activity.  The CSA is a therefore a standard that supports the disaggregation of Arm based compute system into chiplet based parts.

The specification is intended to act as a foundation for standardisation and forms the basis for additional layers of standards such as protocols, physical interface definitions and reusable chiplet products. The CSA does not specify any new interface functionality. The CSA is looking to guide both the 'system designer' looking to make a system using chiplets, and standalone chiplet designers, either within the same organization or third party entities by defining how chiplets are safely aggregated. The design activities of a CSA based chiplet are focused on the needs of the compute system, such as how interrupts of the processor are handled or data safely accessed within the memory system, to enable tasks in the single software based system.  

The CSA addresses the major difference in current compute system architectures by defining two System Types: 

• Compute and Hub System: A hub coordinates the interchange between the various compute, memory and I/O parts of the system. The hub tends to be a limiting factor in system wide performance.
• Compute Tile System: A network or switch fabric handles the interchange between the various compute, memory and I/O parts of the system in a more decentralised model. This offers greater scalability at the expense of additional complexity. 

Within a system the CSA defines Chiplet Types:

• Hub: Provides connected compute chiplets with access to system main memory and I/O as well as system level security and system management functions. In a system with multiple chiplets of this type one is designated the primary coordinator.
• Compute 1 Chiplet: Provides compute resources that do not have direct access to system main memory or I/O peripherals, these are provided via a Hub.
• Compute 2 Chiplet: Provides compute resources with direct access to system main memory and I/O peripherals. In a system with multiple chiplets of this type one is designated the primary coordinator.
• Fully Coherent Expansion: Provides system expansion for application specific acceleration, with coherent system memory access using physical addresses (PA), generated using translation in a memory management unit (MMU) within the chiplet.
• Fully Coherent Expansion (Remote Translation): Provides system expansion for application specific acceleration, address translation is undertaken externally for this chiplet type but can cache translations locally.
• I/O Coherent Expansion (Translated): Provides system expansion for application specific acceleration and I/O with coherent system memory access using physical addresses (PA), generated using translation in a memory management unit (MMU) within the chiplet.
• I/O Coherent Expansion (Untranslated): Provides system expansion for I/O coherent Arm Advanced Microcontroller Bus Architecture devices, interrupt handling and address translation are not managed within this chiplet type.
• I/O Coherent Expansion (Remote Translation): Provides system expansion for complex I/O Coherent devices, address translation is undertaken externally for this chiplet type but can cache translations.
• I/O: Provides system expansion I/O devices, interrupt handling and address translation are not managed within this chiplet type.
• I/O Controller: Provides high speed I/O to the system (including PHYs) for interfaces such as DDR SDRAM, USB, MIPI CSI, HDMI. There are no coherent agents in such chiplets.

For each type of chiplet the CSA defines 1) the conditions for system memory access and how the memory management units interact to handle memory address translation, coherency across any cached data in the system and necessary security protection functionality; 2) how interrupts are managed; 3) how debug and trace data is transmitted; 4) the system control is undertaken; 5) security is maintained; and 6) time is synchronised across the system.

Between the various Chiplet Types  the CSA defines Chiplet Interfaces between them:

• Hub to Hub; Hub to Compute
• Compute 1 to Compute 1; Compute 2 to Compute 2
• Fully Coherent Expansion; Fully Coherent Expansion (Remote Translation)
• I/O Coherent Expansion (Translated); I/O Coherent Expansion (Untranslated); I/O Coherent Expansion (Remote Translation)
• I/O; I/O Controller

Through these interfaces the chiplets within a system must operate in a coordinated way through various activity states of the system, 1) Inactive, 2) Secure boot; where trusted security agents are synchronised, 3) Initial boot; where various security, system control and compute system parts are initiated, 4) Operational; where the chiplets work in a coordinated manner to provide system function, 5) Power saving, where some functions are placed in power saving state.

The CSA also defines some Common Chiplet Components:

• System Controller: Provides SoC specific management tasks such as power and thermal management with control functionality distributed over chiplets, with a system controller component in each chiplet.
• System Control Agent: Supports system control by effecting system control policy such as transitioning power modes, or changing clock gating.
• Trusted Security Agent: Hardware Enforced Security (HES) functionality across the system is provided by by one or more Trusted Security Agents in each chiplet.
• System Counter: Provides a measure of time in real-time allowing a uniform view of time across a system of chiplets.

The CSA is a high level system architecture definition. It does not specify how a system is implemented in detail, instead it references the many existing detailed Arm Architecture Specifications for implementation. It contains rules, such as:

"A Compute & Hub System contains one or more Compute 1 Chiplets"

These rules govern how the Chiplet Types, Interfaces and Components work to make up a viable system of a System Type. The CSA has levels of compliance, a chiplet or system can claim compliance with a specific level based on what is implemented.  Level 0 is the baseline for integrating Chiplets based on CSA Chiplet Types and Chiplet Interfaces. Level 1 ensures memory coherency, Full Compliance full alignment to all the CSA requirements.

Arm has contributed a vendor neutral version of the CSA to the Open Compute Project. 

This is obviously an important piece of overarching design guidance for the architectural partitioning of the compute subsystem functionality across chiplets within a heterogenous integrated system. It is however only covering the compute subsystem .