]> China Mobile

No.32 Xuanwumen West Street Beijing China yangpenglin@chinamobile.com

Huawei Technology Co.,Ltd.

127 Jinye Road, Yanta District Xi'an China pangting@huawei.com

AntGroup

World Financial Center, No.1 East 3rd Ring Middle Road Beijing China zhangxiaomeng.zxm@antgroup.com

Security TEEP Internet-Draft Big data computing framework uses Master-Worker structure to provision applications and process data. When process remote attestation for this kind of framework in TEE cluster, Verifier or Relying party needs to know the dynamic reference value of Worker nodes. This document shows how to use Master to relay Worker provision information to Verifier or Relying Party and its relevant protocol.

Introduction In big data area, different from stand-alone application, distributed application like machine learning always need to be split by Master-Worker framework like MESOS, YARM, Spark or kubernets . The Master node in charges of splitting distributed application to computing tasks. Then these tasks will be deployed on Workers for execution. TEE could be used to protect the application and its secret data, if only the whole process lifecycle is covered by TEE cluster. If a Verifier or Relying Party would like to attest distributed application running in TEE cluster, it has to verify the Worker nodes since the Worker nodes are the real place in where specific tasks and data are processing. The Master node conducts the splitting process and provision specific tasks or executable code to Workers after running the main part of application. Master node could relay these tasks or executable code information to Verifier or Relying party for remote attestation. As a result, Verifier or Relying Party could assess the trustworthiness of the lifecycle of distributed application in TEE cluster. This document specifies the method and protocol of this distributed TEE provision relay.

Terminology The following terms are used:

• Master: The node that runs the main application and in charges of splitting and distributing tasks to Worker nodes.
• Worker: The node that runs the specific tasks that generated by Master.
• Relay: Master transfer the original provision message to Verifier/Relying party for remote attestation.

Use Cases In privacy-preserve computing, participants want to create a unified machine learning model without leaking private data owned by each other. TEE cluster as a trusted party could make sure data inside this environment is integrated and confidential. If the federated machine learning participants trust this TEE cluster and application, they could transfer their data in that TEE cluster and generate the final machine learning model. The method and protocol described in this document could help privacy-preserve computing participants to process remote attestation of TEE cluster during runtime.

Architecture The following figure shows the architecture of TEE distributed provisioning relay. In this architecture, Master runs the main part of distributed application, which will generate tasks or executable code and provision to Workers. Master accepts remote attestation challenge from Verifier and response Evidence of Master. Then, Master relay provisioning information of Workers to Verifier/Relying Party. In the end, Evidence of Workers will be transfered to Verifier/Relying Party by Master. The security or trust logic of this architecture is to build a trust chain between Master and Workers, in which the Verifier/Relying party could trust the Master first and let the Master to relay the provisioning information of Workers to Verifier/Relying Party.

Distributed Application Remote Attestation

The steps of TEE distributed provisioning relay is described below. - Verifier/Relying party sends challenge to Master and tries to assess the trustworthiness of this distributed application runs in TEE cluster. - Master generates Evidence of itself and return to Verifier/Relying Party. - Master generates tasks and provisions to Workers. - Meanwhile, Master transfer the tasks provisioning information and Worker information back to Verifier/Relying Party to generate reference value of Workers. - Master challenges Worker for Evidence. - Master sends the Evidence of Workers back to Verifier/Relying Party for attestation.

Worker Runtime Provisioning Relay Worker runtime provisioning relay message is sent from Master to Verifier/Relying Party. It's function is to provide Verifier/Relying Party information to generate reference value of Workers. This message can only be sent after remote attestation of Master.

Rationality In big data computing framework, Reference Value of Workers may change during runtime based on computing stages and input data source. Thus it is not rational to generate Reference Value of Workers statically. Instead, Master is the manage and monitor center of the tasks in Workers. The following reason shows that Master is the reasonable choice to generate and relay Worker runtime provisioning message.

• Master runs the main part of distributed application which reference value is static and could be pre-generated.
• Master is task generator, it in charges of generating different tasks or executable code for Workers in different processing stage.
• Master is computing resource manager of TEE cluster, it in charges of orchestrating TEE hardware resource.

Worker Runtime Provisioning Relay by CDDL Worker runtime provisioning relay message includes 3 message type, task-serialized, hardware-manifest and software-manifest. The following figure shows the message framework of Worker runtime provisioning relay in CDDL.

TEE Distributed Provisioning Relay Message Framework any) $tee-dpr-message-type /= tee-task-serialized $tee-dpr-message-type /= tee-worker-hardware-manifest $tee-dpr-message-type /= tee-worker-software-manifest $tee-dpr-type = uint .size 1 TEE-task-serialized = 1 TEE-worker-hardware-manifest = 2 TEE-worker-software-manifest = 3 ]]>

Task serialized Task serialized represents the execution content like executable code of task. Master generate and distribute tasks to workers by its default mechanism. Meanwhile Master needs to relay this tee-task-serialized message to Verifier/Relying Party for remote attestation of Workers.

Worker Hardware Manifest Worker hardware manifest message needs to be relayed to Verifier before remote attestation. Worker hardware manifest is intent to describe the specific hardware environment of Workers, which includes CPU, memory, and other necessary information.

Worker Software Manifest Worker software manifest message includes operation system version, memory allocation method, memory protection method, etc. This message needs to be relayed to Verifier before remote attestation. Worker software manifest message is intent to describe the specific software environment of Worker, which includes operation system, memory, and other necessary information. Verifier will use this message to generate reference value.

Security Considerations This protocol is used to relay Worker TEE provisioning message between Master and Verifier, between which secure channel could be built by TLS and etc. The secure channel between Worker and Verifier could also use TLS or other secure protocol to protect provisioning message.

IANA Considerations This document does not require actions by IANA.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Broadcom Amazon Microsoft ALAXALA Networks Corp. This document specifies a protocol that installs, updates, and deletes Trusted Components in a device with a Trusted Execution Environment (TEE). This specification defines an interoperable protocol for managing the lifecycle of Trusted Components. Informative References Broadcom Arm Limited Microsoft Amazon A Trusted Execution Environment (TEE) is an environment that enforces that any code within that environment cannot be tampered with, and that any data used by such code cannot be read or tampered with by any code outside that environment. This architecture document motivates the design and standardization of a protocol for managing the lifecycle of trusted applications running inside such a TEE. China Mobile China Mobile China Mobile Huawei Technology Co.,Ltd. Confidential computing is the protection of data in use by performing computation in a hardware-based Trusted Execution Environment. Confidential computing could provide integrity and confidentiality for users who want to run applications and process data in that environment. When confidential computing is used in scenarios which need network to provision user data and applications in the TEE environment, TEEP architecture[I-D.ietf-teep-architecture] and protocol [I-D.ietf-teep-protocol] could be used. This document focuses on using TEEP to provision Network User data and applications in confidential computing. This document is a use case and extension of TEEP and could provide guidance for cloud computing, [MEC] and other scenarios to use confidential computing in network.

Appendix 1 Full CDDL of TEE DP protocol The full CDDL of TEE distributed provisioning protocol is shown below. any) $tee-dpr-message-type /= tee-task-serialized $tee-dpr-message-type /= tee-worker-hardware-manifest $tee-dpr-message-type /= tee-worker-software-manifest $tee-dpr-type = uint .size 1 TEE-task-serialized = 1 TEE-worker-hardware-manifest = 2 TEE-worker-software-manifest = 3 tee-task-serialized = [ type: TEE-task-serialized options:{         Recommended-serialized-type: uint .bits serialized-type   Serialized-payload = bstr } ] serialized-type = &(     json: 0 protobuf: 1     thrift: 2 pre-defined: 3 ) tee-worker-hardware-manifest = [     type: TEE-worker-hardware-manifest     options:{         CPU-core-number: uint .size 1         CPU-architecture: text .size(1..128)         CPU-version: text .size(1..128)         CPU-frequency: unit .size 2         ;the cpu frequency unit is MHZ         MEMORY-size: uint .size 4         ;the memory size unit is MB     } ] tee-worker-software-manifest = [     type: TEE-worker-software-manifest     options:{         OS-version: text .size(1..128)         Memory-allocation: text .size(1..128)         Sadr: bool         Canary: bool Worker-executor-version: text .size(1..128) Time-stamp: text .size(1..128) } ] ; labels of mapkey for tee dp message parameters, uint (0..15) Recommended-serialized-type = 0 Serialized-payload = 1 CPU-core-number = 2 CPU-architecture = 3 CPU-version = 4 CPU-frequency = 5 MEMORY-size = 6 OS-version = 7 Memory-allocation = 8 Sadr = 9 Canary = 10 Worker-executor-version = 11 Time-stamp = 12 ]]>