There is no support of the 128 bit integers (128uint/128int) in Windows and Linux SGX SDK Great notice about basics of SGX threading Intel Management Engine (Intel ME) for the SGX remote attestation. The Intel SGX SDK’s solution to this problem is to copy the contents of data buffers into and out of enclaves, and have the ECALLs and OCALLs operate on these copies of the original memory buffer. When you pass a pointer into an enclave, you specify in the EDL whether the buffer referenced by the pointer is being pass into the call, out of. Compared to other SGX software stacks, the official Intel SGX SDK seems to be the most secure open-source solution, designed for security and with defenses against Spectre attacks. Azure also exclusivelysupportstheIntelSGXSDKinitscloudenvironment.
An enclave is born when the system software issues the ECREATE
instruction, which turns a free EPC page into the SECS for the new enclave.
ECREATE
copies an SECS structure outside the EPC into an SECS page inside the EPC. The internal structure of SECS is not accessible to software.
Software sets the following fields in the source structure: SECS:BASEADDR
, SECS:SIZE
, and ATTRIBUTES
.
ECREATE
validates the information used to initialize the SECS, and results in a page fault (#PF) or general protection fault (#PF) if the information is not valid.ECREATE
will also fault if SECS target page is in use; already valid; outside the EPC; adresses are not aligned; unused PAGEINFO fields are not zero.
The system software can use EADD
instructions to load the initial code and data into the enclave. EADD
is used to create both TCS pages and regular pages.
This function copies a source page from non-enclave memory into the EPC, associates the EPC page with an SECS page residing in the EPC, and stores the linear address and security attributes in EPCM.
EADD
reads its input data from a Page Information (PAGEINFO) structure.
The PAGEINFO structure contains
LINADDR
)SRCPGE
)SECS
).RT_REG
or PT_TCS
).{: .center-image}
EADD
validates its inputs, and modifies the newly allocated EPC page and its EPCM entry.
EADD
ensures
Simulation implementation might be different from hardware implementation. In simulation mode, ECREATE
allocates all EPC pages via mmap()
.
As SGX simulation simulates SGX behavior by software, it copies EPC page data with virtual address. Hence _EADD()
does not have detailed information about picking a physical EPC page.
SGX simulation code can’t tell EPC page allocation in detail, as it is also a software, so it cannot use the physical address.
To understand actual implementation, I first tried to understand Intel SGX Programming Reference deeply. There is a table explaining inputs for the instruction.
Table. Instruction Operand Encoding{: .center}
Op/En | EAX | RBX | RCX |
---|---|---|---|
IR | EADD (in) | Address of PAGEINFO (in) | Address of the destination EPC page (in) |
As you see the above table, addresses of PAGEINFO and target EPC page should be saved in the register RBX
and RCX
, respectively. The target EPC page is already determined, which means system software is responsible for selecting one. This is also explained in ISCA ‘15 tutorial slide. [link]
{: .center-image width=“600px”}
Then this means I need to search a code that calls EADD
instruction.
Simulation function which calls EADD
instruction is add_enclave_page()
in sdk/simulation/driver_api/driver_api.cpp
. This is a simulation code because it is in simulation
directory.
Then there should be a function with the same name for hardware?
And yes. There is.
Note that PSW (Platform SoftWare) is for actual hardware. It calls ioctl()
, which calls sgx_ioctl_enclave_add_page()
in linux-sgx-driver. linux-sgx-driver is separately provided in [here].
In linux-sgx-driver/isgx_ioctl.c
,
ioctl()
from enclave_creator_hw.cpp
isgx_ioctl_enclave_add_page()
at linux-sgx-driver/isgx_ioctl.c:548
__enclave_add_page()
at linux-sgx-driver/isgx_ioctl.c:440
construct_enclave_page
at linux-sgx-driver/isgx_ioctl.c:122
isgx_alloc_epc_page()
at linux-sgx-driver/isgx_page_cache.c:429
isgx_alloc_epc_page_fast()
at linux-sgx-driver/isgx_page_cache.c:411
isgx Linux SGX driver manages EPC page instances by using the linked list(static LIST_HEAD(isgx_free_list)
at isgx_page_cache.c:25
) and the number of free EPC pages(unsigned int isgx_nr_free_epc_pages
at isgx_page_cache.c:31
).
The type of EPC pages is struct isgx_epc_page
, defined as follows.
Here, pa
is the physical address for the EPC page. How pa
is determined?
Each EPC page that is put into free list is allocated in the function isgx_page_cache_init()
at linux-sgx-driver/isgx_page_cache.c:360
.
Each EPC page has the physical address as start + i
. start
is the first parameter of the function.
The function isgx_page_cache_init()
is called in isgx_init()
at linux-sgx-driver/isgx_main.c:190
.
As shown above, start
is the value of isgx_epc_base
variable.isgx_epc_base
is initialized by the function isgx_init_platform()
at linux-sgx-driver/isgx_main.c:133
.
You can see what isgx_epc_base
value is in your machine, as SGX driver prints it in kernel message buffer by default.
{: .center-image}
My machine tells that 32MiB of EPC is allocated with the physical base memory address 0x80000000
.
All EPC page instance struct isgx_epc_page
has a pa
variable, which contains the physical address of the EPC page, and is initialized as EPC base address + offset.
And, managing page mapping table is also a responsibility of system software. Linux SGX driver insert PTE via calling vm_insert_pfn()
at linux-sgx-driver/isgx_util.c:67
by using epc_page->pa
and expected virtual address enclave_page->addr
as follows.
To be concluded, when a SGX platform is initialized, several EPC page instances (struct isgx_epc_page
) are allocated to represent all EPC pages. System software manages them as a linked list, called isgx_free_list
. When EADD
is called, system software picks a free EPC page instance from the list, and create a page table entry, pointing the physical address that is saved in epc_page->pa
, with the expected virtual address enclave_page->addr
, a part of user enclave’s ELRANGE.
While loading an enclave, the system software will also use the EEXTEND
instruction, which updates the enclave’s measurement used in the software attestation process.
It updates the MRENCLAVE measurement register of an SECS with the measurement of an EXTEND string compromising of “EEXTEND” || ENCLAVEOFFSET || PADDING || 256 bytes of the enclave page.
RCX register contains the effective address of the 256 byte region of an EPC page to be measured.
From Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 3D, Part 4:Section 39.1.2. EADD and EEXTEND Interaction
Software can measure a 256 byte region as determined by the by the developer by invoking EEXTEND. Thus to measure an entire 4KB page, system software must execute EEXTEND 16 times. Each invocation of EEXTEND adds to the cryptographic log information about which region is being measured and the measurement of the section.
This function is the final instruction executed in the enclave build process. After EINIT, the MRENCLAVE measurement is cimplete, and the enclave is ready to start user code execution using EENTER instruction.
When EINIT
completes successfully, it sets the enclave’s INIT attribute to true. This opens the way for ring 3 application software to execute the enclave’s code, using the SGX instructions.
On the other hand, once INIT is set to true, EADD
cannot be invoked on that enclave anymore, so the system software must load all the pages that make up the enclave’s initial state before executing the EINIT
instruction.
All source codes are from Intel SGX SDK Github repository and Intel SGX Linux driver Github repository, released under BSD License 2.0 and GNU General Public License 2.0, respectively.
Copyright (C) 2011-2017 Intel Corporation. All rights reserved.
Redistribution and use in source and binary forms, with or withoutmodification, are permitted provided that the following conditions are met:
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” ANDANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIEDWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AREDISCLAIMED. IN NO EVENT SHALL
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License.