第8章 多处理器管理

  • CHPTER 8 Multiple-processor management
    • 8.1 LOCKED ATOMIC OPERATIONS 锁定的原子操作
      • 8.1.1 Guaranteed Atomic Operations 保证的原子操作
      • 8.1.2 Bus Locking 总线锁定
        • 8.1.2.1 Automatic Locking自动锁定
        • 8.1.2.2 Software Controlled Bus Locking 软件控制总线锁定
    • 8.4 MULTIPLE-PROCESSOR (MP) INITIALIZATION 多处理器(MP)初始化
      • 8.4.1 BSP and AP Processors BSP和AP处理器
      • 8.4.2 MP Initialization Protocol Requirements and RestrictionsMP 初始化协议要求和限制
      • 8.4.3 MP Initialization Protocol Algorithm for MP Systems mp系统的mp初始化协议算法
      • 8.4.4 MP Initialization Example MP初始化示例
        • 8.4.4.1 Typical BSP Initialization Sequence 典型的bsp初始化序列
        • 8.4.4.2 Typical AP Initialization Sequence 典型AP初始化序列
      • 8.4.5 Identifying Logical Processors in an MP System MP系统中逻辑处理器的识别
    • 8.5 INTEL ® HYPER-THREADING TECHNOLOGY AND INTEL ® MULTI-CORE TECHNOLOGY 英特尔超线程技术与英特尔多核技术
    • 8.6 DETECTING HARDWARE MULTI-THREADING SUPPORT AND TOPOLOGY
      • 8.6.1 Initializing Processors Supporting Hyper-Threading Technology 初始化支持超线程技术的处理器
    • 8.7 INTEL ® HYPER-THREADING TECHNOLOGY ARCHITECTURE 英特尔超线程技术体系结构
    • 8.8 MULTI-CORE ARCHITECTURE 多核架构
      • 8.8.1 Logical Processor Support 逻辑处理器支持
      • 8.8.2 Memory Type Range Registers (MTRR) 存储器类型范围寄存器
      • 8.8.3 Performance Monitoring Counters 性能监视计数器
      • 8.8.4 IA32_MISC_ENABLE MSR
      • 8.8.5 Microcode Update Resources 微码更新资源
    • 8.9 PROGRAMMING CONSIDERATIONS FOR HARDWARE MULTI-THREADING CAPABLE PROCESSORS 硬件多线程处理器的编程考虑
      • 8.9.1 Hierarchical Mapping of Shared Resources 共享资源的分层映射
      • 8.9.2 Hierarchical Mapping of CPUID Extended Topology Leaf cpuid扩展拓扑叶的层次映射
      • 8.9.3 Hierarchical ID of Logical Processors in an MP System mp系统中逻辑处理器的层次id
        • 8.9.3.1 Hierarchical ID of Logical Processors with x2APIC ID 具有x2apic id的逻辑处理器的层次id
      • 8.9.5 Identifying Topological Relationships in a MP System mp系统中拓扑关系的识别
    • 8.10 MANAGEMENT OF IDLE AND BLOCKED CONDITIONS 空闲和阻塞条件的管理
    • 8.11 MP INITIALIZATION FOR P6 FAMILY PROCESSORS P6系列处理器的MP初始化

CHPTER 8 Multiple-processor management

The Intel 64 and IA-32 architectures provide mechanisms for managing and improving the performance of multiple processors connected to the same system bus. These include:
Intel 64和IA-32体系结构提供了管理和提高连接到同一系统总线的多个处理器性能的机制。其中包括:

  • Bus locking and/or cache coherency management for performing atomic operations on system memory.
    总线锁定和/或缓存一致性管理,用于对系统内存执行原子操作。

  • Serializing instructions.
    序列化指令。

  • An advance programmable interrupt controller (APIC) located on the processor chip (see Chapter 10, Advanced Programmable Interrupt Controller (APIC)”). This feature was introduced by the Pentium processor.

    位于处理器芯片上的高级可编程中断控制器(APIC)(见第10章,高级可编程中断控制器(APIC))。此功能是由奔腾处理器引入的。

  • A second-level cache (level 2, L2). For the Pentium 4, Intel Xeon, and P6 family processors, the L2 cache is included in the processor package and is tightly coupled to the processor. For the Pentium and Intel486 processors, pins are provided to support an external L2 cache.
    二级缓存(二级,二级)。对于奔腾4、英特尔至强和P6系列处理器,二级缓存包含在处理器包中,并与处理器紧密耦合。对于奔腾和Intel486处理器,提供了管脚以支持外部二级缓存。

  • A third-level cache (level 3, L3). For Intel Xeon processors, the L3 cache is included in the processor package and is tightly coupled to the processor.
    三级缓存(三级,三级)。对于英特尔至强处理器,L3缓存包含在处理器包中,并与处理器紧密耦合。

  • Intel Hyper-Threading Technology. This extension to the Intel 64 and IA-32 architectures enables a single processor core to execute two or more threads concurrently (see Section 8.5, “Intel ® Hyper-Threading Technology and Intel ® Multi-Core Technology”).
    英特尔超线程技术。对Intel 64和IA-32体系结构的扩展使单处理器内核能够同时执行两个或多个线程(请参阅第8.5节“英特尔超线程技术与英特尔多核技术”)。

These mechanisms are particularly useful in symmetric-multiprocessing (SMP) systems. However, they can also be used when an Intel 64 or IA-32 processor and a special-purpose processor (such as a communications, graphics, or video processor) share the system bus.
这些机制在对称多处理(SMP)系统中特别有用。但是,当Intel 64或IA-32处理器与专用处理器(如通信、图形或视频处理器)共享系统总线时,也可以使用它们。

These multiprocessing mechanisms have the following characteristics:
这些多处理机制具有以下特征:

  • To maintain system memory coherency — When two or more processors are attempting simultaneously to access the same address in system memory, some communication mechanism or memory access protocol must be available to promote data coherency and, in some instances, to allow one processor to temporarily lock a memory location.
    要保持系统内存一致性-当两个或多个处理器试图同时访问系统内存中的同一地址时,必须提供一些通信机制或内存访问协议来提高数据一致性,并且在某些情况下,允许一个处理器暂时锁定内存位置。
  • To maintain cache consistency — When one processor accesses data cached on another processor, it must not receive incorrect data. If it modifies data, all other processors that access that data must receive the modified data.
    为了保持缓存一致性-当一个处理器访问另一个处理器上缓存的数据时,它不能接收不正确的数据。如果它修改数据,则访问该数据的所有其他处理器都必须接收修改后的数据。
  • To allow predictable ordering of writes to memory — In some circumstances, it is important that memory writes be observed externally in precisely the same order as programmed.
    为了允许对内存的写操作有可预测的顺序——在某些情况下,在外部观察内存写操作的顺序与编程的顺序完全相同是很重要的。
  • To distribute interrupt handling among a group of processors — When several processors are operating in a system in parallel, it is useful to have a centralized mechanism for receiving interrupts and distributing them to available processors for servicing.
    要在一组处理器之间分配中断处理—当多个处理器在一个系统中并行运行时,有一个集中式机制来接收中断并将其分配给可用的处理器进行服务是很有用的。
  • To increase system performance by exploiting the multi-threaded and multi-process nature of contemporary operating systems and applications.
    利用现代操作系统和应用程序的多线程和多进程特性来提高系统性能。

The caching mechanism and cache consistency of Intel 64 and IA-32 processors are discussed in Chapter 11. The APIC architecture is described in Chapter 10. Bus and memory locking, serializing instructions, memory ordering, and Intel Hyper-Threading Technology are discussed in the following sections.
第11章讨论了intel 64和ia-32处理器的缓存机制和缓存一致性。apic架构在第10章中进行了描述。总线和内存锁定、序列化指令、内存排序和英特尔超线程技术将在以下各节中讨论。

8.1 LOCKED ATOMIC OPERATIONS 锁定的原子操作

The 32-bit IA-32 processors support locked atomic operations on locations in system memory. These operations are typically used to manage shared data structures (such as semaphores, segment descriptors, system segments, or page tables) in which two or more processors may try simultaneously to modify the same field or flag. The processor uses three interdependent mechanisms for carrying out locked atomic operations:
32位IA-32处理器支持对系统内存中的位置执行锁定原子操作。这些操作通常用于管理共享数据结构(如信号量、段描述符、系统段或页表),其中两个或多个处理器可能同时尝试修改同一字段或标志。处理器使用三种相互依赖的机制来执行锁定的原子操作:

  • Guaranteed atomic operations
    保证的原子操作
  • Bus locking, using the LOCK# signal and the LOCK instruction prefix
    总线锁定,使用锁定信号和锁定指令前缀
  • Cache coherency protocols that ensure that atomic operations can be carried out on cached data structures (cache lock); this mechanism is present in the Pentium 4, Intel Xeon, and P6 family processors
    缓存一致性协议,确保可以在缓存数据结构(缓存锁)上执行原子操作;此机制存在于奔腾4、英特尔至强和P6系列处理器中

These mechanisms are interdependent in the following ways. Certain basic memory transactions (such as reading or writing a byte in system memory) are always guaranteed to be handled atomically. That is, once started, the processor guarantees that the operation will be completed before another processor or bus agent is allowed access to the memory location. The processor also supports bus locking for performing selected memory operations (such as a read-modify-write operation in a shared area of memory) that typically need to be handled atomically, but are not automatically handled this way. Because frequently used memory locations are often cached in a processor’s L1 or L2 caches, atomic operations can often be carried out inside a processor’s caches without asserting the bus lock. Here the processor’s cache coherency protocols ensure that other processors that are caching the same memory locations are managed properly while atomic operations are performed on cached memory locations.
这些机制在以下方面相互依存。某些基本的内存事务(例如在系统内存中读或写字节)总是可以保证以原子方式处理。也就是说,一旦启动,处理器保证在允许另一个处理器或总线代理访问内存位置之前完成操作。处理器还支持总线锁定,以执行选定的内存操作(例如内存共享区域中的读-修改-写操作),这些操作通常需要原子处理,但不会以这种方式自动处理。由于经常使用的内存位置通常缓存在处理器的l1或l2缓存中,因此原子操作通常可以在处理器的缓存中执行,而无需断言总线锁。在这里,处理器的缓存一致性协议确保在对缓存的内存位置执行原子操作时,对缓存相同内存位置的其他处理器进行正确的管理。

NOTE 注意
Where there are contested lock accesses, software may need to implement algorithms that ensure fair access to resources in order to prevent lock starvation. The hardware provides no resource that guarantees fairness to participating agents. It is the responsibility of software to manage the fairness of semaphores and exclusive locking functions.
如果存在有争议的锁访问,则软件可能需要实现确保公平访问资源的算法,以防止锁饥饿。硬件不提供保证参与代理公平性的资源。软件的责任是管理信号量的公平性和独占锁定功能。

The mechanisms for handling locked atomic operations have evolved with the complexity of IA-32 processors. More recent IA-32 processors (such as the Pentium 4, Intel Xeon, and P6 family processors) and Intel 64 provide a more refined locking mechanism than earlier processors. These mechanisms are described in the following sections.
处理锁定原子操作的机制随着IA-32处理器的复杂性而发展。较新的IA-32处理器(如奔腾4、英特尔至强和P6系列处理器)和英特尔64提供了比早期处理器更精细的锁定机制。这些机制将在以下各节中介绍。

8.1.1 Guaranteed Atomic Operations 保证的原子操作

The Intel486 processor (and newer processors since) guarantees that the following basic memory operations will always be carried out atomically:
Intel486处理器(以及此后更新的处理器)保证始终以原子方式执行以下基本内存操作:

  • Reading or writing a byte 读取或写入字节
  • Reading or writing a word aligned on a 16-bit boundary 读或写在16位边界上对齐的单词
  • Reading or writing a doubleword aligned on a 32-bit boundary 读或写在32位边界上对齐的双字

The Pentium processor (and newer processors since) guarantees that the following additional memory operations will always be carried out atomically:
奔腾处理器(以及此后更新的处理器)保证始终以原子方式执行以下附加内存操作:

  • Reading or writing a quadword aligned on a 64-bit boundary 读取或写入在64位边界上对齐的四字
  • 16-bit accesses to uncached memory locations that fit within a 32-bit data bus
    16位访问适合32位数据总线的未缓存内存位置

The P6 family processors (and newer processors since) guarantee that the following additional memory operation will always be carried out atomically:
P6系列处理器(以及此后更新的处理器)保证始终以原子方式执行以下附加内存操作:

  • Unaligned 16-, 32-, and 64-bit accesses to cached memory that fit within a cache line
    未对齐的16位、32位和64位访问缓存线中的缓存

Accesses to cacheable memory that are split across cache lines and page boundaries are not guaranteed to be atomic by the Intel Core 2 Duo, Intel ® AtomTM, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, P6 family, Pentium, and Intel486 processors. The Intel Core 2 Duo, Intel Atom, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, and P6 family processors provide bus control signals that permit external memory subsystems to make split accesses atomic; however, nonaligned data accesses will seriously impact the performance of the processor and should be avoided.
英特尔酷睿2双核、英特尔AtomTM、英特尔酷睿双核、奔腾M、奔腾4、英特尔至强、P6系列、奔腾和Intel486处理器不能保证对跨越缓存线和页面边界的可缓存内存的访问是原子的。Intel Core 2 Duo、Intel Atom、Intel Core Duo、Pentium M、Pentium 4、Intel Xeon和P6系列处理器提供总线控制信号,允许外部内存子系统进行原子分离访问;但是,不对齐的数据访问将严重影响处理器的性能,应避免。

An x87 instruction or an SSE instructions that accesses data larger than a quadword may be implemented using multiple memory accesses. If such an instruction stores to memory, some of the accesses may complete (writing to memory) while another causes the operation to fault for architectural reasons (e.g. due an page-table entry that is marked “not present”). In this case, the effects of the completed accesses may be visible to software even though the overall instruction caused a fault. If TLB invalidation has been delayed (see Section 4.10.4.4), such page faults may occur even if all accesses are to the same page.
访问大于四字的数据的x87指令或sse指令可以使用多个内存访问来实现。如果这样的指令存储到内存中,则某些访问可能完成(写入内存),而另一个访问可能由于体系结构原因(例如,由于标记为“不存在”的页表条目)导致操作出错。在这种情况下,即使整个指令导致了错误,软件也可以看到已完成访问的效果。如果TLB失效被延迟(见第4.10.4.4节),即使所有访问都是对同一页的访问,也可能发生这种页错误。

8.1.2 Bus Locking 总线锁定

Intel 64 and IA-32 processors provide a LOCK# signal that is asserted automatically during certain critical memory operations to lock the system bus or equivalent link. While this output signal is asserted, requests from other processors or bus agents for control of the bus are blocked. Software can specify other occasions when the LOCK semantics are to be followed by prepending the LOCK prefix to an instruction.
Intel 64和IA-32处理器提供一个锁定信号,该信号在某些关键内存操作期间自动断言,以锁定系统总线或等效链路。在断言此输出信号时,来自其他处理器或总线代理的控制总线的请求将被阻止。软件可以指定在锁语义后面加上锁前缀的其他情况。

In the case of the Intel386, Intel486, and Pentium processors, explicitly locked instructions will result in the asser-tion of the LOCK# signal. It is the responsibility of the hardware designer to make the LOCK# signal available in system hardware to control memory accesses among processors.
对于Intel386、Intel486和奔腾处理器,显式锁定指令将导致锁定信号的产生。硬件设计者的责任是使系统硬件中的锁信号可用,以控制处理器之间的内存访问。

8.1.2.1 Automatic Locking自动锁定

The operations on which the processor automatically follows the LOCK semantics are as follows:
处理器自动遵循锁语义的操作如下:

  • When executing an XCHG instruction that references memory.
    执行引用内存的xchg指令时。
  • When setting the B (busy) flag of a TSS descriptor — The processor tests and sets the busy flag in the type field of the TSS descriptor when switching to a task. To ensure that two processors do not switch to the same task simultaneously, the processor follows the LOCK semantics while testing and setting this flag.
    设置TSS描述符的B(忙)标志时—处理器在切换到任务时测试并设置TSS描述符的类型字段中的忙标志。为了确保两个处理器不会同时切换到同一任务,处理器在测试和设置此标志时遵循锁语义。
  • When updating segment descriptors — When loading a segment descriptor, the processor will set the accessed flag in the segment descriptor if the flag is clear. During this operation, the processor follows the LOCK semantics so that the descriptor will not be modified by another processor while it is being updated. For this action to be effective, operating-system procedures that update descriptors should use the following steps:
    更新段描述符时-加载段描述符时,如果该标记清除,处理器将在段描述符中设置访问标志。在此操作过程中,处理器遵循锁语义,以便在更新描述符时不会被另一个处理器修改。要使此操作有效,更新描述符的操作系统过程应使用以下步骤:

    • Use a locked operation to modify the access-rights byte to indicate that the segment descriptor is not-present, and specify a value for the type field that indicates that the descriptor is being updated.
      使用锁定操作修改访问权限字节以指示段描述符不存在,并为指示描述符正在更新的类型字段指定一个值。
    • Update the fields of the segment descriptor. (This operation may require several memory accesses; therefore, locked operations cannot be used.)
      更新段描述符的字段。(此操作可能需要多次内存访问;因此,不能使用锁定操作。)
    • Use a locked operation to modify the access-rights byte to indicate that the segment descriptor is valid and present.
      使用锁定操作修改访问权限字节,以指示段描述符有效且存在。
  • The Intel386 processor always updates the accessed flag in the segment descriptor, whether it is clear or not. The Pentium 4, Intel Xeon, P6 family, Pentium, and Intel486 processors only update this flag if it is not already set.
    不管段描述符中的访问标志是否清除,intel386处理器总是更新它。奔腾4、英特尔至强、P6系列、奔腾和Intel486处理器仅在尚未设置此标志时更新此标志。
  • When updating page-directory and page-table entries — When updating page-directory and page-table entries, the processor uses locked cycles to set the accessed and dirty flag in the page-directory and page-table entries.
    更新页目录和页表条目时-更新页目录和页表条目时,处理器使用锁定周期设置页目录和页表条目中的访问和脏标志。
  • Acknowledging interrupts — After an interrupt request, an interrupt controller may use the data bus to send the interrupt’s vector to the processor. The processor follows the LOCK semantics during this time to ensure that no other data appears on the data bus while the vector is being transmitted.
    确认中断-在中断请求之后,中断控制器可以使用数据总线将中断向量发送到处理器。在此期间,处理器遵循锁语义,以确保在传输矢量时数据总线上不会出现其他数据。

8.1.2.2 Software Controlled Bus Locking 软件控制总线锁定

8.4 MULTIPLE-PROCESSOR (MP) INITIALIZATION 多处理器(MP)初始化

The IA-32 architecture (beginning with the P6 family processors) defines a multiple-processor (MP) initialization protocol called the Multiprocessor Specification Version 1.4. This specification defines the boot protocol to be used by IA-32 processors in multiple-processor systems. (Here, multiple processors is defined as two or more processors.) The MP initialization protocol has the following important features:
IA-32体系结构(从P6系列处理器开始)定义了称为多处理器规范1.4版的多处理器(MP)初始化协议。本规范定义了IA-32处理器在多处理器系统中使用的引导协议。(这里,多个处理器定义为两个或多个处理器。)MP初始化协议具有以下重要功能:

  • It supports controlled booting of multiple processors without requiring dedicated system hardware.
    它支持多个处理器的控制引导,而不需要专用的系统硬件。
  • It allows hardware to initiate the booting of a system without the need for a dedicated signal or a predefined boot processor.
    它允许硬件启动系统的引导,而无需专用信号或预定义的引导处理器。
  • It allows all IA-32 processors to be booted in the same manner, including those supporting Intel Hyper-Threading Technology.
    它允许所有IA-32处理器以相同的方式启动,包括那些支持英特尔超线程技术的处理器。
  • The MP initialization protocol also applies to MP systems using Intel 64 processors.
    MP初始化协议也适用于使用Intel 64处理器的MP系统。

The mechanism for carrying out the MP initialization protocol differs depending on the Intel processor generations. The following bullets summarizes the evolution of the changes:
执行MP初始化协议的机制因英特尔处理器的不同代而异。以下项目符号总结了这些变化的演变:

  • For P6 family or older processors supporting MP operations— The selection of the BSP and APs (see Section 8.4.1, “BSP and AP Processors”) is handled through arbitration on the APIC bus, using BIPI and FIPI messages. These processor generations have CPUID signatures of (family=06H, extended_model=0, model<=0DH), or family <06H. See Section 8.11.1, “Overview of the MP Initialization Process For P6 Family Processors” for a complete discussion of MP initialization for P6 family processors.
    对于支持MP操作的P6系列或更早的处理器-选择BSP和AP(请参阅第8.4.1节“BSP和AP处理器”)是通过APIC总线上的仲裁处理的,使用的是BIPI和FIPI消息。这些处理器代具有(family=06h,extended_model=0,model<=0dh)或family<06h的cpuid签名。有关P6系列处理器的MP初始化的完整讨论,请参阅第8.11.1节“P6系列处理器的MP初始化过程概述”。
  • Early generations of IA processors with family 0FH — The selection of the BSP and APs (see Section 8.4.1, “BSP and AP Processors”) is handled through arbitration on the system bus, using BIPI and FIPI messages (see Section 8.4.3, “MP Initialization Protocol Algorithm for MP Systems”). These processor generations have CPUID signatures of family=0FH, model=0H, stepping<=09H.
    具有0FH系列的早期IA处理器-BSP和AP的选择(见第8.4.1节“BSP和AP处理器”)通过系统总线上的仲裁,使用BIPI和FIPI消息进行处理(见第8.4.3节“MP系统的MP初始化协议算法”)。这些处理器代具有cpuid签名,家族=0fh,型号=0h,步进<=09h。
  • Later generations of IA processors with family 0FH, and IA processors with system bus — The selection of the BSP and APs is handled through a special system bus cycle, without using BIPI and FIPI message arbitration (see Section 8.4.3, “MP Initialization Protocol Algorithm for MP Systems”). These processor generations have CPUID signatures of family=0FH with (model=0H, stepping>=0AH) or (model >0, all steppings); or family=06H, extended_model=0, model>=0EH.
    具有0FH系列的下一代IA处理器和具有系统总线的IA处理器-BSP和AP的选择通过特殊的系统总线周期来处理,而不使用BIPI和FIPI消息仲裁(见第8.4.3节,“MP系统的MP初始化协议算法”)。这些处理器代具有cpuid签名:family=0fh(model=0h,stepping>=0ah)或(model>0,all steppings);family=06h,extended_model=0,model>=0eh。
  • All other modern IA processor generations supporting MP operations— The selection of the BSP and APs in the system is handled by platform-specific arrangement of the combination of hardware, BIOS, and/or configuration input options. The basis of the selection mechanism is similar to those of the Later generations of family 0FH and other Intel processor using system bus (see Section 8.4.3, “MP Initialization Protocol Algorithm for MP Systems”). These processor generations have CPUID signatures of family=06H, extended_model>0.
    所有其他支持MP操作的现代IA处理器代—系统中BSP和AP的选择由硬件、BIOS和/或配置输入选项组合的特定平台安排来处理。选择机制的基础类似于0FH系列和其他使用系统总线的Intel处理器的后世(见第8.4.3节“MP系统的MP初始化协议算法”)。这些处理器代具有cpuid签名,家族=06h,扩展的_模型>0。

The family, model, and stepping ID for a processor is given in the EAX register when the CPUID instruction is executed with a value of 1 in the EAX register.
当cpuid指令在eax寄存器中的值为1时,处理器的系列、型号和步进id在eax寄存器中给出。

8.4.1 BSP and AP Processors BSP和AP处理器

The MP initialization protocol defines two classes of processors: the bootstrap processor (BSP) and the application processors (APs). Following a power-up or RESET of an MP system, system hardware dynamically selects one of the processors on the system bus as the BSP. The remaining processors are designated as APs.
mp初始化协议定义了两类处理器:引导处理器(bsp)和应用处理器(aps)。MP系统通电或复位后,系统硬件动态选择系统总线上的一个处理器作为BSP。其余的处理器被指定为ap。

As part of the BSP selection mechanism, the BSP flag is set in the IA32_APIC_BASE MSR (see Figure 10-5) of the BSP, indicating that it is the BSP. This flag is cleared for all other processors.
作为bsp选择机制的一部分,bsp标志设置在bsp的ia32_apic_base msr(见图10-5)中,表示它是bsp。对于所有其他处理器,此标志被清除。

The BSP executes the BIOS’s boot-strap code to configure the APIC environment, sets up system-wide data structures, and starts and initializes the APs. When the BSP and APs are initialized, the BSP then begins executing the operating-system initialization code.
bsp执行bios的引导带代码来配置apic环境,设置系统范围的数据结构,并启动和初始化ap。当bsp和ap初始化时,bsp开始执行操作系统初始化代码。

Following a power-up or reset, the APs complete a minimal self-configuration, then wait for a startup signal (a SIPI message) from the BSP processor. Upon receiving a SIPI message, an AP executes the BIOS AP configuration code, which ends with the AP being placed in halt state.
在通电或复位之后,ap完成最小的自我配置,然后等待来自bsp处理器的启动信号(sipi消息)。在接收到sipi消息时,ap执行bios ap配置代码,该代码以ap处于halt状态结束。

For Intel 64 and IA-32 processors supporting Intel Hyper-Threading Technology, the MP initialization protocol treats each of the logical processors on the system bus or coherent link domain as a separate processor (with a unique APIC ID). During boot-up, one of the logical processors is selected as the BSP and the remainder of the logical processors are designated as APs.
对于支持英特尔超线程技术的英特尔64与IA-32处理器,MP初始化协议将系统总线或相干链路域上的每个逻辑处理器视为单独的处理器(具有唯一的APIC ID)。在引导期间,选择其中一个逻辑处理器作为bsp,其余逻辑处理器指定为ap。

8.4.2 MP Initialization Protocol Requirements and RestrictionsMP 初始化协议要求和限制

The MP initialization protocol imposes the following requirements and restrictions on the system:
MP初始化协议对系统施加以下要求和限制:

  • The MP protocol is executed only after a power-up or RESET. If the MP protocol has completed and a BSP is chosen, subsequent INITs (either to a specific processor or system wide) do not cause the MP protocol to be repeated. Instead, each logical processor examines its BSP flag (in the IA32_APIC_BASE MSR) to determine whether it should execute the BIOS boot-strap code (if it is the BSP) or enter a wait-for-SIPI state (if it is an AP).
    MP协议仅在通电或复位后执行。如果MP协议已完成并选择了BSP,则后续初始化(到特定处理器或系统范围内)不会导致MP协议重复。相反,每个逻辑处理器都会检查其bsp标志(在ia32 apic_base msr中),以确定是执行bios引导带代码(如果是bsp)还是进入等待sipi状态(如果是ap)。
  • All devices in the system that are capable of delivering interrupts to the processors must be inhibited from doing so for the duration of the MP initialization protocol. The time during which interrupts must be inhibited includes the window between when the BSP issues an INIT-SIPI-SIPI sequence to an AP and when the AP responds to the last SIPI in the sequence.
    在MP初始化协议期间,必须禁止系统中能够向处理器传送中断的所有设备这样做。必须禁止中断的时间包括bsp向ap发出init-sipi-sipi序列与ap响应序列中最后一个sipi之间的窗口。

8.4.3 MP Initialization Protocol Algorithm for MP Systems mp系统的mp初始化协议算法

Following a power-up or RESET of an MP system, the processors in the system execute the MP initialization protocol algorithm to initialize each of the logical processors on the system bus or coherent link domain. In the course of executing this algorithm, the following boot-up and initialization operations are carried out:
MP系统通电或复位后,系统中的处理器执行MP初始化协议算法,以初始化系统总线或相干链路域上的每个逻辑处理器。在执行此算法的过程中,将执行以下启动和初始化操作:

  1. Each logical processor is assigned a unique APIC ID, based on system topology. The unique ID is a 32-bit value if the processor supports CPUID leaf 0BH, otherwise the unique ID is an 8-bit value. (see Section 8.4.5, “Identifying Logical Processors in an MP System”).
    每个逻辑处理器都根据系统拓扑结构分配一个唯一的APIC ID。如果处理器支持cpuid leaf 0bh,则unique id是32位值,否则unique id是8位值。(见第8.4.5节,“识别MP系统中的逻辑处理器”)。
  2. Each logical processor is assigned a unique arbitration priority based on its APIC ID.
    每个逻辑处理器根据其apic id被分配一个唯一的仲裁优先级。
  3. Each logical processor executes its internal BIST simultaneously with the other logical processors in the system.
    每个逻辑处理器与系统中的其他逻辑处理器同时执行其内部BIST。
  4. Upon completion of the BIST, the logical processors use a hardware-defined selection mechanism to select the BSP and the APs from the available logical processors on the system bus. The BSP selection mechanism differs depending on the family, model, and stepping IDs of the processors, as follows:
    bist完成后,逻辑处理器使用硬件定义的选择机制从系统总线上可用的逻辑处理器中选择bsp和ap。BSP选择机制因处理器的系列、型号和步进ID而异,如下所示:
  • Later generations of IA processors within family 0FH (see Section 8.4), IA processors with system bus (family=06H, extended_model=0, model>=0EH), or all other modern Intel processors (family=06H, extended_model>0):
    系列0FH(见第8.4节)、带系统总线的IA处理器(系列=06H,扩展U型号=0,型号>=0eh)或所有其他现代英特尔处理器(系列=06H,扩展U型号>0)中的下一代IA处理器:

    • The logical processors begin monitoring the BNR# signal, which is toggling. When the BNR# pin stops toggling, each processor attempts to issue a NOP special cycle on the system bus.
      逻辑处理器开始监视正在切换的bnr信号。当bnr引脚停止切换时,每个处理器都试图在系统总线上发出nop特殊周期。
    • The logical processor with the highest arbitration priority succeeds in issuing a NOP special cycle and is nominated the BSP. This processor sets the BSP flag in its IA32_APIC_BASE MSR, then fetches and begins executing BIOS boot-strap code, beginning at the reset vector (physical address FFFF FFF0H).
      具有最高仲裁优先级的逻辑处理器成功地发出nop特殊周期,并被指定为bsp。此处理器在其ia32 apic_base msr中设置bsp标志,然后从重置向量(物理地址ffff fff0h)开始获取并开始执行bios引导带代码。
    • The remaining logical processors (that failed in issuing a NOP special cycle) are designated as APs. They leave their BSP flags in the clear state and enter a “wait-for-SIPI state.”
      其余的逻辑处理器(无法发出nop特殊周期)被指定为ap。它们将bsp标志保留在clear状态,并输入“wait for sipi state”。
  • Early generations of IA processors within family 0FH (family=0FH, model=0H, stepping<=09H), P6 family or older processors supporting MP operations (family=06H, extended_model=0, model<=0DH; or family <06H):
    系列0FH(系列=0FH,型号=0H,步进<=09H)、P6系列或更旧的支持MP操作的处理器(系列=06H,扩展U型号=0,型号<=0DH;或系列<06H)中的早期IA处理器:

    • Each processor broadcasts a BIPI to “all including self.” The first processor that broadcasts a BIPI (and thus receives its own BIPI vector), selects itself as the BSP and sets the BSP flag in its IA32_APIC_BASE MSR. (See Section 8.11.1, “Overview of the MP Initialization Process For P6 Family Processors” for a description of the BIPI, FIPI, and SIPI messages.)
      每个处理器将一个bipi广播到“all including self”。广播bipi的第一个处理器(从而接收自己的bipi向量),选择自己作为bsp,并在其ia32_apic_base msr中设置bsp标志。(有关BIPI、FIPI和SIPI消息的说明,请参阅第8.11.1节“P6系列处理器的MP初始化过程概述”。)
    • The remainder of the processors (which were not selected as the BSP) are designated as APs. They leave their BSP flags in the clear state and enter a “wait-for-SIPI state.”
      其余处理器(未被选作bsp)被指定为ap。它们将bsp标志保留在clear状态,并输入“wait for sipi state”。
    • The newly established BSP broadcasts an FIPI message to “all including self,” which the BSP and APs treat as an end of MP initialization signal. Only the processor with its BSP flag set responds to the FIPI message. It responds by fetching and executing the BIOS boot-strap code, beginning at the reset vector (physical address FFFF FFF0H).
      新建立的bsp向“all including self”广播fipi消息,bsp和ap将其视为mp初始化信号的结束。只有设置了bsp标志的处理器才响应fipi消息。它通过从重置向量(物理地址ffff fff0h)开始获取并执行bios引导带代码来响应。
  1. As part of the boot-strap code, the BSP creates an ACPI table and/or an MP table and adds its initial APIC ID to these tables as appropriate.
    作为引导带代码的一部分,bsp创建一个acpi表和/或一个mp表,并根据需要将其初始apic id添加到这些表中。
  2. At the end of the boot-strap procedure, the BSP sets a processor counter to 1, then broadcasts a SIPI message to all the APs in the system. Here, the SIPI message contains a vector to the BIOS AP initialization code (at 000VV000H, where VV is the vector contained in the SIPI message).
    在引导程序结束时,bsp将处理器计数器设置为1,然后向系统中的所有ap广播sipi消息。这里,sipi消息包含bios ap初始化代码的向量(在000vv000h,其中vv是sipi消息中包含的向量)。
  3. The first action of the AP initialization code is to set up a race (among the APs) to a BIOS initialization semaphore. The first AP to the semaphore begins executing the initialization code. (See Section 8.4.4, “MP Initialization Example,” for semaphore implementation details.) As part of the AP initialization procedure, the AP adds its APIC ID number to the ACPI and/or MP tables as appropriate and increments the processor counter by 1. At the completion of the initialization procedure, the AP executes a CLI instruction and halts itself.
    ap初始化代码的第一个操作是设置bios初始化信号量的竞争(在ap之间)。信号量的第一个ap开始执行初始化代码。(有关信号量实现的详细信息,请参阅第8.4.4节“MP初始化示例”。)作为AP初始化过程的一部分,AP将其APIC ID号酌情添加到ACPI和/或MP表中,并将处理器计数器增加1。初始化过程完成后,ap执行cli指令并停止自身。
  4. When each of the APs has gained access to the semaphore and executed the AP initialization code, the BSP establishes a count for the number of processors connected to the system bus, completes executing the BIOS boot-strap code, and then begins executing operating-system boot-strap and start-up code.
    当每个ap都获得对信号量的访问并执行ap初始化代码时,bsp建立连接到系统总线的处理器数量的计数,完成执行bios引导带代码,然后开始执行操作系统引导带和启动代码。
  5. While the BSP is executing operating-system boot-strap and start-up code, the APs remain in the halted state. In this state they will respond only to INITs, NMIs, and SMIs. They will also respond to snoops and to assertions of the STPCLK# pin.
    当bsp执行操作系统引导程序和启动代码时,ap仍处于暂停状态。在这种状态下,它们将只响应inits、nmis和smis。他们还将回应窥探和STPCLK引脚的断言。

The following section gives an example (with code) of the MP initialization protocol for of multiple processors operating in an MP configuration.
以下部分给出了在MP配置中运行的多个处理器的MP初始化协议的示例(带代码)。

Chapter 2, “Model-Specific Registers (MSRs)” in the Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 4 describes how to program the LINT[0:1] pins of the processor’s local APICs after an MP configuration has been completed.
《英特尔64与IA-32体系结构软件开发人员手册》第4卷第2章“型号专用寄存器(MSR)”介绍了在MP配置完成后,如何编程处理器本地APIC的lint[0:1]管脚。

8.4.4 MP Initialization Example MP初始化示例

The following example illustrates the use of the MP initialization protocol used to initialize processors in an MP system after the BSP and APs have been established. The code runs on Intel 64 or IA-32 processors that use a protocol. This includes P6 Family processors, Pentium 4 processors, Intel Core Duo, Intel Core 2 Duo and Intel Xeon processors.
下面的示例说明了在建立了bsp和aps之后使用mp初始化协议来初始化mp系统中的处理器。代码运行在使用协议的英特尔64或IA-32处理器上。这包括P6系列处理器、奔腾4处理器、Intel Core Duo、Intel Core 2 Duo和Intel Xeon处理器。

The following constants and data definitions are used in the accompanying code examples. They are based on the addresses of the APIC registers defined in Table 10-1.
下面的常量和数据定义将在附带的代码示例中使用。它们基于表10-1中定义的APIC寄存器的地址。

ICR_LOW                                  EQU 0FEE00300H
SVR                                            EQU 0FEE000F0H
APIC_ID                                    EQU 0FEE00020H
LVT3                                           EQU 0FEE00370H
APIC_ENABLED                     EQU 0100H
BOOT_ID                                  DD ?
COUNT                                      EQU 00H
VACANT                                    EQU 00H

8.4.4.1 Typical BSP Initialization Sequence 典型的bsp初始化序列

After the BSP and APs have been selected (by means of a hardware protocol, see Section 8.4.3, “MP Initialization Protocol Algorithm for MP Systems”), the BSP begins executing BIOS boot-strap code (POST) at the normal IA-32 architecture starting address (FFFF FFF0H). The boot-strap code typically performs the following operations:
在选择了bsp和ap之后(通过硬件协议,参见第8.4.3节“mp系统的mp初始化协议算法”),bsp开始在正常的ia-32架构起始地址(ffff fff0h)执行bios引导带代码(post)。引导带代码通常执行以下操作:

  1. Initializes memory.
    初始化内存。

  2. Loads the microcode update into the processor.
    将微码更新加载到处理器中。

  3. Initializes the MTRRs.
    初始化mtrrs。

  4. Enables the caches.
    启用缓存。

  5. Executes the CPUID instruction with a value of 0H in the EAX register, then reads the EBX, ECX, and EDX registers to determine if the BSP is “GenuineIntel.”
    在eax寄存器中执行值为0h的cpuid指令,然后读取ebx、ecx和edx寄存器,以确定bsp是否为“genuineintel”。

  6. Executes the CPUID instruction with a value of 1H in the EAX register, then saves the values in the EAX, ECX, and EDX registers in a system configuration space in RAM for use later.
    在eax寄存器中执行值为1h的cpuid指令,然后将eax、ecx和edx寄存器中的值保存在ram的系统配置空间中,供以后使用。

  7. Loads start-up code for the AP to execute into a 4-KByte page in the lower 1 MByte of memory.
    加载AP的启动代码,以便在较低的1 MB内存中执行到4 KB的页中。

  8. Switches to protected mode and ensures that the APIC address space is mapped to the strong uncacheable (UC) memory type.
    切换到保护模式并确保APIC地址空间映射到强不可缓存(UC)内存类型。

  9. Determine the BSP’s APIC ID from the local APIC ID register (default is 0), the code snippet below is an example that applies to logical processors in a system whose local APIC units operate in xAPIC mode that APIC registers are accessed using memory mapped interface:
    从本地apic id寄存器确定bsp的apic id(默认值为0),下面的代码片段是一个应用于系统中的逻辑处理器的示例,该系统的本地apic单元以xapic模式运行,apic寄存器使用内存映射接口访问:

MOV ESI, APIC_ID; Address of local APIC ID register
MOV EAX, [ESI];
AND EAX, 0FF000000H; Zero out all other bits except APIC ID
MOV BOOT_ID, EAX; Save in memory

Saves the APIC ID in the ACPI and/or MP tables and optionally in the system configuration space in RAM.
将apic id保存在acpi和/或mp表中,也可以选择保存在ram的系统配置空间中。

  1. Converts the base address of the 4-KByte page for the AP’s bootup code into 8-bit vector. The 8-bit vector defines the address of a 4-KByte page in the real-address mode address space (1-MByte space). For example, a vector of 0BDH specifies a start-up memory address of 000BD000H.
    将AP启动代码的4-kbyte页的基址转换为8位向量。8位向量定义实际地址模式地址空间(1-mbyte空间)中4-kbyte页的地址。例如,向量0bdh指定启动内存地址000bd000h。

  2. Enables the local APIC by setting bit 8 of the APIC spurious vector register (SVR).
    通过设置apic伪向量寄存器(svr)的位8启用局部apic。

    MOV ESI, SVR; Address of SVR
    MOV EAX, [ESI];
    OR EAX, APIC_ENABLED; Set bit 8 to enable (0 on reset)
    MOV [ESI], EAX;
    
  3. Sets up the LVT error handling entry by establishing an 8-bit vector for the APIC error handler.
    通过为APIC错误处理程序建立8位向量来设置LVT错误处理项。

    MOV ESI, LVT3;
    MOV EAX, [ESI];
    AND EAX, FFFFFF00H; Clear out previous vector.
    OR EAX, 000000xxH; xx is the 8-bit vector the APIC error handler.
    MOV [ESI], EAX;
    
  4. Initializes the Lock Semaphore variable VACANT to 00H. The APs use this semaphore to determine the order in which they execute BIOS AP initialization code.
    将锁信号量变量初始化为空的00h。ap使用此信号量来确定它们执行bios ap初始化代码的顺序。

  5. Performs the following operation to set up the BSP to detect the presence of APs in the system and the number of processors (within a finite duration, minimally 100 milliseconds):
    执行以下操作以设置bsp以检测系统中是否存在ap和处理器数量(在有限的持续时间内,至少100毫秒):

    • Sets the value of the COUNT variable to 1.
      将count变量的值设置为1。

    • In the AP BIOS initialization code, the AP will increment the COUNT variable to indicate its presence. The finite duration while waiting for the COUNT to be updated can be accomplished with a timer. When the timer expires, the BSP checks the value of the COUNT variable. If the timer expires and the COUNT variable has not been incremented, no APs are present or some error has occurred.
      在ap bios初始化代码中,ap将递增count变量以指示其存在。等待计数更新的有限持续时间可以通过计时器完成。当计时器过期时,bsp检查count变量的值。如果计时器过期且计数变量未递增,则不存在任何AP或发生了某些错误。

  6. Broadcasts an INIT-SIPI-SIPI IPI sequence to the APs to wake them up and initialize them. If software knows how many logical processors it expects to wake up, it may choose to poll the COUNT variable. If the expected processors show up before the 100 millisecond timer expires, the timer can be canceled and skip to step 16. The left-hand-side of the procedure illustrated in Table 8-1 provides an algorithm when the expected processor count is unknown. The right-hand-side of Table 8-1 can be used when the expected processor count is known.
    将init-sipi-sipi ipi序列广播到ap以唤醒它们并初始化它们。如果软件知道它希望唤醒多少个逻辑处理器,它可以选择轮询count变量。如果预期的处理器在100毫秒计时器过期之前出现,则可以取消计时器并跳到步骤16。当期望的处理器计数未知时,表8-1所示过程的左侧提供了一个算法。当预期的处理器计数已知时,可以使用表8-1的右侧。

    Table 8-1. Broadcast INIT-SIPI-SIPI Sequence and Choice of Timeouts

  7. Reads and evaluates the COUNT variable and establishes a processor count.
    读取并计算count变量并建立处理器计数。

  8. If necessary, reconfigures the APIC and continues with the remaining system diagnostics as appropriate.
    如有必要,重新配置APIC,并继续进行剩余的系统诊断(视情况而定)。

8.4.4.2 Typical AP Initialization Sequence 典型AP初始化序列

When an AP receives the SIPI, it begins executing BIOS AP initialization code at the vector encoded in the SIPI. The AP initialization code typically performs the following operations:
当ap接收到sipi时,它开始在sipi中编码的向量处执行bios ap初始化代码。ap初始化代码通常执行以下操作:

  1. Waits on the BIOS initialization Lock Semaphore. When control of the semaphore is attained, initialization continues.
    等待BIOS初始化锁定信号量。当达到对信号量的控制时,初始化将继续。
  2. Loads the microcode update into the processor.
    将微码更新加载到处理器中。
  3. Initializes the MTRRs (using the same mapping that was used for the BSP).
    初始化mtrrs(使用与bsp相同的映射)。
  4. Enables the cache.
    启用缓存。
  5. Executes the CPUID instruction with a value of 0H in the EAX register, then reads the EBX, ECX, and EDX registers to determine if the AP is “GenuineIntel.”
    在eax寄存器中执行值为0h的cpuid指令,然后读取ebx、ecx和edx寄存器,以确定ap是否为“genuineintel”。
  6. Executes the CPUID instruction with a value of 1H in the EAX register, then saves the values in the EAX, ECX, and EDX registers in a system configuration space in RAM for use later.
    在eax寄存器中执行值为1h的cpuid指令,然后将eax、ecx和edx寄存器中的值保存在ram的系统配置空间中,供以后使用。
  7. Switches to protected mode and ensures that the APIC address space is mapped to the strong uncacheable (UC) memory type.
    切换到保护模式并确保APIC地址空间映射到强不可缓存(UC)内存类型。
  8. Determines the AP’s APIC ID from the local APIC ID register, and adds it to the MP and ACPI tables and optionally to the system configuration space in RAM.
    从本地apic id寄存器确定ap的apic id,并将其添加到mp和acpi表中,还可以选择添加到ram中的系统配置空间中。
  9. Initializes and configures the local APIC by setting bit 8 in the SVR register and setting up the LVT3 (error LVT) for error handling (as described in steps 9 and 10 in Section 8.4.4.1, “Typical BSP Initialization Sequence”).
    通过在SVR寄存器中设置位8并设置用于错误处理的LVT3(错误LVT)来初始化和配置本地APIC(如第8.4.4.1节“典型BSP初始化序列”中的步骤9和10所述)。
  10. Configures the APs SMI execution environment. (Each AP and the BSP must have a different SMBASE address.)
    配置APS SMI执行环境。(每个ap和bsp必须具有不同的smbase地址。)
  11. Increments the COUNT variable by 1.
    将count变量增加1。
  12. Releases the semaphore.
    释放信号量。
  13. Executes one of the following:
    执行下列操作之一:

    • the CLI and HLT instructions (if MONITOR/MWAIT is not supported), or
      cli和hlt指令(如果不支持monitor/mwait),或
    • the CLI, M ONITOR and MWAIT sequence to enter a deep C-state.
      进入深c状态的cli、m onitor和mwait序列。
  14. Waits for an INIT IPI.
    等待初始IPI。

8.4.5 Identifying Logical Processors in an MP System MP系统中逻辑处理器的识别

After the BIOS has completed the MP initialization protocol, each logical processor can be uniquely identified by its local APIC ID. Software can access these APIC IDs in either of the following ways:
BIOS完成MP初始化协议后,每个逻辑处理器都可以通过其本地APIC ID唯一标识。软件可以通过以下任一方式访问这些APIC ID:

  • Read APIC ID for a local APIC — Code running on a logical processor can read APIC ID in one of two ways depending on the local APIC unit is operating in x2APIC mode (see Intel® 64 Architecture x2APIC Specification)or in xAPIC mode:
    读取本地APIC的APIC ID-逻辑处理器上运行的代码可以通过以下两种方式之一读取APIC ID,具体取决于本地APIC单元是在X2APIC模式(请参阅英特尔64体系结构X2APIC规范)下运行还是在Xapic模式下运行:

    • If the local APIC unit supports x2APIC and is operating in x2APIC mode, 32-bit APIC ID can be read by executing a RDMSR instruction to read the processor’s x2APIC ID register. This method is equivalent to executing CPUID leaf 0BH described below.
      如果本地apic单元支持x2apic并在x2apic模式下运行,则可以通过执行rdmsr指令读取处理器的x2apic id寄存器来读取32位apic id。此方法相当于执行下面描述的cpuid leaf 0bh。
    • If the local APIC unit is operating in xAPIC mode, 8-bit APIC ID can be read by executing a MOV instruction to read the processor’s local APIC ID register (see Section 10.4.6, “Local APIC ID”). This is the ID to use for directing physical destination mode interrupts to the processor.
      如果本地apic单元在xapic模式下运行,则可以通过执行mov指令读取处理器的本地apic id寄存器来读取8位apic id(请参阅第10.4.6节“本地apic id”)。这是用于将物理目标模式中断定向到处理器的ID。
  • Read ACPI or MP table — As part of the MP initialization protocol, the BIOS creates an ACPI table and an MP table. These tables are defined in the Multiprocessor Specification Version 1.4 and provide software with a list of the processors in the system and their local APIC IDs. The format of the ACPI table is derived from the ACPI specification, which is an industry standard power management and platform configuration specification for MP systems.
    读取ACPI或MP表-作为MP初始化协议的一部分,BIOS创建一个ACPI表和一个MP表。这些表在多处理器规范1.4版中定义,并为软件提供系统中处理器及其本地APIC ID的列表。acpi表的格式源自acpi规范,acpi规范是mp系统的行业标准电源管理和平台配置规范。
  • Read Initial APIC ID (If the processor does not support CPUID leaf 0BH) — An APIC ID is assigned to a logical processor during power up. This is the initial APIC ID reported by CPUID.1:EBX[31:24] and may be different from the current value read from the local APIC. The initial APIC ID can be used to determine the topological relationship between logical processors for multi-processor systems that do not support CPUID leaf 0BH. Bits in the 8-bit initial APIC ID can be interpreted using several bit masks. Each bit mask can be used to extract an identifier to represent a hierarchical level of the multi-threading resource topology in an MP system (See Section 8.9.1, “Hierarchical Mapping of Shared Resources”). The initial APIC ID may consist of up to four bit-fields. In a non-clustered MP system, the field consists of up to three bit fields.
    读取初始APIC ID(如果处理器不支持cpuid leaf 0bh)-在通电期间,APIC ID被分配给逻辑处理器。这是cpuid.1:ebx[31:24]报告的初始apic id,可能与从本地apic读取的当前值不同。初始apic id可用于确定不支持cpuid leaf 0bh的多处理器系统的逻辑处理器之间的拓扑关系。8位初始apic id中的位可以使用多个位掩码进行解释。每个位掩码可用于提取标识符,以表示MP系统中多线程资源拓扑的层次级别(请参阅第8.9.1节“共享资源的层次映射”)。初始apic id最多可包含四个位字段。在非群集MP系统中,字段最多由三个位字段组成。
  • Read 32-bit APIC ID from CPUID leaf 0BH (If the processor supports CPUID leaf 0BH) — A unique APIC ID is assigned to a logical processor during power up. This APIC ID is reported by CPUID.0BH:EDX[31:0] as a 32-bit value. Use the 32-bit APIC ID and CPUID leaf 0BH to determine the topological relationship between logical processors if the processor supports CPUID leaf 0BH. Bits in the 32-bit x2APIC ID can be extracted into sub-fields using CPUID leaf 0BH parameters. (See Section 8.9.1, “Hierarchical Mapping of Shared Resources”).
    从cpuid leaf 0bh中读取32位apic id(如果处理器支持cpuid leaf 0bh)-在通电期间为逻辑处理器分配唯一的apic id。此APIC ID由cpuid.0bh:edx[31:0]报告为32位值。如果处理器支持cpuid leaf 0bh,则使用32位apic id和cpuid leaf 0bh确定逻辑处理器之间的拓扑关系。可以使用cpuid leaf 0bh参数将32位x2apic id中的位提取到子字段中。(见第8.9.1节,“共享资源的分层映射”)。

Figure 8-2 shows two examples of APIC ID bit fields in earlier single-core processors. In single-core Intel Xeon processors, the APIC ID assigned to a logical processor during power-up and initialization is 8 bits. Bits 2:1 form a 2-bit physical package identifier (which can also be thought of as a socket identifier). In systems that configure physical processors in clusters, bits 4:3 form a 2-bit cluster ID. Bit 0 is used in the Intel Xeon processor MP to identify the two logical processors within the package (see Section 8.9.3, “Hierarchical ID of Logical Processors in an MP System”). For Intel Xeon processors that do not support Intel Hyper-Threading Technology, bit 0 is always set to 0; for Intel Xeon processors supporting Intel Hyper-Threading Technology, bit 0 performs the same function as it does for Intel Xeon processor MP.
图8-2显示了早期单核处理器中APIC ID位字段的两个示例。在单核Intel Xeon处理器中,在通电和初始化期间分配给逻辑处理器的APIC ID为8位。位2:1构成2位物理包标识符(也可以认为是套接字标识符)。在群集配置物理处理器的系统中,位4:3构成2位群集ID。“英特尔至强”处理器MP中使用位0来标识包中的两个逻辑处理器(请参阅第8.9.3节“MP系统中逻辑处理器的层次ID”)。对于不支持英特尔超线程技术的英特尔至强处理器,位0始终设置为0;对于支持英特尔超线程技术的英特尔至强处理器,位0执行与英特尔至强处理器MP相同的功能。

For more recent multi-core processors, see Section 8.9.1, “Hierarchical Mapping of Shared Resources” for a complete description of the topological relationships between logical processors and bit field locations within an initial APIC ID across Intel 64 and IA-32 processor families.
有关最新的多核处理器,请参阅第8.9.1节“共享资源的分层映射”,以获取有关英特尔64与IA-32处理器系列中初始APIC ID内逻辑处理器与位字段位置之间拓扑关系的完整描述。

Note the number of bit fields and the width of bit-fields are dependent on processor and platform hardware capabilities. Software should determine these at runtime. When initial APIC IDs are assigned to logical processors, the value of APIC ID assigned to a logical processor will respect the bit-field boundaries corresponding core, physical package, etc. Additional examples of the bit fields in the initial APIC ID of multi-threading capable systems are shown in Section 8.9.
注意,位字段的数量和位字段的宽度取决于处理器和平台硬件功能。软件应该在运行时确定这些。当初始apic id被分配给逻辑处理器时,分配给逻辑处理器的apic id的值将尊重对应于核心、物理包等的位字段边界。支持多线程系统的初始apic id中的位字段的附加示例如第8.9节所示。
For P6 family processors, the APIC ID that is assigned to a processor during power-up and initialization is 4 bits (see Figure 8-2). Here, bits 0 and 1 form a 2-bit processor (or socket) identifier and bits 2 and 3 form a 2-bit cluster ID.
对于P6系列处理器,在通电和初始化期间分配给处理器的APIC ID为4位(见图8-2)。这里,位0和1构成2位处理器(或套接字)标识符,位2和3构成2位集群id。

8.5 INTEL ® HYPER-THREADING TECHNOLOGY AND INTEL ® MULTI-CORE TECHNOLOGY 英特尔超线程技术与英特尔多核技术

Intel Hyper-Threading Technology and Intel multi-core technology are extensions to Intel 64 and IA-32 architectures that enable a single physical processor to execute two or more separate code streams (called threads) concurrently. In Intel Hyper-Threading Technology, a single processor core provides two logical processors that share execution resources (see Section 8.7, “Intel ® Hyper-Threading Technology Architecture”). In Intel multi-core technology, a physical processor package provides two or more processor cores. Both configurations require chip-sets and a BIOS that support the technologies.
“英特尔超线程技术”和“英特尔多核技术”是英特尔64与IA-32体系结构的扩展,使单个物理处理器能够同时执行两个或多个独立的代码流(称为线程)。在英特尔超线程技术中,单处理器内核提供两个共享执行资源的逻辑处理器(请参阅第8.7节“英特尔超线程技术体系结构”)。在英特尔多核技术中,物理处理器包提供两个或多个处理器核。这两种配置都需要支持这些技术的芯片组和bios。

Software should not rely on processor names to determine whether a processor supports Intel Hyper-Threading Technology or Intel multi-core technology. Use the CPUID instruction to determine processor capability (see Section 8.6.2, “Initializing Multi-Core Processors”).
软件不应依赖处理器名称来确定处理器是否支持“英特尔超线程技术”或“英特尔多核技术”。使用cpuid指令确定处理器能力(请参阅第8.6.2节“初始化多核处理器”)。

8.6 DETECTING HARDWARE MULTI-THREADING SUPPORT AND TOPOLOGY

Use the CPUID instruction to detect the presence of hardware multi-threading support in a physical processor. Hardware multi-threading can support several varieties of multigrade and/or Intel Hyper-Threading Technology. CPUID instruction provides several sets of parameter information to aid software enumerating topology information. The relevant topology enumeration parameters provided by CPUID include:
使用cpuid指令检测物理处理器中是否存在硬件多线程支持。硬件多线程可以支持多种多级和/或英特尔超线程技术。cpuid指令提供多组参数信息,以帮助软件枚举拓扑信息。cpuid提供的相关拓扑枚举参数包括:

  • **Hardware Multi-Threading feature flag (CPUID.1:EDX[28] = 1) **— Indicates when set that the physical package is capable of supporting Intel Hyper-Threading Technology and/or multiple cores.
    硬件多线程功能标志(cpuid.1:edx[28]=1)—当设置为物理包能够支持“英特尔超线程技术”和/或多核时表示。
  • Processor topology enumeration parameters for 8-bit APIC ID:
    8位APIC ID的处理器拓扑枚举参数:

    • Addressable IDs for Logical processors in the same Package (CPUID.1:EBX[23:16]) — Indicates the maximum number of addressable ID for logical processors in a physical package. Within a physical package, there may be addressable IDs that are not occupied by any logical processors. This parameter does not represents the hardware capability of the physical processor.
      同一软件包中的逻辑处理器的可寻址ID(CPUID 1:EBX[23∶16])表示物理包中逻辑处理器的可寻址ID的最大数目。在物理包中,可能有任何逻辑处理器都不占用的可寻址标识。此参数不表示物理处理器的硬件功能。
  • Addressable IDs for processor cores in the same Package 7 (CPUID.(EAX=4, ECX=0 8 ):EAX[31:26] + 1 = Y) — Indicates the maximum number of addressable IDs attributable to processor cores (Y) in the physical package.
    同一包7中的处理器内核的可寻址ID(CPUID((= 4,8 = 0)):表示物理包中的处理器核心(Y)的可寻址ID的最大数量(IDS)1。
  • Extended Processor Topology Enumeration parameters for 32-bit APIC ID: Intel 64 processors supporting CPUID leaf 0BH will assign unique APIC IDs to each logical processor in the system. CPUID leaf 0BH reports the 32-bit APIC ID and provide topology enumeration parameters. See CPUID instruction reference pages in Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 2A.
    32位APIC ID的扩展处理器拓扑枚举参数:支持CPuid叶0bh的英特尔64处理器将为系统中的每个逻辑处理器分配唯一的APIC ID。cpuid leaf 0bh报告32位apic id并提供拓扑枚举参数。请参阅英特尔64与IA-32体系结构软件开发人员手册第2A卷中的CPuid指令参考页。

The CPUID feature flag may indicate support for hardware multi-threading when only one logical processor available in the package. In this case, the decimal value represented by bits 16 through 23 in the EBX register will have a value of 1.
当包中只有一个逻辑处理器可用时,cpuid特性标志可能表示支持硬件多线程。在这种情况下,由ebx寄存器中的位16到23表示的十进制值将具有1的值。

Software should note that the number of logical processors enabled by system software may be less than the value of “Addressable IDs for Logical processors”. Similarly, the number of cores enabled by system software may be less than the value of “Addressable IDs for processor cores”.
软件应注意,由系统软件启用的逻辑处理器的数量可能小于“逻辑处理器的可寻址id”的值。类似地,由系统软件启用的核心数可以小于“处理器核心的可寻址id”的值。

Software can detect the availability of the CPUID extended topology enumeration leaf (0BH) by performing two steps:
软件可以通过执行两个步骤检测CPuid扩展拓扑枚举叶(0bh)的可用性:

  • Check maximum input value for basic CPUID information by executing CPUID with EAX= 0. If CPUID.0H:EAX is greater than or equal or 11 (0BH), then proceed to next step,
    通过使用EAX=0执行CPUID检查基本CPUID信息的最大输入值。如果cpuid.0h:eax大于或等于11(0bh),则继续下一步,
  • Check CPUID.EAX=0BH, ECX=0H:EBX is non-zero.
    检查cpuid.eax=0bh,ecx=0h:ebx为非零。

If both of the above conditions are true, extended topology enumeration leaf is available. Note the presence of CPUID leaf 0BH in a processor does not guarantee support that the local APIC supports x2APIC. If CPUID.(EAX=0BH, ECX=0H):EBX returns zero and maximum input value for basic CPUID information is greater than 0BH, then CPUID.0BH leaf is not supported on that processor.
如果上述两个条件均为真,则扩展拓扑枚举叶可用。注意,处理器中存在cpuid leaf 0bh并不保证本地apic支持x2apic。如果CPUID(EAX= 0BH,ECX= 0H):EBX返回零,并且基本CPUID信息的最大输入值大于0BH,那么处理器上不支持CPUID.0BH叶。

8.6.1 Initializing Processors Supporting Hyper-Threading Technology 初始化支持超线程技术的处理器

8.7 INTEL ® HYPER-THREADING TECHNOLOGY ARCHITECTURE 英特尔超线程技术体系结构

8.8 MULTI-CORE ARCHITECTURE 多核架构

This section describes the architecture of Intel 64 and IA-32 processors supporting dual-core and quad-core technology. The discussion is applicable to the Intel Pentium processor Extreme Edition, Pentium D, Intel Core Duo, Intel Core 2 Duo, Dual-core Intel Xeon processor, Intel Core 2 Quad processors, and quad-core Intel Xeon processors. Features vary across different microarchitectures and are detectable using CPUID.
本节介绍支持双核和四核技术的Intel 64和IA-32处理器的体系结构。此讨论适用于英特尔奔腾处理器极限版、奔腾D、英特尔酷睿双核、英特尔酷睿2双核、英特尔至强双核、英特尔酷睿2四核以及英特尔至强四核处理器。特性在不同的微体系结构中不同,并且可以使用cpuid检测。

In general, each processor core has dedicated microarchitectural resources identical to a single-processor implementation of the underlying microarchitecture without hardware multi-threading capability. Each logical processor in a dual-core processor (whether supporting Intel Hyper-Threading Technology or not) has its own APIC functionality, PAT, machine check architecture, debug registers and extensions. Each logical processor handles serialization instructions or self-modifying code on its own. Memory order is handled the same way as in Intel Hyper-Threading Technology.
一般来说,每个处理器核心都有专用的微体系结构资源,与底层微体系结构的单处理器实现相同,但没有硬件多线程功能。双核处理器中的每个逻辑处理器(无论是否支持英特尔超线程技术)都有自己的APIC功能、PAT、机器检查体系结构、调试寄存器和扩展。每个逻辑处理器单独处理序列化指令或自修改代码。内存顺序的处理方式与英特尔超线程技术相同。

The topology of the cache hierarchy (with respect to whether a given cache level is shared by one or more processor cores or by all logical processors in the physical package) depends on the processor implementation. Software must use the deterministic cache parameter leaf of CPUID instruction to discover the cache-sharing topology between the logical processors in a multi-threading environment.
缓存层次结构的拓扑结构(关于给定的缓存级别是由一个或多个处理器核心共享还是由物理包中的所有逻辑处理器共享)取决于处理器实现。软件必须使用cpuid指令的deterministic cache参数leaf来发现多线程环境中逻辑处理器之间的缓存共享拓扑。

8.8.1 Logical Processor Support 逻辑处理器支持

The topological composition of processor cores and logical processors in a multi-core processor can be discovered using CPUID. Within each processor core, one or more logical processors may be available.
利用cpuid可以发现多核处理器中处理器核和逻辑处理器的拓扑结构。在每个处理器核心中,可以有一个或多个逻辑处理器可用。

System software must follow the requirement MP initialization sequences (see Section 8.4, “Multiple-Processor (MP) Initialization”) to recognize and enable logical processors. At runtime, software can enumerate those logical processors enabled by system software to identify the topological relationships between these logical processors. (See Section 8.9.5, “Identifying Topological Relationships in a MP System”).
系统软件必须遵循MP初始化序列要求(见第8.4节“多处理器(MP)初始化”),以识别和启用逻辑处理器。在运行时,软件可以枚举那些由系统软件启用的逻辑处理器,以识别这些逻辑处理器之间的拓扑关系。(见第8.9.5节,“识别MP系统中的拓扑关系”)。

8.8.2 Memory Type Range Registers (MTRR) 存储器类型范围寄存器

MTRR is shared between two logical processors sharing a processor core if the physical processor supports Intel Hyper-Threading Technology. MTRR is not shared between logical processors located in different cores or different physical packages.
如果物理处理器支持英特尔超线程技术,则在共享处理器核心的两个逻辑处理器之间共享MTRR。MTRR不在位于不同内核或不同物理包中的逻辑处理器之间共享。

The Intel 64 and IA-32 architectures require that all logical processors in an MP system use an identical MTRR memory map. This gives software a consistent view of memory, independent of the processor on which it is running.
Intel 64和IA-32体系结构要求MP系统中的所有逻辑处理器使用相同的MTRR内存映射。这给软件一个一致的内存视图,独立于运行它的处理器。

See Section 11.11, “Memory Type Range Registers (MTRRs).”

8.8.3 Performance Monitoring Counters 性能监视计数器

Performance counters and their companion control MSRs are shared between two logical processors sharing a processor core if the processor core supports Intel Hyper-Threading Technology and is based on Intel NetBurst microarchitecture. They are not shared between logical processors in different cores or different physical packages. As a result, software must manage the use of these resources, based on the topology of performance monitoring resources. Performance counter interrupts, events, and precise event monitoring support can be set up and allocated on a per thread (per logical processor) basis.
如果处理器核心支持英特尔超线程技术并基于英特尔NetBurst微体系结构,则性能计数器及其相应的控制MSR在共享处理器核心的两个逻辑处理器之间共享。它们不在不同内核或不同物理包中的逻辑处理器之间共享。因此,软件必须根据性能监视资源的拓扑结构来管理这些资源的使用。可以基于每个线程(每个逻辑处理器)设置和分配性能计数器中断、事件和精确的事件监视支持。

See Section 18.6.4, “Performance Monitoring and Intel Hyper-Threading Technology in Processors Based on Intel NetBurst ® Microarchitecture.”
请参阅第18.6.4节,“基于英特尔NetBurst微体系结构的处理器中的性能监视与英特尔超线程技术”。

8.8.4 IA32_MISC_ENABLE MSR

Some bit fields in IA32_MISC_ENABLE MSR (MSR address 1A0H) may be shared between two logical processors sharing a processor core, or may be shared between different cores in a physical processor. See Chapter 2, “Model-Specific Registers (MSRs)” in the Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 4.
ia32_misc_enable msr(msr address 1a0h)中的某些位字段可以在共享处理器核心的两个逻辑处理器之间共享,也可以在物理处理器中的不同核心之间共享。请参阅“英特尔64与IA-32体系结构软件开发人员手册”第4卷第2章“特定型号寄存器(MSR)”。

8.8.5 Microcode Update Resources 微码更新资源

Microcode update facilities are shared between two logical processors sharing a processor core if the physical package supports Intel Hyper-Threading Technology. They are not shared between logical processors in different cores or different physical packages. Either logical processor that has access to the microcode update facility can initiate an update.
如果物理包支持“英特尔超线程技术”,则在共享处理器核心的两个逻辑处理器之间共享微码更新功能。它们不在不同内核或不同物理包中的逻辑处理器之间共享。有权访问微码更新工具的任何一个逻辑处理器都可以启动更新。

Each logical processor has its own BIOS signature MSR (IA32_BIOS_SIGN_ID at MSR address 8BH). When a logical processor performs an update for the physical processor, the IA32_BIOS_SIGN_ID MSRs for resident logical processors are updated with identical information.
每个逻辑处理器都有自己的bios签名msr(msr地址8bh处的ia32 bios签名id)。当逻辑处理器对物理处理器执行更新时,常驻逻辑处理器的ia32 bios符号msr将使用相同的信息进行更新。

All microcode update steps during processor initialization should use the same update data on all cores in all physical packages of the same stepping. Any subsequent microcode update must apply consistent update data to all cores in all physical packages of the same stepping. If the processor detects an attempt to load an older microcode update when a newer microcode update had previously been loaded, it may reject the older update to stay with the newer update.
处理器初始化期间的所有微码更新步骤应在相同步骤的所有物理包中的所有核心上使用相同的更新数据。任何后续的微码更新都必须将一致的更新数据应用于同一步进的所有物理包中的所有核心。如果处理器在先前加载较新的微码更新时检测到尝试加载较旧的微码更新,则它可能会拒绝较旧的更新以保留较新的更新。

NOTE 注意
Some processors (prior to the introduction of Intel 64 Architecture and based on Intel NetBurst microarchitecture) do not support simultaneous loading of microcode update to the sibling logical processors in the same core. All other processors support logical processors initiating an update simultaneously. Intel recommends a common approach that the microcode loader use the sequential technique described in Section 9.11.6.3.
某些处理器(在引入英特尔64体系结构之前,基于英特尔NetBurst微体系结构)不支持将微码更新同时加载到同一内核中的同级逻辑处理器。所有其他处理器都支持同时启动更新的逻辑处理器。英特尔推荐一种常见的方法,即微码加载器使用第9.11.6.3节中描述的顺序技术。

8.9 PROGRAMMING CONSIDERATIONS FOR HARDWARE MULTI-THREADING CAPABLE PROCESSORS 硬件多线程处理器的编程考虑

In a multi-threading environment, there may be certain hardware resources that are physically shared at some level of the hardware topology. In the multi-processor systems, typically bus and memory sub-systems are physically shared between multiple sockets. Within a hardware multi-threading capable processors, certain resources are provided for each processor core, while other resources may be provided for each logical processors (see Section 8.7, “Intel ® Hyper-Threading Technology Architecture,” and Section 8.8, “Multi-Core Architecture”).
在多线程环境中,某些硬件资源可能在硬件拓扑的某个级别上物理共享。在多处理器系统中,总线和内存子系统通常在多个套接字之间物理共享。在支持硬件多线程的处理器中,为每个处理器核心提供某些资源,而为每个逻辑处理器提供其他资源(请参阅第8.7节“英特尔超线程技术体系结构”和第8.8节“多核体系结构”)。

From a software programming perspective, control transfer of processor operation is managed at the granularity of logical processor (operating systems dispatch a runnable task by allocating an available logical processor on the platform). To manage the topology of shared resources in a multi-threading environment, it may be useful for software to understand and manage resources that are shared by more than one logical processors.
从软件编程的角度来看,处理器操作的控制传输是在逻辑处理器的粒度上进行管理的(操作系统通过在平台上分配一个可用的逻辑处理器来分派一个可运行的任务)。要在多线程环境中管理共享资源的拓扑结构,软件可能需要了解和管理由多个逻辑处理器共享的资源。

8.9.1 Hierarchical Mapping of Shared Resources 共享资源的分层映射

The APIC_ID value associated with each logical processor in a multi-processor system is unique (see Section 8.6, “Detecting Hardware Multi-Threading Support and Topology”). This 8-bit or 32-bit value can be decomposed into sub-fields, where each sub-field corresponds a hierarchical level of the topological mapping of hardware resources.
与多处理器系统中的每个逻辑处理器关联的apic_id值是唯一的(请参阅第8.6节“检测硬件多线程支持和拓扑”)。这个8位或32位的值可以分解成子字段,其中每个子字段对应硬件资源拓扑映射的层次。

The decomposition of an APIC_ID may consist of several sub fields representing the topology within a physical processor package, the higher-order bits of an APIC ID may also be used by cluster vendors to represent the topology of cluster nodes of each coherent multiprocessor systems:
apic_id的分解可以由表示物理处理器包内拓扑的多个子字段组成,集群供应商也可以使用apic id的高阶位来表示每个相关多处理器系统的集群节点的拓扑:

  • Cluster — Some multi-threading environments consists of multiple clusters of multi-processor systems. The CLUSTER_ID sub-field is usually supported by vendor firmware to distinguish different clusters. For non-clustered systems, CLUSTER_ID is usually 0 and system topology is reduced.
    群集-一些多线程环境由多个多处理器系统群集组成。供应商固件通常支持cluster_id子字段来区分不同的集群。对于非集群系统,集群id通常为0,并且减少了系统拓扑。
  • Package — A physical processor package mates with a socket. A package may contain one or more software visible die. The PACKAGE_ID sub-field distinguishes different physical packages within a cluster.
    package-物理处理器包与套接字匹配。一个包可以包含一个或多个软件可见模具。package_id子字段区分集群中的不同物理包。
  • Die — A software-visible chip inside a package. The DIE_ID sub-field distinguishes different die within a package. If there are no software visible die, the width of this bit field is 0.
    die-软件包中的可见芯片。“模具ID”子字段用于区分包装中的不同模具。如果没有软件可见的模具,则此位字段的宽度为0。
  • Tile — A set of cores, possibly within modules, that share certain resources. The TILE_ID sub-field distinguishes different tiles. If there are no software visible tiles, the width of this bit field is 0.
    tile-一组核心,可能在模块内,共享某些资源。“平铺ID”子字段区分不同的平铺。如果没有软件可见的分幅,则此位字段的宽度为0。
  • Module — A set of cores that share certain resources. The MODULE_ID sub-field distinguishes different modules. If there are no software visible modules, the width of this bit field is 0.
    模块-共享某些资源的一组核心。module_id子字段区分不同的模块。如果没有软件可见模块,则此位字段的宽度为0。
  • Core — Processor cores may be contained within modules, within tiles, on software-visible die, or appear directly at the package level. The CORE_ID sub-field distinguishes processor cores. For a single-core processor, the width of this bit field is 0.
    核心-处理器核心可以包含在模块内、分块内、软件可见模具上,或者直接出现在包级别。core_id子字段用于区分处理器核心。对于单核处理器,此位字段的宽度为0。
  • SMT — A processor core provides one or more logical processors sharing execution resources. The SMT_ID sub-field distinguishes logical processors in a core. The width of this bit field is non-zero if a processor core provides more than one logical processors.
    SMT-处理器核心提供一个或多个共享执行资源的逻辑处理器。smt_id子字段区分核心中的逻辑处理器。如果处理器核心提供多个逻辑处理器,则此位字段的宽度为非零。

SMT and CORE sub-fields are bit-wise contiguous in the APIC_ID field (see Figure 8-5).
smt和core子字段在apic_id字段中按位连续(见图8-5)。
If the processor supports CPUID leaf 0BH and leaf 1FH, the 32-bit APIC ID can represent cluster plus several levels of topology within the physical processor package. The exact number of hierarchical levels within a physical processor package must be enumerated through CPUID leaf 0BH and leaf 1FH. Common processor families may employ topology similar to that represented by 8-bit Initial APIC ID. In general, CPUID leaf 0BH and leaf 1FH can support a topology enumeration algorithm that decompose a 32-bit APIC ID into more than four sub-fields (see Figure 8-6).
如果处理器支持cpuid leaf 0bh和leaf 1fh,则32位apic id可以表示集群以及物理处理器包中的多个拓扑级别。物理处理器包中层次结构级别的确切数量必须通过cpuid leaf 0bh和leaf 1fh枚举。公共处理器系列可能采用与8位初始APIC ID表示的拓扑相似的拓扑。通常,cpuid leaf 0bh和leaf 1fh可以支持将32位APIC ID分解为四个子字段以上的拓扑枚举算法(见图8-6)。

NOTE
CPUID leaf 0BH and leaf 1FH can have differences in the number of level types reported (CPUID leaf 1FH defines additional level types). If the processor supports CPUID leaf 1FH, usage of this leaf is preferred over leaf 0BH. CPUID leaf 0BH is available for legacy compatibility going forward.
cpuid leaf 0bh和leaf 1fh在报告的级别类型数量上可能存在差异(cpuid leaf 1fh定义了其他级别类型)。如果处理器支持cpuid leaf 1fh,则优先使用此leaf而不是leaf 0bh。cpuid叶0bh可用于以后的旧版兼容性。

The width of each sub-field depends on hardware and software configurations. Field widths can be determined at runtime using the algorithm discussed below (Example 8-16 through Example 8-21).
每个子字段的宽度取决于硬件和软件配置。可以使用下面讨论的算法在运行时确定字段宽度(示例8-16到示例8-21)。

Figure 8-6 depicts the relationships of three of the hierarchical sub-fields in a hypothetical MP system. The value of valid APIC_IDs need not be contiguous across package boundary or core boundaries.
图8-6描述了假设MP系统中三个层次子字段的关系。有效apic_id的值不必跨包边界或核心边界连续。

8.9.2 Hierarchical Mapping of CPUID Extended Topology Leaf cpuid扩展拓扑叶的层次映射

CPUID leaf 0BH and leaf 1FH provide enumeration parameters for software to identify each hierarchy of the processor topology in a deterministic manner. Each hierarchical level of the topology starting from the SMT level is represented numerically by a sub-leaf index within the CPUID 0BH leaf and 1FH leaf. Each level of the topology is mapped to a sub-field in the APIC ID, following the general relationship depicted in Figure 8-6. This mechanism allows software to query the exact number of levels within a physical processor package and the bit-width of each sub-field of x2APIC ID directly. For example,
cpuid leaf 0bh和leaf 1fh为软件提供枚举参数,以确定的方式标识处理器拓扑的每个层次结构。从smt级别开始的拓扑的每个层次级别都由cpuid 0bh叶和1fh叶中的子叶索引以数字形式表示。拓扑的每个级别都映射到APIC ID中的子字段,遵循图8-6所示的一般关系。此机制允许软件直接查询物理处理器包中的确切级别数和x2apic id的每个子字段的位宽度。例如,

  • Starting from sub-leaf index 0 and incrementing ECX until CPUID.(EAX=0BH or 1FH, ECX=N):ECX[15:8] returns an invalid “level type” encoding. The number of levels within the physical processor package is “N” (excluding PACKAGE). Using Figure 8-6 as an example, CPUID.(EAX=0BH or 1FH, ECX=4):ECX[15:8] will report 00H, indicating sub leaf 04H is invalid. This is also depicted by a pseudo code example:
    从子叶索引0开始,递增ecx直到cpuid。(eax=0bh或1fh,ecx=n):ecx[15:8]返回无效的“级别类型”编码。物理处理器包中的级别数为“n”(不包括包)。以图8-6为例,cpuid.(eax=0bh或1fh,ecx=4):ecx[15:8]将报告00h,指示子叶04h无效。伪代码示例也描述了这一点:

Example 8-16. Number of Levels Below the Physical Processor Package
例 8-16. 低于物理处理器包的级别数

Byte type = 1;s = 0;While ( type ) {EAX = 0BH or 1FH; // query each sub leaf of CPUID leaf 0BH or 1FH; CPUID leaf 1FH is preferred over leaf 0BH if availableECX = s;CPUID;type = ECX[15:8]; // examine level type encodings ++;}
N = ECX[7:0];
  • Sub-leaf index 0 (ECX= 0 as input) provides enumeration parameters to extract the SMT sub-field of x2APIC ID. If EAX = 0BH or 1FH, and ECX =0 is specified as input when executing CPUID, CPUID.(EAX=0BH or 1FH, ECX=0):EAX[4:0] reports a value (a right-shift count) that allow software to extract part of x2APIC ID to distinguish the next higher topological entities above the SMT level. This value also corresponds to the bit-width of the sub-field of x2APIC ID corresponding the hierarchical level with sub-leaf index 0.
    子叶索引0(ecx=0作为输入)提供枚举参数以提取x2apic id的smt子字段。如果在执行cpuid、cpuid时eax=0bh或1fh,并且ecx=0指定为输入(eax=0bh或1fh,ecx=0:eax[4:0]报告一个值(右移计数),该值允许软件提取x2apic id的一部分,以区分smt级别以上的下一个更高拓扑实体。该值还对应于x2apic id的子字段的位宽度,该子字段对应于具有子叶索引0的层次级别。

  • For each subsequent higher sub-leaf index m, CPUID.(EAX=0BH or 1FH, ECX=m):EAX[4:0] reports the right-shift count that will allow software to extract part of x2APIC ID to distinguish higher-level topological entities. This means the right-shift value at of sub-leaf m, corresponds to the least significant (m+1) subfields of the 32-bit x2APIC ID.
    对于随后的每个较高的子叶索引m,cpuid。(eax=0bh或1fh,ecx=m):eax[4:0]报告右移位计数,这将允许软件提取x2apic id的一部分以区分较高级别的拓扑实体。这意味着子叶m的右移位值at对应于32位x2apic id的最低有效(m+1)子字段。

    For m = 0, m < N, m ++;
    { cumulative_width[m] = CPUID.(EAX=0BH or 1FH, ECX= m): EAX[4:0]; }
    BitWidth[0] = cumulative_width[0];
    For m = 1, m < N, m ++;BitWidth[m] = cumulative_width[m] - cumulative_width[m-1];
    

    NOTE
    CPUID leaf 1FH is a preferred superset to leaf 0BH. Leaf 1FH defines additional level types, and it must be parsed by an algorithm that can handle the addition of future level types.
    cpuid leaf 1fh是叶0bh的首选超集。leaf 1fh定义了额外的级别类型,它必须由能够处理未来级别类型的添加的算法来解析。

Previously, only the following encoding of hierarchical level types were defined: 0 (invalid), 1 (SMT), and 2 (core). With the additional hierarchical level types available (see Section 8.9.1, “Hierarchical Mapping of Shared Resources” and Figure 8-5, “Generalized Seven Level Interpretation of the APIC ID” ) software must not assume any “level type” encoding value to be related to any sub-leaf index, except sub-leaf 0.
以前,只定义了以下层次级别类型的编码:0(无效)、1(SMT)和2(核心)。有了额外的层次级别类型可用(见第8.9.1节,“共享资源的层次映射”和图8-5,“APIC ID的通用七级解释”)软件不得假设任何“级别类型”编码值与任何子叶索引相关,子叶0除外。

Example 8-18. Support Routines for Identifying Package, Die, Core and Logical Processors from 32-bit x2APIC ID
例8-18。支持从32位x2apic id识别包、芯片、核心和逻辑处理器的例程

a. Derive the extraction bitmask for logical processors in a processor core and associated mask offset for different cores.
a.导出处理器内核中逻辑处理器的提取位掩码以及不同内核的相关掩码偏移量。

//
// This example shows how to enumerate CPU topology level types (level types may or may not be known/supported by the software)
//
// Below is the list of sample level types used in the example.
// Refer to the CPUID Leaf 1FH definition for the actual level type numbers: “V2 Extended Topology Enumeration Leaf” .
//
// SMT
// CORE
// MODULE
// TILE
// DIE
// PACKAGE
//
// The example shows how to identify and derive the extraction bitmask for the levels with identify type SMT/CORE/DIE/PACKAGE
//
int DeriveSMT_Mask_Offsets (void)
{IF (!HWMTSupported()) return -1;execute cpuid with EAX = 0BH or 1FH, ECX = 0;IF (returned level type encoding in EXC[15:8] does not match SMT) return -1;Mask_SMT_shift = EAX[4:0];//# bits shift right of APIC ID to distinguish different coresSMT_MASK =~( (-1) << Mask_SMT_shift);//shift left to derive extraction bitmask for SMT_IDreturn 0;
}

b. Derive the extraction bitmask for processor cores in a physical processor package and associated mask offset for different packages.
b.导出物理处理器包中处理器核心的提取位掩码以及不同包的相关掩码偏移量。

int DeriveCore_Mask_Offsets (void)
{IF (!HWMTSupported()) return -1;execute cpuid with EAX = 0BH or 1FH, ECX = 0;WHILE( ECX[15:8] ) {                                                    //level type encoding is validMask_last_known_shift = EAX[4:0]IF (returned level type encoding in ECX[15:8] matches CORE) {Mask_Core_shift = EAX[4:0];}ELSE IF (returned level type encoding in ECX[15:8] matches DIE {Mask_Die_shift = EAX[4:0];}//// Keep enumerating. Check if the next level is the desired level and if not, keep enumerating until you reach a known level 继续枚举。检查下一个级别是否为所需级别,如果不是,则继续枚举,直到达到已知级别// or the invalid level (“0” level type). If there are more levels between DIE and PACKAGE, the unknown levels will be ignored// and treated as an extension of the last known level (i.e., DIE in this case).或无效级别(“0”级别类型)。如果模具和包装之间有更多级别,未知级别将被忽略//ECX++;execute cpuid with EAX = 0BH or 1FH;}COREPlusSMT_MASK = ~( (-1) << Mask_Core_shift);DIEPlusCORE_MASK = ~( (-1) << Mask_Die_shift);//// Treat levels between DIE and physical package as an extension of DIE for software choosing not to implement or recognize these unknown levels.将die和物理包之间的级别视为die的扩展,以便软件选择不实现或识别这些未知级别。//CORE_MASK = COREPlusSMT_MASK ^ SMT_MASK;DIE_MASK = DIEPlusCORE_MASK ^ COREPlusSMT_MASK;PACKAGE_MASK = (-1) << Mask_last_known_shift;return -1;
}

8.9.3 Hierarchical ID of Logical Processors in an MP System mp系统中逻辑处理器的层次id

For Intel 64 and IA-32 processors, system hardware establishes an 8-bit initial APIC ID (or 32-bit APIC ID if the processor supports CPUID leaf 0BH) that is unique for each logical processor following power-up or RESET (see Section 8.6.1). Each logical processor on the system is allocated an initial APIC ID. BIOS may implement features that tell the OS to support less than the total number of logical processors on the system bus. Those logical processors that are not available to applications at runtime are halted during the OS boot process. As a result, the number valid local APIC_IDs that can be queried by affinitizing-current-thread-context (See Example 8-23) is limited to the number of logical processors enabled at runtime by the OS boot process.
对于Intel 64和IA-32处理器,系统硬件建立8位初始APIC ID(或32位APIC ID,如果处理器支持CPuid叶0bh),该ID在通电或重置后对每个逻辑处理器都是唯一的(请参阅第8.6.1节)。系统上的每个逻辑处理器都分配了一个初始APIC ID。BIOS可以实现一些功能,这些功能告诉操作系统支持的逻辑处理器少于系统总线上的逻辑处理器总数。那些在运行时不可用于应用程序的逻辑处理器在操作系统引导过程中被暂停。因此,可以通过关联当前线程上下文(参见示例8-23)来查询的有效本地apic_id的数量被限制为在运行时由os引导进程启用的逻辑处理器的数量。

Table 8-2 shows an example of the 8-bit APIC IDs that are initially reported for logical processors in a system with four Intel Xeon MP processors that support Intel Hyper-Threading Technology (a total of 8 logical processors, each physical package has two processor cores and supports Intel Hyper-Threading Technology). Of the two logical processors within a Intel Xeon processor MP, logical processor 0 is designated the primary logical processor and logical processor 1 as the secondary logical processor.
表8-2显示了一个8位APIC ID的示例,在一个具有四个支持英特尔超线程技术的英特尔至强MP处理器(总共8个逻辑处理器,每个物理包有两个处理器内核,支持英特尔超线程技术)的系统中,逻辑处理器最初会报告这些APIC ID。在英特尔至强处理器MP中的两个逻辑处理器中,逻辑处理器0被指定为主逻辑处理器,逻辑处理器1被指定为辅助逻辑处理器。

  1. Because information on the number of processor cores in a physical package was not available in early single-core processors supporting Intel Hyper-Threading Technology, the core ID can be treated as 0.
    由于在早期支持“英特尔超线程技术”的单核处理器中,物理包中的处理器内核数信息不可用,因此内核ID可以被视为0。

Table 8-3 shows the initial APIC IDs for a hypothetical situation with a dual processor system. Each physical package providing two processor cores, and each processor core also supporting Intel Hyper-Threading Technology.
表8-3显示了双处理器系统假设情况下的初始APIC ID。每个物理包提供两个处理器核,每个处理器核还支持英特尔超线程技术。

8.9.3.1 Hierarchical ID of Logical Processors with x2APIC ID 具有x2apic id的逻辑处理器的层次id

Table 8-4 shows an example of possible x2APIC ID assignments for a dual processor system that support x2APIC. Each physical package providing four processor cores, and each processor core also supporting Intel Hyper-Threading Technology. Note that the x2APIC ID need not be contiguous in the system.
表8-4显示了支持X2APIC的双处理器系统的可能X2APIC ID分配示例。每个物理包提供四个处理器核,每个处理器核还支持英特尔超线程技术。注意,系统中的x2apic id不需要是连续的。
### 8.9.4 Algorithm for Three-Level Mappings of APIC_ID apic_id的三层映射算法

Software can gather the initial APIC_IDs for each logical processor supported by the operating system at runtime 9 and extract identifiers corresponding to the three levels of sharing topology (package, core, and SMT). The three-level algorithms below focus on a non-clustered MP system for simplicity. They do not assume APIC IDs are contiguous or that all logical processors on the platform are enabled.
软件可以在运行时9为操作系统支持的每个逻辑处理器收集初始apic_id,并提取对应于三个共享拓扑级别(包、核心和smt)的标识符。为了简单起见,下面的三级算法将重点放在非集群mp系统上。它们不假定apic id是连续的,也不假定平台上的所有逻辑处理器都已启用。

Intel supports multi-threading systems where all physical processors report identical values in CPUID leaf 0BH, CPUID.1:EBX[23:16]), CPUID.4 10 :EAX[31:26], and CPUID.4 11 :EAX[25:14]. The algorithms below assume the target system has symmetry across physical package boundaries with respect to the number of logical processors per package, number of cores per package, and cache topology within a package.
英特尔支持多线程系统,所有物理处理器在cpuid leaf 0bh、cpuid.1:ebx[23:16])、cpuid.4 10:eax[31:26]和cpuid.4 11:eax[25:14]中报告相同的值。下面的算法假设目标系统在每个包的逻辑处理器数、每个包的核心数和包内的缓存拓扑方面具有跨越物理包边界的对称性。

Software can choose to assume three level hierarchy if it was developed to understand only three levels. However, software implementation needs to ensure it does not break if it runs on systems that have more levels in the hierarchy even if it does not recognize them.
如果软件开发时只理解三个层次,则可以选择采用三级层次结构。但是,软件实现需要确保它在层次结构中具有更多级别的系统上运行时不会中断,即使它无法识别这些级别。

The extraction algorithm (for three-level mappings from an APIC ID) uses the general procedure depicted in Example 8-19, and is supplemented by more detailed descriptions on the derivation of topology enumeration parameters for extraction bit masks:
提取算法(对于来自apic id的三级映射)使用示例8-19中描述的一般过程,并辅以关于提取位掩码的拓扑枚举参数的推导的更详细描述:

  1. Detect hardware multi-threading support in the processor.
    检测处理器中的硬件多线程支持。

  2. Derive a set of bit masks that can extract the sub ID of each hierarchical level of the topology. The algorithm to derive extraction bit masks for SMT_ID/CORE_ID/PACKAGE_ID differs based on APIC ID is 32-bit (see step 3 below) or 8-bit (see step 4 below):
    派生一组位掩码,这些位掩码可以提取拓扑的每个层次级别的子ID。导出smt_id/core_id/package_id的提取位掩码的算法根据apic id的不同是32位(请参阅下面的步骤3)或8位(请参阅下面的步骤4):

  3. If the processor supports CPUID leaf 0BH, each APIC ID contains a 32-bit value, the topology enumeration parameters needed to derive three-level extraction bit masks are:
    如果处理器支持cpuid leaf 0bh,则每个apic id包含一个32位值,则派生三级提取位掩码所需的拓扑枚举参数为:

  • a. Query the right-shift value for the SMT level of the topology using CPUID leaf 0BH with ECX =0H as input. The number of bits to shift-right on x2APIC ID (EAX[4:0]) can distinguish different higher-level entities above SMT (e.g. processor cores) in the same physical package. This is also the width of the bit mask to extract the SMT_ID.
    a.使用cpuid leaf 0bh(ecx=0h)作为输入,查询拓扑的smt级别的右移值。在x2apic id(eax[4:0])上右移的位数可以区分同一物理包中高于smt的不同高级实体(例如处理器核)。这也是用于提取smt_id的位掩码的宽度。

  • b. Enumerate until the desired level is found (i.e. processor cores). Determine if the next level is the expected level. If the next level is not known to the software, keep enumerating until the next known or the last level. Software should use the previous level before this to represent the last previously known level (i.e. processor cores). If the software does not recognize or implement certain hierarchical levels, it should assume these unknown levels as an extension of the last known level.
    b.枚举直到找到所需级别(即处理器核心)。确定下一个级别是否为预期级别。如果软件不知道下一个级别,请继续枚举,直到下一个已知级别或最后一个级别。软件应在此之前使用上一级别来表示上一个已知级别(即处理器核心)。如果软件不能识别或实现某些层次级别,它应该假设这些未知级别是最后一个已知级别的扩展。

  • c. Query CPUID leaf 0BH for the amount of bit shift to distinguish next higher-level entities (e.g. physical processor packages) in the system. This describes an explicit three-level-topology situation for commonly available processors. Consult Example 8-17 to adapt to situations beyond three-level topology of a physical processor. The width of the extraction bit mask can be used to derive the cumulative extraction bitmask to extract the sub IDs of logical processors (including different processor cores) in the same physical package. The extraction bit mask to distinguish merely different processor cores can be derived by xor’ing the SMT extraction bit mask from the cumulative extraction bit mask.
    c.查询cpuid leaf 0bh中的位偏移量,以区分系统中的下一个更高级别实体(例如物理处理器包)。这描述了常用处理器的显式三级拓扑情况。参考示例8-17,以适应物理处理器三级拓扑以外的情况。提取位掩码的宽度可用于导出累积提取位掩码,以提取同一物理包中逻辑处理器(包括不同处理器内核)的子标识。通过将smt提取位掩码与累积提取位掩码进行异或,可以导出仅区分不同处理器核的提取位掩码。

  • d. Query the 32-bit x2APIC ID for the logical processor where the current thread is executing.
    d.查询当前线程正在执行的逻辑处理器的32位x2apic id。

  • e. Derive the extraction bit masks corresponding to SMT_ID, CORE_ID, and PACKAGE_ID, starting from SMT_ID.
    e.从smt_id派生与smt_id、core_id和package_id对应的提取位掩码。

  • f. Apply each extraction bit mask to the 32-bit x2APIC ID to extract sub-field IDs.

    f.将每个提取位掩码应用于32位x2apic id以提取子字段id。

  1. If the processor does not support CPUID leaf 0BH, each initial APIC ID contains an 8-bit value, the topology enumeration parameters needed to derive extraction bit masks are:
    如果处理器不支持cpuid leaf 0bh,则每个初始apic id都包含一个8位值,则导出提取位掩码所需的拓扑枚举参数为:

    • a. Query the size of address space for sub IDs that can accommodate logical processors in a physical processor package. This size parameters (CPUID.1:EBX[23:16]) can be used to derive the width of an extraction bitmask to enumerate the sub IDs of different logical processors in the same physical package.
      a.查询能够容纳物理处理器包中逻辑处理器的子标识的地址空间大小。此大小参数(cpuid.1:ebx[23:16])可用于派生提取位掩码的宽度,以枚举同一物理包中不同逻辑处理器的子标识。
    • b. Query the size of address space for sub IDs that can accommodate processor cores in a physical processor package. This size parameters can be used to derive the width of an extraction bitmask to enumerate the sub IDs of processor cores in the same physical package.
      b.查询能够容纳物理处理器包中处理器核心的子标识的地址空间大小。此大小参数可用于派生提取位掩码的宽度,以枚举同一物理包中处理器核心的子ID。
    • c. Query the 8-bit initial APIC ID for the logical processor where the current thread is executing.
      查询当前线程正在执行的逻辑处理器的8位初始APIC ID。
    • d. Derive the extraction bit masks using respective address sizes corresponding to SMT_ID, CORE_ID, and PACKAGE_ID, starting from SMT_ID.
      d.使用对应于smt_id、core_id和package_id的相应地址大小,从smt_id开始导出提取位掩码。
    • e. Apply each extraction bit mask to the 8-bit initial APIC ID to extract sub-field IDs.
      e.将每个提取位掩码应用于8位初始apic id以提取子字段id。

Example 8-19. Support Routines for Detecting Hardware Multi-Threading and Identifying the Relationships Between Package, Core and Logical Processors
例8-19。支持检测硬件多线程和识别包、核心和逻辑处理器之间关系的例程

1.Detect support for Hardware Multi-Threading Support in a processor.
1.检测处理器中对硬件多线程支持的支持。

//Returns a non-zero value if CPUID reports the presence of hardware multi-threading support in the physical package where the current logical processor is located. This does not guarantee BIOS or OS will enable all logical processors in the physical package and make them available to applications.Returns zero if hardware multi-threading is not present.
//如果cpuid报告当前逻辑处理器所在的物理包中存在硬件多线程支持,则返回非零值。这不能保证BIOS或OS将启用物理包中的所有逻辑处理器并使它们对应用程序可用。如果不存在硬件多线程,则返回零。
#define HWMT_BIT 10000000H
unsigned int HWMTSupported(void)
{// ensure cpuid instruction is supported确保支持cpuid指令execute cpuid with eax = 0 to get vendor stringexecute cpuid with eax = 1 to get feature flag and signature// Check to see if this a Genuine Intel Processor检查这是不是真正的英特尔处理器if (vendor string EQ GenuineIntel) {return (feature_flag_edx & HWMT_BIT); // bit 28}return 0;
}

Example 8-20. Support Routines for Identifying Package, Core and Logical Processors from 32-bit x2APIC ID
例8-20。支持从32位x2apic id识别包、核心和逻辑处理器的例程

a. Derive the extraction bitmask for logical processors in a processor core and associated mask offset for different cores.
a.导出处理器内核中逻辑处理器的提取位掩码以及不同内核的相关掩码偏移量。

int DeriveSMT_Mask_Offsets (void)
{if (!HWMTSupported()) return -1;execute cpuid with eax = 11, ECX = 0;If (returned level type encoding in ECX[15:8] does not match SMT) return -1;Mask_SMT_shift = EAX[4:0];   // # bits shift right of APIC ID to distinguish different cores//位在apic id右移以区分不同的核SMT_MASK = ~( (-1) << Mask_SMT_shift); // shift left to derive extraction bitmask for SMT_ID//左移以派生smt_id的提取位掩码return 0;
}

b. Derive the extraction bitmask for processor cores in a physical processor package and associated mask offset for different packages.
b.导出物理处理器包中处理器核心的提取位掩码以及不同包的相关掩码偏移量。

int DeriveCore_Mask_Offsets (void)
{if (!HWMTSupported()) return -1;execute cpuid with eax = 11, ECX = 0;while( ECX[15:8] ) {// level type encoding is valid级别类型编码有效Mask_Core_shift = EAX[4:0];// needed to distinguish different physical packages需要区分不同的物理包ECX ++;execute cpuid with eax = 11;}COREPlusSMT_MASK = ~( (-1) << Mask_Core_shift);// treat levels between core and physical package as a core for software choosing not to implement or recognize these unknown levels//将核心和物理包之间的级别作为软件选择不实现或不识别这些未知级别的核心CORE_MASK = COREPlusSMT_MASK ^ SMT_MASK;PACKAGE_MASK = (-1) << Mask_Core_shift;return -1;
}

c. Query the x2APIC ID of a logical processor.
c. 查询逻辑处理器的x2apic id。

APIC_IDs for each logical processor.
每个逻辑处理器的apic_id。

unsigned char Getx2APIC_ID (void)
{unsigned reg_edx = 0;execute cpuid with eax = 11, ECX = 0store returned value of edxreturn (unsigned) (reg_edx) ;
}

Example 8-21. Support Routines for Identifying Package, Core and Logical Processors from 8-bit Initial APIC ID
例8-21。支持从8位初始apic id识别包、核心和逻辑处理器的例程

a. Find the size of address space for logical processors in a physical processor package.
a.找出物理处理器包中逻辑处理器的地址空间大小。

#define NUM_LOGICAL_BITS 00FF0000H
// Use the mask above and CPUID.1.EBX[23:16] to obtain the max number of addressable IDs for logical processors in a physical package,使用上面的掩码和cpuid.1.ebx[23:16]获取物理包中逻辑处理器的最大可寻址id数,
//Returns the size of address space of logical processors in a physical processor package;返回物理处理器包中逻辑处理器的地址空间大小;
// Software should not assume the value to be a power of 2.软件不应假定值为2的幂。
unsigned char MaxLPIDsPerPackage(void)
{if (!HWMTSupported()) return 1;execute cpuid with eax = 1store returned value of ebxreturn (unsigned char) ((reg_ebx & NUM_LOGICAL_BITS) >> 16);
}

b. Find the size of address space for processor cores in a physical processor package.
b.找出物理处理器包中处理器核心的地址空间大小。

// Returns the max number of addressable IDs for processor cores in a physical processor package;
// 返回物理处理器包中处理器核心的最大可寻址ID数;
// Software should not assume cpuid reports this value to be a power of 2.
// 软件不应假定cpuid报告此值为2的幂。
unsigned MaxCoreIDsPerPackage(void)
{if (!HWMTSupported()) return (unsigned char) 1;if cpuid supports leaf number 4{   // we can retrieve multi-core topology info using leaf 4//我们可以使用leaf 4检索多核拓扑信息execute cpuid with eax = 4, ecx = 0store returned value of eaxreturn (unsigned) ((reg_eax >> 26) +1);}else // must be a single-core processorreturn 1;
}

c. Query the initial APIC ID of a logical processor.
c. 查询逻辑处理器的初始APIC ID。

#define INITIAL_APIC_ID_BITS FF000000H // CPUID.1.EBX[31:24] initial APIC ID
// Returns the 8-bit unique initial APIC ID for the processor running the code.
// 返回运行代码的处理器的8位唯一初始APIC ID。
// Software can use OS services to affinitize the current thread to each logical processor available under the OS to gather the initial APIC_IDs for each logical processor.
// 软件可以使用操作系统服务将当前线程关联到操作系统下可用的每个逻辑处理器,以收集每个逻辑处理器的初始apic_id。
unsigned GetInitAPIC_ID (void)
{unsigned int reg_ebx = 0;execute cpuid with eax = 1store returned value of ebxreturn (unsigned) ((reg_ebx & INITIAL_APIC_ID_BITS) >> 24;
}

d. Find the width of an extraction bitmask from the maximum count of the bit-field (address size).
d.从位字段的最大计数(地址大小)找到提取位掩码的宽度。

// Returns the mask bit width of a bit field from the maximum count that bit field can represent.
// 从位字段可以表示的最大计数返回位字段的掩码位宽度。
// This algorithm does not assume ‘address size’ to have a value equal to power of 2.
// 此算法不假定“地址大小”的值等于2的幂。
// Address size for SMT_ID can be calculated from MaxLPIDsPerPackage()/MaxCoreIDsPerPackage()
// 可以从maxlpidsperpackage()/maxCoreidsperpackage()计算smt_id的地址大小
// Then use the routine below to derive the corresponding width of SMT extraction bitmask
// 然后使用下面的例程导出smt提取位掩码的相应宽度
// Address size for CORE_ID is MaxCoreIDsPerPackage(),
// 核心ID的地址大小为maxCoreIDsperPackage(),
// Derive the bitwidth for CORE extraction mask similarly
// 类似地导出核心提取掩码的位宽度
unsigned FindMaskWidth(Unsigned Max_Count)
{unsigned int mask_width, cnt = Max_Count;__asm {mov eax, cntmov ecx, 0mov mask_width, ecxdec eaxbsr cx, axjz nextinc cxmov mask_width, ecxnext:mov eax, mask_width}return mask_width;
}

e. Extract a sub ID from an 8-bit full ID, using address size of the sub ID and shift count.
e. 使用子id的地址大小和移位计数,从8位完整id中提取子id。

// The routine below can extract SMT_ID, CORE_ID, and PACKAGE_ID respectively from the init APIC_ID
// 下面的例程可以分别从init apic_id中提取smt_id、core_id和package_id
// To extract SMT_ID, MaxSubIDvalue is set to the address size of SMT_ID, Shift_Count = 0
// 要提取smt_id,将maxsubidvalue设置为smt_id的地址大小,shift_count=0
// To extract CORE_ID, MaxSubIDvalue is the address size of CORE_ID, Shift_Count is width of SMT extraction bitmask.
// 要提取core_id,maxsubidvalue是core_id的地址大小,shift_count是smt提取位掩码的宽度。
// Returns the value of the sub ID, this is not a zero-based value
// 返回子ID的值,这不是基于零的值
Unsigned char GetSubID(unsigned char Full_ID, unsigned char MaxSubIDvalue, unsigned char Shift_Count)
{MaskWidth = FindMaskWidth(MaxSubIDValue);MaskBits = ((uchar) (FFH << Shift_Count)) ^ ((uchar) (FFH << Shift_Count + MaskWidth)) ;SubID = Full_ID & MaskBits;Return SubID;
}

Software must not assume local APIC_ID values in an MP system are consecutive. Non-consecutive local APIC_IDs may be the result of hardware configurations or debug features implemented in the BIOS or OS.
软件不能假定mp系统中的本地apic_id值是连续的。非连续的本地apic_id可能是在bios或os中实现的硬件配置或调试功能的结果。

An identifier for each hierarchical level can be extracted from an 8-bit APIC_ID using the support routines illustrated in Example 8-21. The appropriate bit mask and shift value to construct the appropriate bit mask for each level must be determined dynamically at runtime.
使用示例8-21中所示的支持例程,可以从8位apic_id中提取每个层次级别的标识符。必须在运行时动态确定为每个级别构造适当位掩码的适当位掩码和移位值。

8.9.5 Identifying Topological Relationships in a MP System mp系统中拓扑关系的识别

To detect the number of physical packages, processor cores, or other topological relationships in a MP system, the following procedures are recommended:
要检测MP系统中物理包、处理器核心或其他拓扑关系的数量,建议执行以下步骤:

  • Extract the three-level identifiers from the APIC ID of each logical processor enabled by system software. The sequence is as follows (See the pseudo code shown in Example 8-22 and support routines shown in Example 8-19):
    从系统软件启用的每个逻辑处理器的apic id中提取三级标识符。顺序如下(参见示例8-22中所示的伪代码和示例8-19中所示的支持例程):

    • The extraction start from the right-most bit field, corresponding to SMT_ID, the innermost hierarchy in a three-level topology (See Figure 8-7). For the right-most bit field, the shift value of the working mask is zero. The width of the bit field is determined dynamically using the maximum number of logical processor per core, which can be derived from information provided from CPUID.
      提取从最右边的位字段开始,对应于smt_id,这是三层拓扑中最里面的层次结构(参见图8-7)。对于最右边的位字段,工作掩码的移位值为零。比特字段的宽度是使用每个核心的逻辑处理器的最大数目动态确定的,这可以从CPUID提供的信息中导出。

    • To extract the next bit-field, the shift value of the working mask is determined from the width of the bit mask of the previous step. The width of the bit field is determined dynamically using the maximum number of cores per package.
      为了提取下一个位字段,工作掩码的移位值由上一步的位掩码的宽度确定。使用每个包的最大磁芯数目动态地确定位字段的宽度。

    • To extract the remaining bit-field, the shift value of the working mask is determined from the maximum number of logical processor per package. So the remaining bits in the APIC ID (excluding those bits already extracted in the two previous steps) are extracted as the third identifier. This applies to a non-clustered MP system, or if there is no need to distinguish between PACKAGE_ID and CLUSTER_ID.
      为了提取剩余比特字段,从每个包的逻辑处理器的最大数目确定工作掩码的移位值。因此,apic id中的剩余位(不包括在前面两步中已经提取的位)被提取为第三个标识符。这适用于非群集MP系统,或者如果不需要区分包ID和群集ID。

      If there is need to distinguish between PACKAGE_ID and CLUSTER_ID, PACKAGE_ID can be extracted using an algorithm similar to the extraction of CORE_ID, assuming the number of physical packages in each node of a clustered system is symmetric.
      如果需要区分package_id和cluster_id,则可以使用类似于core_id提取的算法提取package_id,前提是集群系统中每个节点中的物理包数是对称的。

  • Assemble the three-level identifiers of SMT_ID, CORE_ID, PACKAGE_IDs into arrays for each enabled logical processor. This is shown in Example 8-23a.
    为每个启用的逻辑处理器将smt_id、core_id、package_id的三级标识符组装到数组中。如例8-23a所示。

  • To detect the number of physical packages: use PACKAGE_ID to identify those logical processors that reside in the same physical package. This is shown in Example 8-23b. This example also depicts a technique to construct a mask to represent the logical processors that reside in the same package.
    要检测物理包的数量:使用package_id标识驻留在同一物理包中的逻辑处理器。这在示例8-23b中显示。该示例还描述了一种构造掩码以表示驻留在同一包中的逻辑处理器的技术。

  • To detect the number of processor cores: use CORE_ID to identify those logical processors that reside in the same core. This is shown in Example 8-23. This example also depicts a technique to construct a mask to represent the logical processors that reside in the same core.
    要检测处理器内核的数量:使用core_id标识驻留在同一个内核中的逻辑处理器。如例8-23所示。本例还描述了一种构造掩码的技术,该掩码表示驻留在同一内核中的逻辑处理器。

In Example 8-22, the numerical ID value can be obtained from the value extracted with the mask by shifting it right by shift count. Algorithms below do not shift the value. The assumption is that the SubID values can be compared for equivalence without the need to shift.
在实施例8-22中,通过移位计数向右移位,可以从用掩模提取的值中获得数字id值。下面的算法不会移动该值。假设子id值可以进行等价性比较,而无需移位。

Example 8-22. Pseudo Code Depicting Three-level Extraction Algorithm
例8-22。描述三层抽取算法的伪码

For Each local_APIC_ID{// Calculate SMT_MASK, the bit mask pattern to extract SMT_ID,计算smt_mask,用于提取smt_id的位掩码模式,// SMT_MASK is determined using topology enumertaion parameters from CPUID leaf 0BH (Example 8-20);// smt_mask使用cpuid leaf 0bh的拓扑枚举参数确定(示例8-20);// otherwise, SMT_MASK is determined using CPUID leaf 01H and leaf 04H (Example 8-21).// 否则,使用cpuid leaf 01h和leaf 04h(示例8-21)确定smt_mask。// This algorithm assumes there is symmetry across core boundary, i.e. each core within a package has the same number of logical processors// 该算法假设核边界上存在对称性,即包中的每个核具有相同数量的逻辑处理器// SMT_ID always starts from bit 0, corresponding to the right-most bit-field// smt_id总是从位0开始,对应于最右边的位字段SMT_ID = APIC_ID & SMT_MASK;
// Extract CORE_ID:// CORE_MASK is determined in Example 8-20 or Example 8-21// 在实施例8-20或实施例8-21中确定Core_MaskCORE_ID = (APIC_ID & CORE_MASK) ;// Extract PACKAGE_ID:// Assume single cluster.假设是单个集群。// Shift out the mask width for maximum logical processors per package// 为每个包移出最大逻辑处理器的掩码宽度// PACKAGE_MASK is determined in Example 8-20 or Example 8-21// 示例8-20或示例8-21中确定包掩码PACKAGE_ID = (APIC_ID & PACKAGE_MASK) ;
}

Example 8-23. Compute the Number of Packages, Cores, and Processor Relationships in a MP System
例8-23。计算MP系统中包、核心和处理器关系的数量

a) Assemble lists of PACKAGE_ID, CORE_ID, and SMT_ID of each enabled logical processors
a)汇编每个启用的逻辑处理器的包ID、核心ID和SMT ID的列表

//The BIOS and/or OS may limit the number of logical processors available to applications after system boot. The below algorithm will compute topology for the processors visible to the thread that is computing it.
// bios和/或os可以限制系统引导后应用程序可用的逻辑处理器的数量。下面的算法将为计算它的线程可见的处理器计算拓扑。
// Extract the 3-levels of IDs on every processor  在每个处理器上提取3级id
// SystemAffinity is a bitmask of all the processors started by the OS. Use OS specific APIs to obtain it.
// SystemAffinity是操作系统启动的所有处理器的位掩码。使用特定于操作系统的api来获取它。
// ThreadAffinityMask is used to affinitize the topology enumeration thread to each processor using OS specific APIs.
// threadaffinitymask用于使用操作系统特定的api将拓扑枚举线程关联到每个处理器。
// Allocate per processor arrays to store the Package_ID, Core_ID and SMT_ID for every started processor.
// 为每个已启动的处理器分配每个处理器阵列以存储包ID、核心ID和SMT ID。
ThreadAffinityMask = 1;
ProcessorNum = 0;
while (ThreadAffinityMask ≠ 0 && ThreadAffinityMask <= SystemAffinity) {// Check to make sure we can utilize this processor first.检查以确保我们可以先使用此处理器。if (ThreadAffinityMask & SystemAffinity){Set thread to run on the processor specified in ThreadAffinityMaskWait if necessary and ensure thread is running on specified processorAPIC_ID = GetAPIC_ID(); // 32 bit ID in Example 8-20 or 8-bit ID in Example 8-21 Extract the Package_ID, Core_ID and SMT_ID as explained in three level extraction algorithm of Example 8-22// 例8-20中的32位id或例8-21中的8位id提取包id、核心id和smt id,如例8-22的三级提取算法所述PackageID[ProcessorNUM] = PACKAGE_ID;CoreID[ProcessorNum] = CORE_ID;SmtID[ProcessorNum] = SMT_ID;ProcessorNum++;}ThreadAffinityMask <<= 1;
}
NumStartedLPs = ProcessorNum;

b) Using the list of PACKAGE_ID to count the number of physical packages in a MP system and construct, for each package, a multi-bit mask corresponding to those logical processors residing in the same package.
b)使用包id的列表来计算mp系统中的物理包的数量,并为每个包构造一个对应于驻留在同一包中的那些逻辑处理器的多位掩码。

// Compute the number of packages by counting the number of processors with unique PACKAGE_IDs in the PackageID array.通过计算packageid数组中具有唯一包id的处理器数来计算包数。
// Compute the mask of processors in each package.计算每个包中处理器的掩码。
PackageIDBucket is an array of unique PACKAGE_ID values. Allocate an array of NumStartedLPs count of entries in this array.   packageidbucket是一组唯一的包id值。分配一个numStartedLPS数组此数组中的条目数。
PackageProcessorMask is a corresponding array of the bit mask of processors belonging to the same package, these are processors with the same PACKAGE_ID
packageprocessormask是属于同一个包的处理器的位掩码的对应数组,这些处理器具有相同的包id
The algorithm below assumes there is symmetry across package boundary if more than one socket is populated in an MP system.
下面的算法假设如果在mp系统中填充了多个套接字,则包边界上存在对称性。
// Bucket Package IDs and compute processor mask for every package.每个包的bucket包id和计算处理器掩码。
PackageNum = 1;
PackageIDBucket[0] = PackageID[0];
ProcessorMask = 1;
PackageProcessorMask[0] = ProcessorMask;
For (ProcessorNum = 1; ProcessorNum < NumStartedLPs; ProcessorNum++) {ProcessorMask << = 1;For (i=0; i < PackageNum; i++) {// we may be comparing bit-fields of logical processors residing in different packages, the code below assume package symmetry我们可能在比较位于不同包中的逻辑处理器的位字段,下面的代码假设包对称If (PackageID[ProcessorNum] = PackageIDBucket[i]) {PackageProcessorMask[i] |= ProcessorMask;Break; // found in existing bucket, skip to next iteration 在现有的桶中找到,跳到下一个迭代}}if (i =PackageNum) {//PACKAGE_ID did not match any bucket, start new bucket包ID与任何存储桶都不匹配,请启动新存储桶PackageIDBucket[i] = PackageID[ProcessorNum];PackageProcessorMask[i] = ProcessorMask;PackageNum++;}
}
// PackageNum has the number of Packages started in OS
// PackageProcessorMask[] array has the processor set of each package

c) Using the list of CORE_ID to count the number of cores in a MP system and construct, for each core, a multi-bit mask corresponding to those logical processors residing in the same core.
c)使用core_id的列表来计算mp系统中的核数,并为每个核构造对应于驻留在同一核中的那些逻辑处理器的多位掩码。

Processors in the same core can be determined by bucketing the processors with the same PACKAGE_ID and CORE_ID. Note that code below can BIT OR the values of PACKGE and CORE ID because they have not been shifted right.
同一个核心中的处理器可以通过将具有相同包ID和核心ID的处理器进行bucking来确定。请注意,下面的代码可以位或packge和core ID的值,因为它们没有右移。

The algorithm below assumes there is symmetry across package boundary if more than one socket is populated in an MP system.
下面的算法假设如果在mp系统中填充了多个套接字,则包边界上存在对称性。

//Bucketing PACKAGE and CORE IDs and computing processor mask for every core每个核心的bucketing包、核心id和计算处理器掩码
CoreNum = 1;
CoreIDBucket[0] = PackageID[0] | CoreID[0];
ProcessorMask = 1;
CoreProcessorMask[0] = ProcessorMask;
For (ProcessorNum = 1; ProcessorNum < NumStartedLPs; ProcessorNum++) {ProcessorMask << = 1;For (i=0; i < CoreNum; i++) {// we may be comparing bit-fields of logical processors residing in different packages, the code below assume package symmetry我们可能在比较位于不同包中的逻辑处理器的位字段,下面的代码假设包对称If ((PackageID[ProcessorNum] | CoreID[ProcessorNum]) = CoreIDBucket[i]) {CoreProcessorMask[i] |= ProcessorMask;Break; // found in existing bucket, skip to next iteration在现有的桶中找到,跳到下一个迭代}}if (i = CoreNum) {//Did not match any bucket, start new bucket 不匹配任何桶,启动新桶CoreIDBucket[i] = PackageID[ProcessorNum] | CoreID[ProcessorNum];CoreProcessorMask[i] = ProcessorMask;CoreNum++;}
}
// CoreNum has the number of cores started in the OS             corenum具有操作系统中启动的核心数
// CoreProcessorMask[] array has the processor set of each   corecoreprocessormask[]数组具有每个核心的处理器集

Other processor relationships such as processor mask of sibling cores can be computed from set operations of the PackageProcessorMask[] and CoreProcessorMask[].
其他处理器关系(如同级核心的处理器掩码)可以从包processor mask[]和coreprocessormask[]的设置操作计算。

The algorithm shown above can be adapted to work with earlier generations of single-core IA-32 processors that support Intel Hyper-Threading Technology and in situations that the deterministic cache parameter leaf is not supported (provided CPUID supports initial APIC ID). A reference code example is available (see Intel® 64 Architecture Processor Topology Enumeration).
上述算法可适用于支持英特尔超线程技术的早期单核IA-32处理器,以及不支持确定性缓存参数叶的情况(前提是CPuid支持初始APIC ID)。提供了一个参考代码示例(请参阅“英特尔64体系结构处理器拓扑枚举”)。

8.10 MANAGEMENT OF IDLE AND BLOCKED CONDITIONS 空闲和阻塞条件的管理

When a logical processor in an MP system (including multi-core processor or processors supporting Intel Hyper-Threading Technology) is idle (no work to do) or blocked (on a lock or semaphore), additional management of the core execution engine resource can be accomplished by using the HLT (halt), PAUSE, or the MONITOR/MWAIT instructions.
当MP系统中的逻辑处理器(包括多核处理器或支持“英特尔超线程技术”的处理器)空闲(无需工作)或被阻塞(锁定或信号量)时,可以使用HLT(暂停)、PAUSE或MONITOR/MWAIT指令来完成对核心执行引擎资源的额外管理。

8.11 MP INITIALIZATION FOR P6 FAMILY PROCESSORS P6系列处理器的MP初始化

This section describes the MP initialization process for systems that use multiple P6 family processors. This process uses the MP initialization protocol that was introduced with the Pentium Pro processor (see Section 8.4, “Multiple-Processor (MP) Initialization”). For P6 family processors, this protocol is typically used to boot 2 or 4 processors that reside on single system bus; however, it can support from 2 to 15 processors in a multi-clustered system when the APIC busses are tied together. Larger systems are not supported.
本节介绍使用多个P6系列处理器的系统的MP初始化过程。此过程使用奔腾Pro处理器引入的MP初始化协议(请参阅第8.4节“多处理器(MP)初始化”)。对于P6系列处理器,此协议通常用于引导位于单个系统总线上的2或4个处理器;但是,当APIC总线绑定在一起时,它可以支持多集群系统中的2到15个处理器。不支持较大的系统。

Intel® 64 and IA-32 Architectures Software Developer's Manual CHPTER 8 Multiple-processor management相关推荐

  1. Intel 64 and IA-32 Architectures Instruction Format 指令格式

    摘自<Intel® 64 and IA-32 Architectures Software Developer's Manual Combined Volumes1, 2A, 2B, 2C, 2 ...

  2. 《翻译》Intel 64 与 IA-32 架构软件开发者手册卷1翻译

    <前言> 翻译自官方的PDF版手册,可以从下述站点下载英文原版: http://www.intel.com/content/www/us/en/processors/architectur ...

  3. Intel 64/x86_64/IA-32/x86处理器 - 指令格式(2) - 8086/16位指令格式概述

    本节简单地介绍一下x86指令集的指令格式和特点,当优化文档偶尔提及指令格式时,可供参考.完整的指令集格式在Intel Software Developer's Manual的第二卷Instructio ...

  4. Intel software developer manual

    baidu 搜索:"intel software developer manual" 或者 "Intel SDM" 可以打开Intel SDM网站: https ...

  5. 64位ie加载java失败_java-无法在AMD 64位平台上加载IA 32位.dll

    java-无法在AMD 64位平台上加载IA 32位.dll 我正在尝试通过此页面上的JNI包装器使用Java中的SVMLight: static { System.loadLibrary(" ...

  6. Intel 64/x86_64/IA-32/x86处理器基本执行环境 (1) - 32位执行环境概述

    Basic Execution Environment Overview IA-32处理器提供了一套完整的资源,在处理器上运行的程序/任务可以执行指令,存储代码,数据以及状态信息.这些资源(如下简要的 ...

  7. Intel 64/x86_64/IA-32/x86处理器 - 锁原子操作(2) - 总线封锁/缓存封锁

    Bus Locking Intel 64和IA-32处理器提供了LOCK#信号,在某些关键的访存操作时会自动地激活assert这个信号,用于封锁系统总线或类似的链接.当这个输出信号被激活时,就会阻塞来 ...

  8. CPU 64位和32位

    不同的CPU都能够解释的机器语言的体系称为指令集架构(ISA,Instruction Set Architecture),也可以称为指令集(instruction set).Intel将x86系列CP ...

  9. X86 64位和32位

    不同的CPU都能够解释的机器语言的体系称为指令集架构(ISA,Instruction Set Architecture),也可以称为指令集(instruction set).Intel将x86系列CP ...

最新文章

  1. VINS-Mono(经典论文阅读)
  2. Asterisk 1.8 sip 协议栈分析
  3. js学习-DOM之动态创建元素的三种方式、插入元素、onkeydown与onkeyup两个事件整理...
  4. esp8266环境搭建
  5. 均线带角度的指标_指标:均线斜率角度计算
  6. 【论文笔记】One Millisecond Face Alignment with an Ensemble of Regression Trees
  7. Drawing绘图halcon算子,持续更新
  8. java语法定制混淆,由撇号混淆的textpad语法高亮显示
  9. KM算法(最优匹配)
  10. java访问win10共享盘失败_Win10不能访问共享磁盘的解决方法(亲测能用)
  11. Jmeter压力测试实战
  12. 热启动计算机的快捷键,电脑快捷键的用法
  13. Python Day05
  14. 2.3.1 TextView(文本框)详解
  15. 跨境电商个人物品清单申报开发代码
  16. JM8.5中的高精度象素运动估计 1
  17. Vue3大菠萝pinia笔记
  18. 什么是个人热点?安卓手机怎么开启个人热点?
  19. 图形学基本知识整理(概述 + 硬件知识)
  20. 移动端跨平台开发Flutter 与 React Native对比

热门文章

  1. 通过C++类方法地址调用类的虚方法
  2. 嵌入式实时操作系统11——操作系统内核运行原理
  3. 呕心沥血梳理C++新标准超好用的新特性(实战必备)
  4. Hive启动报错 java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang
  5. C语言_指针动态内存分布
  6. Guava系列:Shorts、Doubles、Chars、Floats、Ints、Longs、Bytes使用方法
  7. minpack.error: Result from function call is not a proper array of floats.
  8. 《您的设计模式》(CBF4LIFE)之“访问者模式”【整理】
  9. 我的编程之路之初出茅庐(二)
  10. 原生php开发多管理员留言板系统源码