Appendix: ARM Memory Management

This appendix includes:

This appendix describes various features and restrictions related to the Neutrino implementation on ARM/Xscale processors:

For an overview of how Neutrino manages memory, see the introduction to the Finding Memory Errors chapter of the IDE User's Guide.

ARM-specific restrictions and issues

This section describes the major restrictions and issues raised by the Neutrino implementation on ARM/Xscale:

_NTO_TCTL_IO behavior

Device drivers in Neutrino use ThreadCtl() with the _NTO_TCTL_IO flag to obtain I/O privileges. This mechanism allows direct access to I/O ports and the ability to control processor interrupt masking.

On ARM platforms, all I/O access is memory-mapped, so this flag is used primarily to allow manipulation of the processor interrupt mask.

Normal user processes execute in the processor's User mode, and the processor silently ignores any attempts to manipulate the interrupt mask in the CPSR register (i.e. they don't cause any protection violation, and simply have no effect on the mask).

The _NTO_TCTL_IO flag makes the calling thread execute in the processor's System mode. This is a privileged mode that differs only from the Supervisor mode in its use of banked registers.

This means that such privileged user processes execute with all the access permission of kernel code:

The major consequence of this is that buggy programs using _NTO_TCTL_IO can corrupt kernel memory.

Implications of the ARM Cache Architecture

All currently supported ARM/Xscale processors implement a virtually indexed cache. This has a number of software-visible consequences:

As mentioned, Neutrino uses the MMU Fast Context Switch Extension (FCSE) to avoid cache-flushing during context switches. Since the cost of this cache-flushing can be significant (potentially many thousands of cycles), this is crucial to a microkernel system like Neutrino because context switches are much more frequent than in a monolithic (e.g. UNIX-like) OS:

The FCSE implementation works by splitting the 4 GB virtual address space into a number of 32 MB slots. Each address space appears to have a virtual address space range of 0 - 32 MB, but the MMU transparently remaps this to a a "real" virtual address by putting the slot index into the top 7 bits of the virtual address.

For example, consider two processes: process 1 has slot index 1; process 2 has slot index 2. Each process appears to have an address space 0 - 32 MB, and their code uses those addresses for execution, loads and stores.

In reality, the virtual addresses seen by the MMU (cache and TLB) are:

This mechanism imposes a number of restrictions:

ARM-specific features

This section describes the ARM-specific behavior of certain operations that are provided via a processor-independent interface:

shm_ctl() behavior

The Neutrino implementation on ARM uses various shm_ctl() flags to provide some workarounds for the restrictions imposed by the MMU FCSE implementation, to provide a "global" address space above 0x80000000 that lets processes map objects that wouldn't otherwise fit into the (private) 32 MB process-address space.

The following flags supplied to shm_ctl() create a shared memory object that you can subsequently mmap() with special properties:

To create these special mappings:

  1. Create and initialize the object:
    fd = shm_open(name, ...)
    shm_ctl(fd, ...)
      

    Note that you must be root to use shm_ctl().

  2. Map the object:
    fd = shm_open(name, ...)
    mmap( ..., fd, ...)
      

    Any process that can use shm_open() on the object can map it, not just the process that created the object.

The following table summarizes the effect of the various combinations of flags passed to shm_ctl():

Flags Object type Effect of mmap()
SHMCTL_ANON Anonymous memory (not contiguous) Mapped into normal process address space. PROT_NOCACHE is forced.
SHMCTL_ANON | SHMCTL_PHYS Anonymous memory (physically contiguous) Mapped into normal process address space. PROT_NOCACHE is forced.
SHMCTL_ANON | SHMCTL_GLOBAL Anonymous memory (not contiguous) Mapped into global address space. PROT_NOCACHE isn't forced. All processes receive the same mapping.
SHMCTL_ANON | SHMCTL_GLOBAL | SHMCTL_PHYS Anonymous memory (not contiguous) Mapped into global address space. PROT_NOCACHE isn't forced. All processes receive the same mapping.
SHMCTL_PHYS Physical memory range Mapped into global address space. PROT_NOCACHE is forced. Processes receive unique mappings.
SHMCTL_PHYS | SHMCTL_GLOBAL Physical memory range Mapped into global address space. PROT_NOCACHE isn't forced. All processes receive the same mapping.

Note that by default, mmap() creates privileged access mappings, so the caller must have _NTO_TCTL_IO privilege to access them.

Flags may specify SHMCTL_LOWERPROT to create user-accessible mappings. However, this allows any process to access these mappings if they're in the global address space.