Single-stream convolution with envelope using the XDSP API

In this topic, we will walk through the expected code flow about how to use hardware Convolution reverb on XBOX Series X devices by using one of the samples included with the Microsoft Game Development Kit (GDK).

Hardware connection and startup

A connection has to be established with the hardware acceleration unit via XDspConnect or XDspConnectWithMaximumStreamLimit. The caller sets the requirements for the connection by using the following parameters:

baseBuffer - A pointer to the user allocated memory enough to hold input, output, envelope buffer and/or block multiplier buffers that will be passed to the hardware. This memory should be allocated via XMemAlloc with these attributes: XALLOC_MEMTYPE_PHYSICAL_CACHEABLE, XALLOC_PAGESIZE_64KB, XALLOC_ALIGNMENT_64K. These attributes should be set using MAKE_XALLOC_ATTRIBUTES(), as shown in the example below. This memory should not be freed until XDspDisconnect call succeeds. This buffer should be at least 64KB in size.
baseBufferLength - The length of the baseBuffer in bytes. This buffer should be at least 64KB and no more than 500 MB.
aggregateImpulseResponseInSeconds - It represents the overall duration of all the individual impulse responses that will be activated simultaneously using XDspActivate with XDspProcessType::Convolution. If this value is 0, no streams with XDspProcessType::Convolution can be activated.
handle - a pointer to the XDspClientHandle to hold the handle returned by this call.

If the call succeeds, the baseBuffer can be used by the caller to pass input data and retrieve the output data of the streams via XDspCommand parameters. Any buffer that is passed in the XDspCommand should be 16 byte aligned. The following example of a simple buffer manager can be used for managing the input and output buffers.

class ConvolveOneBufferManager
{
private:
    float const* _outputBuffer = nullptr;
    uint32_t _maxBufferFrameCount = 0;
    uint32_t _bufferStride = 0;
    uint32_t _maxBufferCount = 0;
    uint32_t _acquiredBufferCount = 0;
    uint32_t _channelCount = 1;

public:
    void Reset(float const* outputBuffer, uint32_t maxBufferCount, uint32_t maxBufferFrameCount, uint32_t channelCount) {
        _outputBuffer = outputBuffer;
        _maxBufferCount = maxBufferCount;
        _maxBufferFrameCount = maxBufferFrameCount;
        _channelCount = channelCount;
    }

    uint32_t GetMaxBufferFrameCount() const {
        return _maxBufferFrameCount;
    }

    bool IsFull() const {
        return _acquiredBufferCount >= _maxBufferCount;
    }

    void Acquire() {
        if (_acquiredBufferCount < _maxBufferCount) {
            _acquiredBufferCount++;
        }
    }

    void Release() {
        if (_acquiredBufferCount > 0) {
            _acquiredBufferCount--;
        }
    }

    float* GetOutputBuffer(uint32_t index) {
        return const_cast<float*>(_outputBuffer) + (((index % _maxBufferCount) * _maxBufferFrameCount * _channelCount));
    }

};

The envelope buffers for each stream should be part of the baseBuffer that is passed to XDspConnect or XDspConnectWithMaximumStreamLimit similar to the input and output buffers. The length of each envelope buffer can be calculated as follows.

#define ROUND_UP_TO_16BYTE_ALIGNED(x) (((x) + (15)) & ~(0xF))
uint32_t blockSize = 1024; // can be 512 or 1024
uint32_t irLengthInComplexes = impulseResponseLengthInFloats/2;
uint32_t numFilterBlocks = irLengthInComplexes/blockSize;

// In case of a stereo impulse response filter, the envelope buffer length will be 2 * envelopeLengthInBytes assuming impulseResponseLengthInFloats is the impule response length of a single channel.
uint32_t envelopeLengthInBytes = ROUND_UP_TO_16BYTE_ALIGNED(numFilterBlocks * sizeof(float));
uint32_t envelopeGainCount = envelopeLengthInBytes/sizeof(float);

Following is the code example for the first step to allocate memory and establish a connection. Note that XDspConnectWithMaximumStreamLimit can be used in place of XDspConnect to reduce the internal memory usage if the number of streams that can be active is below 256.

static const ULONGLONG XMemAllocAttributes = MAKE_XALLOC_ATTRIBUTES(allocatorId,
                      0,
                      XALLOC_MEMTYPE_PHYSICAL_CACHEABLE,
                      XALLOC_PAGESIZE_64KB,
                      XALLOC_ALIGNMENT_64K,
                      FALSE);

float* baseBuffer = (float *)XMemAlloc(baseBufferLength, XMemAllocAttributes);

Where XMemAllocAttributes represents the attributes mentioned above. The following code sets the aggregateImpulseResponse to 32 seconds.

hr = XDspConnect(baseBuffer, baseBufferLength, 32 /*aggregateImpulseResponseInSeconds*/, &clientHandle);

Activation and deactivation of streams

After the connection is established with the device, the caller should send a stream activation command to engage a convolution, FFT or IFFT. The caller must specify the XDspProcessType, block frame count, channel count, impulse response length in floats and a pointer to the impulse response data in frequency domain in the XDspActivationParameters. Only mono or stereo streams are allowed. For XDspProcessType::ForwardFourierTransform and XDspProcessType::inverseFourierTransform, XDspActivationParameters::impulseResponse should be nullptr and XDspActivationParameters::impulseResponseLengthInFloats = 0. XDspActivationOptions should be set appropriately. Following is the code to activate one stream for convolution.

XDspActivationParameters params;
XDspStatus* status;
XDspStreamHandle* streamHandle;
params.type = XDspProcessType::Convolution;
params.blockFrameCount = maxBufferFrameCount;
params.channelCount = channelCount; // mono or stereo
params.impulseResponseLengthInFloats = filterFrameCount;
params.impulseResponse = filterBuffer;

if (channelCount == 2 && deinterleaved)
{
    params.options |= XDspActivationOptions::Deinterleaved;
}
if (stereoImpulseResponse)
{
    params.options |= XDspActivationOptions::StereoImpulseResponse;
}
hr = XDspActivate(clientHandle, &params, &status, &streamHandle); 

XDspActivate returns a XDspStatus buffer as one of its parameters. This XDspStatus buffer should be used to find the status of the commands processed by the hardware for this stream. After the activation result is received, the caller can start submitting data for Convolution/FFT/IFFT. This should be done by using the XDspSubmitCommand or XDspSubmitCommandWithEnvelope API. On each call, the caller must do the following:

Fill the input buffer with exactly blockFrameCount of data.
Fill in the XDspCommand with appropriate values and input and output buffers that are 16 byte aligned.
Call XDspSubmitCommand to submit the command to the hardware. This function returns a command sequence number that can be used to track the number of commands sent to the hardware.
To apply an envelope, fill the envelope buffer with gain values for each block of the impulse response filter and call XDspSubmitCommandWithEnvelope and continue making this call with the same envelope buffer until enveloping is no longer needed or the envelope needs to be changed.
When enveloping is no longer required, call XDspSubmitCommand or XDspSubmitCommandWithEnvelope with a nullptr for envelopeBuffer. Note that XDspSubmitCommandWithEnvelope with a nullptr for envelopeBuffer is equivalent to XDspSubmitCommand.
When switching the envelope, the current envelopeBuffer can only be updated for the next envelope if the hardware processed all the commands using that envelope and the results have been picked up. If not, please use a different envelope buffer.

XDspCommand command;

command.beginScale = 1.0f;
command.endScale = 1.0f;
command.inputBlockBuffer = inputDataBuffer;
command.outputBlockBuffer =  bufferManager.GetOutputBuffer(_totalCommandsSubmitted + 1);
command.blockBufferMultiplier = nullptr;

uint32_t sequence = 0;
hr = XDspSubmitCommand(streamHandle, &command, &sequence);

if (SUCCEEDED(hr))
{
    totalCommandsSubmitted++;
}

The following code snippet shows an example of filling the envelope buffer with gain values for each block of the impulse response filter. Here it shows how to use only the first half of the impulse response filter with the gain values for the first half of the impulse response blocks starting with 1.0f, linearly decreasing and the second half filled with a gain value of 0.0f. For a stereo impulse response filter, the entire filter for the left channel is followed by that of the right channel - the envelope buffer for the stereo impulse response filter follows the same pattern with the gain values for the entire left channel followed by the gain values for the entire right channel. Please see Overview of Enveloping using XDSP API for information on calculating the size of the envelope buffer.

// Please see the code snippet at the beginning of this example for information on the envelope buffer size and envelope gain counts.
void CreateEnvelopeParams(
    float* envelopeBuffer, 
    uint32_t envelopeGainCount, 
    uint32_t numFilterBlocks,
    bool stereoImpulseResponse)
{
    const uint32_t envelopeRatio = 0.5f; // Use only half of the impulse response filter
    uint32_t numGains = (uint32_t)(envelopeRatio * numFilterBlocks);
    float gainDecrement = 1.0f/numGains;

    envelopeBuffer[0] = 1.0f;
        
    for (uint32_t i = 1; i < envelopeGainCount; i++)
    {
        if (i < numGains)
        {
            envelopeBuffer[i] = envelopeBuffer[i-1] - gainDecrement;
        }
        else
        {
            envelopeBuffer[i] = 0.0f;
        }
    }

    if (stereoImpulseResponse)
    {
        // Populate gain values for the right channel envelope
        memcpy((void*)&envelopeBuffer[envelopeGainCount], (void*)&envelopeBuffer[0], envelopeGainCount * sizeof(float));
    }
}

The following code snippet shows applying the envelope by calling XDspSubmitCommandWithEnvelope with a valid envelopeBuffer. This is same as the snippet that shows how to submit a command using XDspSubmitCommand except passing in the additional envelopeBuffer parameter. This should be repeated until envelope is no longer needed at which point either XDspSubmitCommand or XDspSubmitCommandWithEnvelope with a nullptr for envelopeBuffer should be called.

XDspCommand command;

command.beginScale = 1.0f;
command.endScale = 1.0f;
command.inputBlockBuffer = inputDataBuffer;
command.outputBlockBuffer =  bufferManager.GetOutputBuffer(_totalCommandsSubmitted + 1);
command.blockBufferMultiplier = nullptr;

uint32_t sequence = 0;
hr = XDspSubmitCommandWithEnvelope(streamHandle, &command, envelopeBuffer, &sequence);

if (SUCCEEDED(hr))
{
    totalCommandsSubmitted++;
}

Next, XDspStatus can be checked to find the status of the commands submitted to the hardware as follows:

while (status->lastProcessedCommandSequence > totalResponsesReceived)
{
    totalResponsesReceived++;
 
    // Hardware status is only updated when there is a hardware fault and the
    // error is going to persist for the duration of the stream. But, all the
    // commands that have already been submitted will still be processed by
    // the hardware and we have to wait for these commands to be processed 
    // before calling XDspDeactivate.
    hr = status->result; 
     
    float* outBuffer = bufferManager.GetOutputBuffer(totalResponsesReceived);
    // The output will be in the outBuffer
    // Do something with the outBuffer
    bufferManager.Release();
}

The packets in the stream are processed or the hardware returns an error in XDspStatus result, XDspDeactivate should be called.

while (totalResponsesReceived < totalCommandsSubmitted)
{
    // run the above code in the while loop to obtain all the responses
}
hr = XDspDeactivate(streamHandle);

XDspDeactivate returns XDSP_E_PENDING_RESULTS if the hardware is not done processing all commands submitted for this stream.

Termination

After XDspDeactivate succeeds, the caller must call XDspDisconnect to disconnect from the audio hardware acceleration unit to free all the allocated resources. If all streams are not deactivated, XDspDisconnect will return XDSP_E_NOT_ALL_HANDLES_DEACTIVATED error.

hr = XDspDisconnect(clientHandle);
XMemFree(baseBuffer, XMemAllocAttributes);

Reference API documentation

XDspAudio (API contents)
- Functions
- Structures

GDK and Game Engines Integration

Steam porting guide

Game Principles

Core Features

Console Features

Single-stream convolution with envelope using the XDSP API

Hardware connection and startup

Activation and deactivation of streams

Termination

Reference API documentation

See also

​Hardware connection and startup

​Activation and deactivation of streams

​Termination

​Reference API documentation

​See also

Hardware connection and startup

Activation and deactivation of streams

Termination

Reference API documentation

See also