Single-stream Opus audio decoding with the XAPU API

In this topic, we will walk through the expected code flow about how to use Opus hardware accelerated decoding on XBOX Series X devices by using one of the samples included with the Microsoft Game Development Kit (GDK). This sample is called DecodeOne.cpp and covers a single-stream decoding scenario with one client.

Audio hardware connection and startup

Decoding starts by establishing a connection with the hardware acceleration unit via XApuConnect. The caller sets the requirements for the connection by using XApuConnectInputParameters to set up the following:

The processing type (in this case, XApuProcessType::DecodeOpus).
The number of streams (in this case, 1).
The total memory that’s required to pass and retrieve the input and the output data, in this case, 1024 for MaxInputBufferLength (must be equal to or greater than the largest Opus packet in the stream). The number must also be a multiple of 16 and 7680. This is 20 ms of data at 48 K, which is 960 × 2 × sizeof(float) = 7680 for MaxOutputBufferLength. The output also needs to be a multiple of 16.

If this call succeeds, the XApuConnectOutputParameters::baseData parameter points to a memory location that’s allocated by the operating system in such a way that the hardware acceleration unit can directly access without additional marshaling or copying. This memory should be used by the caller to pass the input data and retrieve the output data. The following example of a simple memory manager can be used to set XApuConnectInputParameters and partition XApuConnectOutputParameters::baseData into two pointers: one for the input data and the other for the output data.

class DecodeOneSimpleMemoryManager
{
public:
    static const uint32_t MaxStreamCount = 1;
    const uint32_t MaxInputBufferLength = 1024; // Must be equal to or greater than the largest Opus packet in the stream. The number must also be a multiple of 16.
    const uint32_t MaxOutputBufferLength = 7680; // This is 20 ms of data at 48 K, which is 960 x 2 x sizeof(float) = 7680 for MaxOutputBufferLength. The output also needs to be multiple of 16.

private:
    XApuConnectInputParameters _inParam = {
        XApuProcessType::DecodeOpus,
        MaxStreamCount,
        1,
        MaxStreamCount * (MaxInputBufferLength + MaxOutputBufferLength),
        0,
        XApuConnectOptions::None };

    XApuConnectOutputParameters _outParam = {};

public:
    XApuConnectInputParameters * const GetConnectParametersRef() {
        return &_inputParam;
    }

    XApuConnectOutputParameters * const GetConnectionDataRef() {
        return &_outParam;
    }

    uint8_t* GetRawInputBuffer(uint32_t& maxByteCount) {
        maxByteCount = MaxInputBufferLength;
        return reinterpret_cast<uint8_t*>(_outParam.baseData);
    }

    uint8_t* GetRawOutputBuffer(uint32_t& maxByteCount) {
        maxByteCount = MaxOutputBufferLength;
        return reinterpret_cast<uint8_t*>(_outParam.baseData) + MaxInputBufferLength;
    }
};

Following is the code example for the first step to establish a connection.

XApuConnectInputParameters * const connectParameters = memoryManager.GetConnectParametersRef();
XApuConnectOutputParameters * const connectionData = memoryManager.GetConnectionDataRef();
hr = XApuConnect(connectParameters, connectionData, &xapuHandle);

Activation and deactivation of streams

After the connection is established with the device, the caller should send a stream activation command to engage a single decoding engine. The caller must specify the stream index and the channel count. Only mono or stereo streams are allowed. The caller should call XApuDequeueResult to get the activation result. Because XApuConnectInputParameters::maxQueuedCommandsPerStream is set to 1 in this sample, the caller has to wait for each result before submitting the next command. This includes the Activate, Process, and Deactivate commands (for details, see XApuCommandType). Following is the code to activate one stream.

XApuDecodeConvertActivateCommand command = {};
command.id.type = XApuCommandType::Activate;
command.id.streamIndex = 0;
command.id.sequence = 0;
command.channelCount = channelCount;

hr = XApuEnqueueCommand(xapuHandle, &command.id, nullptr);
XApuResult result = {};

do {
    hr = XApuDequeueResult(handle, &result);
} while (hr == XAPU_E_PENDING_RESULTS);

After the activation result is received, the caller can start submitting Opus packets for decoding. This should be done by using the XApuCommandType::Process command. For each Process command, the caller must do the following:

Fill the input buffer with exactly one Opus packet.
Specify the length of the packet.
Provide the location to the output buffer that will be filled with decoded data by the audio hardware acceleration unit.
Specify the maximum length of the output buffer.

Also, the caller must specify the streamIndex and frameCount values. If all the decoded data is required to be copied to the output buffer, the frameCount value can be set to any value that’s equal to or greater than the number of frames that are encoded in the packet (for example, for a 20 ms packet, frameCount can be set between 960 and 0xFFFFFFFF).

XApuDecodeConvertCommand command = {};
command.id.streamIndex = 0;
command.id.type = XApuCommandType::Process;
command.id.sequence = packetIndex;
command.frameCount = 0xFFFFFFFF;
command.inputData = memoryManager.GetRawInputBuffer(inputMaxByteCount);
command.inputDataLength = packetLength;
command.outputData = memoryManager.GetRawOutputBuffer(outputMaxByteCount);
command.maxOutputDataLength = outputMaxByteCount;

hr = XApuEnqueueCommand(xapuHandle, &command.id, nullptr);

Next, XApuDequeueResult should be called to get the decoded output of the packet. After XApuResult is successfully retrieved, the output data will be stored in result.outputData. After it’s processed, the next packet should be submitted for decoding with a new call to XApuEnqueueCommand with the XApuCommandType::Process command.

XApuResult result = {};

do {
    hr = XApuDequeueResult(handle, &result);
    if (SUCCEEDED(hr))
    {
         if (result.id.type == XApuCommandType::Process)
         {
              fwrite(result.outputData, 1, result.outputDataLength, fileOutput);
         }
    }
    
} while (hr == XAPU_E_PENDING_RESULTS);

After all the packets in the stream are decoded, an XApuCommandType::Deactivate command should be submitted.

XApuCommandId id;
id.type = XApuCommandType::Deactivate;
id.streamIndex = 0;
id.sequence = 0;

hr = XApuEnqueueCommand(xapuHandle, &id, nullptr);

do {
    hr = XApuDequeueResult(handle, &result);
} while (hr == XAPU_E_PENDING_RESULTS);

Termination

After the result for the deactivation is received, the caller must call XApuDisconnect to disconnect from the audio hardware acceleration unit to free all the allocated resources.

hr = XApuDisconnect(xapuHandle);

GDK and Game Engines Integration

Steam porting guide

Game Principles

Core Features

Console Features

Audio hardware connection and startup

Activation and deactivation of streams

Termination

​Audio hardware connection and startup

​Activation and deactivation of streams

​Termination

Audio hardware connection and startup

Activation and deactivation of streams

Termination