> ## Documentation Index > Fetch the complete documentation index at: https://devdocs.xbox.com/llms.txt > Use this file to discover all available pages before exploring further. # 使用实时音频操作应用自定义语音效果 > 使用 PlayFab Party 实时音频操作流截获语音聊天缓冲区，并应用自定义效果，如空间音频和语音过滤器。 PlayFab Party 是一款实时联网和语音聊天解决方案。配置为语音聊天时，PlayFab Party 会传输麦克风音频并原样播放。有些游戏需要访问语音聊天音频缓冲区来实现自定义音频效果，如空间音频或语音过滤器。本文档提供了如何使用实时音频操作功能在 PlayFab Party 中截获和修改语音聊天音频的演练。 ## 先决条件 本演练假定你对 [PlayFab Party 中的语音聊天](/services/playfab/community/voice-communications/concepts-chat)有基本了解。 ## 平台支持 并非所有平台都可以使用实时音频操作。虽然与实时音频操作相关的方法存在于统一的跨平台标头中，但它们目前仅针对 Windows、XBOX 和 PlayStation® 5 实现。这些方法在其他平台上将返回错误。 ## 音频流 实时音频操作引入了音频流的概念，用于从库中检索音频或将音频提交到库中。有两种类型的音频流。第一种是**源流**。源流用于从聊天控件检索音频。每个聊天控件只能有一个源流，称为**语音流**。对于本地聊天控件，这用于检索麦克风输入；对于远程聊天控件，这用于检索传入的语音音频。如果聊天控件存在语音流，则库将把该聊天控件的源音频重定向到其语音流，而不是自动处理音频。对于本地聊天控件，这意味着将麦克风音频重定向到语音流，而不是自动编码并传输它；对于远程聊天控件，这意味着将传入的语音音频重定向到语音流，而不是自动将其提交到每个本地聊天控件进行播放。源流由 [`PartyAudioManipulationSourceStream`](/services/playfab/multiplayer/networking/reference/classes/PartyAudioManipulationSourceStream/partyaudiomanipulationsourcestream) 表示。 第二种类型的流是**接收流**。接收流用于向聊天控件提交音频。只有本地聊天控件可以拥有接收流，每个可以有两个。它们分别称为**捕获流**和**渲染流**。如果聊天控件存在捕获流，则库将从捕获流拉取音频进行编码并传输到其他聊天控件，而不是麦克风。如果聊天控件存在渲染流，则库将从渲染流拉取音频并播放，*除了*自动从远程聊天控件播放的语音聊天音频以外。提交到捕获流的音频用作本地聊天控件的麦克风输入；提交到渲染流的音频将播放或“渲染”到本地聊天控件的音频输出设备。接收流由 [`PartyAudioManipulationSinkStream`](/services/playfab/multiplayer/networking/reference/classes/PartyAudioManipulationSinkStream/partyaudiomanipulationsinkstream) 表示。 ### 配置音频流 默认情况下，库处理音频检索、传输和播放。因此，创建聊天控件时不带任何音频流。你可以通过流配置方法为聊天控件创建一个或多个流——[`PartyLocalChatControl::ConfigureAudioManipulationCaptureStream()`](/services/playfab/multiplayer/networking/reference/classes/PartyLocalChatControl/methods/partylocalchatcontrol_configureaudiomanipulationcapturestream)、[`PartyLocalChatControl::ConfigureAudioManipulationRenderStream()`](/services/playfab/multiplayer/networking/reference/classes/PartyLocalChatControl/methods/partylocalchatcontrol_configureaudiomanipulationrenderstream) 和 [`PartyChatControl::ConfigureAudioManipulationVoiceStream()`](/services/playfab/multiplayer/networking/reference/classes/PartyChatControl/methods/partychatcontrol_configureaudiomanipulationvoicestream)。配置后，可以通过 [`PartyLocalChatControl::GetAudioManipulationCaptureStream()`](/services/playfab/multiplayer/networking/reference/classes/PartyLocalChatControl/methods/partylocalchatcontrol_getaudiomanipulationcapturestream)、[`PartyLocalChatControl::GetAudioManipulationRenderStream()`](/services/playfab/multiplayer/networking/reference/classes/PartyLocalChatControl/methods/partylocalchatcontrol_getaudiomanipulationrenderstream) 和 [`PartyChatControl::GetAudioManipulationVoiceStream()`](/services/playfab/multiplayer/networking/reference/classes/PartyChatControl/methods/partychatcontrol_getaudiomanipulationvoicestream) 检索流 每个流配置方法允许你指定将从流中检索或提交到流的音频格式。有关支持的格式的更多信息，请参见每种流配置方法的参考文档。 ### 从源流中检索音频 你可以通过 [`PartyAudioManipulationSourceStream::GetNextBuffer()`](/services/playfab/multiplayer/networking/reference/classes/PartyAudioManipulationSourceStream/methods/partyaudiomanipulationsourcestream_getnextbuffer) 从源流中检索音频。检测到语音活动后，大约每 40 毫秒会有新缓冲区可用。如果没有可用缓冲区，则调用将成功并提供长度为零的缓冲区。可以通过 [`PartyAudioManipulationSourceStream::GetAvailableBufferCount()`](/services/playfab/multiplayer/networking/reference/classes/PartyAudioManipulationSourceStream/methods/partyaudiomanipulationsourcestream_getavailablebuffercount) 检索瞬时可用的缓冲区总数。 为了效率，`GetNextBuffer()` 提供一个指向库内存的缓冲区，而不是复制整个缓冲区。可以选择就地修改它。处理完缓冲区后，应通过 [`PartyAudioManipulationSourceStream::ReturnBuffer()`](/services/playfab/multiplayer/networking/reference/classes/PartyAudioManipulationSourceStream/methods/partyaudiomanipulationsourcestream_returnbuffer) 释放它，以便库可以回收其内存。可以在归还任何缓冲区之前检索多个缓冲区，并且缓冲区不必按其被检索的顺序归还。 ### 向接收流提交音频 你可以通过 [`PartyAudioManipulationSinkStream::SubmitBuffer()`](/services/playfab/multiplayer/networking/reference/classes/PartyAudioManipulationSinkStream/methods/partyaudiomanipulationsinkstream_submitbuffer) 向接收流提交音频。缓冲区由库复制，可以在调用完成后立即释放。 每 40 毫秒，库会消费已提交到接收流的 40 毫秒的音频。为了防止音频卡顿，应以恒定速率提交音频。 ## 场景 ### 麦克风音频操作，又名预编码缓冲区操作 麦克风音频操作是在将麦克风音频传输到其他聊天控件之前截获并更改它的行为。这有时被称为“预编码缓冲区操作”，因为麦克风音频在被编码并传输到其他聊天控件之前被修改。如果你希望为本地聊天控件实现此场景，请首先为本地聊天控件配置语音流和捕获流。配置后，专用音频线程的每次滴答所调用的处理该单个聊天控件的麦克风音频的函数可能类似于下面所示。 ```cpp theme={null} // An app-defined function that takes a microphone buffer and generates a new // buffer that should be transmitted to other chat controls. std::vector ProcessLocalVoiceBuffer( PartyMutableDataBuffer* inputBuffer ); void ProcessLocalMicrophoneAudioForSingleChatControl( PartyLocalChatControl* chatControl ) { // Get the voice stream from which we want to retrieve audio. This provides // the audio generated by the chat control's input device. PartyAudioManipulationSourceStream* voiceStream; RETURN_VOID_IF_FAILED(chatControl->GetAudioManipulationVoiceStream(&voiceStream)); // Get the capture stream to which we want to submit audio. This is used to // submit audio that will be transmitted to other chat controls. PartyAudioManipulationSinkStream* captureStream; RETURN_VOID_IF_FAILED(chatControl->GetAudioManipulationCaptureStream(&captureStream)); // Get the next audio buffer from the voice stream. PartyMutableDataBuffer buffer; RETURN_VOID_IF_FAILED(voiceStream->GetNextBuffer(&buffer)); // If we retrieved a buffer, process it. if (buffer.bufferByteCount > 0) { // Use the buffer we retrieved to generate a new buffer that will be // treated as the "real" capture input and transmitted to other chat // controls. std::vector processedBuffer = ProcessLocalVoiceBuffer(&buffer); // Convert the buffer to a Party type. PartyDataBuffer partyBuffer; partyBuffer.bufferByteCount = static_cast(processedBuffer.size()); partyBuffer.buffer = processedBuffer.data(); // Submit the processed buffer to the capture stream. PartyError error = captureStream->SubmitBuffer(&partyBuffer); if (PARTY_FAILED(error)) { printf("Failed to submit buffer to sink stream! error = 0x%08x", error); } // Return the original buffer back to the voice stream. error = voiceStream->ReturnBuffer(buffer.buffer); if (PARTY_FAILED(error)) { printf("Failed to return buffer to source stream! error = 0x%08x", error); } } } ``` ### 远程音频操作，又名解码后缓冲区操作 远程音频操作是在将传入音频渲染到每个本地聊天控件之前截获并更改它的行为。这有时被称为“解码后缓冲区操作”，因为传入音频在解码后但渲染前被修改。如果你希望实现此场景，请先为每个远程聊天控件配置语音流，并为每个本地聊天控件配置渲染流。然后，音频线程的每次滴答都应从每个语音流中拉取音频，将音频混合成单个流，同时选择性地应用效果，并将混合后的缓冲区提交到每个渲染流。根据你的游戏场景，你可能需要将缓冲区混合成每个本地聊天控件的不同流。专用音频线程的每次滴答所调用的处理传入语音音频的函数可能类似于下面所示。 ```cpp theme={null} // This is an app-defined function that takes a local chat control and list of remote voice buffers and generates // a single mixed buffer to submit to the local chat control's audio output. std::vector GetOutputMixBuffer( PartyLocalChatControl& localChatControl, const std::map& remoteVoiceBuffers ); void ProcessRemoteVoiceAudio( const std::vector& remoteChatControls, const std::vector& localChatControls ) { std::map remoteVoiceBuffers; // Acquire voice buffers from each remote chat control. for (auto remoteChatControl : remoteChatControls) { // Get the voice stream for this chat control from which we will retrieve audio. PartyAudioManipulationSourceStream* voiceStream; PartyError error = remoteChatControl->GetAudioManipulationVoiceStream(&voiceStream); if (PARTY_FAILED(error)) { printf("Failed to get voice stream! error = 0x%08x", error); continue; } // Get the next audio buffer from the voice stream. PartyMutableDataBuffer buffer; error = voiceStream->GetNextBuffer(&buffer); if (PARTY_FAILED(error)) { printf("Failed to get next buffer! error = 0x%08x", error); continue; } // If we retrieved a buffer, cache it in the map for mixing. if (buffer.bufferByteCount > 0) { remoteVoiceBuffers[voiceStream] = buffer; } } // If we didn't acquire any source buffers, we don't have anything to mix. if (remoteVoiceBuffers.empty()) { return; } // Mix the voice buffers and submit to each render stream. for (auto localChatControl : localChatControls) { // Get the render stream for this chat control to which we will submit audio. PartyAudioManipulationSinkStream* renderStream; PartyError error = localChatControl->GetAudioManipulationRenderStream(&renderStream); if (PARTY_FAILED(error)) { printf("Failed to get render stream! error = 0x%08x", error); continue; } // Mix the buffers the buffers to generate a new, mixed buffer. std::vector mixedBuffer = GetOutputMixBuffer(*localChatControl, remoteVoiceBuffers); // Convert the buffer to a party type. PartyDataBuffer partyBuffer; partyBuffer.bufferByteCount = static_cast(mixedBuffer.size()); partyBuffer.buffer = mixedBuffer.data(); // Submit the mixed buffer to the render stream. error = renderStream->SubmitBuffer(&partyBuffer); if (PARTY_FAILED(error)) { printf("Failed to submit buffer to render stream! error = 0x%08x", error); } } // Release the voice buffers. for (auto voiceBuffer : remoteVoiceBuffers) { // Return the voice buffer that we had cached from this voice stream. PartyError error = voiceBuffer.first->ReturnBuffer(voiceBuffer.second.buffer); if (PARTY_FAILED(error)) { printf("Failed to return buffer! error = 0x%08x", error); } } } ``` #### 隐私和混合考虑事项 只要远程聊天控件正在生成音频，并且[聊天权限](/services/playfab/community/voice-communications/concepts-chat-permissions-and-muting#chat-permissions)和[静音](/services/playfab/community/voice-communications/concepts-chat-permissions-and-muting#muting)配置允许音频至少由一个本地聊天控件播放，则库将通过远程聊天控件的语音流提供音频。如果音频应向某个本地聊天控件播放而不向另一个播放，则必须将其从后者聊天控件的音频混合中排除。 #### 聊天指示器注意事项 为远程聊天控件配置语音流不会影响其[聊天指示器](/services/playfab/community/voice-communications/concepts-audio-troubleshooting#check-the-chat-indicators)。你可能需要实现逻辑来协调聊天指示器与你的混合逻辑之间的差异，以选择正确的 UI 指示器。例如，聊天指示器可能表示聊天控件正在说话，但自定义混合逻辑可能选择丢弃该音频。 ### 混合场景 在某些场景中，你可能希望为某些聊天控件启用音频操作，而不对其他聊天控件启用。例如，你可能希望对游戏比赛中遇到的对手玩家应用效果，但不对同一团队的玩家应用效果。在这种情况下，你可以按照之前概述的远程音频操作步骤操作，同时仅为你希望应用音频效果的聊天控件配置语音流。只要静音和权限配置允许，其余远程聊天控件的音频将自动渲染到本地聊天控件。 ## Related topics - [PlayFab Party 发行说明](/zh-CN/services/playfab/multiplayer/networking/release-notes.md) - [PlayFab Party 功能](/zh-CN/services/playfab/multiplayer/networking/party-features.md) - [实时音频处理](/zh-CN/services/xbox-services/multiplayer/chat/game-chat2/real-time-audio-manipulation.md) - [Game Chat 2](/zh-CN/services/xbox-services/multiplayer/chat/game-chat2/index.md) - [PlayFab Party 概览](/zh-CN/services/playfab/multiplayer/networking/index.md)