當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

WebRTC 音频发送和接收处理过程

發布時間：2024/4/11 编程问答 49 豆豆

生活随笔收集整理的這篇文章主要介紹了 WebRTC 音频发送和接收处理过程小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

曾經整理過一個 WebRTC 音頻發送和接收處理的關鍵過程，WebRTC Audio 接收和發送的關鍵過程，不過之前的分析是基于比較老的版本做的。分析所基于的應用程序，依然選擇 WebRTC 的示例應用 peerconnection_client。

這里基于 WebRTC 比較新的 M96 版的代碼，再來看下音頻發送和接收處理過程。

1. 創建 JsepTransportController

#0 webrtc::JsepTransportController::JsepTransportController(rtc::Thread*, cricket::PortAllocator*, webrtc::AsyncDnsResolverFactoryInterface*, webrtc::JsepTransportController::Config) (this=0x0, network_thread=0x0, port_allocator=0x7fffdb3f94f0, async_dns_resolver_factory=0x7fffe4004bc0, config=...)at ../../pc/jsep_transport_controller.cc:41 #1 0x00005555561bdcc7 in webrtc::PeerConnection::InitializeTransportController_n(webrtc::PeerConnectionInterface::RTCConfiguration const&, webrtc::PeerConnectionDependencies const&) (this=0x7fffe40043c0, configuration=..., dependencies=...) at ../../pc/peer_connection.cc:696 #2 0x00005555561ba922 in webrtc::PeerConnection::<lambda()>::operator()(void) const (__closure=0x7fffeeffc030) at ../../pc/peer_connection.cc:615 #3 0x00005555561e0998 in rtc::FunctionView<void()>::CallVoidPtr<webrtc::PeerConnection::Initialize(const webrtc::PeerConnectionInterface::RTCConfiguration&, webrtc::PeerConnectionDependencies)::<lambda()> >(rtc::FunctionView<void()>::VoidUnion) (vu=...) at ../../api/function_view.h:109 #4 0x000055555605c164 in rtc::FunctionView<void ()>::operator()() const (this=0x7fffeeffbec8) at ../../api/function_view.h:95

2. 創建并注冊 transport

#0 webrtc::JsepTransportCollection::RegisterTransport(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<cricket::JsepTransport, std::default_delete<cricket::JsepTransport> >)(this=0x7fffdb3f8f50, mid="", transport=std::unique_ptr<cricket::JsepTransport> = {...}) at ../../pc/jsep_transport_collection.cc:158 #1 0x0000555556b9e732 in webrtc::JsepTransportController::MaybeCreateJsepTransport(bool, cricket::ContentInfo const&, cricket::SessionDescription const&) (this=0x7fffc0000d10, local=false, content_info=..., description=...) at ../../pc/jsep_transport_controller.cc:1082 #2 0x0000555556b9acae in webrtc::JsepTransportController::ApplyDescription_n(bool, webrtc::SdpType, cricket::SessionDescription const*)(this=0x7fffc0000d10, local=false, type=webrtc::SdpType::kOffer, description=0x5555586ca200) at ../../pc/jsep_transport_controller.cc:583 #3 0x0000555556b950e7 in webrtc::JsepTransportController::SetRemoteDescription(webrtc::SdpType, cricket::SessionDescription const*)(this=0x7fffc0000d10, type=webrtc::SdpType::kOffer, description=0x5555586ca200) at ../../pc/jsep_transport_controller.cc:105 #4 0x0000555556b94db6 in webrtc::JsepTransportController::<lambda()>::operator()(void) const (__closure=0x7fffef7fc690)at ../../pc/jsep_transport_controller.cc:101 #5 0x0000555556ba10cd in rtc::FunctionView<webrtc::RTCError()>::CallVoidPtr<webrtc::JsepTransportController::SetRemoteDescription(webrtc::SdpType, const cricket::SessionDescription*)::<lambda()> >(rtc::FunctionView<webrtc::RTCError()>::VoidUnion) (vu=...) at ../../api/function_view.h:109

3. 獲取 RTP transport 并創建 VoiceChannel

在 SdpOfferAnswerHandler::CreateVoiceChannel() 中根據 mid 通過 PeerConnection 獲得 RTP transport

cricket::VoiceChannel* SdpOfferAnswerHandler::CreateVoiceChannel(const std::string& mid) {TRACE_EVENT0("webrtc", "SdpOfferAnswerHandler::CreateVoiceChannel");RTC_DCHECK_RUN_ON(signaling_thread());if (!channel_manager()->media_engine())return nullptr;RtpTransportInternal* rtp_transport = pc_->GetRtpTransport(mid);// TODO(bugs.webrtc.org/11992): CreateVoiceChannel internally switches to the// worker thread. We shouldn't be using the `call_ptr_` hack here but simply// be on the worker thread and use `call_` (update upstream code).return channel_manager()->CreateVoiceChannel(pc_->call_ptr(), pc_->configuration()->media_config, rtp_transport,signaling_thread(), mid, pc_->SrtpRequired(), pc_->GetCryptoOptions(),&ssrc_generator_, audio_options()); }

PeerConnection 將根據 mid 通過 JsepTransportController 獲取 RTP transport：

RtpTransportInternal* PeerConnection::GetRtpTransport(const std::string& mid) {RTC_DCHECK_RUN_ON(signaling_thread());return network_thread()->Invoke<RtpTransportInternal*>(RTC_FROM_HERE, [this, &mid] {auto rtp_transport = transport_controller_->GetRtpTransport(mid);RTC_DCHECK(rtp_transport);return rtp_transport;}); }

然后創建創建 VoiceChannel：

#0 cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)(this=0x7fffeeffaec0, call=0x5555561f550e <rtc::FunctionView<void ()>::CallVoidPtr<rtc::Thread::Invoke<webrtc::RtpTransportInternal*, void>(rtc::Location const&, rtc::FunctionView<webrtc::RtpTransportInternal* ()>)::{lambda()#1}>(rtc::FunctionView<void ()>::VoidUnion)>, media_config=..., rtp_transport=0x55555825bef0 <vtable for rtc::MessageHandler+16>, signaling_thread=0xd68a928ef375e700, content_name="", srtp_required=255, crypto_options=..., ssrc_generator=0xd68a928ef375e700, options=...)at ../../pc/channel_manager.cc:150 #1 0x0000555556272a7a in webrtc::SdpOfferAnswerHandler::CreateVoiceChannel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (this=0x7fffe4005350, mid="0") at ../../pc/sdp_offer_answer.cc:4618 #2 0x0000555556263bfb in webrtc::SdpOfferAnswerHandler::UpdateTransceiverChannel(rtc::scoped_refptr<webrtc::RtpTransceiverProxyWithInternal<webrtc::RtpTransceiver> >, cricket::ContentInfo const&, cricket::ContentGroup const*)(this=0x7fffe4005350, transceiver=..., content=..., bundle_group=0x555558513270) at ../../pc/sdp_offer_answer.cc:3389 #3 0x0000555556260c59 in webrtc::SdpOfferAnswerHandler::UpdateTransceiversAndDataChannels(cricket::ContentSource, webrtc::SessionDescriptionInterface const&, webrtc::SessionDescriptionInterface const*, webrtc::SessionDescriptionInterface const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, cricket::ContentGroup const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, cricket::ContentGroup const*> > > const&)(this=0x7fffe4005350, source=cricket::CS_REMOTE, new_session=..., old_local_description=0x0, old_remote_description=0x0, bundle_groups_by_mid=std::map with 2 elements = {...}) at ../../pc/sdp_offer_answer.cc:3209 #4 0x000055555624d8f5 in webrtc::SdpOfferAnswerHandler::ApplyRemoteDescription(std::unique_ptr<webrtc::SessionDescriptionInterface, std::default_delete<webrtc::SessionDescriptionInterface> >, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, cricket::ContentGroup const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, cricket::ContentGroup const*> > > const&)(this=0x7fffe4005350, desc=std::unique_ptr<webrtc::SessionDescriptionInterface> = {...}, bundle_groups_by_mid=std::map with 2 elements = {...}) at ../../pc/sdp_offer_answer.cc:1599 #5 0x00005555562546ff in webrtc::SdpOfferAnswerHandler::DoSetRemoteDescription(std::unique_ptr<webrtc::SessionDescriptionInterface, std::default_delete<webrtc::SessionDescriptionInterface> >, rtc::scoped_refptr<webrtc::SetRemoteDescriptionObserverInterface>) (this=0x7fffe4005350, desc=std::unique_ptr<webrtc::SessionDescriptionInterface> = {...}, observer=...)at ../../pc/sdp_offer_answer.cc:2196 #6 0x000055555624cbbb in webrtc::SdpOfferAnswerHandler::<lambda(std::function<void()>)>::operator()(std::function<void()>)(__closure=0x7fffeeffc340, operations_chain_callback=...) at ../../pc/sdp_offer_answer.cc:1510 #7 0x00005555562813b9 in rtc::rtc_operations_chain_internal::OperationWithFunctor<webrtc::SdpOfferAnswerHandler::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*)::<lambda(std::function<void()>)> >::Run(void) (this=0x7fffe40185d0) at ../../rtc_base/operations_chain.h:71 #8 0x0000555556278f74 in rtc::OperationsChain::ChainOperation<webrtc::SdpOfferAnswerHandler::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*)::<lambda(std::function<void()>)> >(webrtc::SdpOfferAnswerHandler::<lambda(std::function<void()>)> &&) (this=0x7fffe4005670, functor=...) at ../../rtc_base/operations_chain.h:154 #9 0x000055555624ce45 in webrtc::SdpOfferAnswerHandler::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*) (this=0x7fffe4005350, observer=0x55555881d6e0, desc_ptr=0x5555585ca780) --Type <RET> for more, q to quit, c to continue without paging--.cc:1494 #10 0x00005555561cc02f in webrtc::PeerConnection::SetRemoteDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*)(this=0x7fffe4004580, observer=0x55555881d6e0, desc_ptr=0x5555585ca780) at ../../pc/peer_connection.cc:1362 #11 0x0000555556178768 in webrtc::ReturnType<void>::Invoke<webrtc::PeerConnectionInterface, void (webrtc::PeerConnectionInterface::*)(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*), webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>(webrtc::PeerConnectionInterface*, void (webrtc::PeerConnectionInterface::*)(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*), webrtc::SetSessionDescriptionObserver*&&, webrtc::SessionDescriptionInterface*&&) (this=0x7fffffffc580, c=0x7fffe4004580, m=&virtual table offset 320)at ../../pc/proxy.h:119 #12 0x000055555617412f in webrtc::MethodCall<webrtc::PeerConnectionInterface, void, webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>::Invoke<0ul, 1ul>(std::integer_sequence<unsigned long, 0ul, 1ul>) (this=0x7fffffffc560) at ../../pc/proxy.h:153 #13 0x00005555561835b8 in webrtc::MethodCall<webrtc::PeerConnectionInterface, void, webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>::Run() (this=0x7fffffffc560) at ../../pc/proxy.h:146 #14 0x00005555560575f4 in rtc::Thread::QueuedTaskHandler::OnMessage(rtc::Message*) (this=0x5555585ace98, msg=0x7fffeeffcab0)at ../../rtc_base/thread.cc:1042 #15 0x0000555556055398 in rtc::Thread::Dispatch(rtc::Message*) (this=0x5555585acd70, pmsg=0x7fffeeffcab0) at ../../rtc_base/thread.cc:711

ChannelManager::CreateVoiceChannel() 的實現如下：

VoiceChannel* ChannelManager::CreateVoiceChannel(webrtc::Call* call,const MediaConfig& media_config,webrtc::RtpTransportInternal* rtp_transport,rtc::Thread* signaling_thread,const std::string& content_name,bool srtp_required,const webrtc::CryptoOptions& crypto_options,rtc::UniqueRandomIdGenerator* ssrc_generator,const AudioOptions& options) {RTC_DCHECK(call);RTC_DCHECK(media_engine_);// TODO(bugs.webrtc.org/11992): Remove this workaround after updates in// PeerConnection and add the expectation that we're already on the right// thread.if (!worker_thread_->IsCurrent()) {return worker_thread_->Invoke<VoiceChannel*>(RTC_FROM_HERE, [&] {return CreateVoiceChannel(call, media_config, rtp_transport,signaling_thread, content_name, srtp_required,crypto_options, ssrc_generator, options);});}RTC_DCHECK_RUN_ON(worker_thread_);VoiceMediaChannel* media_channel = media_engine_->voice().CreateMediaChannel(call, media_config, options, crypto_options);if (!media_channel) {return nullptr;}auto voice_channel = std::make_unique<VoiceChannel>(worker_thread_, network_thread_, signaling_thread,absl::WrapUnique(media_channel), content_name, srtp_required,crypto_options, ssrc_generator);voice_channel->Init_w(rtp_transport);VoiceChannel* voice_channel_ptr = voice_channel.get();voice_channels_.push_back(std::move(voice_channel));return voice_channel_ptr; }

4. 為 BaseChannel/VoiceChannel 設置 RtpTransport

為 BaseChannel 設置 RtpTransport 的動作在 BaseChannel 中 BaseChannel::Init_w() 函數里的一個 lambda 表達式中完成：

#0 cricket::BaseChannel::SetRtpTransport(webrtc::RtpTransportInternal*)(this=0x555556ba2534 <cricket::JsepTransport::rtp_transport() const+52>, rtp_transport=0x7fffdb3f9430)at ../../pc/channel.cc:222 #1 0x0000555556b6f1f6 in cricket::BaseChannel::<lambda()>::operator()(void) const (__closure=0x7fffdabf8300)at ../../pc/channel.cc:200 #2 0x0000555556b7d400 in rtc::FunctionView<void()>::CallVoidPtr<cricket::BaseChannel::Init_w(webrtc::RtpTransportInternal*)::<lambda()> >(rtc::FunctionView<void()>::VoidUnion) (vu=...) at ../../api/function_view.h:109

BaseChannel::Init_w() 的調用過程如下：

#0 cricket::BaseChannel::Init_w(webrtc::RtpTransportInternal*) (this=0x7fffdabf8328, rtp_transport=0x7fffdabf82c0)at ../../pc/channel.cc:196 #1 0x0000555556b8dd8a in cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)(this=0x7fffc408c160, call=0x7fffc4090080, media_config=..., rtp_transport=0x7fffc0003e00, signaling_thread=0x5555586bfd90, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe0007710, options=...)at ../../pc/channel_manager.cc:177 #2 0x0000555556b8d86d in cricket::ChannelManager::<lambda()>::operator()(void) const (__closure=0x7fffef7fbf30)at ../../pc/channel_manager.cc:158 #3 0x0000555556b8f4dd in rtc::FunctionView<cricket::VoiceChannel*()>::CallVoidPtr<cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, const cricket::MediaConfig&, webrtc::RtpTransportInternal*, rtc::Thread*, const string&, bool, const webrtc::CryptoOptions&, rtc::UniqueRandomIdGenerator*, const cricket::AudioOptions&)::<lambda()> >(rtc::FunctionView<cricket::VoiceChannel*()>::VoidUnion) (vu=...) at ../../api/function_view.h:109

BaseChannel::Init_w() 的代碼如下：

void BaseChannel::Init_w(webrtc::RtpTransportInternal* rtp_transport) {RTC_DCHECK_RUN_ON(worker_thread());network_thread_->Invoke<void>(RTC_FROM_HERE, [this, rtp_transport] {SetRtpTransport(rtp_transport);// Both RTP and RTCP channels should be set, we can call SetInterface on// the media channel and it can set network options.media_channel_->SetInterface(this);}); }

這段代碼將數據包發送過程中的多個組件連接起來。MediaChannel::SetInterface(NetworkInterface* iface) 接口需要一個 MediaChannel::NetworkInterface 接口的實現，MediaChannel 可以通過這個接口發送數據包：

class MediaChannel {public:class NetworkInterface {public:enum SocketType { ST_RTP, ST_RTCP };virtual bool SendPacket(rtc::CopyOnWriteBuffer* packet,const rtc::PacketOptions& options) = 0;virtual bool SendRtcp(rtc::CopyOnWriteBuffer* packet,const rtc::PacketOptions& options) = 0;virtual int SetOption(SocketType type,rtc::Socket::Option opt,int option) = 0;virtual ~NetworkInterface() {}};

BaseChannel 實現 MediaChannel::NetworkInterface 接口。BaseChannel::Init_w() 將 MediaChannel 、BaseChannel 和 RtpTransportInternal 這三個組件連接起來。

5. MediaChannel 是 WebRtcVoiceMediaChannel

BaseChannel 的 MediaChannel 在對象構造的時候傳入：

#0 cricket::BaseChannel::BaseChannel(rtc::Thread*, rtc::Thread*, rtc::Thread*, std::unique_ptr<cricket::MediaChannel, std::default_delete<cricket::MediaChannel> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions, rtc::UniqueRandomIdGenerator*)(this=0x7fffdabf80e0, worker_thread=0x7fffdabf8100, network_thread=0x7fffdabf80f0, signaling_thread=0x7fffdabf80e0, media_channel=std::unique_ptr<cricket::MediaChannel> = {...}, content_name="", srtp_required=255, crypto_options=..., ssrc_generator=0xc4d93a8ca0272e00) at ../../pc/channel.cc:117 #1 0x0000555556b77d44 in cricket::VoiceChannel::VoiceChannel(rtc::Thread*, rtc::Thread*, rtc::Thread*, std::unique_ptr<cricket::VoiceMediaChannel, std::default_delete<cricket::VoiceMediaChannel> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions, rtc::UniqueRandomIdGenerator*)(this=0x7fffc4092460, worker_thread=0x7fffe4003030, network_thread=0x7fffe4002600, signaling_thread=0x555558805600, media_channel=std::unique_ptr<cricket::VoiceMediaChannel> = {...}, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe40077b0) at ../../pc/channel.cc:797 #2 0x0000555556b9001a in std::make_unique<cricket::VoiceChannel, rtc::Thread* const&, rtc::Thread* const&, rtc::Thread*&, std::unique_ptr<cricket::VoiceMediaChannel, std::default_delete<cricket::VoiceMediaChannel> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*&>(rtc::Thread* const&, rtc::Thread* const&, rtc::Thread*&, std::unique_ptr<cricket::VoiceMediaChannel, std::default_delete<cricket::VoiceMediaChannel> >&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*&) () at /usr/include/c++/9/bits/unique_ptr.h:857 #3 0x0000555556b8dd4f in cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)(this=0x7fffc408c160, call=0x7fffc4090090, media_config=..., rtp_transport=0x7fffc0003f10, signaling_thread=0x555558805600, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe40077b0, options=...) at ../../pc/channel_manager.cc:175 #4 0x0000555556b8d86d in cricket::ChannelManager::<lambda()>::operator()(void) const (__closure=0x7fffeeffaf30) at ../../pc/channel_manager.cc:158

ChannelManager 中通過 WebRtcVoiceEngine/VoiceEngine 創建 VoiceChannel/BaseChannel 的過程如下：

MediaChannel/VoiceMediaChannel 是 WebRtcVoiceMediaChannel:

#0 webrtc::internal::Call::worker_thread() const (this=0x5555557d146f <webrtc::MutexLock::~MutexLock()+31>) at ../../call/call.cc:1285 #1 0x00005555560d9e8f in cricket::WebRtcVoiceEngine::CreateMediaChannel(webrtc::Call*, cricket::MediaConfig const&, cricket::AudioOptions const&, webrtc::CryptoOptions const&) (this=0x55555844a800, call=0x7fffc4090090, config=..., options=..., crypto_options=...)at ../../media/engine/webrtc_voice_engine.cc:432 #2 0x0000555556b8dcc1 in cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, cricket::MediaConfig const&, webrtc::RtpTransportInternal*, rtc::Thread*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, webrtc::CryptoOptions const&, rtc::UniqueRandomIdGenerator*, cricket::AudioOptions const&)(this=0x7fffc408c160, call=0x7fffc4090090, media_config=..., rtp_transport=0x7fffc0003f20, signaling_thread=0x55555861bdd0, content_name="0", srtp_required=true, crypto_options=..., ssrc_generator=0x7fffe40055b0, options=...) at ../../pc/channel_manager.cc:166 #3 0x0000555556b8d86d in cricket::ChannelManager::<lambda()>::operator()(void) const (__closure=0x7fffeeffaf50) at ../../pc/channel_manager.cc:158 #4 0x0000555556b8f4dd in rtc::FunctionView<cricket::VoiceChannel*()>::CallVoidPtr<cricket::ChannelManager::CreateVoiceChannel(webrtc::Call*, const cricket::MediaConfig&, webrtc::RtpTransportInternal*, rtc::Thread*, const string&, bool, const webrtc::CryptoOptions&, rtc::UniqueRandomIdGenerator*, const cricket::AudioOptions&)::<lambda()> >(rtc::FunctionView<cricket::VoiceChannel*()>::VoidUnion) (vu=...) at ../../api/function_view.h:109

6. 音頻數據包的發送處理

音頻數據包的編碼發送，分為幾個階段。

第一階段，錄制線程將錄制的麥克風音頻數據拋上來，送進編碼線程。

#0 webrtc::voe::(anonymous namespace)::ChannelSend::ProcessAndEncodeAudio(std::unique_ptr<webrtc::AudioFrame, std::default_delete<webrtc::AudioFrame> >) (this=0x5555561264fa <webrtc::AudioProcessingImpl::ApmStatsReporter::GetStatistics()+106>, audio_frame=std::unique_ptr<webrtc::AudioFrame> = {...})at ../../audio/channel_send.cc:815 #1 0x000055555698a90e in webrtc::internal::AudioSendStream::SendAudioData(std::unique_ptr<webrtc::AudioFrame, std::default_delete<webrtc::AudioFrame> >) (this=0x7fffc40dab90, audio_frame=std::unique_ptr<webrtc::AudioFrame> = {...}) at ../../audio/audio_send_stream.cc:422 #2 0x000055555699e2e4 in webrtc::AudioTransportImpl::SendProcessedData(std::unique_ptr<webrtc::AudioFrame, std::default_delete<webrtc::AudioFrame> >)(this=0x7fffc4024f60, audio_frame=std::unique_ptr<webrtc::AudioFrame> = {...}) at ../../audio/audio_transport_impl.cc:190 #3 0x000055555699df46 in webrtc::AudioTransportImpl::RecordedDataIsAvailable(void const*, unsigned long, unsigned long, unsigned long, unsigned int, unsigned int, int, unsigned int, bool, unsigned int&)(this=0x7fffc4024f60, audio_data=0x7fff84001030, number_of_frames=441, bytes_per_sample=4, number_of_channels=2, sample_rate=44100, audio_delay_milliseconds=207, key_pressed=false) at ../../audio/audio_transport_impl.cc:171 #4 0x00005555565b7862 in webrtc::AudioDeviceBuffer::DeliverRecordedData() (this=0x7fffc4002c28)at ../../modules/audio_device/audio_device_buffer.cc:270 #5 0x00005555565a90c0 in webrtc::AudioDeviceLinuxPulse::ProcessRecordedData(signed char*, unsigned int, unsigned int)(this=0x7fffc4004de0, bufferData=0x7fffb0910374 "\r", bufferSizeInSamples=441, recDelay=10)at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:1968 #6 0x00005555565a8f80 in webrtc::AudioDeviceLinuxPulse::ReadRecordedData(void const*, unsigned long)(this=0x7fffc4004de0, bufferData=0x7fffb0910374, bufferSize=4408) at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:1926 #7 0x00005555565abcc1 in webrtc::AudioDeviceLinuxPulse::RecThreadProcess() (this=0x7fffc4004de0)at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:2247 #8 0x000055555659e509 in webrtc::AudioDeviceLinuxPulse::<lambda()>::operator()(void) const (__closure=0x7fffc4003740)at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:166 #9 0x00005555565abf53 in std::_Function_handler<void(), webrtc::AudioDeviceLinuxPulse::Init()::<lambda()> >::_M_invoke(const std::_Any_data &)(__functor=...) at /usr/include/c++/9/bits/std_function.h:300

第二階段，編碼線程將編碼的數據送進 PacedSender 的隊列里。

#0 webrtc::PacingController::EnqueuePacket(std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >)(this=0x55555692fb9a <std::tuple<webrtc::RtpPacketToSend*, std::default_delete<webrtc::RtpPacketToSend> >::tuple<webrtc::RtpPacketToSend*&, std::default_delete<webrtc::RtpPacketToSend>, true>(webrtc::RtpPacketToSend*&, std::default_delete<webrtc::RtpPacketToSend>&&)+72>, packet=std::unique_ptr<webrtc::RtpPacketToSend> = {...}) at ../../modules/pacing/pacing_controller.cc:235 #1 0x000055555692edbc in webrtc::PacedSender::EnqueuePackets(std::vector<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >, std::allocator<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> > > >)(this=0x7fffc408e530, packets=std::vector of length 1, capacity 1 = {...}) at ../../modules/pacing/paced_sender.cc:128 #2 0x00005555569b1a6b in webrtc::voe::(anonymous namespace)::RtpPacketSenderProxy::EnqueuePackets(std::vector<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >, std::allocator<std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> > > >) (this=0x7fffc40db280, packets=std::vector of length 0, capacity 0) at ../../audio/channel_send.cc:270 #3 0x0000555557425210 in webrtc::RTPSender::SendToNetwork(std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >)(this=0x7fffc40f35d0, packet=std::unique_ptr<webrtc::RtpPacketToSend> = {...}) at ../../modules/rtp_rtcp/source/rtp_sender.cc:499 #4 0x000055555742b1bb in webrtc::RTPSenderAudio::SendAudio(webrtc::AudioFrameType, signed char, unsigned int, unsigned char const*, unsigned long, long)(this=0x7fffc40f4870, frame_type=webrtc::AudioFrameType::kAudioFrameSpeech, payload_type=111 'o', rtp_timestamp=1692995353, payload_data=0x7fff9c000b90 "x\v\350A1[\355\301e?\257~-rs\252\261\206\314\333X\315\"N}\022\251y\235\352X\234\021x\223\260\206\261\255\243wR7T\317\353\177\035\366\211", <incomplete sequence \362\274>, payload_size=56, absolute_capture_timestamp_ms=-1) at ../../modules/rtp_rtcp/source/rtp_sender_audio.cc:316 #5 0x00005555569b27a4 in webrtc::voe::(anonymous namespace)::ChannelSend::SendRtpAudio(webrtc::AudioFrameType, uint8_t, uint32_t, rtc::ArrayView<unsigned char const, -4711>, int64_t)(this=0x7fffc40db010, frameType=webrtc::AudioFrameType::kAudioFrameSpeech, payloadType=111 'o', rtp_timestamp=0, payload=..., absolute_capture_timestamp_ms=-1) at ../../audio/channel_send.cc:442 #6 0x00005555569b20f7 in webrtc::voe::(anonymous namespace)::ChannelSend::SendData(webrtc::AudioFrameType, uint8_t, uint32_t, uint8_t const*, size_t, int64_t)(this=0x7fffc40db010, frameType=webrtc::AudioFrameType::kAudioFrameSpeech, payloadType=111 'o', rtp_timestamp=0, payloadData=0x7fff9c000b90 "x\v\350A1[\355\301e?\257~-rs\252\261\206\314\333X\315\"N}\022\251y\235\352X\234\021x\223\260\206\261\255\243wR7T\317\353\177\035\366\211", <incomplete sequence \362\274>, payloadSize=56, absolute_capture_timestamp_ms=-1) at ../../audio/channel_send.cc:370 #7 0x00005555569c9b0c in webrtc::(anonymous namespace)::AudioCodingModuleImpl::Encode(webrtc::(anonymous namespace)::AudioCodingModuleImpl::InputData const&, absl::optional<long>) (this=0x7fffc40dca00, input_data=..., absolute_capture_timestamp_ms=...)at ../../modules/audio_coding/acm2/audio_coding_module.cc:302 #8 0x00005555569c9c9d in webrtc::(anonymous namespace)::AudioCodingModuleImpl::Add10MsData(webrtc::AudioFrame const&)(this=0x7fffc40dca00, audio_frame=...) at ../../modules/audio_coding/acm2/audio_coding_module.cc:339 #9 0x00005555569b8269 in webrtc::voe::(anonymous namespace)::ChannelSend::<lambda()>::operator()(void) (__closure=0x7fff880e1d98)at ../../audio/channel_send.cc:864 #10 0x00005555569bd044 in webrtc::webrtc_new_closure_impl::ClosureTask<webrtc::voe::(anonymous namespace)::ChannelSend::ProcessAndEncodeAudio(std::unique_ptr<webrtc::AudioFrame>)::<lambda()> >::Run(void) (this=0x7fff880e1d90) at ../../rtc_base/task_utils/to_queued_task.h:32

第三階段，PacedSender 將 RTP 包通過 MediaChannel 發送出去。

#0 cricket::MediaChannel::SendRtp(unsigned char const*, unsigned long, webrtc::PacketOptions const&)(this=0x7fff9c001648, data=0xffffffffffffffff <error: Cannot access memory at address 0xffffffffffffffff>, len=93825004092765, options=...)at ../../media/base/media_channel.cc:169 #1 0x00005555560f11dc in cricket::WebRtcVoiceMediaChannel::SendRtp(unsigned char const*, unsigned long, webrtc::PacketOptions const&)(this=0x7fffc4091a90, data=0x7fff9c0016d0 "\220\357\061&d\351\017\031\342f\335\\\276", <incomplete sequence \336>, len=84, options=...)at ../../media/engine/webrtc_voice_engine.cc:2572 #2 0x0000555557430fd4 in webrtc::RtpSenderEgress::SendPacketToNetwork(webrtc::RtpPacketToSend const&, webrtc::PacketOptions const&, webrtc::PacedPacketInfo const&) (this=0x7fffc40f3310, packet=..., options=..., pacing_info=...) at ../../modules/rtp_rtcp/source/rtp_sender_egress.cc:555 #3 0x000055555742e285 in webrtc::RtpSenderEgress::SendPacket(webrtc::RtpPacketToSend*, webrtc::PacedPacketInfo const&)(this=0x7fffc40f3310, packet=0x7fff9c0015f0, pacing_info=...) at ../../modules/rtp_rtcp/source/rtp_sender_egress.cc:273 #4 0x000055555741cc89 in webrtc::ModuleRtpRtcpImpl2::TrySendPacket(webrtc::RtpPacketToSend*, webrtc::PacedPacketInfo const&)(this=0x7fffc40f2800, packet=0x7fff9c0015f0, pacing_info=...) at ../../modules/rtp_rtcp/source/rtp_rtcp_impl2.cc:376 #5 0x0000555556936fd5 in webrtc::PacketRouter::SendPacket(std::unique_ptr<webrtc::RtpPacketToSend, std::default_delete<webrtc::RtpPacketToSend> >, webrtc::PacedPacketInfo const&) (this=0x7fffc408df18, packet=std::unique_ptr<webrtc::RtpPacketToSend> = {...}, cluster_info=...)at ../../modules/pacing/packet_router.cc:160 #6 0x00005555569347ca in webrtc::PacingController::ProcessPackets() (this=0x7fffc408e598) at ../../modules/pacing/pacing_controller.cc:590 #7 0x000055555692f1e3 in webrtc::PacedSender::Process() (this=0x7fffc408e530) at ../../modules/pacing/paced_sender.cc:183 #8 0x000055555692f6cb in webrtc::PacedSender::ModuleProxy::Process() (this=0x7fffc408e548) at ../../modules/pacing/paced_sender.h:152 #9 0x00005555573aaafe in webrtc::ProcessThreadImpl::Process() (this=0x7fffc408daa0) at ../../modules/utility/source/process_thread_impl.cc:257 #10 0x00005555573a8e9b in webrtc::ProcessThreadImpl::<lambda()>::operator()(void) const (__closure=0x7fffc4095e50)at ../../modules/utility/source/process_thread_impl.cc:86

MediaChannel::SendRtp() 會將發送轉到網絡發送線程中去。

void MediaChannel::SendRtp(const uint8_t* data,size_t len,const webrtc::PacketOptions& options) {auto send =[this, packet_id = options.packet_id,included_in_feedback = options.included_in_feedback,included_in_allocation = options.included_in_allocation,packet = rtc::CopyOnWriteBuffer(data, len, kMaxRtpPacketLen)]() mutable {rtc::PacketOptions rtc_options;rtc_options.packet_id = packet_id;if (DscpEnabled()) {rtc_options.dscp = PreferredDscp();}rtc_options.info_signaled_after_sent.included_in_feedback =included_in_feedback;rtc_options.info_signaled_after_sent.included_in_allocation =included_in_allocation;SendPacket(&packet, rtc_options);};// TODO(bugs.webrtc.org/11993): ModuleRtpRtcpImpl2 and related classes (e.g.// RTCPSender) aren't aware of the network thread and may trigger calls to// this function from different threads. Update those classes to keep// network traffic on the network thread.if (network_thread_->IsCurrent()) {send();} else {network_thread_->PostTask(ToQueuedTask(network_safety_, std::move(send)));} }

第四階段，通過 socket 接口將數據包發送到網絡。

#0 rtc::PhysicalSocket::DoSendTo(int, char const*, int, int, sockaddr const*, unsigned int)(this=0x5555563da778 <rtc::SocketAddress::ToSockAddrStorage(sockaddr_storage*) const+58>, socket=32767, buf=0x7fffdb3f8770 "\001\001", len=-616594960, flags=0, dest_addr=0x0, addrlen=16) at ../../rtc_base/physical_socket_server.cc:509 #1 0x00005555560463f3 in rtc::PhysicalSocket::SendTo(void const*, unsigned long, rtc::SocketAddress const&)(this=0x7fffc00069f8, buffer=0x7fffe0003560, length=100, addr=...) at ../../rtc_base/physical_socket_server.cc:375 #2 0x0000555557346e8e in rtc::AsyncUDPSocket::SendTo(void const*, unsigned long, rtc::SocketAddress const&, rtc::PacketOptions const&)(this=0x7fffc0005210, pv=0x7fffe0003560, cb=100, addr=..., options=...) at ../../rtc_base/async_udp_socket.cc:84 #3 0x000055555731241a in cricket::UDPPort::SendTo(void const*, unsigned long, rtc::SocketAddress const&, rtc::PacketOptions const&, bool)(this=0x7fffc00196a0, data=0x7fffe0003560, size=100, addr=..., options=..., payload=true) at ../../p2p/base/stun_port.cc:286 #4 0x000055555730d814 in cricket::ProxyConnection::Send(void const*, unsigned long, rtc::PacketOptions const&)(this=0x7fffc001a7f0, data=0x7fffe0003560, size=100, options=...) at ../../p2p/base/connection.cc:1371 #5 0x000055555728dfac in cricket::P2PTransportChannel::SendPacket(char const*, unsigned long, rtc::PacketOptions const&, int)(this=0x7fffc0002b20, data=0x7fffe0003560 "\220o1Id\351\222Y\342f\335\\\276", <incomplete sequence \336>, len=100, options=..., flags=0)at ../../p2p/base/p2p_transport_channel.cc:1616 #6 0x000055555726e7de in cricket::DtlsTransport::SendPacket(char const*, unsigned long, rtc::PacketOptions const&, int)(this=0x7fffc00033c0, data=0x7fffe0003560 "\220o1Id\351\222Y\342f\335\\\276", <incomplete sequence \336>, size=100, options=..., flags=1)at ../../p2p/base/dtls_transport.cc:417 #7 0x0000555556bd5522 in webrtc::RtpTransport::SendPacket(bool, rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&, int)(this=0x7fffc0003e00, rtcp=false, packet=0x7fffe00154e8, options=..., flags=1) at ../../pc/rtp_transport.cc:147 #8 0x0000555556bdf39c in webrtc::SrtpTransport::SendRtpPacket(rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&, int)(this=0x7fffc0003e00, packet=0x7fffe00154e8, options=..., flags=1) at ../../pc/srtp_transport.cc:173 #9 0x0000555556b731d8 in cricket::BaseChannel::SendPacket(bool, rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&)(this=0x7fffc4092460, rtcp=false, packet=0x7fffe00154e8, options=...) at ../../pc/channel.cc:437 #10 0x0000555556b710c9 in cricket::BaseChannel::SendPacket(rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&)(this=0x7fffc4092460, packet=0x7fffe00154e8, options=...) at ../../pc/channel.cc:318 #11 0x0000555557257ea2 in cricket::MediaChannel::DoSendPacket(rtc::CopyOnWriteBuffer*, bool, rtc::PacketOptions const&)(this=0x7fffc4091a90, packet=0x7fffe00154e8, rtcp=false, options=...) at ../../media/base/media_channel.cc:163 #12 0x0000555557257496 in cricket::MediaChannel::SendPacket(rtc::CopyOnWriteBuffer*, rtc::PacketOptions const&)(this=0x7fffc4091a90, packet=0x7fffe00154e8, options=...) at ../../media/base/media_channel.cc:71 #13 0x0000555557257f9a in cricket::MediaChannel::<lambda()>::operator()(void) (__closure=0x7fffe00154d8) at ../../media/base/media_channel.cc:184 #14 0x000055555725aeb2 in webrtc::webrtc_new_closure_impl::SafetyClosureTask<cricket::MediaChannel::SendRtp(const uint8_t*, size_t, const webrtc::PacketOptions&)::<lambda()> >::Run(void) (this=0x7fffe00154d0) at ../../rtc_base/task_utils/to_queued_task.h:50

PhysicalSocket::DoSendTo() 實現如下：

int PhysicalSocket::DoSendTo(SOCKET socket,const char* buf,int len,int flags,const struct sockaddr* dest_addr,socklen_t addrlen) {return ::sendto(socket, buf, len, flags, dest_addr, addrlen); }

在 AudioTransportImpl::RecordedDataIsAvailable() 中有如下這段代碼：

RTC_DCHECK_GT(audio_frame->samples_per_channel_, 0);if (async_audio_processing_)async_audio_processing_->Process(std::move(audio_frame));elseSendProcessedData(std::move(audio_frame));return 0; }

這也就意味著，在一些情況下，音頻發送過程會多轉一次線程。

7. 音頻數據包的接收處理

音頻數據包的接收處理分為幾個階段。

第一階段，從網絡接收音頻 RTP 包。

#0 cricket::WebRtcVoiceMediaChannel::OnPacketReceived(rtc::CopyOnWriteBuffer, long)(this=0x5555557d7c73 <rtc::dchecked_cast<long, long>(long)+28>, packet=..., packet_time_us=1640606419828691)at ../../media/engine/webrtc_voice_engine.cc:2217 #1 0x0000555556b736fe in cricket::BaseChannel::OnRtpPacket(webrtc::RtpPacketReceived const&) (this=0x7fffc4092460, parsed_packet=...)at ../../pc/channel.cc:467 #2 0x00005555568bec90 in webrtc::RtpDemuxer::OnRtpPacket(webrtc::RtpPacketReceived const&) (this=0x7fffc00040a8, packet=...)at ../../call/rtp_demuxer.cc:249 #3 0x0000555556bd5d12 in webrtc::RtpTransport::DemuxPacket(rtc::CopyOnWriteBuffer, long)(this=0x7fffc0003f20, packet=..., packet_time_us=1640606419828691) at ../../pc/rtp_transport.cc:194 #4 0x0000555556be0551 in webrtc::SrtpTransport::OnRtpPacketReceived(rtc::CopyOnWriteBuffer, long)(this=0x7fffc0003f20, packet=..., packet_time_us=1640606419828691) at ../../pc/srtp_transport.cc:226 #5 0x0000555556bd6898 in webrtc::RtpTransport::OnReadPacket(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)(this=0x7fffc0003f20, transport=0x7fffc00034e0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, len=106, packet_time_us=@0x7fffdb3f8bc8: 1640606419828691, flags=1) at ../../pc/rtp_transport.cc:268 #6 0x0000555556bd7d41 in sigslot::_opaque_connection::emitter<webrtc::RtpTransport, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(sigslot::_opaque_connection const*, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) (self=0x7fffc0004420)at ../../rtc_base/third_party/sigslot/sigslot.h:342 #7 0x0000555557276ec7 in sigslot::_opaque_connection::emit<rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) const (this=0x7fffc0004420) at ../../rtc_base/third_party/sigslot/sigslot.h:331 #8 0x000055555727656d in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::emit(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)(this=0x7fffc00035d8, args#0=0x7fffc00034e0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, args#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=1) at ../../rtc_base/third_party/sigslot/sigslot.h:566 #9 0x0000555557275b90 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::operator()(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)(this=0x7fffc00035d8, args#0=0x7fffc00034e0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, args#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=1) at ../../rtc_base/third_party/sigslot/sigslot.h:570 #10 0x000055555727172e in cricket::DtlsTransport::OnReadPacket(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)(this=0x7fffc00034e0, transport=0x7fffc0002c40, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, size=106, packet_time_us=@0x7fffdb3f8bc8: 1640606419828691, flags=0) at ../../p2p/base/dtls_transport.cc:627 #11 0x0000555557276d64 in sigslot::_opaque_connection::emitter<cricket::DtlsTransport, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(sigslot::_opaque_connection const*, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) (self=0x7fffc0002340)at ../../rtc_base/third_party/sigslot/sigslot.h:342 #12 0x0000555557276ec7 in sigslot::_opaque_connection::emit<rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int) const (this=0x7fffc0002340) at ../../rtc_base/third_party/sigslot/sigslot.h:331 #13 0x000055555727656d in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::emit(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)(this=0x7fffc0002d38, args#0=0x7fffc0002c40, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, args#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=0) at ../../rtc_base/third_party/sigslot/sigslot.h:566 #14 0x0000555557275b90 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int>::operator()(rtc::PacketTransportInternal*, char const*, unsigned long, long const&, int)(this=0x7fffc0002d38, args#0=0x7fffc0002c40, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, ar--Type <RET> for more, q to quit, c to continue without paging-- gs#3=@0x7fffdb3f8bc8: 1640606419828691, args#4=0) at ../../rtc_base/third_party/sigslot/sigslot.h:570 #15 0x0000555557295a6d in cricket::P2PTransportChannel::OnReadPacket(cricket::Connection*, char const*, unsigned long, long)(this=0x7fffc0002c40, connection=0x7fffc0031650, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, len=106, packet_time_us=1640606419828691) at ../../p2p/base/p2p_transport_channel.cc:2228 #16 0x00005555572a4e14 in sigslot::_opaque_connection::emitter<cricket::P2PTransportChannel, cricket::Connection*, char const*, unsigned long, long>(sigslot::_opaque_connection const*, cricket::Connection*, char const*, unsigned long, long) (self=0x7fffc40ccb20)at ../../rtc_base/third_party/sigslot/sigslot.h:342 #17 0x000055555730fcb0 in sigslot::_opaque_connection::emit<cricket::Connection*, char const*, unsigned long, long>(cricket::Connection*, char const*, unsigned long, long) const (this=0x7fffc40ccb20) at ../../rtc_base/third_party/sigslot/sigslot.h:331 #18 0x000055555730f612 in sigslot::signal_with_thread_policy<sigslot::single_threaded, cricket::Connection*, char const*, unsigned long, long>::emit(cricket::Connection*, char const*, unsigned long, long)(this=0x7fffc0031720, args#0=0x7fffc0031650, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, args#3=1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:566 #19 0x000055555730ed67 in sigslot::signal_with_thread_policy<sigslot::single_threaded, cricket::Connection*, char const*, unsigned long, long>::operator()(cricket::Connection*, char const*, unsigned long, long)(this=0x7fffc0031720, args#0=0x7fffc0031650, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, args#3=1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:570 #20 0x00005555573004c7 in cricket::Connection::OnReadPacket(char const*, unsigned long, long)(this=0x7fffc0031650, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, size=106, packet_time_us=1640606419828691) at ../../p2p/base/connection.cc:465 #21 0x00005555573134b8 in cricket::UDPPort::OnReadPacket(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)(this=0x7fffc002aa60, socket=0x7fffc0009fa0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, size=106, remote_addr=..., packet_time_us=@0x7fffdb3f9260: 1640606419828691) at ../../p2p/base/stun_port.cc:394 #22 0x0000555557312fa4 in cricket::UDPPort::HandleIncomingPacket(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long)(this=0x7fffc002aa60, socket=0x7fffc0009fa0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, size=106, remote_addr=..., packet_time_us=1640606419828691) at ../../p2p/base/stun_port.cc:335 #23 0x00005555572d854f in cricket::AllocationSequence::OnReadPacket(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)(this=0x7fffc0009e60, socket=0x7fffc0009fa0, data=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, size=106, remote_addr=..., packet_time_us=@0x7fffdb3f9600: 1640606419828691) at ../../p2p/client/basic_port_allocator.cc:1641 #24 0x00005555572e857e in sigslot::_opaque_connection::emitter<cricket::AllocationSequence, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&>(sigslot::_opaque_connection const*, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&) (self=0x7fffc001a1c0) at ../../rtc_base/third_party/sigslot/sigslot.h:342 #25 0x00005555572eff15 in sigslot::_opaque_connection::emit<rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&>(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&) const (this=0x7fffc001a1c0)at ../../rtc_base/third_party/sigslot/sigslot.h:331 #26 0x00005555572efda3 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&>::emit(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)(this=0x7fffc0009ff0, args#0=0x7fffc0009fa0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, args#3=..., args#4=@0x7fffdb3f9600: 1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:566 #27 0x00005555572efbe0 in sigslot::signal_with_thread_policy<sigslot::single_threaded, rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketA--Type <RET> for more, q to quit, c to continue without paging-- ddress const&, long const&>::operator()(rtc::AsyncPacketSocket*, char const*, unsigned long, rtc::SocketAddress const&, long const&)(this=0x7fffc0009ff0, args#0=0x7fffc0009fa0, args#1=0x7fffc000a1a0 "\220o\\\026_\372\242\367\006~?\276", <incomplete sequence \336>, args#2=106, args#3=..., args#4=@0x7fffdb3f9600: 1640606419828691) at ../../rtc_base/third_party/sigslot/sigslot.h:570 #28 0x00005555573475ff in rtc::AsyncUDPSocket::OnReadEvent(rtc::Socket*) (this=0x7fffc0009fa0, socket=0x7fffc00096e8)at ../../rtc_base/async_udp_socket.cc:132 #29 0x00005555573477e9 in sigslot::_opaque_connection::emitter<rtc::AsyncUDPSocket, rtc::Socket*>(sigslot::_opaque_connection const*, rtc::Socket*)(self=0x7fffc0009b70) at ../../rtc_base/third_party/sigslot/sigslot.h:342 #30 0x000055555604eb9d in sigslot::_opaque_connection::emit<rtc::Socket*>(rtc::Socket*) const (this=0x7fffc0009b70)at ../../rtc_base/third_party/sigslot/sigslot.h:331 #31 0x000055555604d8e6 in sigslot::signal_with_thread_policy<sigslot::multi_threaded_local, rtc::Socket*>::emit(rtc::Socket*)(this=0x7fffc00096f0, args#0=0x7fffc00096e8) at ../../rtc_base/third_party/sigslot/sigslot.h:566 #32 0x000055555604ccb5 in sigslot::signal_with_thread_policy<sigslot::multi_threaded_local, rtc::Socket*>::operator()(rtc::Socket*)(this=0x7fffc00096f0, args#0=0x7fffc00096e8) at ../../rtc_base/third_party/sigslot/sigslot.h:570 #33 0x0000555556047f11 in rtc::SocketDispatcher::OnEvent(unsigned int, int) (this=0x7fffc00096e0, ff=1, err=0)at ../../rtc_base/physical_socket_server.cc:831 #34 0x00005555560492d4 in rtc::ProcessEvents(rtc::Dispatcher*, bool, bool, bool)(dispatcher=0x7fffc00096e0, readable=true, writable=false, check_error=false) at ../../rtc_base/physical_socket_server.cc:1222 #35 0x000055555604b232 in rtc::PhysicalSocketServer::WaitEpoll(int) (this=0x7fffe4002cd0, cmsWait=42) at ../../rtc_base/physical_socket_server.cc:1454 #36 0x0000555556049193 in rtc::PhysicalSocketServer::Wait(int, bool) (this=0x7fffe4002cd0, cmsWait=42, process_io=true)at ../../rtc_base/physical_socket_server.cc:1169 #37 0x00005555560546a0 in rtc::Thread::Get(rtc::Message*, int, bool) (this=0x7fffe4001820, pmsg=0x7fffdb3f9ab0, cmsWait=-1, process_io=true)at ../../rtc_base/thread.cc:547

第二階段，將接收到的音頻 RTP 包異步插入 NetEQ 的包緩沖區里。

#0 webrtc::NetEqImpl::InsertPacketInternal(webrtc::RTPHeader const&, rtc::ArrayView<unsigned char const, -4711l>)(this=0x7fffc0094a80, rtp_header=..., payload=...) at ../../modules/audio_coding/neteq/neteq_impl.cc:563 #1 0x00005555569d7cbb in webrtc::NetEqImpl::InsertPacket(webrtc::RTPHeader const&, rtc::ArrayView<unsigned char const, -4711l>) (this=0x7fffc0094a80, rtp_header=..., payload=...) at ../../modules/audio_coding/neteq/neteq_impl.cc:192 #2 0x00005555569c5425 in webrtc::acm2::AcmReceiver::InsertPacket(webrtc::RTPHeader const&, rtc::ArrayView<unsigned char const, -4711l>) (this=0x7fffc009a958, rtp_header=..., incoming_payload=...) at ../../modules/audio_coding/acm2/acm_receiver.cc:136 #3 0x00005555569a0e4a in webrtc::voe::(anonymous namespace)::ChannelReceive::OnReceivedPayloadData(rtc::ArrayView<unsigned char const, -4711>, webrtc::RTPHeader const&) (this=0x7fffc009a790, payload=..., rtpHeader=...)at ../../audio/channel_receive.cc:340 #4 0x00005555569a48a3 in webrtc::voe::(anonymous namespace)::ChannelReceive::ReceivePacket(uint8_t const*, size_t, webrtc::RTPHeader const&)(this=0x7fffc009a790, packet=0x7fffe0069600 "\220o\003\227\372l\023\347?A(\276", <incomplete sequence \336>, packet_length=80, header=...) at ../../audio/channel_receive.cc:719 #5 0x00005555569a41bc in webrtc::voe::(anonymous namespace)::ChannelReceive::OnRtpPacket(webrtc::RtpPacketReceived const&)(this=0x7fffc009a790, packet=...) at ../../audio/channel_receive.cc:669 #6 0x00005555568bec90 in webrtc::RtpDemuxer::OnRtpPacket(webrtc::RtpPacketReceived const&) (this=0x7fffc0090340, packet=...)at ../../call/rtp_demuxer.cc:249 #7 0x00005555568bae55 in webrtc::RtpStreamReceiverController::OnRtpPacket(webrtc::RtpPacketReceived const&)(this=0x7fffc00902e0, packet=...) at ../../call/rtp_stream_receiver_controller.cc:52 #8 0x0000555556879062 in webrtc::internal::Call::DeliverRtp(webrtc::MediaType, rtc::CopyOnWriteBuffer, long)(this=0x7fffc0090090, media_type=webrtc::MediaType::AUDIO, packet=..., packet_time_us=1640601570497083)at ../../call/call.cc:1587 #9 0x0000555556879641 in webrtc::internal::Call::DeliverPacket(webrtc::MediaType, rtc::CopyOnWriteBuffer, long)(this=0x7fffc0090090, media_type=webrtc::MediaType::AUDIO, packet=..., packet_time_us=1640601570497083)at ../../call/call.cc:1618 #10 0x00005555560ebede in cricket::WebRtcVoiceMediaChannel::<lambda()>::operator()(void) const (__closure=0x7fffe005a768)at ../../media/engine/webrtc_voice_engine.cc:2229 #11 0x00005555560f2b74 in webrtc::webrtc_new_closure_impl::SafetyClosureTask<cricket::WebRtcVoiceMediaChannel::OnPacketReceived(rtc::CopyOnWriteBuffer, int64_t)::<lambda()> >::Run(void) (this=0x7fffe005a760)at ../../rtc_base/task_utils/to_queued_task.h:50 #12 0x00005555560575f4 in rtc::Thread::QueuedTaskHandler::OnMessage(rtc::Message*) (this=0x7fffe4002728, msg=0x7fffdb3f9ab0)at ../../rtc_base/thread.cc:1042 #13 0x0000555556055398 in rtc::Thread::Dispatch(rtc::Message*) (this=0x7fffe4002600, pmsg=0x7fffdb3f9ab0)at ../../rtc_base/thread.cc:711 #14 0x000055555605842a in rtc::Thread::ProcessMessages(int) (this=0x7fffe4002600, cmsLoop=-1) at ../../rtc_base/thread.cc:1149 #15 0x000055555605694f in rtc::Thread::Run() (this=0x7fffe4002600) at ../../rtc_base/thread.cc:901 #16 0x0000555556056914 in rtc::Thread::PreRun(void*) (pv=0x7fffe4002600) at ../../rtc_base/thread.cc:890 #17 0x00007ffff7e55609 in start_thread (arg=<optimized out>) at pthread_create.c:477

第三階段，播放線程從 NetEQ 里獲取音頻包，解碼并播放出來。

#0 webrtc::NetEqImpl::GetAudioInternal(webrtc::AudioFrame*, bool*, absl::optional<webrtc::NetEq::Operation>)(this=0x0, audio_frame=0x0, muted=0x0, action_override=...) at ../../modules/audio_coding/neteq/neteq_impl.cc:836 #1 0x00005555569d804f in webrtc::NetEqImpl::GetAudio(webrtc::AudioFrame*, bool*, int*, absl::optional<webrtc::NetEq::Operation>)(this=0x7fffc4094ab0, audio_frame=0x7fffc4100b60, muted=0x7fffcd9f3aa7, current_sample_rate_hz=0x7fffcd9efc40, action_override=...)at ../../modules/audio_coding/neteq/neteq_impl.cc:261 #2 0x00005555569c5896 in webrtc::acm2::AcmReceiver::GetAudio(int, webrtc::AudioFrame*, bool*)(this=0x7fffc409a958, desired_freq_hz=16000, audio_frame=0x7fffc4100b60, muted=0x7fffcd9f3aa7)at ../../modules/audio_coding/acm2/acm_receiver.cc:151 #3 0x00005555569a1d8b in webrtc::voe::(anonymous namespace)::ChannelReceive::GetAudioFrameWithInfo(int, webrtc::AudioFrame*)(this=0x7fffc409a790, sample_rate_hz=16000, audio_frame=0x7fffc4100b60) at ../../audio/channel_receive.cc:388 #4 0x0000555556984783 in webrtc::internal::AudioReceiveStream::GetAudioFrameWithInfo(int, webrtc::AudioFrame*)(this=0x7fffc409a530, sample_rate_hz=16000, audio_frame=0x7fffc4100b60) at ../../audio/audio_receive_stream.cc:393 #5 0x0000555556b5c3d9 in webrtc::AudioMixerImpl::GetAudioFromSources(int) (this=0x7fffc40208f0, output_frequency=16000)at ../../modules/audio_mixer/audio_mixer_impl.cc:205 #6 0x0000555556b5be4c in webrtc::AudioMixerImpl::Mix(unsigned long, webrtc::AudioFrame*)(this=0x7fffc40208f0, number_of_channels=2, audio_frame_for_mixing=0x7fffc4025068) at ../../modules/audio_mixer/audio_mixer_impl.cc:175 #7 0x000055555699ef20 in webrtc::AudioTransportImpl::NeedMorePlayData(unsigned long, unsigned long, unsigned long, unsigned int, void*, unsigned long&, long*, long*)(this=0x7fffc4024f60, nSamples=441, nBytesPerSample=4, nChannels=2, samplesPerSec=44100, audioSamples=0x7fff94001030, nSamplesOut=@0x7fffcd9f4348: 0, elapsed_time_ms=0x7fffcd9f4350, ntp_time_ms=0x7fffcd9f4358) at ../../audio/audio_transport_impl.cc:215 #8 0x00005555565b7e9b in webrtc::AudioDeviceBuffer::RequestPlayoutData(unsigned long) (this=0x7fffc4002c28, samples_per_channel=441)at ../../modules/audio_device/audio_device_buffer.cc:302 #9 0x00005555565aa394 in webrtc::AudioDeviceLinuxPulse::PlayThreadProcess() (this=0x7fffc4004de0)at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:2106 #10 0x000055555659e52d in webrtc::AudioDeviceLinuxPulse::<lambda()>::operator()(void) const (__closure=0x7fffc4007890)at ../../modules/audio_device/linux/audio_device_pulse_linux.cc:174 #11 0x00005555565ac068 in std::_Function_handler<void(), webrtc::AudioDeviceLinuxPulse::Init()::<lambda()> >::_M_invoke(const std::_Any_data &)(__functor=...) at /usr/include/c++/9/bits/std_function.h:300

這里還想要捋出來 WebRTC 中音頻發送和接收處理過程，各個部分的邊界。WebRTC 的整體架構大概如下圖所示：

最下層的是媒體處理核心功能模塊，上面是媒體引擎，再上面是 PeerConnection 和 API，此外，WebRTC 還提供了大量的基礎組件，如 socket，線程，鎖等等等。前面我們說明的音頻發送和接收處理過程，除了最后 6. 音頻數據包的發送處理 和 7. 音頻數據包的接收處理 兩個過程，其它都完全屬于 PeerConnection。對于 6. 音頻數據包的發送處理 過程，通過 BaseChannel 實現 MediaChannel::NetworkInterface 接口，并注冊給 MediaChannel，將 PeerConnection 的發送能力和媒體處理核心功能組件連接起來。對于接收處理過程，邊界則在 RtpTransport::DemuxPacket()，RtpTransport 是 PeerConnection 傳輸引擎的一部分，它通過 webrtc::RtpDemuxer 將收到的音頻包送進核心的媒體處理功能中去。

總結

以上是生活随笔為你收集整理的WebRTC 音频发送和接收处理过程的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

上一篇： mediasoup-demo 运行实战
下一篇： WebRTC 的 AudioSource