前言

当媒体服务器发行故障后，先后通过分析SIP信令及媒体服务器抓包及服务器日志，定位媒体服务器网元出现故障。
通过abrt抓取进程异常的core文件，使用sipp模拟大呼进行故障场景复现。采用valgrind进行内存泄漏定位。最终定位问题并修复该问题。

提示：以下是本篇文章正文内容

一、abrt获取core文件

上上周，负责江苏现场交付的同事联系我，客户有反馈通话过程中存在通话中断的现象。
我先让现场的抓取了客户的通话信令，发现通话中断时，收到了核心网交换机发送了拆线的指令。原因值是RAP/RTCP TIME OUT。
然后我让现场的同事采用gdb命令在媒体服务器进行网卡抓包，并进行实际拨测，当通话中断时，停止网络抓包。
通过网络包分析，发现媒体服务器存在媒体包丢包情况。
于是判断媒体服务器存在问题了，通过分析进城的运行日志，发现存在重启现象。当中午发生时会导致媒体包丢包，导致通话中断。
至此问题陡然升级，先联系现场配置abrt服务（service arbtd restart），当进程重启时，抓取到coredump。
经和客户沟通后，先下掉媒体服务器，通话旁路。

二、gdb分析coredump文件

gdb ms coredump

然后输入where命令，frame及相关的print等命令，逐一查看相关变量。

三、sipp模拟呼叫

于是编写sipp的uac和uas的模拟桩，uac模拟主叫，uas模拟被叫。同时uac和uas的模拟桩中加入rtp流，让场景变得更真实。
uac脚本如下：

<?xml version="1.0" encoding="ISO-8859-1" ?><!-- This program is free software; you can redistribute it and/or      -->
<!-- modify it under the terms of the GNU General Public License as     -->
<!-- published by the Free Software Foundation; either version 2 of the -->
<!-- License, or (at your option) any later version.                    -->
<!--                                                                    -->
<!-- This program is distributed in the hope that it will be useful,    -->
<!-- but WITHOUT ANY WARRANTY; without even the implied warranty of     -->
<!-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the      -->
<!-- GNU General Public License for more details.                       -->
<!--                                                                    -->
<!-- You should have received a copy of the GNU General Public License  -->
<!-- along with this program; if not, write to the                      -->
<!-- Free Software Foundation, Inc.,                                    -->
<!-- 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA             -->
<!--                                                                    -->
<!--                 Sipp default 'uac' scenario.                       -->
<!--                                                                    --><scenario name="Basic Sipstone UAC"><!-- In client mode (sipp placing calls), the Call-ID MUST be         --><!-- generated by sipp. To do so, use [call_id] token.                --><send><![CDATA[INVITE sip:[field1]@[remote_ip]:[remote_port] SIP/2.0Via: SIP/2.0/[transport] [local_ip]:[local_port]From: sipp <sip:[field0]@[local_ip]:[local_port]>;tag=[call_number]To: sut <sip:[field1]@[remote_ip]:[remote_port]>Call-ID: [call_id]Cseq: [cseq] INVITEContact: sip:sipp@[local_ip]:[local_port]Route: <sip:tagscpxf03.js.ctcims.cn:[remote_port];call=ldmp;lr>Route: <sip:[local_ip]:8063;lr>Max-Forwards: 70P-Asserted-Identity: <sip:[field0]@cintel.com.cn>Subject: Performance TestContent-Type: application/sdpContent-Length: [len]v=0o=- 53655765 2353687637 IN IP4 [media_ip]s=SIPPc=IN IP4 [media_ip]t=0 0m=audio [media_port] RTP/AVP 115 103 102 101 8 96 97 18 0b=AS:80b=RS:600b=RR:2000a=rtpmap:115 AMR-WB/16000/1a=fmtp:115 mode-change-capability=2;max-red=0a=rtpmap:103 AMR-WB/16000/1a=fmtp:103 octet-align=1;mode-change-capability=2;max-red=0a=rtpmap:102 AMR/8000/1a=fmtp:102 mode-change-capability=2;max-red=0a=rtpmap:101 AMR/8000/1a=fmtp:101 octet-align=1;mode-change-capability=2;max-red=0a=rtpmap:8 PCMA/8000a=rtpmap:96 telephone-event/16000a=fmtp:96 0-15a=rtpmap:97 telephone-event/8000a=fmtp:97 0-15a=curr:qos local nonea=curr:qos remote nonea=des:qos mandatory local sendrecva=des:qos optional remote sendrecva=sendrecva=maxptime:240a=ptime:20a=rtpmap:18 G729/8000a=fmtp:18 annexb=noa=rtpmap:0 PCMU/8000]]></send><recv response="100" ></recv><recv response="183" ></recv><send><![CDATA[PRACK sip:[field1]@[remote_ip]:[remote_port] SIP/2.0Via: SIP/2.0/[transport] [local_ip]:[local_port]From: <sip:[field0]@[local_ip]:[local_port]>;tag=[call_number]To: <sip:[field1]@192.168.2.228:8700>[peer_tag_param]Call-ID: [call_id]CSeq: 2 PRACKRAck: 1 1 INVITEContact: <sip:[local_ip]@[local_ip]:[local_port]>      Max-Forwards: 70Content-Length: 0 ]]></send><recv response="200" ></recv><nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/amrwb1_115.pcap"/></action></nop>   <send><![CDATA[UPDATE sip:[remote_ip]:[remote_port] SIP/2.0Via: SIP/2.0/[transport] [local_ip]:[local_port][last_Call-ID:]From: sipp <sip:[field0]@[local_ip]:[local_port]>;tag=[call_number]To: sut <sip:[field1]@[remote_ip]:[remote_port]>[peer_tag_param]Allow: INVITE,ACK,CANCEL,BYE,UPDATE,PRACK,MESSAGE,REFER,NOTIFY,INFO,OPTIONS        Cseq: 3 UPDATEContact: <sip:[local_ip]@[local_ip]:[local_port]>      Require: preconditionSupported: timerContent-Length: [len]Content-Type: application/sdpv=0o=HuaweiATS9900 171392092 171392093 IN IP4 5.2.169.97s=SBC callc=IN IP4 [media_ip]b=AS:49b=RS:600b=RR:2000t=0 0m=audio [media_port] RTP/AVP 115 102 8 96 18 0 97b=AS:80b=RS:600b=RR:2000a=rtpmap:115 AMR-WB/16000/1a=fmtp:115 mode-change-capability=2;max-red=0a=rtpmap:102 AMR/8000a=fmtp:102 mode-change-capability=2;max-red=0a=rtpmap:8 PCMA/8000a=rtpmap:96 telephone-event/16000a=fmtp:96 0-15a=curr:qos local sendrecva=curr:qos remote nonea=des:qos mandatory local sendrecva=des:qos mandatory remote sendrecva=sendrecva=maxptime:240a=ptime:20a=rtpmap:18 G729/8000a=fmtp:18 annexb=noa=rtpmap:0 PCMU/8000a=rtpmap:97 telephone-event/8000a=fmtp:97 0-15         ]]>          </send><recv response="200"></recv><nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/g711a.pcap"/></action></nop> <recv response="180"></recv><recv response="200"></recv><send><![CDATA[ACK sip:[remote_ip]:[remote_port];transport=udp;Hpt=8f28_16;CxtId=3;TRC=ffffffff-ffffffff SIP/2.0Via: SIP/2.0/[transport] [local_ip]:[local_port]From: sipp <sip:[field0]@[local_ip]:[local_port]>;tag=[call_number]To: sut <sip:[field1]@[remote_ip]:[remote_port]>[peer_tag_param]Call-ID: [call_id]Cseq: [cseq] ACKContact: sip:sipp@[local_ip]:[local_port]Max-Forwards: 70Subject: Performance TestContent-Length: 0]]></send><nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/amr102.pcap"/></action></nop> <!-- This delay can be customized by the -d command-line option       --><!-- or by adding a 'milliseconds = "value"' option here.             --><pause milliseconds = "500"/><send><![CDATA[INVITE sip:[field1]@[remote_ip]:[remote_port] SIP/2.0Via: SIP/2.0/[transport] [local_ip]:[local_port]From: sipp <sip:[field0]@[local_ip]:[local_port]>;tag=[call_number]To: sut <sip:[field1]@[remote_ip]:[remote_port]>[peer_tag_param]Call-ID: [call_id]Cseq: 4 INVITEContact: sip:sipp@[local_ip]:[local_port]Max-Forwards: 66P-Early-Media: supported      Subject: Performance TestSession-Expires: 1800;refresher=uasMin-SE: 600Accept-Contact: *;+g.3gpp.icsi-ref="urn%3Aurn-7%3A3gpp-service.ims.icsi.mmtel";videoContent-Type: application/sdpContent-Length: [len]v=0o=HuaweiATS9900 171392092 171392094 IN IP4 5.2.169.97s=SBC callc=IN IP4 [media_ip]b=AS:2211b=RS:8600b=RR:8000t=0 0m=audio [media_port] RTP/AVP 115 102 8 96 18 0 97b=AS:80b=RS:600b=RR:2000a=rtpmap:115 AMR-WB/16000/1a=fmtp:104 mode-change-capability=2;max-red=0a=rtpmap:102 AMR/8000a=fmtp:102 mode-change-capability=2;max-red=0a=rtpmap:8 PCMA/8000a=rtpmap:96 telephone-event/16000a=fmtp:96 0-15a=sendrecva=maxptime:240a=ptime:20a=rtpmap:18 G729/8000a=fmtp:18 annexb=noa=rtpmap:0 PCMU/8000a=rtpmap:97 telephone-event/8000a=fmtp:97 0-15m=video 52998 RTP/AVP 114 113b=AS:2162b=RS:8000b=RR:6000a=rtpmap:114 H264/90000a=fmtp:114 profile-level-id=42C01F;packetization-mode=1;sar-understood=16;sar-supported=1;sprop-parameter-sets=Z0LAH9oC0ChoBtChNQ==,aM4G4g==a=imageattr:114 send a=rtpmap:113 H264/90000a=fmtp:113 profile-level-id=42C01F;packetization-mode=0;sar-understood=16;sar-supported=1;sprop-parameter-sets=Z0LAH9oC0ChoBtChNQ==,aM4G4g==a=imageattr:113 send a=rtcp-fb:* nacka=rtcp-fb:* nack plia=rtcp-fb:* ccm fira=rtcp-fb:* ccm tmmbra=sendrecva=tcap:1 RTP/AVPFa=pcfg:1 t=1a=extmap:2 urn:3gpp:video-orientation]]></send><recv response="100" ></recv><recv response="200" ></recv><send><![CDATA[ACK sip:[remote_ip]:[remote_port];transport=udp;Hpt=8f28_16;CxtId=3;TRC=ffffffff-ffffffff SIP/2.0Via: SIP/2.0/[transport] [local_ip]:[local_port]From: sipp <sip:[field0]@[local_ip]:[local_port]>;tag=[call_number]To: sut <sip:[field1]@[remote_ip]:[remote_port]>[peer_tag_param]Call-ID: [call_id]Cseq: 4 ACKContact: sip:sipp@[local_ip]:[local_port]Max-Forwards: 70Subject: Performance TestContent-Length: 0]]></send>  <nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/amrwb1_115.pcap"/></action></nop>   <pause milliseconds = "30000"/> <!-- The 'crlf' option inserts a blank line in the statistics report. --><send><![CDATA[BYE sip:[field1]@[remote_ip]:[remote_port] SIP/2.0Via: SIP/2.0/[transport] [local_ip]:[local_port]From: sipp <sip:[field0]@[local_ip]:[local_port]>;tag=[call_number]To: sut <sip:[field1]@[remote_ip]:[remote_port]>[peer_tag_param]Call-ID: [call_id]Cseq: 1 BYEContact: sip:sipp@[local_ip]:[local_port]Max-Forwards: 70Subject: Performance TestContent-Length: 0]]></send><recv response="200" crlf="true"></recv><!-- definition of the response time repartition table (unit is ms)   --><ResponseTimeRepartition value="10, 20, 30, 40, 50, 100, 150, 200"/><!-- definition of the call length repartition table (unit is ms)     --><CallLengthRepartition value="10, 50, 100, 500, 1000, 5000, 10000"/></scenario>

uac的启动脚本如下：

./sipp -sf uac_jsfhgk.xml -inf calling_number_graylist_speech_template_performance.txt -i 192.168.2.179 -p 5788 192.168.2.112:5062 -m 10000 -r 10 -trace_screen -screen_overwrite true -screen_file uac_jsfhgk.log

uas脚本如下：

<?xml version="1.0" encoding="ISO-8859-1" ?><!-- This program is free software; you can redistribute it and/or      -->
<!-- modify it under the terms of the GNU General Public License as     -->
<!-- published by the Free Software Foundation; either version 2 of the -->
<!-- License, or (at your option) any later version.                    -->
<!--                                                                    -->
<!-- This program is distributed in the hope that it will be useful,    -->
<!-- but WITHOUT ANY WARRANTY; without even the implied warranty of     -->
<!-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the      -->
<!-- GNU General Public License for more details.                       -->
<!--                                                                    -->
<!-- You should have received a copy of the GNU General Public License  -->
<!-- along with this program; if not, write to the                      -->
<!-- Free Software Foundation, Inc.,                                    -->
<!-- 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA             -->
<!--                                                                    -->
<!--                 Sipp default 'uas' scenario.                       -->
<!--                                                                    --><scenario name="Basic UAS responder"><!-- By adding rrs="true" (Record Route Sets), the route sets         --><!-- are saved and used for following messages sent. Useful to test   --><!-- against stateful SIP proxies/B2BUAs.                             --><recv request="INVITE" crlf="true"><action><ereg regexp=".*" search_in="hdr" header="CSeq:" assign_to="cseq_invite" /><ereg regexp=".*" search_in="hdr" header="Via:" assign_to="via_invite" /><!--ereg regexp="<.*>" search_in="hdr" header="Contact:" assign_to="uac_addr" /--><!--ereg regexp=".*" search_in="hdr" header="From:" assign_to="from_invite" /--></action></recv><!-- The '[last_*]' keyword is replaced automatically by the          --><!-- specified header if it was present in the last message received  --><!-- (except if it was a retransmission). If the header was not       --><!-- present or if no message has been received, the '[last_*]'       --><!-- keyword is discarded, and all bytes until the end of the line    --><!-- are also discarded.                                              --><!--                                                                  --><!-- If the specified header was present several times in the         --><!-- message, all occurences are concatenated (CRLF seperated)        --><!-- to be used in place of the '[last_*]' keyword.                   --><send><![CDATA[SIP/2.0 100 Trying[last_Via:][last_From:]To: <sip:[field1]@[remote_ip]:[remote_port]>;tag=[call_number][last_Call-ID:][last_CSeq:]Contact: <sip:[local_ip]:[local_port];transport=[transport]>Content-Length: 0]]></send><send><![CDATA[SIP/2.0 183 Session Progress[last_Via:][last_From:]To: <sip:[field1]@[remote_ip]:[remote_port]>;tag=[call_number][last_Call-ID:][last_CSeq:]Require: 100rel,preconditionRSeq: 1P-Charging-Vector: icid-value="pbac22bhw.19e.627a.20211025080346";orig-ioi=js.ims.mnc011.mcc460.3gppnetwork.org;term-ioi=js.chinamobile.comContact: <sip:[local_ip]:[local_port];transport=[transport]>Content-Length: [len]Content-Type: application/sdpv=0o=- 1635149026 1635149026 IN IP4 5.92.12.134s=SBC callc=IN IP4 [media_ip]b=AS:41b=RR:1837b=RS:600t=0 0m=audio [media_port] RTP/AVP 115 96b=AS:41b=RR:1837b=RS:600a=rtpmap:115 AMR-WB/16000/1a=fmtp:115 mode-change-capability=2;max-red=0a=ptime:20a=maxptime:240a=sendrecva=curr:qos local nonea=curr:qos remote nonea=des:qos mandatory local sendrecva=des:qos mandatory remote sendrecva=conf:qos remote sendrecva=rtpmap:96 telephone-event/16000a=fmtp:96 0-15]]></send><recv request="PRACK"></recv><send><![CDATA[SIP/2.0 200 OK[last_Via:][last_From:]To: <sip:01082012002@[remote_ip]:[remote_port]>;tag=[call_number][last_Call-ID:]CSeq: 2 PRACKContact: <sip:[local_ip]:[local_port];transport=[transport]>Content-Length: [len]]]></send><nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/amrwb1_115.pcap"/></action></nop>  <recv request="UPDATE"></recv><send><![CDATA[SIP/2.0 200 OK[last_Via:][last_Call-ID:][last_From:][last_To:][last_CSeq:]Allow: INVITE,ACK,OPTIONS,BYE,CANCEL,REGISTER,INFO,UPDATE,PRACK,REFER,SUBSCRIBE,NOTIFY,MESSAGEContact: <sip:[field1]@[local_ip]:[local_port];transport=udp;Hpt=8ec2_16;CxtId=3;TRC=ffffffff-ffffffff;srti=d0_1025>;video;+g.3gpp.icsi-ref="urn%3Aurn-7%3A3gpp-service.ims.icsi.mmtel";+g.3gpp.mid-call;+g.3gpp.srvcc-alertingServer: Ericsson MTAS - CXP2010134/1 R20G11Supported: timer,100rel,histinfo,join,norefersub,replacesSession-Expires: 1800;refresher=uasP-Access-Network-Info: 3GPP-E-UTRAN;utran-cell-id-3gpp="460115E13675381B";access-domain=bac01.+86591.fj.ctcims.cn;network-providedP-Charging-Vector: icid-value=sg6.psbc12ber.fz.fj.node.epc.-1608-718164-185570-946;orig-ioi=ericsson.com;term-ioi=fj.ctcims.cn;icid-generated-at=sg6.psbc12ber.fz.fj.node.epc.mnc011.mcc460.3gppnetwork.orgSupported: timerP-Access-Network-Info: 3GPP-E-UTRAN;utran-cell-id-3gpp=4600069167E8F712;sbc-domain=sbc7.0791.100.jx.chinamobile.com;ue-ip=[local_ip];ue-port=[local_port];network-providedP-Charging-Vector: icid-value="cdpsbc21bhw.19c.4185.20180803081658";orig-ioi=cdscscf6bhw.sc.chinamobile.com;term-ioi=jx.chinamobile.comP-Early-Media: gatedContent-Length: [len]Content-Type: application/sdpP-Charging-Vector: icid-value=sg9.psbc21ber.nc.jx.node.epc.mnc0-1589-273998-798645;orig-ioi=ericsson.com;term-ioi=js.ctcims.cnP-Early-Media: sendrecvP-Access-Network-Info: IEEE-802.11;"location-info=5.253.86.82"v=0o=- 1635149026 1635149027 IN IP4 5.92.12.134s=SBC callc=IN IP4 [media_ip]b=AS:41b=RR:1837b=RS:600t=0 0m=audio [media_port] RTP/AVP 8 96b=AS:41b=RR:1837b=RS:600a=rtpmap:8 PCMA/8000a=ptime:20a=maxptime:240a=sendrecva=curr:qos local sendrecva=curr:qos remote sendrecva=des:qos mandatory local sendrecva=des:qos mandatory remote sendrecva=rtpmap:96 telephone-event/16000a=fmtp:96 0-15   ]]></send><nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/g711a.pcap"/></action></nop> <pause milliseconds = "6000"/><send> <![CDATA[SIP/2.0 180 RingingVia:[$via_invite][last_Call-ID:][last_From:][last_To:]CSeq: 1 INVITEAllow: INVITE,UPDATE,BYE,PRACK,INFO,OPTIONS,CANCEL,SUBSCRIBE,ACK,REFER,NOTIFY,REGISTER,PUBLISH,MESSAGECall-Info: <tel:+8613665171969>;purpose=call-completion;m=NRContact: <sip:[local_ip]:[local_port];transport=[transport]>   P-Asserted-Identity: <tel:+8613665171969>P-Charging-Vector: icid-value="pbac22bhw.19e.627a.20211025080346";orig-ioi=js.ims.mnc011.mcc460.3gppnetwork.org;term-ioi=js.chinamobile.comP-Early-Media: inactiveContent-Length: 0        ]]>          </send> <pause milliseconds = "6000"/>   <send><![CDATA[SIP/2.0 200 OKVia:[$via_invite][last_From:]To: <sip:[field1]@[remote_ip]:[remote_port]>;tag=[call_number][last_Call-ID:]CSeq: [$cseq_invite]Contact: <sip:[local_ip]:[local_port];transport=[transport]>Content-Type: application/sdpContent-Length: [len]v=0o=- 1635149026 1635149027 IN IP4 5.92.12.134s=SBC callc=IN IP4 [media_ip]b=AS:41b=RR:1837b=RS:600t=0 0m=audio [media_port] RTP/AVP 102 96b=AS:41b=RR:1837b=RS:600a=rtpmap:102 AMR/8000a=fmtp:102 mode-change-capability=2;max-red=0a=ptime:20a=maxptime:240a=sendrecva=curr:qos local sendrecva=curr:qos remote sendrecva=des:qos mandatory local sendrecva=des:qos mandatory remote sendrecva=rtpmap:96 telephone-event/16000a=fmtp:96 0-15]]></send><recv request="ACK"rtd="true"crlf="true"></recv><nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/amr102.pcap"/></action></nop> <recv request="INVITE"></recv><send><![CDATA[SIP/2.0 100 Trying[last_Via:][last_From:]To: <sip:[field1]@[remote_ip]:[remote_port]>;tag=[call_number][last_Call-ID:][last_CSeq:]Contact: <sip:[local_ip]:[local_port];transport=[transport]>Content-Length: 0]]></send>  <send><![CDATA[SIP/2.0 200 OK[last_Via:][last_From:]To: <sip:[field1]@[remote_ip]:[remote_port]>;tag=[call_number][last_Call-ID:][last_CSeq:]Contact: <sip:[local_ip]:[local_port];transport=[transport]>Content-Type: application/sdpContent-Length: [len]v=0o=- 1635149026 1635149027 IN IP4 5.92.12.134s=SBC callc=IN IP4 [media_ip]b=AS:41b=RR:1837b=RS:600t=0 0m=audio [media_port] RTP/AVP 115 96b=AS:41b=RR:1837b=RS:600a=rtpmap:115 AMR-WB/16000/1a=fmtp:115 mode-change-capability=2;max-red=0a=ptime:20a=maxptime:240a=sendrecva=curr:qos local sendrecva=curr:qos remote sendrecva=des:qos mandatory local sendrecva=des:qos mandatory remote sendrecva=rtpmap:96 telephone-event/16000a=fmtp:96 0-15]]></send> <recv request="ACK"></recv> <nop><action><!--exec play_pcap_audio="pcap/dtmf_2833_0.pcap"/--><exec play_pcap_audio="pcap/amrwb1_115.pcap"/></action></nop>     <recv request="BYE"></recv><send><![CDATA[SIP/2.0 200 OK[last_Via:][last_From:]To: <sip:[field1]@[remote_ip]:[remote_port]>;tag=[call_number][last_Call-ID:][last_CSeq:]Contact: <sip:[local_ip]:[local_port];transport=[transport]>Content-Type: application/sdpContent-Length: 0]]></send><!-- Keep the call open for a while in case the 200 is lost to be     --><!-- able to retransmit it if we receive the BYE again.               --><pause milliseconds="0"/><!-- definition of the response time repartition table (unit is ms)   --><ResponseTimeRepartition value="10, 20, 30, 40, 50, 100, 150, 200"/><!-- definition of the call length repartition table (unit is ms)     --><CallLengthRepartition value="10, 50, 100, 500, 1000, 5000, 10000"/></scenario>

uas的启动脚本如下：

 ./sipp -sf uas_jsfhgk.xml -i 192.168.2.179 -p 8063 -trace_screen -screen_overwrite true -screen_file uas_jsfhgk.log -inf calling_number_graylist_speech_template_performance.txt

三、valrind工具

    通过10 caps的连续呼叫测试，媒体服务器的故障现象开始复现，确定了代码的大致位置后。开始对媒体服务器的代码部分进行修复加锁。加锁完成后，进程不重启了，但是发现有内存泄漏，通过top命令查看，进程占用的内存一直在增长。于是启用了valgrind工具，用valgrind启动媒体服务进程，uac单呼一次，然后分析valgrind日志。通过valgrind的日志发现，提示调用的avcodec_send_frame存在内存泄漏，但是我查了代码，已经调用了av_frame_free释放了内存，于是我判断应该不是这个地方存在内存泄漏。通过百度查询avcodec_send_frame内存泄漏问题时，我注意到一个专家写了一句话“当av_read_frame返回值不小于0时，内部会进行缓存操作，需要外界释放。

总结

提示：这里对文章进行总结：
当媒体服务器发行故障后，先后通过分析SIP信令及媒体服务器抓包及服务器日志，定位媒体服务器网元出现故障。
通过abrt抓取进程异常的core文件，使用sipp模拟大呼进行故障场景复现。采用valgrind进行内存泄漏定位。最终定位问题并修复该问题。
希望这个抢救的案例，能够给其他同行起到借鉴作用。

媒体服务器（MS）抢救记相关推荐

FLV 封装格式解析
本文为作者原创,转载请注明出处:https://www.cnblogs.com/leisure_chn/p/10662941.html FLV (Flash Video) 是由 Adobe 公司推出的 ...
dlna 交互步骤-转发
当下三大主流投屏协议,Airplay.Miracast.DLNA.Airplay(苹果设备).Miracast(安卓设备)都属于镜像投屏,也就是手机显示什么,投屏到大屏幕上也显示同样的内容,唯独DLN ...
14.1.2. EXPLAIN ANALYZE
可以使用EXPLAIN的ANALYZE选项获取时间的执行计划预估.使用该选项,EXPLAIN会实际的执行查询,然后返回实际的返回行及运行时间.例如: EXPLAIN ANALYZE SELECT * ...
【STC开源项目】STC51单片机LRC万用表表制作程序+PCB文件
[STC开源项目]STC51单片机LRC万用表表制作程序+PCB文件申明:本项目来源于51黑单片机论坛:http://www.51hei.com/bbs/dpj-175080-1.html 本篇文章 ...
使用stm32f103c8实现一个简单的运动会计时器
没事做突然想写博客,然后发现没有刷什么有价值的题,就写个做实验的记录吧. 首先是构思的思维导图可以看到,我用到了按键结合中断作为输入,使用定时中断来计数,同时显示使用的是oled屏幕,大小为128* ...
MS讲座：可视化的软件架构设计和Portal Starter Kit挖宝记
昨天下午参加了MS的讲座:可视化的软件架构设计和Portal Starter Kit挖宝记,主讲是广州嘉为的王兴明. 讲座听完了,没有什么特别的感受,"可视化的软件架构设计"讲的是 ...
ms查约束具体代码_记 Arthas 实现一次 CPU 排查与代码热更新
简介:线上代码经常会出现 CPU 占用过高的情况,按以往经验我会使用 top 指令,进一步借助于 jstack 去查看具体信息从而进行问题排查,但基本上都逃不过需要重新发包的局面,及时是一个增量包,应 ...
记一次使用compiz失败及抢救过程
看网上有个compiz能给ubuntu的桌面设置3D特效,一时手痒就试着安装了一下,然后启动栏没了,标题栏没了,窗口也都移不动了,遂重启,但还是一样,但是总算是能打开终端. 于是在终端里输入: sud ...
Java练习 SDUT-2737_小鑫の日常系列故事（六）——奇遇记
小鑫の日常系列故事(六)--奇遇记 Time Limit: 1000 ms Memory Limit: 65536 KiB Problem Description 今天,小鑫在山上玩的时候,意外被推下 ...

媒体服务器（MS）抢救记

文章目录

前言

一、abrt获取core文件

二、gdb分析coredump文件

三、sipp模拟呼叫

三、valrind工具

总结

媒体服务器（MS）抢救记相关推荐

最新文章

热门文章