【GStreamer】基于NTP+SEI的视频流传输时延测量

本文以H.264视频流为例，用GStreamer实现插入和提取SEI(Supplemental Enhancement Information)，实现视频流传输时延的测量。

原理1

用GStreamer实现的方案

sender

#mermaid-svg-sIST8YtPiGVkcH7o .label{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);fill:#333;color:#333}#mermaid-svg-sIST8YtPiGVkcH7o .label text{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .node rect,#mermaid-svg-sIST8YtPiGVkcH7o .node circle,#mermaid-svg-sIST8YtPiGVkcH7o .node ellipse,#mermaid-svg-sIST8YtPiGVkcH7o .node polygon,#mermaid-svg-sIST8YtPiGVkcH7o .node path{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-sIST8YtPiGVkcH7o .node .label{text-align:center;fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .node.clickable{cursor:pointer}#mermaid-svg-sIST8YtPiGVkcH7o .arrowheadPath{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .edgePath .path{stroke:#333;stroke-width:1.5px}#mermaid-svg-sIST8YtPiGVkcH7o .flowchart-link{stroke:#333;fill:none}#mermaid-svg-sIST8YtPiGVkcH7o .edgeLabel{background-color:#e8e8e8;text-align:center}#mermaid-svg-sIST8YtPiGVkcH7o .edgeLabel rect{opacity:0.9}#mermaid-svg-sIST8YtPiGVkcH7o .edgeLabel span{color:#333}#mermaid-svg-sIST8YtPiGVkcH7o .cluster rect{fill:#ffffde;stroke:#aa3;stroke-width:1px}#mermaid-svg-sIST8YtPiGVkcH7o .cluster text{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:12px;background:#ffffde;border:1px solid #aa3;border-radius:2px;pointer-events:none;z-index:100}#mermaid-svg-sIST8YtPiGVkcH7o .actor{stroke:#ccf;fill:#ECECFF}#mermaid-svg-sIST8YtPiGVkcH7o text.actor>tspan{fill:#000;stroke:none}#mermaid-svg-sIST8YtPiGVkcH7o .actor-line{stroke:grey}#mermaid-svg-sIST8YtPiGVkcH7o .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333}#mermaid-svg-sIST8YtPiGVkcH7o .messageLine1{stroke-width:1.5;stroke-dasharray:2, 2;stroke:#333}#mermaid-svg-sIST8YtPiGVkcH7o #arrowhead path{fill:#333;stroke:#333}#mermaid-svg-sIST8YtPiGVkcH7o .sequenceNumber{fill:#fff}#mermaid-svg-sIST8YtPiGVkcH7o #sequencenumber{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o #crosshead path{fill:#333;stroke:#333}#mermaid-svg-sIST8YtPiGVkcH7o .messageText{fill:#333;stroke:#333}#mermaid-svg-sIST8YtPiGVkcH7o .labelBox{stroke:#ccf;fill:#ECECFF}#mermaid-svg-sIST8YtPiGVkcH7o .labelText,#mermaid-svg-sIST8YtPiGVkcH7o .labelText>tspan{fill:#000;stroke:none}#mermaid-svg-sIST8YtPiGVkcH7o .loopText,#mermaid-svg-sIST8YtPiGVkcH7o .loopText>tspan{fill:#000;stroke:none}#mermaid-svg-sIST8YtPiGVkcH7o .loopLine{stroke-width:2px;stroke-dasharray:2, 2;stroke:#ccf;fill:#ccf}#mermaid-svg-sIST8YtPiGVkcH7o .note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-sIST8YtPiGVkcH7o .noteText,#mermaid-svg-sIST8YtPiGVkcH7o .noteText>tspan{fill:#000;stroke:none}#mermaid-svg-sIST8YtPiGVkcH7o .activation0{fill:#f4f4f4;stroke:#666}#mermaid-svg-sIST8YtPiGVkcH7o .activation1{fill:#f4f4f4;stroke:#666}#mermaid-svg-sIST8YtPiGVkcH7o .activation2{fill:#f4f4f4;stroke:#666}#mermaid-svg-sIST8YtPiGVkcH7o .mermaid-main-font{font-family:"trebuchet ms", verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .section{stroke:none;opacity:0.2}#mermaid-svg-sIST8YtPiGVkcH7o .section0{fill:rgba(102,102,255,0.49)}#mermaid-svg-sIST8YtPiGVkcH7o .section2{fill:#fff400}#mermaid-svg-sIST8YtPiGVkcH7o .section1,#mermaid-svg-sIST8YtPiGVkcH7o .section3{fill:#fff;opacity:0.2}#mermaid-svg-sIST8YtPiGVkcH7o .sectionTitle0{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .sectionTitle1{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .sectionTitle2{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .sectionTitle3{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .sectionTitle{text-anchor:start;font-size:11px;text-height:14px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .grid .tick{stroke:#d3d3d3;opacity:0.8;shape-rendering:crispEdges}#mermaid-svg-sIST8YtPiGVkcH7o .grid .tick text{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .grid path{stroke-width:0}#mermaid-svg-sIST8YtPiGVkcH7o .today{fill:none;stroke:red;stroke-width:2px}#mermaid-svg-sIST8YtPiGVkcH7o .task{stroke-width:2}#mermaid-svg-sIST8YtPiGVkcH7o .taskText{text-anchor:middle;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .taskText:not([font-size]){font-size:11px}#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutsideRight{fill:#000;text-anchor:start;font-size:11px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutsideLeft{fill:#000;text-anchor:end;font-size:11px}#mermaid-svg-sIST8YtPiGVkcH7o .task.clickable{cursor:pointer}#mermaid-svg-sIST8YtPiGVkcH7o .taskText.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutsideLeft.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutsideRight.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-sIST8YtPiGVkcH7o .taskText0,#mermaid-svg-sIST8YtPiGVkcH7o .taskText1,#mermaid-svg-sIST8YtPiGVkcH7o .taskText2,#mermaid-svg-sIST8YtPiGVkcH7o .taskText3{fill:#fff}#mermaid-svg-sIST8YtPiGVkcH7o .task0,#mermaid-svg-sIST8YtPiGVkcH7o .task1,#mermaid-svg-sIST8YtPiGVkcH7o .task2,#mermaid-svg-sIST8YtPiGVkcH7o .task3{fill:#8a90dd;stroke:#534fbc}#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutside0,#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutside2{fill:#000}#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutside1,#mermaid-svg-sIST8YtPiGVkcH7o .taskTextOutside3{fill:#000}#mermaid-svg-sIST8YtPiGVkcH7o .active0,#mermaid-svg-sIST8YtPiGVkcH7o .active1,#mermaid-svg-sIST8YtPiGVkcH7o .active2,#mermaid-svg-sIST8YtPiGVkcH7o .active3{fill:#bfc7ff;stroke:#534fbc}#mermaid-svg-sIST8YtPiGVkcH7o .activeText0,#mermaid-svg-sIST8YtPiGVkcH7o .activeText1,#mermaid-svg-sIST8YtPiGVkcH7o .activeText2,#mermaid-svg-sIST8YtPiGVkcH7o .activeText3{fill:#000 !important}#mermaid-svg-sIST8YtPiGVkcH7o .done0,#mermaid-svg-sIST8YtPiGVkcH7o .done1,#mermaid-svg-sIST8YtPiGVkcH7o .done2,#mermaid-svg-sIST8YtPiGVkcH7o .done3{stroke:grey;fill:#d3d3d3;stroke-width:2}#mermaid-svg-sIST8YtPiGVkcH7o .doneText0,#mermaid-svg-sIST8YtPiGVkcH7o .doneText1,#mermaid-svg-sIST8YtPiGVkcH7o .doneText2,#mermaid-svg-sIST8YtPiGVkcH7o .doneText3{fill:#000 !important}#mermaid-svg-sIST8YtPiGVkcH7o .crit0,#mermaid-svg-sIST8YtPiGVkcH7o .crit1,#mermaid-svg-sIST8YtPiGVkcH7o .crit2,#mermaid-svg-sIST8YtPiGVkcH7o .crit3{stroke:#f88;fill:red;stroke-width:2}#mermaid-svg-sIST8YtPiGVkcH7o .activeCrit0,#mermaid-svg-sIST8YtPiGVkcH7o .activeCrit1,#mermaid-svg-sIST8YtPiGVkcH7o .activeCrit2,#mermaid-svg-sIST8YtPiGVkcH7o .activeCrit3{stroke:#f88;fill:#bfc7ff;stroke-width:2}#mermaid-svg-sIST8YtPiGVkcH7o .doneCrit0,#mermaid-svg-sIST8YtPiGVkcH7o .doneCrit1,#mermaid-svg-sIST8YtPiGVkcH7o .doneCrit2,#mermaid-svg-sIST8YtPiGVkcH7o .doneCrit3{stroke:#f88;fill:#d3d3d3;stroke-width:2;cursor:pointer;shape-rendering:crispEdges}#mermaid-svg-sIST8YtPiGVkcH7o .milestone{transform:rotate(45deg) scale(0.8, 0.8)}#mermaid-svg-sIST8YtPiGVkcH7o .milestoneText{font-style:italic}#mermaid-svg-sIST8YtPiGVkcH7o .doneCritText0,#mermaid-svg-sIST8YtPiGVkcH7o .doneCritText1,#mermaid-svg-sIST8YtPiGVkcH7o .doneCritText2,#mermaid-svg-sIST8YtPiGVkcH7o .doneCritText3{fill:#000 !important}#mermaid-svg-sIST8YtPiGVkcH7o .activeCritText0,#mermaid-svg-sIST8YtPiGVkcH7o .activeCritText1,#mermaid-svg-sIST8YtPiGVkcH7o .activeCritText2,#mermaid-svg-sIST8YtPiGVkcH7o .activeCritText3{fill:#000 !important}#mermaid-svg-sIST8YtPiGVkcH7o .titleText{text-anchor:middle;font-size:18px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o g.classGroup text{fill:#9370db;stroke:none;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:10px}#mermaid-svg-sIST8YtPiGVkcH7o g.classGroup text .title{font-weight:bolder}#mermaid-svg-sIST8YtPiGVkcH7o g.clickable{cursor:pointer}#mermaid-svg-sIST8YtPiGVkcH7o g.classGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-sIST8YtPiGVkcH7o g.classGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o .classLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5}#mermaid-svg-sIST8YtPiGVkcH7o .classLabel .label{fill:#9370db;font-size:10px}#mermaid-svg-sIST8YtPiGVkcH7o .relation{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-sIST8YtPiGVkcH7o .dashed-line{stroke-dasharray:3}#mermaid-svg-sIST8YtPiGVkcH7o #compositionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o #compositionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o #aggregationStart{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o #aggregationEnd{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o #dependencyStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o #dependencyEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o #extensionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o #extensionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o .commit-id,#mermaid-svg-sIST8YtPiGVkcH7o .commit-msg,#mermaid-svg-sIST8YtPiGVkcH7o .branch-label{fill:lightgrey;color:lightgrey;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .pieTitleText{text-anchor:middle;font-size:25px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .slice{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o g.stateGroup text{fill:#9370db;stroke:none;font-size:10px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o g.stateGroup text{fill:#9370db;fill:#333;stroke:none;font-size:10px}#mermaid-svg-sIST8YtPiGVkcH7o g.statediagram-cluster .cluster-label text{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o g.stateGroup .state-title{font-weight:bolder;fill:#000}#mermaid-svg-sIST8YtPiGVkcH7o g.stateGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-sIST8YtPiGVkcH7o g.stateGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-sIST8YtPiGVkcH7o .transition{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-sIST8YtPiGVkcH7o .stateGroup .composit{fill:white;border-bottom:1px}#mermaid-svg-sIST8YtPiGVkcH7o .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px}#mermaid-svg-sIST8YtPiGVkcH7o .state-note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-sIST8YtPiGVkcH7o .state-note text{fill:black;stroke:none;font-size:10px}#mermaid-svg-sIST8YtPiGVkcH7o .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.7}#mermaid-svg-sIST8YtPiGVkcH7o .edgeLabel text{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .stateLabel text{fill:#000;font-size:10px;font-weight:bold;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sIST8YtPiGVkcH7o .node circle.state-start{fill:black;stroke:black}#mermaid-svg-sIST8YtPiGVkcH7o .node circle.state-end{fill:black;stroke:white;stroke-width:1.5}#mermaid-svg-sIST8YtPiGVkcH7o #statediagram-barbEnd{fill:#9370db}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-cluster rect{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-cluster rect.outer{rx:5px;ry:5px}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-state .divider{stroke:#9370db}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-state .title-state{rx:5px;ry:5px}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-cluster.statediagram-cluster .inner{fill:white}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-cluster.statediagram-cluster-alt .inner{fill:#e0e0e0}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-cluster .inner{rx:0;ry:0}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-state rect.basic{rx:5px;ry:5px}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#efefef}#mermaid-svg-sIST8YtPiGVkcH7o .note-edge{stroke-dasharray:5}#mermaid-svg-sIST8YtPiGVkcH7o .statediagram-note rect{fill:#fff5ad;stroke:#aa3;stroke-width:1px;rx:0;ry:0}:root{--mermaid-font-family: '"trebuchet ms", verdana, arial';--mermaid-font-family: "Comic Sans MS", "Comic Sans", cursive}#mermaid-svg-sIST8YtPiGVkcH7o .error-icon{fill:#522}#mermaid-svg-sIST8YtPiGVkcH7o .error-text{fill:#522;stroke:#522}#mermaid-svg-sIST8YtPiGVkcH7o .edge-thickness-normal{stroke-width:2px}#mermaid-svg-sIST8YtPiGVkcH7o .edge-thickness-thick{stroke-width:3.5px}#mermaid-svg-sIST8YtPiGVkcH7o .edge-pattern-solid{stroke-dasharray:0}#mermaid-svg-sIST8YtPiGVkcH7o .edge-pattern-dashed{stroke-dasharray:3}#mermaid-svg-sIST8YtPiGVkcH7o .edge-pattern-dotted{stroke-dasharray:2}#mermaid-svg-sIST8YtPiGVkcH7o .marker{fill:#333}#mermaid-svg-sIST8YtPiGVkcH7o .marker.cross{stroke:#333}:root { --mermaid-font-family: "trebuchet ms", verdana, arial;}#mermaid-svg-sIST8YtPiGVkcH7o {color: rgba(0, 0, 0, 0.75);font: ;}

SEI generator
appsrc
funnel
videotestsrc
x264enc
h264parse
rtph264pay
udpsink

第一关键节点是funnel，翻译过来就是漏斗，以GstBuffer为最小粒度对两路数据流做merge，形象点讲就是appsrc产生绿豆，x264enc产生红豆，绿豆红豆一字纵队过漏斗，每颗豆豆代表GstBuffer，是以alignment=au2为最小粒度的NAL。

第二关键节点是appsrc，通过need-data signal3向pipeline中插入SEI

gst-launch语法的pipeline如下：

gst-launch-1.0 funnel name=f \
appsrc name=appsrc-h264-sei do-timestamp=true block=true is-live=true ! video/x-h264, stream-format=byte-stream, alignment=au ! queue ! f. \
videotestsrc is-live=true ! x264enc ! video/x-h264, stream-format=byte-stream, alignment=au, profile=baseline ! queue ! f. \
f. ! queue ! h264parse ! video/x-h264, stream-format=byte-stream, alignment=au ! rtph264pay ! udpsink sync=false clients=127.0.0.1:5004

注册到need-data signal的need_data_callback实现：

static void need_data_callback(GstElement *appsrc, guint unused,gpointer udata) {GST_LOG("need_data_callback");GstBuffer *buffer;GstFlowReturn ret;static uint64_t next_ms_time_insert_sei = 0;struct timespec one_ms;struct timespec rem;uint8_t *h264_sei = NULL;size_t length = 0;one_ms.tv_sec = 0;one_ms.tv_nsec = 1000000;while (now_ms() <= next_ms_time_insert_sei) {GST_TRACE("sleep to wait time trigger");nanosleep(&one_ms, &rem);}if (!h264_sei_ntp_new(&h264_sei, &length)) {GST_ERROR("h264_sei_ntp_new failed");return;}if (NULL != h264_sei && length > 0) {buffer =gst_buffer_new_allocate(NULL, START_CODE_PREFIX_BYTES + length, NULL);if (NULL != buffer) {// fill start_code_prefix: 0x00000001uint8_t start_code_prefix[] = START_CODE_PREFIX;gst_buffer_fill(buffer, 0, start_code_prefix, START_CODE_PREFIX_BYTES);// fill H.264 SEIsize_t bytes_copied =gst_buffer_fill(buffer, START_CODE_PREFIX_BYTES, h264_sei, length);if (bytes_copied == length) {g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);GST_DEBUG("H264 SEI NTP timestamp inserted");} else {GST_ERROR("GstBuffer.fill without all bytes copied");}} else {GST_ERROR("GstBuffer.new_allocate failed");}gst_buffer_unref(buffer);}next_ms_time_insert_sei = now_ms() + 1000;free(h264_sei);
}

receiver

#mermaid-svg-sOEIZ40K8wNa5DPk .label{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);fill:#333;color:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .label text{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .node rect,#mermaid-svg-sOEIZ40K8wNa5DPk .node circle,#mermaid-svg-sOEIZ40K8wNa5DPk .node ellipse,#mermaid-svg-sOEIZ40K8wNa5DPk .node polygon,#mermaid-svg-sOEIZ40K8wNa5DPk .node path{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-sOEIZ40K8wNa5DPk .node .label{text-align:center;fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .node.clickable{cursor:pointer}#mermaid-svg-sOEIZ40K8wNa5DPk .arrowheadPath{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .edgePath .path{stroke:#333;stroke-width:1.5px}#mermaid-svg-sOEIZ40K8wNa5DPk .flowchart-link{stroke:#333;fill:none}#mermaid-svg-sOEIZ40K8wNa5DPk .edgeLabel{background-color:#e8e8e8;text-align:center}#mermaid-svg-sOEIZ40K8wNa5DPk .edgeLabel rect{opacity:0.9}#mermaid-svg-sOEIZ40K8wNa5DPk .edgeLabel span{color:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .cluster rect{fill:#ffffde;stroke:#aa3;stroke-width:1px}#mermaid-svg-sOEIZ40K8wNa5DPk .cluster text{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:12px;background:#ffffde;border:1px solid #aa3;border-radius:2px;pointer-events:none;z-index:100}#mermaid-svg-sOEIZ40K8wNa5DPk .actor{stroke:#ccf;fill:#ECECFF}#mermaid-svg-sOEIZ40K8wNa5DPk text.actor>tspan{fill:#000;stroke:none}#mermaid-svg-sOEIZ40K8wNa5DPk .actor-line{stroke:grey}#mermaid-svg-sOEIZ40K8wNa5DPk .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .messageLine1{stroke-width:1.5;stroke-dasharray:2, 2;stroke:#333}#mermaid-svg-sOEIZ40K8wNa5DPk #arrowhead path{fill:#333;stroke:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .sequenceNumber{fill:#fff}#mermaid-svg-sOEIZ40K8wNa5DPk #sequencenumber{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk #crosshead path{fill:#333;stroke:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .messageText{fill:#333;stroke:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .labelBox{stroke:#ccf;fill:#ECECFF}#mermaid-svg-sOEIZ40K8wNa5DPk .labelText,#mermaid-svg-sOEIZ40K8wNa5DPk .labelText>tspan{fill:#000;stroke:none}#mermaid-svg-sOEIZ40K8wNa5DPk .loopText,#mermaid-svg-sOEIZ40K8wNa5DPk .loopText>tspan{fill:#000;stroke:none}#mermaid-svg-sOEIZ40K8wNa5DPk .loopLine{stroke-width:2px;stroke-dasharray:2, 2;stroke:#ccf;fill:#ccf}#mermaid-svg-sOEIZ40K8wNa5DPk .note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-sOEIZ40K8wNa5DPk .noteText,#mermaid-svg-sOEIZ40K8wNa5DPk .noteText>tspan{fill:#000;stroke:none}#mermaid-svg-sOEIZ40K8wNa5DPk .activation0{fill:#f4f4f4;stroke:#666}#mermaid-svg-sOEIZ40K8wNa5DPk .activation1{fill:#f4f4f4;stroke:#666}#mermaid-svg-sOEIZ40K8wNa5DPk .activation2{fill:#f4f4f4;stroke:#666}#mermaid-svg-sOEIZ40K8wNa5DPk .mermaid-main-font{font-family:"trebuchet ms", verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .section{stroke:none;opacity:0.2}#mermaid-svg-sOEIZ40K8wNa5DPk .section0{fill:rgba(102,102,255,0.49)}#mermaid-svg-sOEIZ40K8wNa5DPk .section2{fill:#fff400}#mermaid-svg-sOEIZ40K8wNa5DPk .section1,#mermaid-svg-sOEIZ40K8wNa5DPk .section3{fill:#fff;opacity:0.2}#mermaid-svg-sOEIZ40K8wNa5DPk .sectionTitle0{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .sectionTitle1{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .sectionTitle2{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .sectionTitle3{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .sectionTitle{text-anchor:start;font-size:11px;text-height:14px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .grid .tick{stroke:#d3d3d3;opacity:0.8;shape-rendering:crispEdges}#mermaid-svg-sOEIZ40K8wNa5DPk .grid .tick text{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .grid path{stroke-width:0}#mermaid-svg-sOEIZ40K8wNa5DPk .today{fill:none;stroke:red;stroke-width:2px}#mermaid-svg-sOEIZ40K8wNa5DPk .task{stroke-width:2}#mermaid-svg-sOEIZ40K8wNa5DPk .taskText{text-anchor:middle;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .taskText:not([font-size]){font-size:11px}#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutsideRight{fill:#000;text-anchor:start;font-size:11px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutsideLeft{fill:#000;text-anchor:end;font-size:11px}#mermaid-svg-sOEIZ40K8wNa5DPk .task.clickable{cursor:pointer}#mermaid-svg-sOEIZ40K8wNa5DPk .taskText.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutsideLeft.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutsideRight.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-sOEIZ40K8wNa5DPk .taskText0,#mermaid-svg-sOEIZ40K8wNa5DPk .taskText1,#mermaid-svg-sOEIZ40K8wNa5DPk .taskText2,#mermaid-svg-sOEIZ40K8wNa5DPk .taskText3{fill:#fff}#mermaid-svg-sOEIZ40K8wNa5DPk .task0,#mermaid-svg-sOEIZ40K8wNa5DPk .task1,#mermaid-svg-sOEIZ40K8wNa5DPk .task2,#mermaid-svg-sOEIZ40K8wNa5DPk .task3{fill:#8a90dd;stroke:#534fbc}#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutside0,#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutside2{fill:#000}#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutside1,#mermaid-svg-sOEIZ40K8wNa5DPk .taskTextOutside3{fill:#000}#mermaid-svg-sOEIZ40K8wNa5DPk .active0,#mermaid-svg-sOEIZ40K8wNa5DPk .active1,#mermaid-svg-sOEIZ40K8wNa5DPk .active2,#mermaid-svg-sOEIZ40K8wNa5DPk .active3{fill:#bfc7ff;stroke:#534fbc}#mermaid-svg-sOEIZ40K8wNa5DPk .activeText0,#mermaid-svg-sOEIZ40K8wNa5DPk .activeText1,#mermaid-svg-sOEIZ40K8wNa5DPk .activeText2,#mermaid-svg-sOEIZ40K8wNa5DPk .activeText3{fill:#000 !important}#mermaid-svg-sOEIZ40K8wNa5DPk .done0,#mermaid-svg-sOEIZ40K8wNa5DPk .done1,#mermaid-svg-sOEIZ40K8wNa5DPk .done2,#mermaid-svg-sOEIZ40K8wNa5DPk .done3{stroke:grey;fill:#d3d3d3;stroke-width:2}#mermaid-svg-sOEIZ40K8wNa5DPk .doneText0,#mermaid-svg-sOEIZ40K8wNa5DPk .doneText1,#mermaid-svg-sOEIZ40K8wNa5DPk .doneText2,#mermaid-svg-sOEIZ40K8wNa5DPk .doneText3{fill:#000 !important}#mermaid-svg-sOEIZ40K8wNa5DPk .crit0,#mermaid-svg-sOEIZ40K8wNa5DPk .crit1,#mermaid-svg-sOEIZ40K8wNa5DPk .crit2,#mermaid-svg-sOEIZ40K8wNa5DPk .crit3{stroke:#f88;fill:red;stroke-width:2}#mermaid-svg-sOEIZ40K8wNa5DPk .activeCrit0,#mermaid-svg-sOEIZ40K8wNa5DPk .activeCrit1,#mermaid-svg-sOEIZ40K8wNa5DPk .activeCrit2,#mermaid-svg-sOEIZ40K8wNa5DPk .activeCrit3{stroke:#f88;fill:#bfc7ff;stroke-width:2}#mermaid-svg-sOEIZ40K8wNa5DPk .doneCrit0,#mermaid-svg-sOEIZ40K8wNa5DPk .doneCrit1,#mermaid-svg-sOEIZ40K8wNa5DPk .doneCrit2,#mermaid-svg-sOEIZ40K8wNa5DPk .doneCrit3{stroke:#f88;fill:#d3d3d3;stroke-width:2;cursor:pointer;shape-rendering:crispEdges}#mermaid-svg-sOEIZ40K8wNa5DPk .milestone{transform:rotate(45deg) scale(0.8, 0.8)}#mermaid-svg-sOEIZ40K8wNa5DPk .milestoneText{font-style:italic}#mermaid-svg-sOEIZ40K8wNa5DPk .doneCritText0,#mermaid-svg-sOEIZ40K8wNa5DPk .doneCritText1,#mermaid-svg-sOEIZ40K8wNa5DPk .doneCritText2,#mermaid-svg-sOEIZ40K8wNa5DPk .doneCritText3{fill:#000 !important}#mermaid-svg-sOEIZ40K8wNa5DPk .activeCritText0,#mermaid-svg-sOEIZ40K8wNa5DPk .activeCritText1,#mermaid-svg-sOEIZ40K8wNa5DPk .activeCritText2,#mermaid-svg-sOEIZ40K8wNa5DPk .activeCritText3{fill:#000 !important}#mermaid-svg-sOEIZ40K8wNa5DPk .titleText{text-anchor:middle;font-size:18px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk g.classGroup text{fill:#9370db;stroke:none;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:10px}#mermaid-svg-sOEIZ40K8wNa5DPk g.classGroup text .title{font-weight:bolder}#mermaid-svg-sOEIZ40K8wNa5DPk g.clickable{cursor:pointer}#mermaid-svg-sOEIZ40K8wNa5DPk g.classGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-sOEIZ40K8wNa5DPk g.classGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk .classLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5}#mermaid-svg-sOEIZ40K8wNa5DPk .classLabel .label{fill:#9370db;font-size:10px}#mermaid-svg-sOEIZ40K8wNa5DPk .relation{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-sOEIZ40K8wNa5DPk .dashed-line{stroke-dasharray:3}#mermaid-svg-sOEIZ40K8wNa5DPk #compositionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk #compositionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk #aggregationStart{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk #aggregationEnd{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk #dependencyStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk #dependencyEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk #extensionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk #extensionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk .commit-id,#mermaid-svg-sOEIZ40K8wNa5DPk .commit-msg,#mermaid-svg-sOEIZ40K8wNa5DPk .branch-label{fill:lightgrey;color:lightgrey;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .pieTitleText{text-anchor:middle;font-size:25px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .slice{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk g.stateGroup text{fill:#9370db;stroke:none;font-size:10px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk g.stateGroup text{fill:#9370db;fill:#333;stroke:none;font-size:10px}#mermaid-svg-sOEIZ40K8wNa5DPk g.statediagram-cluster .cluster-label text{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk g.stateGroup .state-title{font-weight:bolder;fill:#000}#mermaid-svg-sOEIZ40K8wNa5DPk g.stateGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-sOEIZ40K8wNa5DPk g.stateGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-sOEIZ40K8wNa5DPk .transition{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-sOEIZ40K8wNa5DPk .stateGroup .composit{fill:white;border-bottom:1px}#mermaid-svg-sOEIZ40K8wNa5DPk .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px}#mermaid-svg-sOEIZ40K8wNa5DPk .state-note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-sOEIZ40K8wNa5DPk .state-note text{fill:black;stroke:none;font-size:10px}#mermaid-svg-sOEIZ40K8wNa5DPk .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.7}#mermaid-svg-sOEIZ40K8wNa5DPk .edgeLabel text{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .stateLabel text{fill:#000;font-size:10px;font-weight:bold;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-sOEIZ40K8wNa5DPk .node circle.state-start{fill:black;stroke:black}#mermaid-svg-sOEIZ40K8wNa5DPk .node circle.state-end{fill:black;stroke:white;stroke-width:1.5}#mermaid-svg-sOEIZ40K8wNa5DPk #statediagram-barbEnd{fill:#9370db}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-cluster rect{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-cluster rect.outer{rx:5px;ry:5px}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-state .divider{stroke:#9370db}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-state .title-state{rx:5px;ry:5px}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-cluster.statediagram-cluster .inner{fill:white}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-cluster.statediagram-cluster-alt .inner{fill:#e0e0e0}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-cluster .inner{rx:0;ry:0}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-state rect.basic{rx:5px;ry:5px}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#efefef}#mermaid-svg-sOEIZ40K8wNa5DPk .note-edge{stroke-dasharray:5}#mermaid-svg-sOEIZ40K8wNa5DPk .statediagram-note rect{fill:#fff5ad;stroke:#aa3;stroke-width:1px;rx:0;ry:0}:root{--mermaid-font-family: '"trebuchet ms", verdana, arial';--mermaid-font-family: "Comic Sans MS", "Comic Sans", cursive}#mermaid-svg-sOEIZ40K8wNa5DPk .error-icon{fill:#522}#mermaid-svg-sOEIZ40K8wNa5DPk .error-text{fill:#522;stroke:#522}#mermaid-svg-sOEIZ40K8wNa5DPk .edge-thickness-normal{stroke-width:2px}#mermaid-svg-sOEIZ40K8wNa5DPk .edge-thickness-thick{stroke-width:3.5px}#mermaid-svg-sOEIZ40K8wNa5DPk .edge-pattern-solid{stroke-dasharray:0}#mermaid-svg-sOEIZ40K8wNa5DPk .edge-pattern-dashed{stroke-dasharray:3}#mermaid-svg-sOEIZ40K8wNa5DPk .edge-pattern-dotted{stroke-dasharray:2}#mermaid-svg-sOEIZ40K8wNa5DPk .marker{fill:#333}#mermaid-svg-sOEIZ40K8wNa5DPk .marker.cross{stroke:#333}:root { --mermaid-font-family: "trebuchet ms", verdana, arial;}#mermaid-svg-sOEIZ40K8wNa5DPk {color: rgba(0, 0, 0, 0.75);font: ;}

udpsrc
rtph264depay
identity
fakesink

关键节点是identity，对于每个GstBuffer的到来都会触发handoff signal，接下来的识别处理工作就交给handoff_callback处理

gst-launch语法的pipeline如下：

gst-launch-1.0 udpsrc uri=udp://127.0.0.1:5004 caps="application/x-rtp, media=video, encoding-name=H264" ! rtph264depay ! video/x-h264, stream-format=byte-stream, alignment=nal ! identity name=identity ! fakesink

注意到alignment=nal跟sender不一样，因为rtph264depay输出为alignment=au时会把SEI丢弃掉4

注册到handoff signal的handoff_callback实现：

static void handoff_callback(GstElement *identity, GstBuffer *buffer,gpointer user_data) {GST_TRACE("handoff_callback");GstMapInfo info = GST_MAP_INFO_INIT;GstH264NalParser *nalparser = NULL;GstH264NalUnit nalu;if (gst_buffer_map(buffer, &info, GST_MAP_READ)) {nalparser = gst_h264_nal_parser_new();if (NULL != nalparser) {if (GST_H264_PARSER_OK ==gst_h264_parser_identify_nalu_unchecked(nalparser, info.data, 0,info.size, &nalu)) {// if (info.size < 100) GST_LOG("buffer info size %ld", info.size);if (GST_H264_NAL_SEI == nalu.type) {GST_LOG("identify sei nalu with size = %d, offset = %d, sc_offset = %d",nalu.size, nalu.offset, nalu.sc_offset);int64_t delay = -1;if (TRUE ==h264_sei_ntp_parse(nalu.data + nalu.offset, nalu.size, &delay)) {GST_LOG("delay = %ld ms", delay);}}} else {GST_WARNING("gst_h264_parser_identify_nalu_unchecked failed");}gst_h264_nal_parser_free(nalparser);} else {GST_WARNING("gst_h264_nal_parser_new failed");}gst_buffer_unmap(buffer, &info);} else {GST_WARNING("gst_buffer_map failed");}
}

NTP SEI generator and parser

基于aizvorski/h264bitstream实现
关于H.264 SEI的数据结构参考：FFmpeg从入门到精通——进阶篇，SEI那些事儿

// h264_sei_ntp.h
#ifndef H264_SEI_NTP
#define H264_SEI_NTP#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>// #define H264_SEI_UUID_NTP_TIMESTAMP {0x60, 0x2b, 0x0d, 0xb6, 0x2d, 0x3d,
// 0x44, 0xb5, 0xab, 0x9e, 0xec, 0x8a, 0xd7, 0x1f, 0x3f, 0x8e} test
// emulation_prevention_three_byte
#define H264_SEI_UUID_NTP_TIMESTAMP                                            \{                                                                            \0x00, 0x00, 0x01, 0x00, 0x00, 0x02, 0x00, 0x00, 0x03, 0x9e, 0xec, 0x8a,    \0xd7, 0x1f, 0x3f, 0x8e                                                 \}
#define START_CODE_PREFIX_BYTES 4
#define START_CODE_PREFIX                                                      \{ 0x00, 0x00, 0x00, 0x01 }uint64_t now_ms();bool h264_sei_ntp_new(uint8_t **h264_sei, size_t *length);bool h264_sei_ntp_parse(uint8_t *h264_sei, size_t length, int64_t *delay);#endif

// h264_sei_ntp.c
#include "h264_sei_ntp.h"
#include "h264_stream.h" // https://github.com/D-Y-Innovations/h264bitstream/blob/master/h264_stream.h
#include <math.h>
#include <stdlib.h>
#include <time.h>#define NALU_BUFFER_MAX_SIZE 50
#define SEI_PAYLOAD_SIZE 24
#define H264_SEI_NTP_UUID_SIZE 16/*
start_code_prefix_one_3bytes: 0x000001
nal_unit_type: 0x06, Supplemental enhancement information (SEI)
SEI payload_type_byte: 0x05,
SEI payload_size_byte: 0x18
uuid_iso_iec_11578: 602b0db6-2d3d-44b5-ab9e-ec8ad71f3f8e
user_data_payload_byte: 0x0000012345678912
rbsp_trailing_bits(): 0x80emulation_prevention_three_byte: 0x03
*/uint64_t now_ms() {long ms;  // Millisecondstime_t s; // Secondsstruct timespec spec;clock_gettime(CLOCK_REALTIME, &spec);s = spec.tv_sec;ms = round(spec.tv_nsec / 1.0e6); // Convert nanoseconds to millisecondsif (ms > 999) {s++;ms = 0;}return s * 1000 + ms;
}/*** @param h264_sei H264 SEI buffer with start_code_prefix_one_3bytes,* transfer-full, handler should free memory after used* @param length   h264_sei 的长度* @return         如果执行成功则返回true，否则返回false* @since          1.0*/
bool h264_sei_ntp_new(uint8_t **h264_sei, size_t *length) {uint8_t sei_uuid[] = H264_SEI_UUID_NTP_TIMESTAMP;uint8_t buffer[NALU_BUFFER_MAX_SIZE] = {0};uint8_t payloadData[SEI_PAYLOAD_SIZE] = {0};h264_stream_t *h264_nalu_sei = h264_new();h264_nalu_sei->nal->nal_ref_idc = NAL_REF_IDC_PRIORITY_DISPOSABLE;h264_nalu_sei->nal->nal_unit_type = NAL_UNIT_TYPE_SEI;memcpy(payloadData, sei_uuid, sizeof(sei_uuid));uint64_t milli_time = now_ms();memcpy(&(payloadData[16]), &milli_time, sizeof(uint64_t)); // little-endiansei_t *seis[1] = {sei_new()};sei_t *sei = seis[0];sei->payloadType = SEI_TYPE_USER_DATA_UNREGISTERED;sei->payloadSize = SEI_PAYLOAD_SIZE;sei->data = payloadData;h264_nalu_sei->seis = seis;h264_nalu_sei->num_seis = 1;int len = write_nal_unit(h264_nalu_sei, buffer, NALU_BUFFER_MAX_SIZE);if (len <= 0) {return false;}uint8_t *h264_sei_tmp = malloc(len);if (NULL == h264_sei_tmp)return false;memcpy(h264_sei_tmp, buffer, len);*length = len;*h264_sei = h264_sei_tmp;return true;
}/*** @param h264_sei H264 SEI buffer without start_code_prefix_one_3bytes* @param length   h264_sei 的长度* @param delay    返回时延，单位ms* @return         如果执行成功则返回true，否则返回false* @since          1.0*/
bool h264_sei_ntp_parse(uint8_t *h264_sei, size_t length, int64_t *delay) {uint8_t sei_uuid[] = H264_SEI_UUID_NTP_TIMESTAMP;h264_stream_t *h264_nalu_sei = h264_new();if (read_nal_unit(h264_nalu_sei, h264_sei, length) > 0) {if (1 == h264_nalu_sei->num_seis) {sei_t *sei = h264_nalu_sei->seis[0];if (SEI_TYPE_USER_DATA_UNREGISTERED == sei->payloadType) {if (SEI_PAYLOAD_SIZE == sei->payloadSize) {if (0 == memcmp(sei->data, sei_uuid, H264_SEI_NTP_UUID_SIZE)) {uint64_t timestamp = 0;memcpy(&timestamp, sei->data + H264_SEI_NTP_UUID_SIZE,sizeof(timestamp));*delay = now_ms() - timestamp;if (*delay >= 0) {return true;} else {printf("delay <0\n");}} else {printf("memcmp != 0\n");}} else {printf("payloadSize !== SEI_PAYLOAD_SIZE\n");}} else {printf("payloadType !== SEI_TYPE_USER_DATA_UNREGISTERED\n");}} else {printf("num_seis !== 1\n");}} else {printf("read_nal_unit <= 0\n");}return false;
}

  1. 播放器技术分享（5）：延时优化 ↩︎

  2. What is the alignment capability in video/x-h264 ↩︎

  3. appsrc-stream2.c ↩︎

  4. gstrtph264depay.c ↩︎

【GStreamer】基于NTP+SEI的视频流传输时延测量相关推荐

  1. 【Python】基于OpenCV与UDP实现的视频流传输

    文章目录 前言 原理 代码 服务端 客户端 运行效果 参考资料 前言 2021年电赛的测量题(如下)需要实现局域网视频传输,我们的方案是使用gst-rtsp-server 搭建 RTSP 服务器 进行 ...

  2. python实现流媒体传输_基于OpenCV的网络实时视频流传输的实现

    很多小伙伴都不会在家里或者办公室安装网络摄像头或监视摄像头.但是有时,大家又希望能够随时随地观看视频直播. 大多数人会选择使用IP摄像机(Internet协议摄像机)而不是CCTV(闭路电视),因为它 ...

  3. 基于视频流传输 — 在线教育白板技术

    在线教育不同于线下教育, 内容需要经过电子白板展现给用户,如何做出优秀的在线教育白板成为研究的重点.本文来自学而思网校客户端架构负责人赵文杰在LiveVideoStackCon 2018大会上的分享, ...

  4. 第6季2：基于RTSP协议的实时视频流传输的源码分析

    以下内容源于朱有鹏嵌入式课程的学习与整理,如有侵权请告知删除. 前言 博文第一季2:HI3518EV200的初体验中,所提供的测试文件sample_venc实现了基于RTSP协议的实时视频流传输功能. ...

  5. 基于内容的自适应视频传输算法及其应用

    本文内容来自LiveVideoStack线上分享第四季第二期,由湖北经济学院副教授,胡胜红博士为大家介绍如何基于内容分析技术,从用户需求角度标注视频流重要性级别,构建自适应流传输策略,实现语义级QoE ...

  6. 模拟QQ软件的基于多线程的流媒体加密传输软件技术

    模拟QQ软件的基于多线程的流媒体加密传输软件技术 模拟QQ软件,基于多线程编程捕捉摄像头及麦克风实时数据,基于socket通信设计发送端.接收端两个部分的,对音频和视频进行采集.加密或加水印.传输.解 ...

  7. 基于NS-2的网络视频传输仿真平台的建立

    基于NS-2的网络视频传输仿真平台的建立   ----网络视频传输仿真平台的建立 摘 要 随着Internet和多媒体技术的迅猛发展, Internet已逐步从单一的数据传送网向数据.语音.图像等多媒 ...

  8. pcb 布线电容 影响延时_信号在PCB走线中传输时延

    信号在媒质中传播时,其传播速度受信号载体以及周围媒质属性决定.在PCB(印刷电路板)中信号的传输速度就与板材DK(介电常数),信号模式,信号线与信号线间耦合以及绕线方式等有关.随着PCB走线信号速率越 ...

  9. （超）低延迟视频流传输的未来

    作者:Anthony Dantard 翻译:Alex 技术审校:袁荣喜 ▲扫描图中二维码了解音视频技术大会更多信息▲ 影音探索 #013# 用户对服务的期望在不断攀升,并逐渐出现了不满情绪.由于有了Y ...

最新文章

  1. 使用Tensorize评估硬件内部特性
  2. shiny 发布部署到公开网站 ，供别人访问
  3. 在ASP.Net和IIS中删除不必要的HTTP响应头
  4. Linux下php添加新扩展
  5. sony z2 android 5.0,索尼Xperia Z2 5.0 root教程_索尼Z2获取5.0系统的root
  6. 网络流(最大流) HDU 1565 方格取数(1) HDU 1569 方格取数(2)
  7. Python len函数 - Python零基础入门教程
  8. 如何利用小熊派获取MPU6050六轴原始数据
  9. 开源 静态 文件 服务器,Node.js搭建静态服务器
  10. win11组策略如何恢复默认设置 windows11组策略恢复默认设置的步骤方法
  11. 工作流之流程定义存储表
  12. 学习python:模块的建立与发布
  13. canvas特效收集
  14. 小米mix2s主板电路图
  15. 【奶奶看了也不会】AI绘画 Mac安装stable-diffusion-webui绘制AI妹子保姆级教程
  16. 两步解决macbook电池不充电
  17. 承前2021，启后2022
  18. 阿里云服务器cnetos7常用命令
  19. 征信衍生规则中的逾期强变量都有哪些？
  20. 【BZOJ 1925】【SDOI 2010】地精部落

热门文章

  1. 数据结构课程设计报告-职工信息管理系统
  2. pandas函数melt的应用
  3. JavaSE-网络socket编程
  4. SqlDataReader转换为DataTable
  5. UVA140回溯法剪枝剪多少的讨论
  6. 双十二大家都在买哪些书？这份书单请码住
  7. Linux替换jar包中内容
  8. 数据结构 深入理解二叉树的实现
  9. 最系统的大数据挖掘技术及其应用详解！
  10. 北大ACM线上比赛有感