[live555] 谈一谈 SERVER RTP+NALU 分包过程

##前言
live555 处理请求消息四 “PLAY” 续中已经写了start stream 流程，但是没有专门写RTP 包完整的组成过程

当我写完流程图，自己都吓了一跳，这么复杂，所以准备后面每个部分拆分

下面组包的前提，已经读取150000个字节在内存中，在谈一谈 live555 (*.mkv) track1 和 track2 的信息获取，已经读取了. 具体 MatroskaFileParser::parseEBMLIdAndSize 方法中用

而读取内存的指针变量是 curBank()

void StreamParser::ensureValidBytes1(unsigned numBytesNeeded) {
//....// Try to read as many new bytes as will fit in the current bank:unsigned maxNumBytesToRead = BANK_SIZE - fTotNumValidBytes;fInputSource->getNextFrame(&curBank()[fTotNumValidBytes],maxNumBytesToRead,afterGettingBytes, this,onInputClosure, this);
}
unsigned char* curBank() { return fCurBank; }

知道数据的来源后，就可以解析RTP包的数据

RTP Header 组成

void MultiFramedRTPSink::buildAndSendPacket(Boolean isFirstPacket) {nextTask() = NULL;fIsFirstPacket = isFirstPacket;// Set up the RTP header:unsigned rtpHdr = 0x80000000; // RTP version 2; marker ('M') bit not set (by default; it can be set later)rtpHdr |= (fRTPPayloadType<<16);rtpHdr |= fSeqNo; // sequence numberfOutBuf->enqueueWord(rtpHdr);// Note where the RTP timestamp will go.// (We can't fill this in until we start packing payload frames.)fTimestampPosition = fOutBuf->curPacketSize();fOutBuf->skipBytes(4); // leave a hole for the timestampfOutBuf->enqueueWord(SSRC());// Allow for a special, payload-format-specific header following the// RTP header:fSpecialHeaderPosition = fOutBuf->curPacketSize();fSpecialHeaderSize = specialHeaderSize();fOutBuf->skipBytes(fSpecialHeaderSize);// Begin packing as many (complete) frames into the packet as we can:fTotalFrameSpecificHeaderSizes = 0;fNoFramesLeft = False;fNumFramesUsedSoFar = 0;packFrame();
}

上面的代码在第一次开始打包的时候，isFirstPacket = true,

所以
unsigned rtpHdr = 0x80000000; RTP Header 根据RTP协议规定 V=2 占用2bit
rtpHdr |= (fRTPPayloadType<<16); 左移16bit 表示负载 96
rtpHdr |= fSeqNo; 序列号

在初始化RTPSink的时候已经初始化序列号时间戳还有SSRC

  fSeqNo = (u_int16_t)our_random();fSSRC = our_random32();fTimestampBase = our_random32();

最后放入 fOutBuf->enqueueWord(rtpHdr); buffer中

fOutBuf->skipBytes(4); 给时间戳流出4个字节的位置
fOutBuf->enqueueWord(SSRC()); 将SSRC 填充到buffer中

因为第一次 fOutBuf->curPacketSize() = 0 所以后面直接跟 NALU
代表的偏移 fCurOffset

NALU 因为过大，按照 FU-A分片的流程
1）第一个FU-A包的FU indicator：F应该为当前NALU头的F，而NRI应该为当前NALU头的NRI，Type则等于28，表明它是FU-A包。FU header生成方法：S = 1，E = 0，R = 0，Type则等于NALU头中的Type。
2）后续的N个FU-A包的FU indicator和第一个是完全一样的，如果不是最后一个包，则FU header应该为：S = 0，E = 0，R = 0，Type等于NALU头中的Type。
3）最后一个FU-A包FU header应该为：S = 0，E = 1，R = 0，Type等于NALU头中的Type。
同一个NALU分包厚的FU indicator头是完全一致的，FU header只有S以及E位有区别，分别标记开始和结束，它们的RTP分包的序列号应该是依次递增的，并且它们的时间戳必须一致，而负载数据为NALU包去掉1个字节的NALU头后对剩余数据的拆分，这点很关键，你可以认为NALU头被拆分成了FU indicator和FU header，所以不再需要1字节的NALU头了

根据上面流程分析code ：

NALU Indicator & NALU Header

void H264or5Fragmenter::doGetNextFrame() {if (fNumValidDataBytes == 1) { //第一次读取数据 需要从文件中读取 数据fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,afterGettingFrame, this,FramedSource::handleClosure, this);} else { //已经读取了数据 但是根据读取的NALU 第一个字节和第二个字节判断 是NALU 单一模式 还是 分片NALU//这里分成三种1. NALU 单一模式 2. 分片NALU FU-A start 3. FU-A 和 FU-endfMaxSize = fMaxOutputPacketSize;fLastFragmentCompletedNALUnit = True; // by defaultif (fCurDataOffset == 1) { // case 1 or 2if (fNumValidDataBytes - 1 <= fMaxSize) { // case 1  NALU 单一模式 只有NALU Headermemmove(fTo, &fInputBuffer[1], fNumValidDataBytes - 1);envir() <<"NAL unit data exist case 1 fTo = "<< fTo<<" fTo[0] = "<<fTo[0] <<" fInputBuffer[1] = "<< fInputBuffer[1] <<" \n";} else { // case 2 FU-A start FU indicator+FU headerif (fHNumber == 264) {fInputBuffer[0] = (fInputBuffer[1] & 0xE0) | 28; // FU indicatorfInputBuffer[1] = 0x80 | (fInputBuffer[1] & 0x1F); // FU header (with S bit)}memmove(fTo, fInputBuffer, fMaxSize);envir() << "NAL unit data exist case 2 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer[1] = " << fInputBuffer[1] << " \n";}} else { // case 3 FU-A & Fu-A end     FU indicator+FU headerunsigned numExtraHeaderBytes;envir() << "fCurDataOffset =  " << fCurDataOffset  << " \n";if (fHNumber == 264) {fInputBuffer[fCurDataOffset-2] = fInputBuffer[0]; // FU indicatorfInputBuffer[fCurDataOffset-1] = fInputBuffer[1]&~0x80; // FU header (no S bit)numExtraHeaderBytes = 2;} //根据发送包的大小拆分， 判断是中间的FU-A 还是 如果最后包大小小于最大允许发送大小就是 FU-A end if (numBytesToSend > fMaxSize) {// We can't send all of the remaining data this time:numBytesToSend = fMaxSize;fLastFragmentCompletedNALUnit = False;} else {// This is the last fragment:fInputBuffer[fCurDataOffset-1] |= 0x40; // set the E bit in the FU headerfNumTruncatedBytes = fSaveNumTruncatedBytes;}memmove(fTo, &fInputBuffer[fCurDataOffset-numExtraHeaderBytes], numBytesToSend);envir() << "NAL unit data exist case 3 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer["<<fCurDataOffset-numExtraHeaderBytes<<"] = " << fInputBuffer[fCurDataOffset - numExtraHeaderBytes] << " \n";}//.......FramedSource::afterGetting(this);}
}

这里就是处理NALU header 地方并且针对不同的情况加了log

    //第一种情况 NALU 单一模式 不需要分片 只有8个字节的NALU Headerenvir() <<"NAL unit data exist case 1 fTo = "<< fTo<<" fTo[0] = "<<fTo[0] <<" fInputBuffer[1] = "<< fInputBuffer[1] <<" \n";//第二种情况 NALU 需要分片  分片FU-A start  所以会有FU indicator + header 2个字节envir() << "NAL unit data exist case 2 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer[1] = " << fInputBuffer[1] << " \n";//第三种情况 NALU 需要分片  分片FU-A 和 FU-A end 唯一能区分就是包是否小于可允许最大size 如果是 那就是最后一个FU-A，反之分包并发送中间的FU-Aenvir() << "NAL unit data exist case 3 fTo = " << fTo << " fTo[0] = " << fTo[0] << " fTo[1] = " << fTo[1] << " fInputBuffer["<<fCurDataOffset-numExtraHeaderBytes<<"] = " <<fInputBuffer[fCurDataOffset - numExtraHeaderBytes] << " \n";

参考 log如下

若只要fTo[0] 就是NALU Header 值，有fTo[0] 和fTo[1]的值，表示 FUindicator 和 FU header
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 103 fInputBuffer[1] = 103 //SPS
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 104 fInputBuffer[1] = 104 //PPS
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 6 fInputBuffer[1] = 6   //SEI
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 101 fInputBuffer[1] = 101 //IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65 //NON-IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65 //NON-IDR
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1//FU-A start
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 1 fInputBuffer[1442] = 92
//FU-A end
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[2884] = 92
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1
NAL unit data exist case 2 fTo = 00985A34 fTo[0] = 92 fTo[1] = 129 fInputBuffer[1] = 129
NAL unit data exist case 3 fTo = 00985A34 fTo[0] = 92 fTo[1] = 65 fInputBuffer[1442] = 92
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 65 fInputBuffer[1] = 65
NAL unit data exist case 1 fTo = 00985A34 fTo[0] = 1 fInputBuffer[1] = 1....

上面都是根据 fMaxSize 判断是否应该使用分片NALU ,fMaxSize = 1444 ,计算过程如下 1456-12(RTP hdr) ourMaxPacketSize() = 1456

fOurFragmenter = new H264or5Fragmenter(fHNumber, envir(), fSource, OutPacketBuffer::maxSize,ourMaxPacketSize() - 12/*RTP hdr size*/);//OutPacketBuffer::maxSize = 60000
setPacketSizes((RTP_PAYLOAD_PREFERRED_SIZE), (RTP_PAYLOAD_MAX_SIZE));
RTP_PAYLOAD_MAX_SIZE = 1456

具体的参数计算可能需要自己加log 计算后面需要详细知道参数获取过程