Introduction

There is no document that explain fully described RTMP specification. So, I don’t know what RPC methods, and there logic.

In this series’s design, Ingest receive RTMP stream from video streaming sw such as OBS studio, xsplit, or prism. And Ingest passthrough stream into transcoder that depend on ffmpeg.

I don’t know xsplit and prism’s rtmp implementation. But, OBS studio is open source and it use librtmp that rtmp implementation from ffmpeg(exactly, rtmpdump from mplayer).

librtmp is a part of ffmpeg that cover almost media player and servers.

In this post, Summary OBS studio’s RTMP imeplmentation.

Methods

Analyze real streaming network flows (Youtube Live streaming)

sequenceDiagram
	Note over Initiator, Responder: Handshaking done
	Initiator->>Responder: Set Chunk Size(4096)
	
	Initiator->>Responder: Command(connect(live2))
    Responder->>Initiator: Window Acknowledgement Size(2_500_000)
    Responder->>Initiator: Set Peer Bandwidth(10_000_000)
	Responder->>Initiator: Set Chunk Size(256)
    Responder->>Initiator: Command(connect_result(NetConnection.Connect.Success))
	Responder->>Initiator: Command(onBWDone())

    Initiator->>Responder: Command(releaseStream([stream_key]))
    Initiator->>Responder: Command(FCPublish([stream_key]))
    Initiator->>Responder: Command(createStream())
    Initiator->>Responder: Command(_checkbw())
    Responder->>Initiator: Command(createStream_result(1))
	
    Initiator->>Responder: Command(publish([stream_key]))
    Responder->>Initiator: Command(publish_onStatus(NetStream.Publish.Start))
    Initiator->>Responder: Data([Metadata])

    Initiator->>Responder: Audio(AAC)
    Initiator->>Responder: Video(AVC)
    Note over Initiator, Responder: Until the stream is complete

    Initiator->>Responder: Command(FCUnpublish([stream_key]))
    Initiator->>Responder: Command(deleteStream(1))

• connect send app name, target host information
• Window Acknowledgement Size(2_500_000) is librtmp’s default value, so no reply
• only exist Set Chunk Size, Command, Window Acknowledgement Size, Set Peer Bandwidth, Data
  • In RTMP 1.0 Specification, User Control(Stream Begin) exist, but OBS don’t send.

    sequenceDiagram
    	Note over Initiator, Responder: Handshaking done
    
    	Initiator->>Responder: Command(connect)
    	Responder->>Initiator: Window Acknowledgement Size
    	Responder->>Initiator: Set Peer Bandwidth
    	Initiator->>Responder: Window Acknowledgement Size
    	Responder->>Initiator: User Control(Stream Begin)
    	Responder->>Initiator: Command(_result(connect))
    

Follow source codes

Flow of streaming

When start streaming by OBS studio.

sequenceDiagram
	Note over Initiator, Responder: Handshaking done

	alt OBS::try_connect
		alt librtmp::RTMP_Connect
			alt librtmp::RTMP_Connect1
				alt librtmp::SendConnectPacket
					Initiator->>Responder: Set Chunk Size
					Initiator->>Responder: Command(connect)
				end
			end
		end
		alt librtmp::RTMP_ConnectStream
			alt librtmp::RTMP_ClientPacket
				par librtmp::HandleInvoke
					Responder->>Initiator: Command(_result(connect))
					loop
						Initiator->>Responder: Command(ReleaseStream)
					end
					loop
						Initiator->>Responder: Command(FCPublish)
					end
					loop
						Initiator->>Responder: Command(CreateStream)
					end
				and
					Responder->>Initiator: Command(_result(CreateStream))
					Initiator->>Responder: Command(publish)
					Responder->>Initiator: Command(_result(publish))
				end
			end
		end
	end
	alt librtmp::init_send
		alt librtmp::send_meta_data
			Initiator->>Responder: Data(Metadata)
		end
	end
	par
		loop librtmp::send_thread
		    alt librtmp::send_headers
			    Initiator->>Responder: Data
            end
            alt librtmp::send_packet
                Note over Initiator: circlebuf_pop_front()
                Note over Initiator: FLV data(RTMP use FLV's Audio, Video, and Data(AMF0))
                Initiator->>Responder: Audio
                Initiator->>Responder: Video
                Initiator->>Responder: Data
            end
		end
	and
		loop librtmp::rtmp_stream_data
			alt librtmp::add_video_packet
				Note over Initiator: Drop frames
				alt librtmp::add_packet
          Note over Initiator: circlebuf_push_back()
				end
			end
		end
	end
	alt librtmp::RTMP_DeleteStream
		loop
			Initiator->>Responder: Command(FCUnpublish)
		end
		Initiator->>Responder: Command(DeleteStream)
	end

• Reference code lines
  • Command(_result(connect))
  • Command(CreateStream)
  • send_meta_data
  • send_packet
  • circlebuf_push_back

Points

• AMF3 is not supported

  “%s, flex …, size %u bytes, not supported, ignoring”,

  “%s, flex message, size %u bytes, not fully supported”,

• Shared Object (0x13, 0x10) is not supported

RTMP_Log(RTMP_LOGDEBUG, “%s, shared object, not supported, ignoring”,

• Audio, Video, Data(metadata and media header) are defined in FLV

• OBS process special message Silent Reconnect (0x20)

  • OBS reconnect at nearest keyframe
  • Can be useful in server-side for rebalancing connection or graceful shutdown.

• Drop frame is occured by video packet that has exceed threshold.

  • DO NOT drop Audio packet and video keyframe

Discussions

• Twitch support two audio track, Live track and VOD(DCMA-free) track.
  • It may related in these lines

    	while (next) {
    		if (!send_audio_header(stream, i++, &next))
    			return false;
    	}