Transcode is a threaded application. The different threads which are of importance here are one decoder thread, one encoder thread and N frame processing threads. N is a number the user can control via the second argument of transcodes -u option.
Each frame is put into a frame_list_t (/src/framebuffer.h) structure. The frames are buffered internally in a double linked list. The order of an frame in this list denotes the decoding (and therefore) the encoding order (/src/video_buffer.c)
The import module decompresses a frame from the input file into a format the core of transcode likes. Just after the frame arrives in the decoder, the status of the frame is set to FRAME_WAIT. the first frame pre-processing takes place here.
1) preprocess_vid_frame() (/src/frame_preprocess.c)
After the very early clipping the filters get their draw with the tag
When all filters are done with TC_PRE_S_PROCESS, the frame leaves the decoder and the sychronous part and now waits to be picked up by one of the frame processing threads (/src/frame_threads.c). One frame processing thread does the following: It grabs one frame from the frame_list and sets its state to FRAME_LOCK to make sure no other frame processing thread will take this frame. Now the frame enters the filters again
3) TC_PRE_M_PROCESS == TC_PRE_PROCESS
Next, the internal processing routines take place
4) process_vid_frame() (/src/video_trans.c)
The frame wanders off into the filters a third time
5) TC_POST_M_PROCESS == TC_POST_PROCESS
The frame processing thread now sets the status of the frame to FRAME_READY which means the frame can now be picked up by the encoder. The encoder does again some processing and calls the filters for the fourth and last time.
After this a internal post processing takes place
7) postprocess_vid_frame() (/src/frame_postprocess.c)
When the frame has the skip flag set it will not be encoded, all other frames will encoded and freed.
decode() | 1) preprocess() | 2) filters(TC_PRE_S_PROCESS) | _____________/ | \______________ / | | | \ / | | | \ N F R A M E P R O C E S S I N G T H R E A D S / . . . . | . . . . | 3) filters(TC_PRE_M_PROCESS) | 4) process_vid_frame() | 5) filters(TC_POST_M_PROCESS) | | . . . . \ . . . . \_________|______ | ______|_______/ \ | / | 6) filters(TC_POST_S_PROCESS) | 7) postprocess() | encode()
Slot | Abb. | Comment ---------------------+------+---------------------------------- TC_PRE_S_PROCESS | ES | synchronous pre processing TC_PRE_M_PROCESS | EM | multithreaded pre processing TC_POST_M_PROCESS | OM | multithreaded post processing TC_POST_S_PROCESS | OS | synchronous post processingEvery filter in every slot can clone frames but the filter may not be called again with the cloned frame again. This depends on the slot the filter lives in. Note that a filter can live in several slots, for example, it can set the clone flag at ES time and catch the cloned frame at OM time if it likes to do so. The filter must keep in mind, that the cloned frame may have different geometry at OM time because it may be rescaled or resampled.
How to read the following table. The field "Slot" denotes the slot the filter is in when it sets CLONE. The "Slots to pass" field tells the slots the cloned frame will pass.
Slot | Slots to pass | Slots NOT passed ------+---------------+----------------- ES | EM OM OS | ES EM | EM OM OS | ES OM | OS | ES EM OM OS | OS | ES EM OMExample 1: A filter in ES sets CLONE. The filter itself will not see the cloned frame, but all filters in following slots will.
Example 2: A filter in EM sets CLONE. The filter _will_ see the cloned frame again and all consecutive slots will get it again, too.
Notes on Cloning. To clone a frame, the filter adds TC_FRAME_IS_CLONED to the frame pointer attributes. The frame ID never gets incremented to reflect the count of cloned frames. If the filter decides to clone a frame, it gets the frame back with the same ID but with TC_FRAME_WAS_CLONED set, so its easy to distinguish between the frame with the original ID and the duplicated ID.
Slot | slot in which the frame will be dropped ------+---------------------------------------- ES | Before EM EM | After EM, before OM OM | After OM, before OS OS | After OS, before encodeIn other words, a skipped frame gets dropped right after the filter has returned.
However, its not always possible to put a filter in _M_. In _M_ slots the filter may recieve the frames in any order. So if the filter does depend on correct frame ordering it cannot be in _M_. It may be possible to rewrite the filter in a way so that it does not depend on the order of frames, check if it can be done for your filter.
As an example, a deinterlace should always be run as a PRE filter before the the up/down scaling of the frame happens in transcode-core. A denoiser can be either run in PRE or in POST because it just does not matter. One can assume that if the user wants deinterlacing, in POST you can be pretty sure that you work with progressive material.
Always keep in mind that a filter can be in multiple slots. It depends on the filter which tag it wants.