1. The line width limitation on IPIPE module is 2176 pixels per line. As you need 2456 pixels per line for your 5MP resolution, you cannot use IPIPE/RSZ in single pass. You would have to do vertical slicing of the frame and then do the imagepipe/resize operation on each vertical slice.
2. RSZ can take input from DDR or directly from ISIF (parallel input port interface). But when directly taking input from ISIF, the RSZ runs on input pixel clock and this more is calleed continuous mode. In this mode, generally, it is not possible to break the resize operation during the vertical blanking duration.
3. For your usecase, you need to run RSZ in DDR input mode as you need three different resolutions and you have only two resizer ports (RSZA and RSZB) to generate the YUV data.
So my suggestion for your data flow is as follows: