so I want to create a program in which I will compress a video using this method:

Save a frame for a video each second, and for the rest of the frames in that second, save only the changes from that main frame. And then combine all that info to create a smaller-sized video.

So my idea is to iterate through each frame, save that frame in an array, and when I reach the main frame, find the differences and rewrite all the other frames. However I'm not sure how I can create a video using differences and a main frame afterward... Should I use cv2.VideoWriter or that only works with frames that haven't been altered?

This is what I have for now (I haven't saved the frames yet because I'm not sure of the format that I need):

import cv2
import time

imcap = cv2.VideoCapture('test2.mp4')  # test 1
sample_rate = 30
success, img = imcap.read()  # get the next frame
frame_no = 0


fps = imcap.get(cv2.CAP_PROP_FPS)
print("Frames per second: {0}".format(fps))

while success:
    frame_no += 1
    if frame_no % sample_rate == 0:
        cv2.imshow('frame', img)
        cv2.waitKey()

    print(frame_no)
    # read next frame
    success, img = imcap.read()

imcap.release()