MP4 to Streaming Video Conversion

TL;DR / [Geek Summary]:
Protocol Transcoding: Use FFmpeg to slice standalone MP4s into HLS (M3U8+TS) streams for smooth web delivery.
Keyframe Tuning: Force I-frame injection via -force_key_frames to fix slice misalignment and player stuttering.
Full-Stack Logic: Covers advanced CLI tricks plus multi-language (PHP/Python/Go/Rust) snippets for merging TS slices back to MP4.

FFmpeg is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams. It provides software for recording, converting and streaming audio and video.

Download from official website http://www.ffmpeg.org/download.html and configure the environment after decompression

# Convert MP4 to M3U8

M3U8 is the Unicode version of M3U, encoded in UTF-8. Both “M3U” and “M3U8” files are the basis of the HTTP Live Streaming (HLS) protocol format used by Apple, which can be played on devices such as iPhone and Macbook.

Simply put, m3u8 is a video format that divides a video into many small parts, which makes it easier to load the video.

# Simple operation, low efficiency

1
ffmpeg -i input.mp4 -c:v libx264 -c:a aac -strict -2 -f hls -hls_list_size 2 -hls_time 15 output.m3u8

The generated effect is:

Generate a ts file every 15 seconds for the input.mp4 video file, and finally generate an m3u8 file. The m3u8 file is the index file of ts.

We can directly open the m3u8 file with playback software such as VLC media player, just like playing mp4.

The default length of each piece is 2 seconds. The m3u8 file only saves the information of the latest 5 pieces by default, resulting in only the last small part of the last playback (pay special attention when live streaming). -hls_time n sets the length of each piece. The default value is 2, in seconds. -hls_list_size n Set the maximum number of entries saved in the playlist. If set to 0, all the information of the film will be saved. The default value is 5. -hls_wrap n Set the number of films to start overwriting. If set to 0, no overwriting will be done. The default value is 0. This option can avoid storing too many films on the disk and limit the maximum number of films written to the disk. -hls_start_number n Set the value of sequence number in the playlist to number. The default value is 0. Note: The sequence number of the playlist must be unique for each segment, and it cannot be confused with the file name of the film (when using the wrap option, the file name may be reused).
For more parameters, please see the document: ffmpeg.org/ffmpeg.html#Video-Options

# Efficiency optimization, improve efficiency

TS file is an extension of a media, which is a packaging format for Japanese high-definition cameras. MPEG2-TS (Transport Stream; also known as TS, TP, MPEG-TS or M2T) is a communication protocol for audio, image and data, and was first used in real-time transmission of DVD programs. The characteristic of the MPEG2-TS format is that it requires that any segment of the video stream can be independently decoded.

1
2
3
4
# 1. Video overall transcoding ts
ffmpeg -y -i music.mp4 -vcodec copy -acodec copy -vbsf h264_mp4toannexb out\music.ts
# 2. .ts file slicing
ffmpeg -i music.ts -c copy -map 0 -f segment -segment_list out\music.m3u8 -segment_time 10 out\15s_%3d.ts

# hls_time slicing time inaccuracy

To play the ts slice of m3u8, you must download a ts slice completely before you can play it. Set the hls_time time interval as short as possible (depending on the actual situation). In the actual process, set the slicing time interval to 2 seconds and call the following command:

1
ffmpeg -i test.mp4 -c:v libx264 -c:a aac -strict -2 -f hls -hls_time 2 index.m3u8

But the slice is not performed according to the input parameters.

Reason:

The cutting of ts file is also related to the GOP size of the original file video (that is, the time interval between two I frames), because the first frame of any ts segment must be an I frame, otherwise it cannot be played at the fastest speed, and the first frame is not an I frame, which is meaningless to the player and is directly thrown away by the player. Any video stream must obtain the first I frame to successfully decode the picture. Although it is specified to cut a ts file in 1 second, in fact, since the original video stream may have an I frame for several seconds, it must wait until the next I frame to restart the slicing.

Solution:

Since it is known that a ts segment is generated in 1 second, it is necessary to force a key frame to be generated in one second during the slicing process.

Set the key frame interval, and set the interval to 2 seconds as follows: -force_key_frames "expr:gte(t,n_forced*2)“

Complete command:

1
ffmpeg -i test.mp4 -force_key_frames "expr:gte(t,n_forced*2)" -strict -2 -c:a aac -c:v libx264 -hls_time 2 -f hls index.m3u8

# m3u8 format analysis

The complete m3u8 file has three parts:

index.m3u8, saves the basic information of the video and the order of the segmented files;
key, if the video is encrypted, save the key;
data file, the others are video data files.

Specific content analysis:

#EXTM3U, is the beginning of the file
#EXT-X-VERSION, identifies the HLS protocol version number;
#EXT-X-TARGETDURATION, indicates the maximum duration of each video segment (in seconds);
#EXT-X-MEDIA-SEQUENCE, indicates the sequence number of the first URL fragment file in the playlist;
#EXT-X-PLAYLIST-TYPE, indicates the streaming media type;
#EXT-X-KEY, encryption method, here the encryption method is AES-128, and IV is specified, which is required for decryption;
#EXTINF, indicates the duration of the media fragment specified by the subsequent URL (in seconds).

# HLS

The working principle of HLS is to divide the entire stream into small HTTP-based files for download, and only download some at a time.
While a media stream is playing, the client can choose to download the same resource from many different alternative sources at different rates, allowing the streaming session to adapt to different data rates.
When starting a streaming session, the client downloads an extended M3U (m3u8) playlist file containing metadata for finding available media streams.
HLS only requests basic HTTP messages, and unlike the Real-time Transport Protocol (RTP), HLS can pass through any firewall or proxy server that allows HTTP data to pass through.
It is also easy to use a content distribution network to transmit media streams.

# Merge m3u8(ts) into MP4

# Remote file

1
ffmpeg -i “https://xpdbk.com/demo/ffmpeg/hls265/output.m3u8” -vcodec copy -acodec copy -absf aac_adtstoasc output.mp4

# Local file

Open cmd
Enter the command and merge according to the actual path of the file

Merge into ts file copy /b F:\f\*.ts E:\f\new.ts

Merge into MP4 file copy /b F:\f\*.ts E:\f\new.MP4

And through ffmpeg command as follows:

1
2
3
4
5
6
7
8
Direct conversion:  
ffmpeg -i new.ts -c copy -map 0:v -map 0:a output.mp4

Specify audio stream (usually use this):  
ffmpeg -i new.ts -c copy -map 0:v -map 0:a -bsf:a aac_adtstoasc output.mp4

Re-encode video:  
ffmpeg -y -i new.ts -c:v libx264 -c:a copy -bsf:a aac_adtstoasc output.mp4

# PHP implementation code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$url = 'https://******.m3u8?Expires=15853412145&OSSAccessKeyId=******&Signature=******';

$ts_content = file_get_contents($url);
$ts_content = explode(',', $ts_content);
$ts_file = array();

foreach ($ts_content as $key => $value) {
    if($key == 0) continue;
    $value = trim($value);
    $ts_file[] = substr($value, 0, strpos($value, '.ts') + 3);
}

$url_prefix = substr($url, 0, strpos($url, '.m3u8'));
$url_prefix = substr($url, 0, strrpos($url, '/') + 1);
$file_content = '';

foreach ($ts_file as $key => $value) {
    $file_content .= file_get_contents($url_prefix . $value);
}

file_put_contents('tmp_out.ts', $file_content);

// FFMPEG_PATH is the bin path to decompress ffmpeg yourself, for example, mine is F:/ffmpeg/bin/
exec(FFMPEG_PATH . "ffmpeg -i tmp_out.ts tmp_out.mp4");

# Python implementation code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
import os
import sys
import time
from Crypto.Cipher import AES

def fileList(findex):
    rpath = os.path.dirname(os.path.realpath(findex))
    name = rpath.split("\\")[-1]
    fi = open(findex, 'r')
    flag = False
    IV = None
    tl = []
    for line in fi.readlines():
        if line.startswith("#EXT-X-KEY"):
            # If IV exists, extract it;
            if line.split(",")[-1].startswith("IV="):
                IV = line.split(",")[-1][5:]
                IV = bytes.fromhex(IV)
        if line.startswith("#EXTINF"):
            flag = not flag
            continue
        if flag:
            tmp = line.strip().split("/")[-1]
            tmp = os.path.join(rpath, tmp)
            tl.append(tmp)
            flag = not flag
    fi.close()
    fk = open(os.path.join(rpath, "key"), 'rb')
    key = fk.read()
    fk.close()
    return name, tl, key, IV

def aes_decode(data, key, IV):
    # If no IV value is specified, use the key value directly
    if not IV:
        IV = key
    cryptor = AES.new(key, AES.MODE_CBC, IV)
    plain_text = cryptor.decrypt(data)
    return plain_text

def main():
    fp = os.listdir()
    used = [s[:-4] for s in os.listdir("./result/")]
    for ind in fp:
        if not ind.isdigit():
            continue
        if ind in used:
            continue
        try: 
            name, fl, key, IV = fileList(os.path.join(ind, "index.m3u8"))
        except:
            print("-"*30)
            print("[-] Error! file: ", ind)
            print("-"*30)
            continue
        print("[*] Begin process file: ", name)
        start = time.time()
        f = open(os.path.join("./result/", name+".mp4"), 'ab')
        for i in fl:
            with open(i, 'rb') as inf:
                data = inf.read()
                f.write(aes_decode(data, key, IV))
        f.close()
        print("[+] Successfully! Cost time: ", time.time()-start)

main()

# Golang implementation code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
package main

import (
	"io/ioutil"
	"net/http"
	"os/exec"
	"strings"
)

func main() {
	url := "https://******.m3u8?Expires=15853412145&OSSAccessKeyId=******&Signature=******"

	resp, _ := http.Get(url)
	body, _ := ioutil.ReadAll(resp.Body)
	tsContent := strings.Split(string(body), ",")
	var tsFile []string

	for key, value := range tsContent {
		if key == 0 {
			continue
		}
		value = strings.TrimSpace(value)
		tsFile = append(tsFile, value[:strings.Index(value, ".ts")+3])
	}

	urlPrefix := url[:strings.LastIndex(url, "/")+1]
	var fileContent string

	for _, value := range tsFile {
		resp, _ := http.Get(urlPrefix + value)
		body, _ := ioutil.ReadAll(resp.Body)
		fileContent += string(body)
	}

	ioutil.WriteFile("tmp_out.ts", []byte(fileContent), 0644)

	cmd := exec.Command("ffmpeg", "-i", "tmp_out.ts", "tmp_out.mp4")
	cmd.Run()
}

# Rust implementation code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
use std::fs::File;
use std::io::prelude::*;
use std::process::Command;
use reqwest;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let url = "https://******.m3u8?Expires=15853412145&OSSAccessKeyId=******&Signature=******";
    let resp = reqwest::get(url).await?.text().await?;
    let ts_content: Vec<&str> = resp.split(',').collect();
    let mut ts_file = vec![];

    for (i, line) in ts_content.iter().enumerate() {
        if i == 0 { continue; }
        let value = line.trim();
        ts_file.push(&value[0..value.find(".ts").unwrap()+3]);
    }

    let url_prefix = &url[0..url.rfind('/').unwrap()+1];
    let mut file_content = vec![];

    for file in ts_file {
        let resp = reqwest::get(&(url_prefix.to_string() + file)).await?.bytes().await?;
        file_content.extend_from_slice(&resp);
    }

    let mut file = File::create("tmp_out.ts")?;
    file.write_all(&file_content)?;

    Command::new("ffmpeg")
        .arg("-i")
        .arg("tmp_out.ts")
        .arg("tmp_out.mp4")
        .output()
        .expect("Failed to execute command");

    Ok(())
}

# C++ implementation code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <curl/curl.h>

size_t WriteCallback(void* contents, size_t size, size_t nmemb, std::string* s) {
    size_t newLength = size*nmemb;
    size_t oldLength = s->size();
    try {
        s->resize(oldLength + newLength);
    }
    catch(std::bad_alloc &e) {
        return 0;
    }

    std::copy((char*)contents,(char*)contents+newLength,s->begin()+oldLength);
    return size*nmemb;
}

std::string GetFile(const std::string& url) {
    CURL* curl;
    CURLcode res;
    std::string readBuffer;

    curl_global_init(CURL_GLOBAL_DEFAULT);
    curl = curl_easy_init();

    if(curl) {
        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
        res = curl_easy_perform(curl);

        if(res != CURLE_OK)
            fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));

        curl_easy_cleanup(curl);
    }

    curl_global_cleanup();
    return readBuffer;
}

int main() {
    std::string url = "https://******.m3u8?Expires=15853412145&OSSAccessKeyId=******&Signature=******";
    std::string ts_content = GetFile(url);

    std::vector<std::string> ts_file;
    std::string delimiter = ",";
    size_t pos = 0;
    std::string token;

    while ((pos = ts_content.find(delimiter)) != std::string::npos) {
        token = ts_content.substr(0, pos);
        ts_file.push_back(token.substr(0, token.find(".ts") + 3));
        ts_content.erase(0, pos + delimiter.length());
    }

    std::string url_prefix = url.substr(0, url.rfind('/') + 1);
    std::ofstream outfile ("tmp_out.ts",std::ofstream::binary);

    for (auto &value : ts_file) {
        std::string file_content = GetFile(url_prefix + value);
        outfile.write(file_content.c_str(), file_content.size());
    }

    outfile.close();

    system("ffmpeg -i tmp_out.ts tmp_out.mp4");
}

# NASM implementation code

1
I don't know

# NODEJS implementation code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
const https = require('https');
const fs = require('fs');
const { exec } = require('child_process');

const url = 'https://******.m3u8?Expires=15853412145&OSSAccessKeyId=******&Signature=******';

https.get(url, (res) => {
    let tsContent = '';

    res.on('data', (chunk) => {
        tsContent += chunk;
    });

    res.on('end', () => {
        const tsFile = tsContent.split(',').slice(1).map(value => value.trim().split('.ts')[0] + '.ts');
        const urlPrefix = url.split('.m3u8')[0];

        let fileContent = '';

        tsFile.forEach((value, i) => {
            https.get(urlPrefix + value, (res) => {
                let data = '';

                res.on('data', (chunk) => {
                    data += chunk;
                });

                res.on('end', () => {
                    fileContent += data;

                    if (i === tsFile.length - 1) {
                        fs.writeFile('tmp_out.ts', fileContent, (err) => {
                            if (err) throw err;

                            exec('ffmpeg -i tmp_out.ts tmp_out.mp4', (err, stdout, stderr) => {
                                if (err) throw err;
                            });
                        });
                    }
                });
            });
        });
    });
});

# Kotlin implementation code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import java.io.*;
import java.net.URL;
import java.nio.file.*;
import java.util.stream.*;

public class Main {
    public static void main(String[] args) throws IOException {
        String url = "https://******.m3u8?Expires=15853412145&OSSAccessKeyId=******&Signature=******";

        String tsContent = new String(Files.readAllBytes(Paths.get(url)));
        String[] tsFile = tsContent.split(",");

        String urlPrefix = url.substring(0, url.lastIndexOf('/') + 1);
        StringBuilder fileContent = new StringBuilder();

        for (int i = 1; i < tsFile.length; i++) {
            String value = tsFile[i].trim();
            fileContent.append(new String(Files.readAllBytes(Paths.get(urlPrefix + value.substring(0, value.indexOf(".ts") + 3)))));
        }

        Files.write(Paths.get("tmp_out.ts"), fileContent.toString().getBytes());

        try {
            ProcessBuilder pb = new ProcessBuilder("ffmpeg", "-i", "tmp_out.ts", "tmp_out.mp4");
            Process p = pb.start();
            p.waitFor();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}