brooke/hls-test
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

whee

Browse source
This commit is contained in:
Brooke Vibber 2023-04-05 12:05:14 -07:00
parent bd290cc9a5
commit a3e7cd59c1
3 changed files with 157 additions and 63 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

109
fmp4-alt.html
View file

@ -17,64 +17,79 @@
<h3>With fallbacks</h3>
<p>HLS with VP9 (.mp4)/MJPEG (.mov) video tracks and Opus/MP3 audio tracks. Custom MSE wrapper enabled to provide HLS-over-MSE for Mac Safari/Chrome/Firefox.</p>
<p>HLS with VP9 (.mp4)/MJPEG (.mov) video tracks and Opus/MP3 audio tracks. Video.js enabled to provide HLS-over-MSE for Chrome/Firefox/desktop Safari.</p>
<div>
<video controls width=640 height=360>
<source type=application/vnd.apple.mpegurl src=fmp4.vp9-mjpeg.mov.m3u8>
</video>
</div>
<p>Current behavior:</p>
<p>Browsers that play the VP9 track will get sharp video, those that play the MJPEG track will get blurry video. Audio should sound the same either way.</p>
<ul>
<li>MSE-based streaming with VHS
<ul>
<li>Firefox seems to work with VP9 & Opus tracks via video.js</li>
<li>Chrome works (needed to fix an output setting)</li>
<li>Safari uses the MP3 audio tracks and VHS gets confused because it tries to parse them as MP4 (not Safari's fault)</li>
</ul>
</li>
<li>Apple HLS player
<ul>
<li>macOS 13's Safari 16 plays MJPEG on most of my test machines, though macOS 12 ~2015 systems with Intel graphics require lying about the type.</li>
<li>iOS 16 plays VP9 if supported, or MJPEG if no hardware codec</li>
<li>Those last two will also play h.263 or MPEG-4 visual <a href="fmp4-lies.html">IF labeled as if h.264 in the playlist</a>; MJPEG can be properly labeled as "jpeg". I haven't found a supported labeling that is correct yet.</li>
<li><i>no access to iOS 13-15</i></li>
<li>iOS 13 doesn't seem to like mjpeg in .mp4, but .mov is fine</li>
<li>iOS 12 doesn't seem to like any version on an old iPad Air, except with h264 video</li>
<li><i>no access to iOS 11</i></li>
<li>iOS 10 on iPhone 5C plays h.263, or mpeg-4 visual IF labeled as false avc1.blah. It will also play mjpeg if so mislabeled, but only in .mov not in .mp4 as above.</li>
<li>iOS 9 doesn't understand the required version of HLS playlist format, and fails.</li>
</ul>
</li>
</ul>
<!--<script src="node_modules/video.js/dist/video.js"></script>-->
<script src="video.js/dist/alt/video.core.js"></script>
<script src="http-streaming/dist/videojs-http-streaming.js"></script>
<script type="text/javascript">
if (typeof MediaSource !== 'undefined') {
async function process(video) {
let vp9 = 'video/mp4; codecs="vp09.00.41.08"';
let opus = 'audio/mp4; codecs="opus"';
let mp3 = 'audio/mpeg';
// Temporary hack, just load the full tracks
let videoTracks = [
{ type: vp9, src: 'fmp4.480p.vp9.mp4' },
];
let audioTracks = [
{ type: opus, src: 'fmp4.audio.opus.mp4' },
{ type: mp3, src: 'fmp4.audio.mpeg.mp3' },
];
let videoTrack = videoTracks.filter(({type}) => MediaSource.isTypeSupported(type))[0];
let audioTrack = audioTracks.filter(({type}) => MediaSource.isTypeSupported(type))[0];
if (!videoTrack || !audioTrack) {
throw new Error('error no tracks');
let playerConfig = {
responsive: true,
controlBar: {
volumePanel: {
vertical: true,
inline: false
}
console.log(videoTrack);
console.log(audioTrack);
let videoBytes = await (await fetch(videoTrack.src)).arrayBuffer();
// hackhack
videoBytes = videoBytes.slice(0, 878148 + 779);
let audioBytes = await (await fetch(audioTrack.src)).arrayBuffer();
let source = new MediaSource();
source.addEventListener("sourceopen", (event) => {
source.duration = 150; // hack
let videoBuffer = source.addSourceBuffer(videoTrack.type);
let audioBuffer = source.addSourceBuffer(audioTrack.type);
videoBuffer.appendBuffer(videoBytes);
audioBuffer.appendBuffer(audioBytes);
console.log('appended.');
});
video.addEventListener('error', (event) => {
console.log('video error?', video.error);
});
video.src = URL.createObjectURL(source);
console.log('opening...');
}
},
html5: {
vhs: {
// Currently the MP3 audio track fails in Safari
// and it doesn't grok the Opus
// Either fix MP3 handling in vhs or use AAC.
overrideNative: true
}
},
};
videojs.log.level('debug');
console.log(typeof MediaSource)
if (typeof MediaSource !== 'undefined') {
//let vp9 = MediaSource.isTypeSupported('video/mp4; codecs="vp09.00.41.08"');
//let opus = MediaSource.isTypeSupported('video/mp4; codecs="opus"');
//let mp3 = MediaSource.isTypeSupported('audio/mpeg');
//if (vp9 && (opus || mp3)) {
for (let video of document.querySelectorAll('video')) {
process(video);
video.classList.add('video-js');
video.classList.add('vjs-default-skin');
videojs(video, playerConfig);
}
// }
}
</script>
</body>
</html>
</html>

102
index.html
View file

@ -15,18 +15,29 @@
<li><a href="fmp4-tracks.html">see component track list</a></li>
</ul>
<h3>With fallbacks</h3>
<h3>HLS only:</h3>
<p>HLS with VP9 (.mp4)/MJPEG (.mov) video tracks and Opus/MP3 audio tracks. Video.js enabled to provide HLS-over-MSE for Chrome/Firefox.</p>
<p>HLS with correctly-labeled VP9 (.mp4) and MJPEG (.mov) video tracks and Opus (.mp4, for Chrome/Firefox) and MP3 (.mp3, used by Safari) audio tracks. Video.js is enabled but will not yet override the native HLS in Safari Desktop.</p>
<div>
<video controls width=640 height=360>
<source type=application/vnd.apple.mpegurl src=fmp4.vp9-mjpeg.mov.m3u8>
</video>
</div>
<h3>WebM or HLS:</h3>
<p>WebM VP9/Opus, WebM VP8/Vorbis, or HLS. Current versions of desktop Safari should see one or the other WebM, and those that fall back to the HLS may or may not work.</p>
<div>
<video controls width=640 height=360>
<source type="video/webm; codecs=&quot;vp9, opus&quot;" src=flat.480p.vp9-opus.webm>
<source type="video/webm; codecs=&quot;vp8, vorbis&quot;" src=flat.480p.vp8-vorbis.webm>
<source type=application/vnd.apple.mpegurl src=fmp4.vp9-mjpeg.mov.m3u8>
</video>
</div>
<p>Current behavior:</p>
<p>Browsers that play the VP9 track will get sharp video, those that play the MJPEG track will get blurry video. Audio should sound the same either way.</p>
<p>Browsers that play the VP9 track or one of the WebM files will get sharp video, those that play the MJPEG track will get blurry video. Audio should sound the same either way.</p>
<ul>
<li>MSE-based streaming with VHS
<ul>
@ -36,17 +47,78 @@
</ul>
</li>
<li>Apple HLS player
<ul>
<li>macOS 13's Safari 16 plays MJPEG on most of my test machines, except a 2015 MacBook Pro with Intel graphics unless the type is lied about.</li>
<li>iOS 16 plays VP9 if supported, or MJPEG if no hardware codec</li>
<li>Those last two will also play h.263 or MPEG-4 visual <a href="fmp4-lies.html">IF labeled as if h.264 in the playlist</a>; MJPEG can be properly labeled as "jpeg". I haven't found a supported labeling that is correct yet.</li>
<li><i>no access to iOS 13-15</i></li>
<li>iOS 13 doesn't seem to like mjpeg in .mp4, but .mov is fine</li>
<li>iOS 12 doesn't seem to like any version on an old iPad Air, except with h264 video</li>
<li><i>no access to iOS 11</i></li>
<li>iOS 10 on iPhone 5C plays h.263, or mpeg-4 visual IF labeled as false avc1.blah. It will also play mjpeg if so mislabeled, but only in .mov not in .mp4 as above.</li>
<li>iOS 9 doesn't understand the required version of HLS playlist format, and fails.</li>
</ul>
<table border>
<tr>
<th>platform</th>
<td>vp09 in .mp4</td>
<td>mp4v in .mp4</td>
<td>s263 in .3gp</td>
<td>jpeg in .mov</td>
<td>jpeg in .mp4</td>
</tr>
<tr>
<td>
macOS 13 / Safari 16.4<br>
on 2019 Intel MBP, 2020 M1 MBA
</td>
<td></td>
<td>if lie</td>
<td>if lie</td>
<td>works</td>
<td>works in 16.4 only?</td>
</tr>
<tr>
<td>
macOS 12 / Safari 16.4<br>
on 2015 Intel MBP, 2015 Intel MBA
</td>
<td></td>
<td>if lie</td>
<td>if lie</td>
<td>if lie</td>
<td></td>
</tr>
<tr>
<td>
iOS 13+ on A12+
</td>
<td>works</td>
<td>works</td>
<td>if lie</td>
<td>if lie</td>
<td>works in 16.4 only?</td>
</tr>
<tr>
<td>
iOS 13+ on A11+
</td>
<td></td>
<td>works</td>
<td>if lie</td>
<td>if lie</td>
<td>works in 16.4 only?</td>
</tr>
<tr>
<td>
iOS 12 on A7
</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>
iOS 10 on A6
</td>
<td></td>
<td>if lie</td>
<td>if lie</td>
<td>if lie</td>
<td></td>
</tr>
</table>
</li>
</ul>
@ -94,4 +166,4 @@
}
</script>
</body>
</html>
</html>

9
make-fmp4.sh
View file

@ -13,8 +13,10 @@ VIDEO_MPEG4="-vcodec mpeg4 -g 240 $BITRATE_HI $SIZE_MAIN"
VIDEO_H263="-vcodec h263 -g 240 $BITRATE_LO $SIZE_H263"
VIDEO_MJPEG="-vcodec mjpeg $BITRATE_HI $SIZE_SMALL"
VIDEO_VP9="-vcodec libvpx-vp9 -tile-columns 2 -row-mt 1 -cpu-used 3 -g 240 $BITRATE_LO $SIZE_MAIN"
VIDEO_VP8="-vcodec libvpx -slices 4 -cpu-used 3 -g 240 $BITRATE_HI $SIZE_MAIN"
AUDIO_OPUS="-acodec libopus -ac 2 -ar 48000 -ab 96k"
AUDIO_VORBIS="-acodec libvorbis -ac 2 -ar 48000 -ab 128k"
AUDIO_AAC="-ac 2 -ar 48000 -ab 128k"
AUDIO_MP3="-acodec libmp3lame -ac 2 -ar 48000 -ab 128k"
@ -49,10 +51,15 @@ ffmpeg -i $INFILE -vn $AUDIO_MP3 -y fmp4.audio.mpeg.mp3
#ffmpeg -i $INFILE -an $VIDEO_MJPEG $AUDFLAGS -y fmp4.120p.mjpeg.mp4
#ffmpeg -i $INFILE -an $VIDEO_MJPEG $AUDFLAGS -y fmp4.120p.mjpeg.mov
# Flat fallbacks
#ffmpeg -i $INFILE $AUDIO_OPUS $VIDEO_VP9 -pass 1 -y flat.480p.vp9-opus.webm
#ffmpeg -i $INFILE $AUDIO_OPUS $VIDEO_VP9 -pass 2 -y flat.480p.vp9-opus.webm
#ffmpeg -i $INFILE $AUDIO_VORBIS $VIDEO_VP8 -pass 1 -y flat.480p.vp8-vorbis.webm
#ffmpeg -i $INFILE $AUDIO_VORBIS $VIDEO_VP8 -pass 2 -y flat.480p.vp8-vorbis.webm
# Playlist processing
php HLS/rewrite-mp3.php fmp4.audio.mpeg.mp3 > fmp4.audio.mpeg.mp3.m3u8
#php extract-playlist.php fmp4.audio.mpeg.mp3 > fmp4.audio.mpeg.mp3.m3u8
php extract-playlist.php fmp4.audio.mpeg.mp4 > fmp4.audio.mpeg.mp4.m3u8
php extract-playlist.php fmp4.audio.mpeg.mov > fmp4.audio.mpeg.mov.m3u8
php extract-playlist.php fmp4.audio.aac.mp4 > fmp4.audio.aac.mp4.m3u8