Hello All, I heard a BBC documentary on ABC Radio National, in the World Docos segment. I want the audio to pass on to others, and I would strongly recommend it to all here. I can get it to play, but not to save, yet. https://www.bbc.co.uk/programmes/m000jtmv There are five episodes, and the last is the kicker. I do have youtube-dl installed,and command runner firefox extension, but not yet sure how to go about. I would also appreciate examples to be able to use them to download from ABC iView, I want "saving Planet A". Regards, Mark Trickett
On Sun, Aug 16, 2020 at 07:48:57AM +1000, Mark Trickett wrote:
I heard a BBC documentary on ABC Radio National, in the World Docos segment. I want the audio to pass on to others, and I would strongly recommend it to all here. I can get it to play, but not to save, yet.
while there are lots of options with youtube-dl for selecting download format and quality, a basic download using that URL works well here. e.g. (btw, you can ignore the "HTTP Error 403: Forbidden" warnings - some BBC formats are geo-locked to the UK - for BBC Subscribers only, i guess) $ youtube-dl 'https://www.bbc.co.uk/programmes/m000jtmv' [bbc.co.uk] m000jtmv: Downloading video page [bbc.co.uk] m000jtmv: Downloading playlist JSON [bbc.co.uk] m000jtmt: Downloading media selection XML [bbc.co.uk] m000jtmt: Downloading MPD manifest [bbc.co.uk] m000jtmt: Downloading m3u8 information [bbc.co.uk] m000jtmt: Downloading m3u8 information WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden [bbc.co.uk] m000jtmt: Downloading MPD manifest WARNING: Failed to download MPD manifest: HTTP Error 403: Forbidden [bbc.co.uk] m000jtmt: Downloading m3u8 information WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden [bbc.co.uk] m000jtmt: Downloading m3u8 information WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden [bbc.co.uk] m000jtmt: Downloading MPD manifest [bbc.co.uk] m000jtmt: Downloading m3u8 information [bbc.co.uk] m000jtmt: Downloading m3u8 information WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden [bbc.co.uk] m000jtmt: Downloading MPD manifest WARNING: Failed to download MPD manifest: <urlopen error Tunnel connection failed: 403 Forbidden> [bbc.co.uk] m000jtmt: Downloading m3u8 information WARNING: Failed to download m3u8 information: <urlopen error Tunnel connection failed: 403 Forbidden> [bbc.co.uk] m000jtmt: Downloading m3u8 information WARNING: Failed to download m3u8 information: <urlopen error Tunnel connection failed: 403 Forbidden> [dashsegments] Total fragments: 131 [download] Destination: Watching Us, Week 1-m000jtmt.m4a [download] 100% of 9.60MiB in 01:58 [ffmpeg] Correcting container in "Watching Us, Week 1-m000jtmt.m4a" $ mediainfo /tmp/Watching\ Us\,\ Week\ 1-m000jtmt.m4a General Complete name : /tmp/Watching Us, Week 1-m000jtmt.m4a Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/mp41) File size : 9.59 MiB Duration : 13 min 51 s Overall bit rate mode : Constant Overall bit rate : 96.8 kb/s Writing application : Lavf58.45.100 Audio ID : 1 Format : AAC LC SBR Format/Info : Advanced Audio Codec Low Complexity with Spectral Band Replication Commercial name : HE-AAC Format settings : Explicit Codec ID : mp4a-40-2 Duration : 13 min 51 s Bit rate mode : Constant Bit rate : 96.0 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 23.438 FPS (2048 SPF) Compression mode : Lossy Stream size : 9.52 MiB (99%) Language : English Default : Yes Alternate group : 1 I tested the download with smplayer, sounds pretty good.
I do have youtube-dl installed,and command runner firefox extension,
i've never used youtube-dl with a firefox extension, only from the command line.
but not yet sure how to go about. I would also appreciate examples to be able to use them to download from ABC iView, I want "saving Planet A".
i'm going to assume you mean "Fight For Planet A: Our Climate Challenge" with Craig Reucassel from The Chaser, because that's the closest thing that that came up when I searched the iview site. ABC iview is a bit trickier. Finding the exact URL for a recording can be difficult. First, start with the main URL for the show: https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge Use that with `lynx -dump -listonly -nonumbers` and `grep` to find the actual download URL. $ lynx -dump -listonly -nonumbers 'https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge' | grep -i ^http.*/video/ https://www.facebook.com/sharer.php?u=https://iview.abc.net.au/show/fight-fo... https://twitter.com/share?url=https://iview.abc.net.au/show/fight-for-planet... For Planet A: Our Climate Challenge Series 1 Episode 1&via=ABCTV These URLs look like they're for sharing with facebook and twitter. We don't need or want that social media crap in the URL, so we need to strip it. You can do that manually with careful mouse copy-paste, or you can use sed or perl or similar tools. The facebook one looks like a simple one to clean up with sed (we only need to delete everything up to and including the "u="), so let's use that. There's only one instance of "u=" in the URL so greedy regex matches aren't going to be a problem so we can use sed: what's a greedy match vs non-greedy? greedy matches will match as many characters as possible. non-greedy will match as few as possible while still matching the regex pattern (in short: stop at the FIRST possible match). sed doesn't have a non-greedy regex modifier, perl does. $ lynx -dump -listonly -nonumbers 'https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge' | sed -n -e '/facebook.*video/ s/^.*u=//p' https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge/serie... or with perl for non-greedy match (note the ? after the .* - that makes it non-greedy): $ lynx -dump -listonly -nonumbers 'https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge' | perl -n -e 'if (m/facebook.*video/) { s/^.*?u=//; print }' https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge/serie... Anyway, we can now use that with youtube-dl: I'm not going to bother with format or quality selection, but I do like to download and embed the subtitles in videos downloaded from iview, so I'll use a few options to do that: (I'm also going to wrap the download URL in single-quotes. It's not necessary here because it doesn't contain any spaces or shell-metacharacters like ; or &, but it's a good habit to get into when working with URLs on the command line. Those characters are VERY common in URLs and *WILL* be acted on by the shell unless you wrap the URL in single-quotes) $ youtube-dl --write-sub --all-subs --embed-subs 'https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge/serie...' [abc.net.au:iview] DO1904H001S00: Downloading JSON metadata [abc.net.au:iview] DO1904H001S00: Downloading webpage [abc.net.au:iview] DO1904H001S00: Downloading m3u8 information [info] Writing video subtitles to: Series 1 Ep 1-DO1904H001S00.en.vtt [hlsnative] Downloading m3u8 manifest [hlsnative] Total fragments: 397 [download] Destination: Series 1 Ep 1-DO1904H001S00.mp4 [download] 100% of 1.01GiB in 01:45 [ffmpeg] Fixing malformed AAC bitstream in "Series 1 Ep 1-DO1904H001S00.mp4" [ffmpeg] Embedding subtitles in 'Series 1 Ep 1-DO1904H001S00.mp4' Deleting original file Series 1 Ep 1-DO1904H001S00.en.vtt (pass -k to keep) The output filename's a bit shit. You can rename it or read through youtube-dl's man page for the '-o' option and OUTPUT TEMPLATE formatting details. I'm not going to bother. I tested it with smplayer, and it's good. decent video and audio and subtitles. Here's what mediainfo has to say about the file: $ mediainfo Series\ 1\ Ep\ 1-DO1904H001S00.mp4 General Complete name : Series 1 Ep 1-DO1904H001S00.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/avc1/mp41) File size : 997 MiB Duration : 1 h 6 min Overall bit rate mode : Variable Overall bit rate : 2 110 kb/s Writing application : Lavf58.45.100 Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : High@L4 Format settings : CABAC / 4 Ref Frames Format settings, CABAC : Yes Format settings, Reference frames : 4 frames Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 1 h 6 min Bit rate : 1 973 kb/s Width : 1 280 pixels Height : 720 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 25.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.086 Stream size : 933 MiB (94%) Writing library : x264 core 160 Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=22 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=22.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=4372 / vbv_bufsize=4500 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00 Codec configuration box : avcC Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Codec ID : mp4a-40-2 Duration : 1 h 6 min Bit rate mode : Constant Bit rate : 128 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 44.1 kHz Frame rate : 43.066 FPS (1024 SPF) Compression mode : Lossy Stream size : 60.5 MiB (6%) Default : Yes Alternate group : 1 Text ID : 3 Format : Timed Text Muxing mode : sbtl Codec ID : tx3g Duration : 1 h 5 min Bit rate mode : Variable Bit rate : 131 b/s Stream size : 63.1 KiB (0%) Language : English Default : Yes Forced : No Alternate group : 3 The file size is nearly a gigabyte. You can use ffmpeg or handbrake or something to re-encode with x265 to get it smaller (i'd guess that transcoding the video to x265 would probably shrink the file by a third to a half. maybe more). If you're only going to watch it once and delete it, don't bother...but might be worthwhile if you want to archive it for repeat viewing. craig
On Mon, Aug 17, 2020 at 04:13:30PM +1000, Craig Sanders wrote:
The file size is nearly a gigabyte. You can use ffmpeg or handbrake or something to re-encode with x265 to get it smaller (i'd guess that transcoding the video to x265 would probably shrink the file by a third to a half. maybe more). If you're only going to watch it once and delete it, don't bother...but might be worthwhile if you want to archive it for repeat viewing.
I transcoded it with handbrake, using the Matroska H.265 720p30 pre-set, but it only shrunk to 807 MB. It took nearly an hour to transcode...and that's on my 16-core threadripper 1950x. a lot of CPU time and electricity used (and the fans ramped up to deal with the excess heat generated) for not much benefit. This would be significantly faster if my version of handbrake (the debian handbrake 1.3.1+ds1-2 package) supported NVENC for GPU hardware encoding, but it doesn't. It's probably a licensing conflict with GPL vs proprietary nvidia-licensed code that prevents debian from distributing it. I'd have to re-compile the package on my own machine to enable GPU-encoding. Apparently the Ubuntu PPA 1.3.3 version of handbrake has it enabled. Transcoding using the Matroska H.265 576p25 pre-set (i.e. not only transcoding to a better/newer encoder, but also reducing the resolution from 1280x720 to 720x576 - same aspect ratio of 16:9) in handrake reduced the file from 998 MB to 548 MB. There's some reduction in quality, but not much. This took about 40 minutes to transcode. not worth the bother unless you want to archive and are really short on disk space. and even then it would better to use youtube-dl's format and quality options to download a smaller, lower-qualty video in the first place....they've been transcoded from much higher quality master videos. e.g. $ youtube-dl -F https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge/serie... [abc.net.au:iview] DO1904H001S00: Downloading JSON metadata [abc.net.au:iview] DO1904H001S00: Downloading webpage [abc.net.au:iview] DO1904H001S00: Downloading m3u8 information [info] Available formats for DO1904H001S00: format code extension resolution note hls-287 mp4 320x180 287k , avc1.77.30, mp4a.40.2 hls-491 mp4 512x288 491k , avc1.77.30, mp4a.40.2 hls-619 mp4 640x360 619k , avc1.77.30, mp4a.40.2 hls-917 mp4 800x450 917k , avc1.640028, mp4a.40.2 hls-1307 mp4 1024x576 1307k , avc1.640028, mp4a.40.2 hls-1969 mp4 1280x720 1969k , avc1.640028, mp4a.40.2 hls-2099 mp4 1280x720 2099k , avc1.640028, mp4a.40.2 (best) I'm far from sure, but i'm guessing that the 287k, ... 2099k etc mentioned in the notes field (and also in the hls-* format codes) are the bandwith rates for streaming, from 287 Kilobytes per second to 2.1 Mbps. Anyway, you could download the 576p (1024x576) version with: $ youtube-dl -f hls-1307 --write-sub --all-subs --embed-subs https://iview.abc.net.au/show/fight-for-planet-a-our-climate-challenge/serie... [abc.net.au:iview] DO1904H001S00: Downloading JSON metadata [abc.net.au:iview] DO1904H001S00: Downloading webpage [abc.net.au:iview] DO1904H001S00: Downloading m3u8 information [info] Writing video subtitles to: Series 1 Ep 1-DO1904H001S00.en.vtt [hlsnative] Downloading m3u8 manifest [hlsnative] Total fragments: 397 [download] Destination: Series 1 Ep 1-DO1904H001S00.mp4 [download] 100% of 653.70MiB in 01:09 [ffmpeg] Fixing malformed AAC bitstream in "Series 1 Ep 1-DO1904H001S00.mp4" [ffmpeg] Embedding subtitles in 'Series 1 Ep 1-DO1904H001S00.mp4' Deleting original file Series 1 Ep 1-DO1904H001S00.en.vtt (pass -k to keep) It's larger than my transcoded version because it's h.264 rather than the newer/better h.265....but it only took a minute to download with my 100 Mbps NBN connection, much better than 40+ minutes to transcode. The hls-917 (800x450) version is even smaller (438 MB), and still looks OK to watch even when upscaled to fullscreen on my 2560x1440 27" monitor, or my 42" 1920x1080 TV. $ ls -lh Series\ 1\ Ep\ 1-DO1904H001S00*.mp4 -rw-r--r-- 1 cas cas 998M Aug 17 15:16 Series 1 Ep 1-DO1904H001S00-best.mp4 -rw-r--r-- 1 cas cas 623M Aug 18 14:05 Series 1 Ep 1-DO1904H001S00-hls-1307.mp4 -rw-r--r-- 1 cas cas 438M Aug 18 14:10 Series 1 Ep 1-DO1904H001S00-hls-917.mp4 craig -- craig sanders <cas@taz.net.au>
participants (2)
-
Craig Sanders -
Mark Trickett