Compare commits
1 Commits
2019.04.01
...
master-ytd
| Author | SHA1 | Date | |
|---|---|---|---|
|
c03f08929a |
6
.github/ISSUE_TEMPLATE.md
vendored
6
.github/ISSUE_TEMPLATE.md
vendored
@@ -6,8 +6,8 @@
|
||||
|
||||
---
|
||||
|
||||
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2019.04.01*. If it's not, read [this FAQ entry](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
|
||||
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2019.04.01**
|
||||
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2019.03.09*. If it's not, read [this FAQ entry](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
|
||||
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2019.03.09**
|
||||
|
||||
### Before submitting an *issue* make sure you have:
|
||||
- [ ] At least skimmed through the [README](https://github.com/ytdl-org/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/ytdl-org/youtube-dl#faq) and [BUGS](https://github.com/ytdl-org/youtube-dl#bugs) sections
|
||||
@@ -36,7 +36,7 @@ Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl
|
||||
[debug] User config: []
|
||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||
[debug] youtube-dl version 2019.04.01
|
||||
[debug] youtube-dl version 2019.03.09
|
||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||
[debug] Proxy map: {}
|
||||
|
||||
54
ChangeLog
54
ChangeLog
@@ -1,57 +1,3 @@
|
||||
version 2019.04.01
|
||||
|
||||
Core
|
||||
* [utils] Improve int_or_none and float_or_none (#20403)
|
||||
* Check for valid --min-sleep-interval when --max-sleep-interval is specified
|
||||
(#20435)
|
||||
|
||||
Extractors
|
||||
+ [weibo] Extend URL regular expression (#20496)
|
||||
+ [xhamster] Add support for xhamster.one (#20508)
|
||||
+ [mediasite] Add support for catalogs (#20507)
|
||||
+ [teamtreehouse] Add support for teamtreehouse.com (#9836)
|
||||
+ [ina] Add support for audio URLs
|
||||
* [ina] Improve extraction
|
||||
* [cwtv] Fix episode number extraction (#20461)
|
||||
* [npo] Improve DRM detection
|
||||
+ [pornhub] Add support for DASH formats (#20403)
|
||||
* [svtplay] Update API endpoint (#20430)
|
||||
|
||||
|
||||
version 2019.03.18
|
||||
|
||||
Core
|
||||
* [extractor/common] Improve HTML5 entries extraction
|
||||
+ [utils] Introduce parse_bitrate
|
||||
* [update] Hide update URLs behind redirect
|
||||
* [extractor/common] Fix url meta field for unfragmented DASH formats (#20346)
|
||||
|
||||
Extractors
|
||||
+ [yandexvideo] Add extractor
|
||||
* [openload] Improve embed detection
|
||||
+ [corus] Add support for bigbrothercanada.ca (#20357)
|
||||
+ [orf:radio] Extract series (#20012)
|
||||
+ [cbc:watch] Add support for gem.cbc.ca (#20251, #20359)
|
||||
- [anysex] Remove extractor (#19279)
|
||||
+ [ciscolive] Add support for new URL schema (#20320, #20351)
|
||||
+ [youtube] Add support for invidiou.sh (#20309)
|
||||
- [anitube] Remove extractor (#20334)
|
||||
- [ruleporn] Remove extractor (#15344, #20324)
|
||||
* [npr] Fix extraction (#10793, #13440)
|
||||
* [biqle] Fix extraction (#11471, #15313)
|
||||
* [viddler] Modernize
|
||||
* [moevideo] Fix extraction
|
||||
* [primesharetv] Remove extractor
|
||||
* [hypem] Modernize and extract more metadata (#15320)
|
||||
* [veoh] Fix extraction
|
||||
* [escapist] Modernize
|
||||
- [videomega] Remove extractor (#10108)
|
||||
+ [beeg] Add support for beeg.porn (#20306)
|
||||
* [vimeo:review] Improve config url extraction and extract original format
|
||||
(#20305)
|
||||
* [fox] Detect geo restriction and authentication errors (#20208)
|
||||
|
||||
|
||||
version 2019.03.09
|
||||
|
||||
Core
|
||||
|
||||
@@ -44,7 +44,9 @@
|
||||
- **AmericasTestKitchen**
|
||||
- **anderetijden**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
|
||||
- **AnimeOnDemand**
|
||||
- **anitube.se**
|
||||
- **Anvato**
|
||||
- **AnySex**
|
||||
- **APA**
|
||||
- **Aparat**
|
||||
- **AppleConnect**
|
||||
@@ -488,7 +490,6 @@
|
||||
- **Medialaan**
|
||||
- **Mediaset**
|
||||
- **Mediasite**
|
||||
- **MediasiteCatalog**
|
||||
- **Medici**
|
||||
- **megaphone.fm**: megaphone.fm embedded players
|
||||
- **Meipai**: 美拍
|
||||
@@ -697,6 +698,7 @@
|
||||
- **PornoXO**
|
||||
- **PornTube**
|
||||
- **PressTV**
|
||||
- **PrimeShareTV**
|
||||
- **PromptFile**
|
||||
- **prosiebensat1**: ProSiebenSat.1 Digital
|
||||
- **puhutv**
|
||||
@@ -716,7 +718,7 @@
|
||||
- **radio.de**
|
||||
- **radiobremen**
|
||||
- **radiocanada**
|
||||
- **radiocanada:audiovideo**
|
||||
- **RadioCanadaAudioVideo**
|
||||
- **radiofrance**
|
||||
- **RadioJavan**
|
||||
- **Rai**
|
||||
@@ -763,6 +765,7 @@
|
||||
- **RTVS**
|
||||
- **Rudo**
|
||||
- **RUHD**
|
||||
- **RulePorn**
|
||||
- **rutube**: Rutube videos
|
||||
- **rutube:channel**: Rutube channels
|
||||
- **rutube:embed**: Rutube embedded videos
|
||||
@@ -870,7 +873,6 @@
|
||||
- **teachertube:user:collection**: teachertube.com user and collection videos
|
||||
- **TeachingChannel**
|
||||
- **Teamcoco**
|
||||
- **TeamTreeHouse**
|
||||
- **TechTalks**
|
||||
- **techtv.mit.edu**
|
||||
- **ted**
|
||||
@@ -1008,6 +1010,7 @@
|
||||
- **video.mit.edu**
|
||||
- **VideoDetective**
|
||||
- **videofy.me**
|
||||
- **VideoMega**
|
||||
- **videomore**
|
||||
- **videomore:season**
|
||||
- **videomore:video**
|
||||
@@ -1124,7 +1127,6 @@
|
||||
- **yandexmusic:album**: Яндекс.Музыка - Альбом
|
||||
- **yandexmusic:playlist**: Яндекс.Музыка - Плейлист
|
||||
- **yandexmusic:track**: Яндекс.Музыка - Трек
|
||||
- **YandexVideo**
|
||||
- **YapFiles**
|
||||
- **YesJapan**
|
||||
- **yinyuetai:video**: 音悦Tai
|
||||
|
||||
@@ -107,184 +107,6 @@ class TestInfoExtractor(unittest.TestCase):
|
||||
self.assertRaises(ExtractorError, self.ie._download_json, uri, None)
|
||||
self.assertEqual(self.ie._download_json(uri, None, fatal=False), None)
|
||||
|
||||
def test_parse_html5_media_entries(self):
|
||||
# from https://www.r18.com/
|
||||
# with kpbs in label
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.r18.com/',
|
||||
r'''
|
||||
<video id="samplevideo_amateur" class="js-samplevideo video-js vjs-default-skin vjs-big-play-centered" controls preload="auto" width="400" height="225" poster="//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg">
|
||||
<source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_sm_w.mp4" type="video/mp4" res="240" label="300kbps">
|
||||
<source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dm_w.mp4" type="video/mp4" res="480" label="1000kbps">
|
||||
<source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dmb_w.mp4" type="video/mp4" res="740" label="1500kbps">
|
||||
<p>Your browser does not support the video tag.</p>
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_sm_w.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '300kbps',
|
||||
'height': 240,
|
||||
'tbr': 300,
|
||||
}, {
|
||||
'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dm_w.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '1000kbps',
|
||||
'height': 480,
|
||||
'tbr': 1000,
|
||||
}, {
|
||||
'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dmb_w.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '1500kbps',
|
||||
'height': 740,
|
||||
'tbr': 1500,
|
||||
}],
|
||||
'thumbnail': '//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg'
|
||||
})
|
||||
|
||||
# from https://www.csfd.cz/
|
||||
# with width and height
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.csfd.cz/',
|
||||
r'''
|
||||
<video width="770" height="328" preload="none" controls poster="https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360" >
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327358_eac647.mp4" type="video/mp4" width="640" height="360">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327360_3d2646.mp4" type="video/mp4" width="1280" height="720">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327356_91f258.mp4" type="video/mp4" width="1920" height="1080">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327359_962b4a.webm" type="video/webm" width="640" height="360">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327361_6feee0.webm" type="video/webm" width="1280" height="720">
|
||||
<source src="https://video.csfd.cz/files/videos/157/750/157750813/163327357_8ab472.webm" type="video/webm" width="1920" height="1080">
|
||||
<track src="https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt" type="text/x-srt" kind="subtitles" srclang="cs" label="cs">
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327358_eac647.mp4',
|
||||
'ext': 'mp4',
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327360_3d2646.mp4',
|
||||
'ext': 'mp4',
|
||||
'width': 1280,
|
||||
'height': 720,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327356_91f258.mp4',
|
||||
'ext': 'mp4',
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327359_962b4a.webm',
|
||||
'ext': 'webm',
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327361_6feee0.webm',
|
||||
'ext': 'webm',
|
||||
'width': 1280,
|
||||
'height': 720,
|
||||
}, {
|
||||
'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327357_8ab472.webm',
|
||||
'ext': 'webm',
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
}],
|
||||
'subtitles': {
|
||||
'cs': [{'url': 'https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt'}]
|
||||
},
|
||||
'thumbnail': 'https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360'
|
||||
})
|
||||
|
||||
# from https://tamasha.com/v/Kkdjw
|
||||
# with height in label
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://tamasha.com/v/Kkdjw',
|
||||
r'''
|
||||
<video crossorigin="anonymous">
|
||||
<source src="https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4" type="video/mp4" label="AUTO" res="0"/>
|
||||
<source src="https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4" type="video/mp4"
|
||||
label="240p" res="240"/>
|
||||
<source src="https://s-v2.tamasha.com/statics/videos_file/20/00/Kkdjw_200041c66f657fc967db464d156eafbc1ed9fe6f_n_144.mp4" type="video/mp4"
|
||||
label="144p" res="144"/>
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4',
|
||||
}, {
|
||||
'url': 'https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '240p',
|
||||
'height': 240,
|
||||
}, {
|
||||
'url': 'https://s-v2.tamasha.com/statics/videos_file/20/00/Kkdjw_200041c66f657fc967db464d156eafbc1ed9fe6f_n_144.mp4',
|
||||
'ext': 'mp4',
|
||||
'format_id': '144p',
|
||||
'height': 144,
|
||||
}]
|
||||
})
|
||||
|
||||
# from https://www.directvnow.com
|
||||
# with data-src
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.directvnow.com',
|
||||
r'''
|
||||
<video id="vid1" class="header--video-masked active" muted playsinline>
|
||||
<source data-src="https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4" type="video/mp4" />
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'ext': 'mp4',
|
||||
'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
|
||||
}]
|
||||
})
|
||||
|
||||
# from https://www.directvnow.com
|
||||
# with data-src
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.directvnow.com',
|
||||
r'''
|
||||
<video id="vid1" class="header--video-masked active" muted playsinline>
|
||||
<source data-src="https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4" type="video/mp4" />
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
|
||||
'ext': 'mp4',
|
||||
}]
|
||||
})
|
||||
|
||||
# from https://www.klarna.com/uk/
|
||||
# with data-video-src
|
||||
expect_dict(
|
||||
self,
|
||||
self.ie._parse_html5_media_entries(
|
||||
'https://www.directvnow.com',
|
||||
r'''
|
||||
<video loop autoplay muted class="responsive-video block-kl__video video-on-medium">
|
||||
<source src="" data-video-desktop data-video-src="https://www.klarna.com/uk/wp-content/uploads/sites/11/2019/01/KL062_Smooth3_0_DogWalking_5s_920x080_.mp4" type="video/mp4" />
|
||||
</video>
|
||||
''', None)[0],
|
||||
{
|
||||
'formats': [{
|
||||
'url': 'https://www.klarna.com/uk/wp-content/uploads/sites/11/2019/01/KL062_Smooth3_0_DogWalking_5s_920x080_.mp4',
|
||||
'ext': 'mp4',
|
||||
}],
|
||||
})
|
||||
|
||||
def test_extract_jwplayer_data_realworld(self):
|
||||
# from http://www.suffolk.edu/sjc/
|
||||
expect_dict(
|
||||
@@ -752,8 +574,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
# Also tests duplicate representation ids, see
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/15111
|
||||
'float_duration',
|
||||
'http://unknown/manifest.mpd', # mpd_url
|
||||
None, # mpd_base_url
|
||||
'http://unknown/manifest.mpd',
|
||||
[{
|
||||
'manifest_url': 'http://unknown/manifest.mpd',
|
||||
'ext': 'm4a',
|
||||
@@ -833,8 +654,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
), (
|
||||
# https://github.com/ytdl-org/youtube-dl/pull/14844
|
||||
'urls_only',
|
||||
'http://unknown/manifest.mpd', # mpd_url
|
||||
None, # mpd_base_url
|
||||
'http://unknown/manifest.mpd',
|
||||
[{
|
||||
'manifest_url': 'http://unknown/manifest.mpd',
|
||||
'ext': 'mp4',
|
||||
@@ -913,61 +733,15 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
}]
|
||||
), (
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/20346
|
||||
# Media considered unfragmented even though it contains
|
||||
# Initialization tag
|
||||
'unfragmented',
|
||||
'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd', # mpd_url
|
||||
'https://v.redd.it/hw1x7rcg7zl21', # mpd_base_url
|
||||
[{
|
||||
'url': 'https://v.redd.it/hw1x7rcg7zl21/audio',
|
||||
'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
|
||||
'ext': 'm4a',
|
||||
'format_id': 'AUDIO-1',
|
||||
'format_note': 'DASH audio',
|
||||
'container': 'm4a_dash',
|
||||
'acodec': 'mp4a.40.2',
|
||||
'vcodec': 'none',
|
||||
'tbr': 129.87,
|
||||
'asr': 48000,
|
||||
|
||||
}, {
|
||||
'url': 'https://v.redd.it/hw1x7rcg7zl21/DASH_240',
|
||||
'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
|
||||
'ext': 'mp4',
|
||||
'format_id': 'VIDEO-2',
|
||||
'format_note': 'DASH video',
|
||||
'container': 'mp4_dash',
|
||||
'acodec': 'none',
|
||||
'vcodec': 'avc1.4d401e',
|
||||
'tbr': 608.0,
|
||||
'width': 240,
|
||||
'height': 240,
|
||||
'fps': 30,
|
||||
}, {
|
||||
'url': 'https://v.redd.it/hw1x7rcg7zl21/DASH_360',
|
||||
'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
|
||||
'ext': 'mp4',
|
||||
'format_id': 'VIDEO-1',
|
||||
'format_note': 'DASH video',
|
||||
'container': 'mp4_dash',
|
||||
'acodec': 'none',
|
||||
'vcodec': 'avc1.4d401e',
|
||||
'tbr': 804.261,
|
||||
'width': 360,
|
||||
'height': 360,
|
||||
'fps': 30,
|
||||
}]
|
||||
)
|
||||
]
|
||||
|
||||
for mpd_file, mpd_url, mpd_base_url, expected_formats in _TEST_CASES:
|
||||
for mpd_file, mpd_url, expected_formats in _TEST_CASES:
|
||||
with io.open('./test/testdata/mpd/%s.mpd' % mpd_file,
|
||||
mode='r', encoding='utf-8') as f:
|
||||
formats = self.ie._parse_mpd_formats(
|
||||
compat_etree_fromstring(f.read().encode('utf-8')),
|
||||
mpd_base_url=mpd_base_url, mpd_url=mpd_url)
|
||||
mpd_url=mpd_url)
|
||||
self.ie._sort_formats(formats)
|
||||
expect_value(self, formats, expected_formats, None)
|
||||
|
||||
|
||||
@@ -33,13 +33,11 @@ from youtube_dl.utils import (
|
||||
ExtractorError,
|
||||
find_xpath_attr,
|
||||
fix_xml_ampersands,
|
||||
float_or_none,
|
||||
get_element_by_class,
|
||||
get_element_by_attribute,
|
||||
get_elements_by_class,
|
||||
get_elements_by_attribute,
|
||||
InAdvancePagedList,
|
||||
int_or_none,
|
||||
intlist_to_bytes,
|
||||
is_html,
|
||||
js_to_json,
|
||||
@@ -57,7 +55,6 @@ from youtube_dl.utils import (
|
||||
parse_count,
|
||||
parse_iso8601,
|
||||
parse_resolution,
|
||||
parse_bitrate,
|
||||
pkcs1pad,
|
||||
read_batch_urls,
|
||||
sanitize_filename,
|
||||
@@ -470,21 +467,6 @@ class TestUtil(unittest.TestCase):
|
||||
shell_quote(args),
|
||||
"""ffmpeg -i 'ñ€ß'"'"'.mp4'""" if compat_os_name != 'nt' else '''ffmpeg -i "ñ€ß'.mp4"''')
|
||||
|
||||
def test_float_or_none(self):
|
||||
self.assertEqual(float_or_none('42.42'), 42.42)
|
||||
self.assertEqual(float_or_none('42'), 42.0)
|
||||
self.assertEqual(float_or_none(''), None)
|
||||
self.assertEqual(float_or_none(None), None)
|
||||
self.assertEqual(float_or_none([]), None)
|
||||
self.assertEqual(float_or_none(set()), None)
|
||||
|
||||
def test_int_or_none(self):
|
||||
self.assertEqual(int_or_none('42'), 42)
|
||||
self.assertEqual(int_or_none(''), None)
|
||||
self.assertEqual(int_or_none(None), None)
|
||||
self.assertEqual(int_or_none([]), None)
|
||||
self.assertEqual(int_or_none(set()), None)
|
||||
|
||||
def test_str_to_int(self):
|
||||
self.assertEqual(str_to_int('123,456'), 123456)
|
||||
self.assertEqual(str_to_int('123.456'), 123456)
|
||||
@@ -1048,13 +1030,6 @@ class TestUtil(unittest.TestCase):
|
||||
self.assertEqual(parse_resolution('4k'), {'height': 2160})
|
||||
self.assertEqual(parse_resolution('8K'), {'height': 4320})
|
||||
|
||||
def test_parse_bitrate(self):
|
||||
self.assertEqual(parse_bitrate(None), None)
|
||||
self.assertEqual(parse_bitrate(''), None)
|
||||
self.assertEqual(parse_bitrate('300kbps'), 300)
|
||||
self.assertEqual(parse_bitrate('1500kbps'), 1500)
|
||||
self.assertEqual(parse_bitrate('300 kbps'), 300)
|
||||
|
||||
def test_version_tuple(self):
|
||||
self.assertEqual(version_tuple('1'), (1,))
|
||||
self.assertEqual(version_tuple('10.23.344'), (10, 23, 344))
|
||||
|
||||
28
test/testdata/mpd/unfragmented.mpd
vendored
28
test/testdata/mpd/unfragmented.mpd
vendored
@@ -1,28 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
|
||||
<MPD mediaPresentationDuration="PT54.915S" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
|
||||
<Period duration="PT54.915S">
|
||||
<AdaptationSet segmentAlignment="true" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
|
||||
<Representation bandwidth="804261" codecs="avc1.4d401e" frameRate="30" height="360" id="VIDEO-1" mimeType="video/mp4" startWithSAP="1" width="360">
|
||||
<BaseURL>DASH_360</BaseURL>
|
||||
<SegmentBase indexRange="915-1114" indexRangeExact="true">
|
||||
<Initialization range="0-914"/>
|
||||
</SegmentBase>
|
||||
</Representation>
|
||||
<Representation bandwidth="608000" codecs="avc1.4d401e" frameRate="30" height="240" id="VIDEO-2" mimeType="video/mp4" startWithSAP="1" width="240">
|
||||
<BaseURL>DASH_240</BaseURL>
|
||||
<SegmentBase indexRange="913-1112" indexRangeExact="true">
|
||||
<Initialization range="0-912"/>
|
||||
</SegmentBase>
|
||||
</Representation>
|
||||
</AdaptationSet>
|
||||
<AdaptationSet>
|
||||
<Representation audioSamplingRate="48000" bandwidth="129870" codecs="mp4a.40.2" id="AUDIO-1" mimeType="audio/mp4" startWithSAP="1">
|
||||
<AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
|
||||
<BaseURL>audio</BaseURL>
|
||||
<SegmentBase indexRange="832-1007" indexRangeExact="true">
|
||||
<Initialization range="0-831"/>
|
||||
</SegmentBase>
|
||||
</Representation>
|
||||
</AdaptationSet>
|
||||
</Period>
|
||||
</MPD>
|
||||
@@ -166,8 +166,6 @@ def _real_main(argv=None):
|
||||
if opts.max_sleep_interval is not None:
|
||||
if opts.max_sleep_interval < 0:
|
||||
parser.error('max sleep interval must be positive or 0')
|
||||
if opts.sleep_interval is None:
|
||||
parser.error('min sleep interval must be specified, use --min-sleep-interval')
|
||||
if opts.max_sleep_interval < opts.sleep_interval:
|
||||
parser.error('max sleep interval must be greater than or equal to min sleep interval')
|
||||
else:
|
||||
|
||||
30
youtube_dl/extractor/anitube.py
Normal file
30
youtube_dl/extractor/anitube.py
Normal file
@@ -0,0 +1,30 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .nuevo import NuevoBaseIE
|
||||
|
||||
|
||||
class AnitubeIE(NuevoBaseIE):
|
||||
IE_NAME = 'anitube.se'
|
||||
_VALID_URL = r'https?://(?:www\.)?anitube\.se/video/(?P<id>\d+)'
|
||||
|
||||
_TEST = {
|
||||
'url': 'http://www.anitube.se/video/36621',
|
||||
'md5': '59d0eeae28ea0bc8c05e7af429998d43',
|
||||
'info_dict': {
|
||||
'id': '36621',
|
||||
'ext': 'mp4',
|
||||
'title': 'Recorder to Randoseru 01',
|
||||
'duration': 180.19,
|
||||
},
|
||||
'skip': 'Blocked in the US',
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
key = self._search_regex(
|
||||
r'src=["\']https?://[^/]+/embed/([A-Za-z0-9_-]+)', webpage, 'key')
|
||||
|
||||
return self._extract_nuevo(
|
||||
'http://www.anitube.se/nuevo/econfig.php?key=%s' % key, video_id)
|
||||
61
youtube_dl/extractor/anysex.py
Normal file
61
youtube_dl/extractor/anysex.py
Normal file
@@ -0,0 +1,61 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
parse_duration,
|
||||
int_or_none,
|
||||
)
|
||||
|
||||
|
||||
class AnySexIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?anysex\.com/(?P<id>\d+)'
|
||||
_TEST = {
|
||||
'url': 'http://anysex.com/156592/',
|
||||
'md5': '023e9fbb7f7987f5529a394c34ad3d3d',
|
||||
'info_dict': {
|
||||
'id': '156592',
|
||||
'ext': 'mp4',
|
||||
'title': 'Busty and sexy blondie in her bikini strips for you',
|
||||
'description': 'md5:de9e418178e2931c10b62966474e1383',
|
||||
'categories': ['Erotic'],
|
||||
'duration': 270,
|
||||
'age_limit': 18,
|
||||
}
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
video_id = mobj.group('id')
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
|
||||
video_url = self._html_search_regex(r"video_url\s*:\s*'([^']+)'", webpage, 'video URL')
|
||||
|
||||
title = self._html_search_regex(r'<title>(.*?)</title>', webpage, 'title')
|
||||
description = self._html_search_regex(
|
||||
r'<div class="description"[^>]*>([^<]+)</div>', webpage, 'description', fatal=False)
|
||||
thumbnail = self._html_search_regex(
|
||||
r'preview_url\s*:\s*\'(.*?)\'', webpage, 'thumbnail', fatal=False)
|
||||
|
||||
categories = re.findall(
|
||||
r'<a href="http://anysex\.com/categories/[^"]+" title="[^"]*">([^<]+)</a>', webpage)
|
||||
|
||||
duration = parse_duration(self._search_regex(
|
||||
r'<b>Duration:</b> (?:<q itemprop="duration">)?(\d+:\d+)', webpage, 'duration', fatal=False))
|
||||
view_count = int_or_none(self._html_search_regex(
|
||||
r'<b>Views:</b> (\d+)', webpage, 'view count', fatal=False))
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'url': video_url,
|
||||
'ext': 'mp4',
|
||||
'title': title,
|
||||
'description': description,
|
||||
'thumbnail': thumbnail,
|
||||
'categories': categories,
|
||||
'duration': duration,
|
||||
'view_count': view_count,
|
||||
'age_limit': 18,
|
||||
}
|
||||
@@ -9,8 +9,8 @@ from ..utils import (
|
||||
|
||||
|
||||
class BeegIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?beeg\.(?:com|porn(?:/video)?)/(?P<id>\d+)'
|
||||
_TESTS = [{
|
||||
_VALID_URL = r'https?://(?:www\.)?beeg\.com/(?P<id>\d+)'
|
||||
_TEST = {
|
||||
'url': 'http://beeg.com/5416503',
|
||||
'md5': 'a1a1b1a8bc70a89e49ccfd113aed0820',
|
||||
'info_dict': {
|
||||
@@ -24,13 +24,7 @@ class BeegIE(InfoExtractor):
|
||||
'tags': list,
|
||||
'age_limit': 18,
|
||||
}
|
||||
}, {
|
||||
'url': 'https://beeg.porn/video/5416503',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://beeg.porn/5416503',
|
||||
'only_matching': True,
|
||||
}]
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
@@ -2,96 +2,39 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from .vk import VKIE
|
||||
from ..utils import (
|
||||
HEADRequest,
|
||||
int_or_none,
|
||||
)
|
||||
|
||||
|
||||
class BIQLEIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?biqle\.(?:com|org|ru)/watch/(?P<id>-?\d+_\d+)'
|
||||
_TESTS = [{
|
||||
# Youtube embed
|
||||
'url': 'https://biqle.ru/watch/-115995369_456239081',
|
||||
'md5': '97af5a06ee4c29bbf9c001bdb1cf5c06',
|
||||
'url': 'http://www.biqle.ru/watch/847655_160197695',
|
||||
'md5': 'ad5f746a874ccded7b8f211aeea96637',
|
||||
'info_dict': {
|
||||
'id': '8v4f-avW-VI',
|
||||
'id': '160197695',
|
||||
'ext': 'mp4',
|
||||
'title': "PASSE-PARTOUT - L'ete c'est fait pour jouer",
|
||||
'description': 'Passe-Partout',
|
||||
'uploader_id': 'mrsimpsonstef3',
|
||||
'uploader': 'Phanolito',
|
||||
'upload_date': '20120822',
|
||||
},
|
||||
'title': 'Foo Fighters - The Pretender (Live at Wembley Stadium)',
|
||||
'uploader': 'Andrey Rogozin',
|
||||
'upload_date': '20110605',
|
||||
}
|
||||
}, {
|
||||
'url': 'http://biqle.org/watch/-44781847_168547604',
|
||||
'url': 'https://biqle.org/watch/-44781847_168547604',
|
||||
'md5': '7f24e72af1db0edf7c1aaba513174f97',
|
||||
'info_dict': {
|
||||
'id': '168547604',
|
||||
'ext': 'mp4',
|
||||
'title': 'Ребенок в шоке от автоматической мойки',
|
||||
'timestamp': 1396633454,
|
||||
'uploader': 'Dmitry Kotov',
|
||||
'upload_date': '20140404',
|
||||
'uploader_id': '47850140',
|
||||
},
|
||||
'skip': ' This video was marked as adult. Embedding adult videos on external sites is prohibited.',
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
embed_url = self._proto_relative_url(self._search_regex(
|
||||
r'<iframe.+?src="((?:https?:)?//daxab\.com/[^"]+)".*?></iframe>',
|
||||
webpage, 'embed url'))
|
||||
if VKIE.suitable(embed_url):
|
||||
return self.url_result(embed_url, VKIE.ie_key(), video_id)
|
||||
|
||||
self._request_webpage(
|
||||
HEADRequest(embed_url), video_id, headers={'Referer': url})
|
||||
video_id, sig, _, access_token = self._get_cookies(embed_url)['video_ext'].value.split('%3A')
|
||||
item = self._download_json(
|
||||
'https://api.vk.com/method/video.get', video_id,
|
||||
headers={'User-Agent': 'okhttp/3.4.1'}, query={
|
||||
'access_token': access_token,
|
||||
'sig': sig,
|
||||
'v': 5.44,
|
||||
'videos': video_id,
|
||||
})['response']['items'][0]
|
||||
title = item['title']
|
||||
|
||||
formats = []
|
||||
for f_id, f_url in item.get('files', {}).items():
|
||||
if f_id == 'external':
|
||||
return self.url_result(f_url)
|
||||
ext, height = f_id.split('_')
|
||||
formats.append({
|
||||
'format_id': height + 'p',
|
||||
'url': f_url,
|
||||
'height': int_or_none(height),
|
||||
'ext': ext,
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
|
||||
thumbnails = []
|
||||
for k, v in item.items():
|
||||
if k.startswith('photo_') and v:
|
||||
width = k.replace('photo_', '')
|
||||
thumbnails.append({
|
||||
'id': width,
|
||||
'url': v,
|
||||
'width': int_or_none(width),
|
||||
})
|
||||
r'<iframe.+?src="((?:http:)?//daxab\.com/[^"]+)".*?></iframe>', webpage, 'embed url'))
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'formats': formats,
|
||||
'comment_count': int_or_none(item.get('comments')),
|
||||
'description': item.get('description'),
|
||||
'duration': int_or_none(item.get('duration')),
|
||||
'thumbnails': thumbnails,
|
||||
'timestamp': int_or_none(item.get('date')),
|
||||
'uploader': item.get('owner_id'),
|
||||
'view_count': int_or_none(item.get('views')),
|
||||
'_type': 'url_transparent',
|
||||
'url': embed_url,
|
||||
}
|
||||
|
||||
@@ -360,7 +360,7 @@ class CBCWatchVideoIE(CBCWatchBaseIE):
|
||||
|
||||
class CBCWatchIE(CBCWatchBaseIE):
|
||||
IE_NAME = 'cbc.ca:watch'
|
||||
_VALID_URL = r'https?://(?:gem|watch)\.cbc\.ca/(?:[^/]+/)+(?P<id>[0-9a-f-]+)'
|
||||
_VALID_URL = r'https?://watch\.cbc\.ca/(?:[^/]+/)+(?P<id>[0-9a-f-]+)'
|
||||
_TESTS = [{
|
||||
# geo-restricted to Canada, bypassable
|
||||
'url': 'http://watch.cbc.ca/doc-zone/season-6/customer-disservice/38e815a-009e3ab12e4',
|
||||
@@ -386,9 +386,6 @@ class CBCWatchIE(CBCWatchBaseIE):
|
||||
'description': 'Arthur, the sweetest 8-year-old aardvark, and his pals solve all kinds of problems with humour, kindness and teamwork.',
|
||||
},
|
||||
'playlist_mincount': 30,
|
||||
}, {
|
||||
'url': 'https://gem.cbc.ca/media/this-hour-has-22-minutes/season-26/episode-20/38e815a-0108c6c6a42',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
|
||||
@@ -65,8 +65,8 @@ class CiscoLiveBaseIE(InfoExtractor):
|
||||
|
||||
|
||||
class CiscoLiveSessionIE(CiscoLiveBaseIE):
|
||||
_VALID_URL = r'https?://(?:www\.)?ciscolive(?:\.cisco)?\.com/[^#]*#/session/(?P<id>[^/?&]+)'
|
||||
_TESTS = [{
|
||||
_VALID_URL = r'https?://ciscolive\.cisco\.com/on-demand-library/\??[^#]*#/session/(?P<id>[^/?&]+)'
|
||||
_TEST = {
|
||||
'url': 'https://ciscolive.cisco.com/on-demand-library/?#/session/1423353499155001FoSs',
|
||||
'md5': 'c98acf395ed9c9f766941c70f5352e22',
|
||||
'info_dict': {
|
||||
@@ -79,13 +79,7 @@ class CiscoLiveSessionIE(CiscoLiveBaseIE):
|
||||
'uploader_id': '5647924234001',
|
||||
'location': '16B Mezz.',
|
||||
},
|
||||
}, {
|
||||
'url': 'https://www.ciscolive.com/global/on-demand-library.html?search.event=ciscoliveemea2019#/session/15361595531500013WOU',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://www.ciscolive.com/global/on-demand-library.html?#/session/1490051371645001kNaS',
|
||||
'only_matching': True,
|
||||
}]
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
rf_id = self._match_id(url)
|
||||
@@ -94,7 +88,7 @@ class CiscoLiveSessionIE(CiscoLiveBaseIE):
|
||||
|
||||
|
||||
class CiscoLiveSearchIE(CiscoLiveBaseIE):
|
||||
_VALID_URL = r'https?://(?:www\.)?ciscolive(?:\.cisco)?\.com/(?:global/)?on-demand-library(?:\.html|/)'
|
||||
_VALID_URL = r'https?://ciscolive\.cisco\.com/on-demand-library/'
|
||||
_TESTS = [{
|
||||
'url': 'https://ciscolive.cisco.com/on-demand-library/?search.event=ciscoliveus2018&search.technicallevel=scpsSkillLevel_aintroductory&search.focus=scpsSessionFocus_designAndDeployment#/',
|
||||
'info_dict': {
|
||||
@@ -104,9 +98,6 @@ class CiscoLiveSearchIE(CiscoLiveBaseIE):
|
||||
}, {
|
||||
'url': 'https://ciscolive.cisco.com/on-demand-library/?search.technology=scpsTechnology_applicationDevelopment&search.technology=scpsTechnology_ipv6&search.focus=scpsSessionFocus_troubleshootingTroubleshooting#/',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://www.ciscolive.com/global/on-demand-library.html?search.technicallevel=scpsSkillLevel_aintroductory&search.event=ciscoliveemea2019&search.technology=scpsTechnology_dataCenter&search.focus=scpsSessionFocus_bestPractices#/',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
@classmethod
|
||||
|
||||
@@ -44,7 +44,6 @@ from ..utils import (
|
||||
compiled_regex_type,
|
||||
determine_ext,
|
||||
determine_protocol,
|
||||
dict_get,
|
||||
error_to_compat_str,
|
||||
ExtractorError,
|
||||
extract_attributes,
|
||||
@@ -57,16 +56,13 @@ from ..utils import (
|
||||
JSON_LD_RE,
|
||||
mimetype2ext,
|
||||
orderedSet,
|
||||
parse_bitrate,
|
||||
parse_codecs,
|
||||
parse_duration,
|
||||
parse_iso8601,
|
||||
parse_m3u8_attributes,
|
||||
parse_resolution,
|
||||
RegexNotFoundError,
|
||||
sanitized_Request,
|
||||
sanitize_filename,
|
||||
str_or_none,
|
||||
unescapeHTML,
|
||||
unified_strdate,
|
||||
unified_timestamp,
|
||||
@@ -112,13 +108,10 @@ class InfoExtractor(object):
|
||||
for RTMP - RTMP URL,
|
||||
for HLS - URL of the M3U8 media playlist,
|
||||
for HDS - URL of the F4M manifest,
|
||||
for DASH
|
||||
- HTTP URL to plain file media (in case of
|
||||
unfragmented media)
|
||||
- URL of the MPD manifest or base URL
|
||||
representing the media if MPD manifest
|
||||
is parsed froma string (in case of
|
||||
fragmented media)
|
||||
for DASH - URL of the MPD manifest or
|
||||
base URL representing the media
|
||||
if MPD manifest is parsed from
|
||||
a string,
|
||||
for MSS - URL of the ISM manifest.
|
||||
* manifest_url
|
||||
The URL of the manifest file in case of
|
||||
@@ -2144,6 +2137,8 @@ class InfoExtractor(object):
|
||||
bandwidth = int_or_none(representation_attrib.get('bandwidth'))
|
||||
f = {
|
||||
'format_id': '%s-%s' % (mpd_id, representation_id) if mpd_id else representation_id,
|
||||
# NB: mpd_url may be empty when MPD manifest is parsed from a string
|
||||
'url': mpd_url or base_url,
|
||||
'manifest_url': mpd_url,
|
||||
'ext': mimetype2ext(mime_type),
|
||||
'width': int_or_none(representation_attrib.get('width')),
|
||||
@@ -2282,14 +2277,10 @@ class InfoExtractor(object):
|
||||
fragment['duration'] = segment_duration
|
||||
fragments.append(fragment)
|
||||
representation_ms_info['fragments'] = fragments
|
||||
# If there is a fragments key available then we correctly recognized fragmented media.
|
||||
# Otherwise we will assume unfragmented media with direct access. Technically, such
|
||||
# assumption is not necessarily correct since we may simply have no support for
|
||||
# some forms of fragmented media renditions yet, but for now we'll use this fallback.
|
||||
# NB: MPD manifest may contain direct URLs to unfragmented media.
|
||||
# No fragments key is present in this case.
|
||||
if 'fragments' in representation_ms_info:
|
||||
f.update({
|
||||
# NB: mpd_url may be empty when MPD manifest is parsed from a string
|
||||
'url': mpd_url or base_url,
|
||||
'fragment_base_url': base_url,
|
||||
'fragments': [],
|
||||
'protocol': 'http_dash_segments',
|
||||
@@ -2300,10 +2291,6 @@ class InfoExtractor(object):
|
||||
f['url'] = initialization_url
|
||||
f['fragments'].append({location_key(initialization_url): initialization_url})
|
||||
f['fragments'].extend(representation_ms_info['fragments'])
|
||||
else:
|
||||
# Assuming direct URL to unfragmented media.
|
||||
f['url'] = base_url
|
||||
|
||||
# According to [1, 5.3.5.2, Table 7, page 35] @id of Representation
|
||||
# is not necessarily unique within a Period thus formats with
|
||||
# the same `format_id` are quite possible. There are numerous examples
|
||||
@@ -2485,43 +2472,18 @@ class InfoExtractor(object):
|
||||
media_info['thumbnail'] = absolute_url(media_attributes.get('poster'))
|
||||
if media_content:
|
||||
for source_tag in re.findall(r'<source[^>]+>', media_content):
|
||||
s_attr = extract_attributes(source_tag)
|
||||
# data-video-src and data-src are non standard but seen
|
||||
# several times in the wild
|
||||
src = dict_get(s_attr, ('src', 'data-video-src', 'data-src'))
|
||||
source_attributes = extract_attributes(source_tag)
|
||||
src = source_attributes.get('src')
|
||||
if not src:
|
||||
continue
|
||||
f = parse_content_type(s_attr.get('type'))
|
||||
f = parse_content_type(source_attributes.get('type'))
|
||||
is_plain_url, formats = _media_formats(src, media_type, f)
|
||||
if is_plain_url:
|
||||
# width, height, res, label and title attributes are
|
||||
# all not standard but seen several times in the wild
|
||||
labels = [
|
||||
s_attr.get(lbl)
|
||||
for lbl in ('label', 'title')
|
||||
if str_or_none(s_attr.get(lbl))
|
||||
]
|
||||
width = int_or_none(s_attr.get('width'))
|
||||
height = (int_or_none(s_attr.get('height')) or
|
||||
int_or_none(s_attr.get('res')))
|
||||
if not width or not height:
|
||||
for lbl in labels:
|
||||
resolution = parse_resolution(lbl)
|
||||
if not resolution:
|
||||
continue
|
||||
width = width or resolution.get('width')
|
||||
height = height or resolution.get('height')
|
||||
for lbl in labels:
|
||||
tbr = parse_bitrate(lbl)
|
||||
if tbr:
|
||||
break
|
||||
else:
|
||||
tbr = None
|
||||
# res attribute is not standard but seen several times
|
||||
# in the wild
|
||||
f.update({
|
||||
'width': width,
|
||||
'height': height,
|
||||
'tbr': tbr,
|
||||
'format_id': s_attr.get('label') or s_attr.get('title'),
|
||||
'height': int_or_none(source_attributes.get('res')),
|
||||
'format_id': source_attributes.get('label'),
|
||||
})
|
||||
f.update(formats[0])
|
||||
media_info['formats'].append(f)
|
||||
|
||||
@@ -13,9 +13,9 @@ class CorusIE(ThePlatformFeedIE):
|
||||
(?:www\.)?
|
||||
(?P<domain>
|
||||
(?:globaltv|etcanada)\.com|
|
||||
(?:hgtv|foodnetwork|slice|history|showcase|bigbrothercanada)\.ca
|
||||
(?:hgtv|foodnetwork|slice|history|showcase)\.ca
|
||||
)
|
||||
/(?:video/(?:[^/]+/)?|(?:[^/]+/)+(?:videos/[a-z0-9-]+-|video\.html\?.*?\bv=))
|
||||
/(?:video/|(?:[^/]+/)+(?:videos/[a-z0-9-]+-|video\.html\?.*?\bv=))
|
||||
(?P<id>\d+)
|
||||
'''
|
||||
_TESTS = [{
|
||||
@@ -42,12 +42,6 @@ class CorusIE(ThePlatformFeedIE):
|
||||
}, {
|
||||
'url': 'http://www.showcase.ca/eyewitness/video/eyewitness++106/video.html?v=955070531919&p=1&s=da#video',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'http://www.bigbrothercanada.ca/video/1457812035894/',
|
||||
'only_matching': True
|
||||
}, {
|
||||
'url': 'https://www.bigbrothercanada.ca/video/big-brother-canada-704/1457812035894/',
|
||||
'only_matching': True
|
||||
}]
|
||||
|
||||
_TP_FEEDS = {
|
||||
@@ -79,10 +73,6 @@ class CorusIE(ThePlatformFeedIE):
|
||||
'feed_id': '9H6qyshBZU3E',
|
||||
'account_id': 2414426607,
|
||||
},
|
||||
'bigbrothercanada': {
|
||||
'feed_id': 'ChQqrem0lNUp',
|
||||
'account_id': 2269680845,
|
||||
},
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
|
||||
@@ -79,7 +79,7 @@ class CWTVIE(InfoExtractor):
|
||||
season = str_or_none(video_data.get('season'))
|
||||
episode = str_or_none(video_data.get('episode'))
|
||||
if episode and season:
|
||||
episode = episode[len(season):]
|
||||
episode = episode.lstrip(season)
|
||||
|
||||
return {
|
||||
'_type': 'url_transparent',
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import json
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
determine_ext,
|
||||
clean_html,
|
||||
int_or_none,
|
||||
float_or_none,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
@@ -33,7 +36,7 @@ def _decrypt_config(key, string):
|
||||
|
||||
|
||||
class EscapistIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://?(?:(?:www|v1)\.)?escapistmagazine\.com/videos/view/[^/]+/(?P<id>[0-9]+)'
|
||||
_VALID_URL = r'https?://?(?:www\.)?escapistmagazine\.com/videos/view/[^/?#]+/(?P<id>[0-9]+)-[^/?#]*(?:$|[?#])'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.escapistmagazine.com/videos/view/the-escapist-presents/6618-Breaking-Down-Baldurs-Gate',
|
||||
'md5': 'ab3a706c681efca53f0a35f1415cf0d1',
|
||||
@@ -58,12 +61,6 @@ class EscapistIE(InfoExtractor):
|
||||
'duration': 304,
|
||||
'uploader': 'The Escapist',
|
||||
}
|
||||
}, {
|
||||
'url': 'http://escapistmagazine.com/videos/view/the-escapist-presents/6618',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://v1.escapistmagazine.com/videos/view/the-escapist-presents/6618-Breaking-Down-Baldurs-Gate',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
@@ -77,20 +74,19 @@ class EscapistIE(InfoExtractor):
|
||||
video_id = ims_video['videoID']
|
||||
key = ims_video['hash']
|
||||
|
||||
config = self._download_webpage(
|
||||
'http://www.escapistmagazine.com/videos/vidconfig.php',
|
||||
video_id, 'Downloading video config', headers={
|
||||
'Referer': url,
|
||||
}, query={
|
||||
'videoID': video_id,
|
||||
'hash': key,
|
||||
})
|
||||
config_req = sanitized_Request(
|
||||
'http://www.escapistmagazine.com/videos/'
|
||||
'vidconfig.php?videoID=%s&hash=%s' % (video_id, key))
|
||||
config_req.add_header('Referer', url)
|
||||
config = self._download_webpage(config_req, video_id, 'Downloading video config')
|
||||
|
||||
data = self._parse_json(_decrypt_config(key, config), video_id)
|
||||
data = json.loads(_decrypt_config(key, config))
|
||||
|
||||
video_data = data['videoData']
|
||||
|
||||
title = clean_html(video_data['title'])
|
||||
duration = float_or_none(video_data.get('duration'), 1000)
|
||||
uploader = video_data.get('publisher')
|
||||
|
||||
formats = [{
|
||||
'url': video['src'],
|
||||
@@ -103,9 +99,8 @@ class EscapistIE(InfoExtractor):
|
||||
'id': video_id,
|
||||
'formats': formats,
|
||||
'title': title,
|
||||
'thumbnail': self._og_search_thumbnail(webpage) or data.get('poster'),
|
||||
'thumbnail': self._og_search_thumbnail(webpage),
|
||||
'description': self._og_search_description(webpage),
|
||||
'duration': float_or_none(video_data.get('duration'), 1000),
|
||||
'uploader': video_data.get('publisher'),
|
||||
'series': video_data.get('show'),
|
||||
'duration': duration,
|
||||
'uploader': uploader,
|
||||
}
|
||||
|
||||
@@ -38,7 +38,9 @@ from .alphaporno import AlphaPornoIE
|
||||
from .amcnetworks import AMCNetworksIE
|
||||
from .americastestkitchen import AmericasTestKitchenIE
|
||||
from .animeondemand import AnimeOnDemandIE
|
||||
from .anitube import AnitubeIE
|
||||
from .anvato import AnvatoIE
|
||||
from .anysex import AnySexIE
|
||||
from .aol import AolIE
|
||||
from .allocine import AllocineIE
|
||||
from .aliexpress import AliExpressLiveIE
|
||||
@@ -632,10 +634,7 @@ from .massengeschmacktv import MassengeschmackTVIE
|
||||
from .matchtv import MatchTVIE
|
||||
from .mdr import MDRIE
|
||||
from .mediaset import MediasetIE
|
||||
from .mediasite import (
|
||||
MediasiteIE,
|
||||
MediasiteCatalogIE,
|
||||
)
|
||||
from .mediasite import MediasiteIE
|
||||
from .medici import MediciIE
|
||||
from .megaphone import MegaphoneIE
|
||||
from .meipai import MeipaiIE
|
||||
@@ -902,6 +901,7 @@ from .puhutv import (
|
||||
PuhuTVSerieIE,
|
||||
)
|
||||
from .presstv import PressTVIE
|
||||
from .primesharetv import PrimeShareTVIE
|
||||
from .promptfile import PromptFileIE
|
||||
from .prosiebensat1 import ProSiebenSat1IE
|
||||
from .puls4 import Puls4IE
|
||||
@@ -978,6 +978,7 @@ from .rtvnh import RTVNHIE
|
||||
from .rtvs import RTVSIE
|
||||
from .rudo import RudoIE
|
||||
from .ruhd import RUHDIE
|
||||
from .ruleporn import RulePornIE
|
||||
from .rutube import (
|
||||
RutubeIE,
|
||||
RutubeChannelIE,
|
||||
@@ -1117,7 +1118,6 @@ from .teachertube import (
|
||||
)
|
||||
from .teachingchannel import TeachingChannelIE
|
||||
from .teamcoco import TeamcocoIE
|
||||
from .teamtreehouse import TeamTreeHouseIE
|
||||
from .techtalks import TechTalksIE
|
||||
from .ted import TEDIE
|
||||
from .tele5 import Tele5IE
|
||||
@@ -1299,6 +1299,7 @@ from .viddler import ViddlerIE
|
||||
from .videa import VideaIE
|
||||
from .videodetective import VideoDetectiveIE
|
||||
from .videofyme import VideofyMeIE
|
||||
from .videomega import VideoMegaIE
|
||||
from .videomore import (
|
||||
VideomoreIE,
|
||||
VideomoreVideoIE,
|
||||
@@ -1404,7 +1405,7 @@ from .webofstories import (
|
||||
WebOfStoriesPlaylistIE,
|
||||
)
|
||||
from .weibo import (
|
||||
WeiboIE,
|
||||
WeiboIE,
|
||||
WeiboMobileIE
|
||||
)
|
||||
from .weiqitv import WeiqiTVIE
|
||||
@@ -1448,13 +1449,12 @@ from .yahoo import (
|
||||
YahooIE,
|
||||
YahooSearchIE,
|
||||
)
|
||||
from .yandexdisk import YandexDiskIE
|
||||
from .yandexmusic import (
|
||||
YandexMusicTrackIE,
|
||||
YandexMusicAlbumIE,
|
||||
YandexMusicPlaylistIE,
|
||||
)
|
||||
from .yandexvideo import YandexVideoIE
|
||||
from .yandexdisk import YandexDiskIE
|
||||
from .yapfiles import YapFilesIE
|
||||
from .yesjapan import YesJapanIE
|
||||
from .yinyuetai import YinYueTaiIE
|
||||
|
||||
@@ -6,12 +6,10 @@ import uuid
|
||||
|
||||
from .adobepass import AdobePassIE
|
||||
from ..compat import (
|
||||
compat_HTTPError,
|
||||
compat_str,
|
||||
compat_urllib_parse_unquote,
|
||||
)
|
||||
from ..utils import (
|
||||
ExtractorError,
|
||||
int_or_none,
|
||||
parse_age_limit,
|
||||
parse_duration,
|
||||
@@ -50,7 +48,6 @@ class FOXIE(AdobePassIE):
|
||||
'url': 'https://www.fox.com/watch/30056b295fb57f7452aeeb4920bc3024/',
|
||||
'only_matching': True,
|
||||
}]
|
||||
_GEO_BYPASS = False
|
||||
_HOME_PAGE_URL = 'https://www.fox.com/'
|
||||
_API_KEY = 'abdcbed02c124d393b39e818a4312055'
|
||||
_access_token = None
|
||||
@@ -61,22 +58,9 @@ class FOXIE(AdobePassIE):
|
||||
}
|
||||
if self._access_token:
|
||||
headers['Authorization'] = 'Bearer ' + self._access_token
|
||||
try:
|
||||
return self._download_json(
|
||||
'https://api2.fox.com/v2.0/' + path,
|
||||
video_id, data=data, headers=headers)
|
||||
except ExtractorError as e:
|
||||
if isinstance(e.cause, compat_HTTPError) and e.cause.status == 403:
|
||||
entitlement_issues = self._parse_json(
|
||||
e.cause.read().decode(), video_id)['entitlementIssues']
|
||||
for e in entitlement_issues:
|
||||
if e.get('errorCode') == 1005:
|
||||
raise ExtractorError(
|
||||
'This video is only available via cable service provider '
|
||||
'subscription. You may want to use --cookies.', expected=True)
|
||||
messages = ', '.join([e['message'] for e in entitlement_issues])
|
||||
raise ExtractorError(messages, expected=True)
|
||||
raise
|
||||
return self._download_json(
|
||||
'https://api2.fox.com/v2.0/' + path,
|
||||
video_id, data=data, headers=headers)
|
||||
|
||||
def _real_initialize(self):
|
||||
if not self._access_token:
|
||||
@@ -97,15 +81,7 @@ class FOXIE(AdobePassIE):
|
||||
|
||||
title = video['name']
|
||||
release_url = video['url']
|
||||
try:
|
||||
m3u8_url = self._download_json(release_url, video_id)['playURL']
|
||||
except ExtractorError as e:
|
||||
if isinstance(e.cause, compat_HTTPError) and e.cause.status == 403:
|
||||
error = self._parse_json(e.cause.read().decode(), video_id)
|
||||
if error.get('exception') == 'GeoLocationBlocked':
|
||||
self.raise_geo_restricted(countries=['US'])
|
||||
raise ExtractorError(error['description'], expected=True)
|
||||
raise
|
||||
m3u8_url = self._download_json(release_url, video_id)['playURL']
|
||||
formats = self._extract_m3u8_formats(
|
||||
m3u8_url, video_id, 'mp4',
|
||||
entry_protocol='m3u8_native', m3u8_id='hls')
|
||||
|
||||
@@ -1,11 +1,18 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import json
|
||||
import time
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import int_or_none
|
||||
from ..compat import compat_urllib_parse_urlencode
|
||||
from ..utils import (
|
||||
ExtractorError,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
class HypemIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?hypem\.com/track/(?P<id>[0-9a-z]{5})'
|
||||
_VALID_URL = r'https?://(?:www\.)?hypem\.com/track/(?P<id>[^/]+)/'
|
||||
_TEST = {
|
||||
'url': 'http://hypem.com/track/1v6ga/BODYWORK+-+TAME',
|
||||
'md5': 'b9cc91b5af8995e9f0c1cee04c575828',
|
||||
@@ -14,36 +21,41 @@ class HypemIE(InfoExtractor):
|
||||
'ext': 'mp3',
|
||||
'title': 'Tame',
|
||||
'uploader': 'BODYWORK',
|
||||
'timestamp': 1371810457,
|
||||
'upload_date': '20130621',
|
||||
}
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
track_id = self._match_id(url)
|
||||
|
||||
response = self._download_webpage(url, track_id)
|
||||
data = {'ax': 1, 'ts': time.time()}
|
||||
request = sanitized_Request(url + '?' + compat_urllib_parse_urlencode(data))
|
||||
response, urlh = self._download_webpage_handle(
|
||||
request, track_id, 'Downloading webpage with the url')
|
||||
|
||||
track = self._parse_json(self._html_search_regex(
|
||||
r'(?s)<script\s+type="application/json"\s+id="displayList-data">(.+?)</script>',
|
||||
response, 'tracks'), track_id)['tracks'][0]
|
||||
html_tracks = self._html_search_regex(
|
||||
r'(?ms)<script type="application/json" id="displayList-data">(.+?)</script>',
|
||||
response, 'tracks')
|
||||
try:
|
||||
track_list = json.loads(html_tracks)
|
||||
track = track_list['tracks'][0]
|
||||
except ValueError:
|
||||
raise ExtractorError('Hypemachine contained invalid JSON.')
|
||||
|
||||
key = track['key']
|
||||
track_id = track['id']
|
||||
title = track['song']
|
||||
|
||||
final_url = self._download_json(
|
||||
'http://hypem.com/serve/source/%s/%s' % (track_id, track['key']),
|
||||
track_id, 'Downloading metadata', headers={
|
||||
'Content-Type': 'application/json'
|
||||
})['url']
|
||||
request = sanitized_Request(
|
||||
'http://hypem.com/serve/source/%s/%s' % (track_id, key),
|
||||
'', {'Content-Type': 'application/json'})
|
||||
song_data = self._download_json(request, track_id, 'Downloading metadata')
|
||||
final_url = song_data['url']
|
||||
artist = track.get('artist')
|
||||
|
||||
return {
|
||||
'id': track_id,
|
||||
'url': final_url,
|
||||
'ext': 'mp3',
|
||||
'title': title,
|
||||
'uploader': track.get('artist'),
|
||||
'duration': int_or_none(track.get('time')),
|
||||
'timestamp': int_or_none(track.get('ts')),
|
||||
'track': title,
|
||||
'uploader': artist,
|
||||
}
|
||||
|
||||
@@ -1,83 +1,36 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
determine_ext,
|
||||
int_or_none,
|
||||
strip_or_none,
|
||||
xpath_attr,
|
||||
xpath_text,
|
||||
)
|
||||
|
||||
|
||||
class InaIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?ina\.fr/(?:video|audio)/(?P<id>[A-Z0-9_]+)'
|
||||
_TESTS = [{
|
||||
_VALID_URL = r'https?://(?:www\.)?ina\.fr/video/(?P<id>I?[A-Z0-9]+)'
|
||||
_TEST = {
|
||||
'url': 'http://www.ina.fr/video/I12055569/francois-hollande-je-crois-que-c-est-clair-video.html',
|
||||
'md5': 'a667021bf2b41f8dc6049479d9bb38a3',
|
||||
'info_dict': {
|
||||
'id': 'I12055569',
|
||||
'ext': 'mp4',
|
||||
'title': 'François Hollande "Je crois que c\'est clair"',
|
||||
'description': 'md5:3f09eb072a06cb286b8f7e4f77109663',
|
||||
}
|
||||
}, {
|
||||
'url': 'https://www.ina.fr/video/S806544_001/don-d-organes-des-avancees-mais-d-importants-besoins-video.html',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://www.ina.fr/audio/P16173408',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://www.ina.fr/video/P16173408-video.html',
|
||||
'only_matching': True,
|
||||
}]
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
info_doc = self._download_xml(
|
||||
'http://player.ina.fr/notices/%s.mrss' % video_id, video_id)
|
||||
item = info_doc.find('channel/item')
|
||||
title = xpath_text(item, 'title', fatal=True)
|
||||
media_ns_xpath = lambda x: self._xpath_ns(x, 'http://search.yahoo.com/mrss/')
|
||||
content = item.find(media_ns_xpath('content'))
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
|
||||
get_furl = lambda x: xpath_attr(content, media_ns_xpath(x), 'url')
|
||||
formats = []
|
||||
for q, w, h in (('bq', 400, 300), ('mq', 512, 384), ('hq', 768, 576)):
|
||||
q_url = get_furl(q)
|
||||
if not q_url:
|
||||
continue
|
||||
formats.append({
|
||||
'format_id': q,
|
||||
'url': q_url,
|
||||
'width': w,
|
||||
'height': h,
|
||||
})
|
||||
if not formats:
|
||||
furl = get_furl('player') or content.attrib['url']
|
||||
ext = determine_ext(furl)
|
||||
formats = [{
|
||||
'url': furl,
|
||||
'vcodec': 'none' if ext == 'mp3' else None,
|
||||
'ext': ext,
|
||||
}]
|
||||
video_id = mobj.group('id')
|
||||
mrss_url = 'http://player.ina.fr/notices/%s.mrss' % video_id
|
||||
info_doc = self._download_xml(mrss_url, video_id)
|
||||
|
||||
thumbnails = []
|
||||
for thumbnail in content.findall(media_ns_xpath('thumbnail')):
|
||||
thumbnail_url = thumbnail.get('url')
|
||||
if not thumbnail_url:
|
||||
continue
|
||||
thumbnails.append({
|
||||
'url': thumbnail_url,
|
||||
'height': int_or_none(thumbnail.get('height')),
|
||||
'width': int_or_none(thumbnail.get('width')),
|
||||
})
|
||||
self.report_extraction(video_id)
|
||||
|
||||
video_url = info_doc.find('.//{http://search.yahoo.com/mrss/}player').attrib['url']
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'formats': formats,
|
||||
'title': title,
|
||||
'description': strip_or_none(xpath_text(item, 'description')),
|
||||
'thumbnails': thumbnails,
|
||||
'url': video_url,
|
||||
'title': info_doc.find('.//title').text,
|
||||
}
|
||||
|
||||
@@ -13,8 +13,6 @@ from ..utils import (
|
||||
ExtractorError,
|
||||
float_or_none,
|
||||
mimetype2ext,
|
||||
str_or_none,
|
||||
try_get,
|
||||
unescapeHTML,
|
||||
unsmuggle_url,
|
||||
url_or_none,
|
||||
@@ -22,11 +20,8 @@ from ..utils import (
|
||||
)
|
||||
|
||||
|
||||
_ID_RE = r'[0-9a-f]{32,34}'
|
||||
|
||||
|
||||
class MediasiteIE(InfoExtractor):
|
||||
_VALID_URL = r'(?xi)https?://[^/]+/Mediasite/(?:Play|Showcase/(?:default|livebroadcast)/Presentation)/(?P<id>%s)(?P<query>\?[^#]+|)' % _ID_RE
|
||||
_VALID_URL = r'(?xi)https?://[^/]+/Mediasite/(?:Play|Showcase/(?:default|livebroadcast)/Presentation)/(?P<id>[0-9a-f]{32,34})(?P<query>\?[^#]+|)'
|
||||
_TESTS = [
|
||||
{
|
||||
'url': 'https://hitsmediaweb.h-its.org/mediasite/Play/2db6c271681e4f199af3c60d1f82869b1d',
|
||||
@@ -114,7 +109,7 @@ class MediasiteIE(InfoExtractor):
|
||||
return [
|
||||
unescapeHTML(mobj.group('url'))
|
||||
for mobj in re.finditer(
|
||||
r'(?xi)<iframe\b[^>]+\bsrc=(["\'])(?P<url>(?:(?:https?:)?//[^/]+)?/Mediasite/Play/%s(?:\?.*?)?)\1' % _ID_RE,
|
||||
r'(?xi)<iframe\b[^>]+\bsrc=(["\'])(?P<url>(?:(?:https?:)?//[^/]+)?/Mediasite/Play/[0-9a-f]{32,34}(?:\?.*?)?)\1',
|
||||
webpage)]
|
||||
|
||||
def _real_extract(self, url):
|
||||
@@ -226,110 +221,3 @@ class MediasiteIE(InfoExtractor):
|
||||
'formats': formats,
|
||||
'thumbnails': thumbnails,
|
||||
}
|
||||
|
||||
|
||||
class MediasiteCatalogIE(InfoExtractor):
|
||||
_VALID_URL = r'''(?xi)
|
||||
(?P<url>https?://[^/]+/Mediasite)
|
||||
/Catalog/Full/
|
||||
(?P<catalog_id>{0})
|
||||
(?:
|
||||
/(?P<current_folder_id>{0})
|
||||
/(?P<root_dynamic_folder_id>{0})
|
||||
)?
|
||||
'''.format(_ID_RE)
|
||||
_TESTS = [{
|
||||
'url': 'http://events7.mediasite.com/Mediasite/Catalog/Full/631f9e48530d454381549f955d08c75e21',
|
||||
'info_dict': {
|
||||
'id': '631f9e48530d454381549f955d08c75e21',
|
||||
'title': 'WCET Summit: Adaptive Learning in Higher Ed: Improving Outcomes Dynamically',
|
||||
},
|
||||
'playlist_count': 6,
|
||||
'expected_warnings': ['is not a supported codec'],
|
||||
}, {
|
||||
# with CurrentFolderId and RootDynamicFolderId
|
||||
'url': 'https://medaudio.medicine.iu.edu/Mediasite/Catalog/Full/9518c4a6c5cf4993b21cbd53e828a92521/97a9db45f7ab47428c77cd2ed74bb98f14/9518c4a6c5cf4993b21cbd53e828a92521',
|
||||
'info_dict': {
|
||||
'id': '9518c4a6c5cf4993b21cbd53e828a92521',
|
||||
'title': 'IUSM Family and Friends Sessions',
|
||||
},
|
||||
'playlist_count': 2,
|
||||
}, {
|
||||
'url': 'http://uipsyc.mediasite.com/mediasite/Catalog/Full/d5d79287c75243c58c50fef50174ec1b21',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
# no AntiForgeryToken
|
||||
'url': 'https://live.libraries.psu.edu/Mediasite/Catalog/Full/8376d4b24dd1457ea3bfe4cf9163feda21',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://medaudio.medicine.iu.edu/Mediasite/Catalog/Full/9518c4a6c5cf4993b21cbd53e828a92521/97a9db45f7ab47428c77cd2ed74bb98f14/9518c4a6c5cf4993b21cbd53e828a92521',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
mediasite_url = mobj.group('url')
|
||||
catalog_id = mobj.group('catalog_id')
|
||||
current_folder_id = mobj.group('current_folder_id') or catalog_id
|
||||
root_dynamic_folder_id = mobj.group('root_dynamic_folder_id')
|
||||
|
||||
webpage = self._download_webpage(url, catalog_id)
|
||||
|
||||
# AntiForgeryToken is optional (e.g. [1])
|
||||
# 1. https://live.libraries.psu.edu/Mediasite/Catalog/Full/8376d4b24dd1457ea3bfe4cf9163feda21
|
||||
anti_forgery_token = self._search_regex(
|
||||
r'AntiForgeryToken\s*:\s*(["\'])(?P<value>(?:(?!\1).)+)\1',
|
||||
webpage, 'anti forgery token', default=None, group='value')
|
||||
if anti_forgery_token:
|
||||
anti_forgery_header = self._search_regex(
|
||||
r'AntiForgeryHeaderName\s*:\s*(["\'])(?P<value>(?:(?!\1).)+)\1',
|
||||
webpage, 'anti forgery header name',
|
||||
default='X-SOFO-AntiForgeryHeader', group='value')
|
||||
|
||||
data = {
|
||||
'IsViewPage': True,
|
||||
'IsNewFolder': True,
|
||||
'AuthTicket': None,
|
||||
'CatalogId': catalog_id,
|
||||
'CurrentFolderId': current_folder_id,
|
||||
'RootDynamicFolderId': root_dynamic_folder_id,
|
||||
'ItemsPerPage': 1000,
|
||||
'PageIndex': 0,
|
||||
'PermissionMask': 'Execute',
|
||||
'CatalogSearchType': 'SearchInFolder',
|
||||
'SortBy': 'Date',
|
||||
'SortDirection': 'Descending',
|
||||
'StartDate': None,
|
||||
'EndDate': None,
|
||||
'StatusFilterList': None,
|
||||
'PreviewKey': None,
|
||||
'Tags': [],
|
||||
}
|
||||
|
||||
headers = {
|
||||
'Content-Type': 'application/json; charset=UTF-8',
|
||||
'Referer': url,
|
||||
'X-Requested-With': 'XMLHttpRequest',
|
||||
}
|
||||
if anti_forgery_token:
|
||||
headers[anti_forgery_header] = anti_forgery_token
|
||||
|
||||
catalog = self._download_json(
|
||||
'%s/Catalog/Data/GetPresentationsForFolder' % mediasite_url,
|
||||
catalog_id, data=json.dumps(data).encode(), headers=headers)
|
||||
|
||||
entries = []
|
||||
for video in catalog['PresentationDetailsList']:
|
||||
if not isinstance(video, dict):
|
||||
continue
|
||||
video_id = str_or_none(video.get('Id'))
|
||||
if not video_id:
|
||||
continue
|
||||
entries.append(self.url_result(
|
||||
'%s/Play/%s' % (mediasite_url, video_id),
|
||||
ie=MediasiteIE.ie_key(), video_id=video_id))
|
||||
|
||||
title = try_get(
|
||||
catalog, lambda x: x['CurrentFolder']['Name'], compat_str)
|
||||
|
||||
return self.playlist_result(entries, catalog_id, title,)
|
||||
|
||||
@@ -1,12 +1,15 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import json
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
clean_html,
|
||||
ExtractorError,
|
||||
int_or_none,
|
||||
sanitized_Request,
|
||||
urlencode_postdata,
|
||||
)
|
||||
|
||||
|
||||
@@ -14,8 +17,8 @@ class MoeVideoIE(InfoExtractor):
|
||||
IE_DESC = 'LetitBit video services: moevideo.net, playreplay.net and videochart.net'
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://(?P<host>(?:www\.)?
|
||||
(?:(?:moevideo|playreplay|videochart)\.net|thesame\.tv))/
|
||||
(?:video|framevideo|embed)/(?P<id>[0-9a-z]+\.[0-9A-Za-z]+)'''
|
||||
(?:(?:moevideo|playreplay|videochart)\.net))/
|
||||
(?:video|framevideo)/(?P<id>[0-9]+\.[0-9A-Za-z]+)'''
|
||||
_API_URL = 'http://api.letitbit.net/'
|
||||
_API_KEY = 'tVL0gjqo5'
|
||||
_TESTS = [
|
||||
@@ -54,26 +57,58 @@ class MoeVideoIE(InfoExtractor):
|
||||
]
|
||||
|
||||
def _real_extract(self, url):
|
||||
host, video_id = re.match(self._VALID_URL, url).groups()
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
video_id = mobj.group('id')
|
||||
|
||||
webpage = self._download_webpage(
|
||||
'http://%s/video/%s' % (host, video_id),
|
||||
'http://%s/video/%s' % (mobj.group('host'), video_id),
|
||||
video_id, 'Downloading webpage')
|
||||
|
||||
title = self._og_search_title(webpage)
|
||||
thumbnail = self._og_search_thumbnail(webpage)
|
||||
description = self._og_search_description(webpage)
|
||||
|
||||
embed_webpage = self._download_webpage(
|
||||
'http://%s/embed/%s' % (host, video_id),
|
||||
video_id, 'Downloading embed webpage')
|
||||
video = self._parse_json(self._search_regex(
|
||||
r'mvplayer\("#player"\s*,\s*({.+})',
|
||||
embed_webpage, 'mvplayer'), video_id)['video']
|
||||
r = [
|
||||
self._API_KEY,
|
||||
[
|
||||
'preview/flv_link',
|
||||
{
|
||||
'uid': video_id,
|
||||
},
|
||||
],
|
||||
]
|
||||
r_json = json.dumps(r)
|
||||
post = urlencode_postdata({'r': r_json})
|
||||
req = sanitized_Request(self._API_URL, post)
|
||||
req.add_header('Content-type', 'application/x-www-form-urlencoded')
|
||||
|
||||
response = self._download_json(req, video_id)
|
||||
if response['status'] != 'OK':
|
||||
raise ExtractorError(
|
||||
'%s returned error: %s' % (self.IE_NAME, response['data']),
|
||||
expected=True
|
||||
)
|
||||
item = response['data'][0]
|
||||
video_url = item['link']
|
||||
duration = int_or_none(item['length'])
|
||||
width = int_or_none(item['width'])
|
||||
height = int_or_none(item['height'])
|
||||
filesize = int_or_none(item['convert_size'])
|
||||
|
||||
formats = [{
|
||||
'format_id': 'sd',
|
||||
'http_headers': {'Range': 'bytes=0-'}, # Required to download
|
||||
'url': video_url,
|
||||
'width': width,
|
||||
'height': height,
|
||||
'filesize': filesize,
|
||||
}]
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'thumbnail': video.get('poster') or self._og_search_thumbnail(webpage),
|
||||
'description': clean_html(self._og_search_description(webpage)),
|
||||
'duration': int_or_none(self._og_search_property('video:duration', webpage)),
|
||||
'url': video['ourUrl'],
|
||||
'thumbnail': thumbnail,
|
||||
'description': description,
|
||||
'duration': duration,
|
||||
'formats': formats,
|
||||
}
|
||||
|
||||
@@ -181,7 +181,10 @@ class NPOIE(NPOBaseIE):
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
return self._get_info(url, video_id) or self._get_old_info(video_id)
|
||||
try:
|
||||
return self._get_info(url, video_id)
|
||||
except ExtractorError:
|
||||
return self._get_old_info(video_id)
|
||||
|
||||
def _get_info(self, url, video_id):
|
||||
token = self._download_json(
|
||||
@@ -203,7 +206,6 @@ class NPOIE(NPOBaseIE):
|
||||
|
||||
player_token = player['token']
|
||||
|
||||
drm = False
|
||||
format_urls = set()
|
||||
formats = []
|
||||
for profile in ('hls', 'dash-widevine', 'dash-playready', 'smooth'):
|
||||
@@ -225,8 +227,7 @@ class NPOIE(NPOBaseIE):
|
||||
if not stream_url or stream_url in format_urls:
|
||||
continue
|
||||
format_urls.add(stream_url)
|
||||
if stream.get('protection') is not None or stream.get('keySystemOptions') is not None:
|
||||
drm = True
|
||||
if stream.get('protection') is not None:
|
||||
continue
|
||||
stream_type = stream.get('type')
|
||||
stream_ext = determine_ext(stream_url)
|
||||
@@ -245,11 +246,6 @@ class NPOIE(NPOBaseIE):
|
||||
'url': stream_url,
|
||||
})
|
||||
|
||||
if not formats:
|
||||
if drm:
|
||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||
return
|
||||
|
||||
self._sort_formats(formats)
|
||||
|
||||
info = {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..compat import compat_urllib_parse_urlencode
|
||||
from ..utils import (
|
||||
int_or_none,
|
||||
qualities,
|
||||
@@ -8,16 +9,16 @@ from ..utils import (
|
||||
|
||||
|
||||
class NprIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?npr\.org/(?:sections/[^/]+/)?\d{4}/\d{2}/\d{2}/(?P<id>\d+)'
|
||||
_VALID_URL = r'https?://(?:www\.)?npr\.org/player/v2/mediaPlayer\.html\?.*\bid=(?P<id>\d+)'
|
||||
_TESTS = [{
|
||||
'url': 'https://www.npr.org/sections/allsongs/2015/10/21/449974205/new-music-from-beach-house-chairlift-cmj-discoveries-and-more',
|
||||
'url': 'http://www.npr.org/player/v2/mediaPlayer.html?id=449974205',
|
||||
'info_dict': {
|
||||
'id': '449974205',
|
||||
'title': 'New Music From Beach House, Chairlift, CMJ Discoveries And More'
|
||||
},
|
||||
'playlist_count': 7,
|
||||
}, {
|
||||
'url': 'https://www.npr.org/sections/deceptivecadence/2015/10/09/446928052/music-from-the-shadows-ancient-armenian-hymns-and-piano-jazz',
|
||||
'url': 'http://www.npr.org/player/v2/mediaPlayer.html?action=1&t=1&islist=false&id=446928052&m=446929930&live=1',
|
||||
'info_dict': {
|
||||
'id': '446928052',
|
||||
'title': "Songs We Love: Tigran Hamasyan, 'Your Mercy is Boundless'"
|
||||
@@ -31,46 +32,30 @@ class NprIE(InfoExtractor):
|
||||
'duration': 402,
|
||||
},
|
||||
}],
|
||||
}, {
|
||||
# mutlimedia, not media title
|
||||
'url': 'https://www.npr.org/2017/06/19/533198237/tigers-jaw-tiny-desk-concert',
|
||||
'info_dict': {
|
||||
'id': '533198237',
|
||||
'title': 'Tigers Jaw: Tiny Desk Concert',
|
||||
},
|
||||
'playlist': [{
|
||||
'md5': '12fa60cb2d3ed932f53609d4aeceabf1',
|
||||
'info_dict': {
|
||||
'id': '533201718',
|
||||
'ext': 'mp4',
|
||||
'title': 'Tigers Jaw: Tiny Desk Concert',
|
||||
'duration': 402,
|
||||
},
|
||||
}],
|
||||
'expected_warnings': ['Failed to download m3u8 information'],
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
playlist_id = self._match_id(url)
|
||||
|
||||
story = self._download_json(
|
||||
'http://api.npr.org/query', playlist_id, query={
|
||||
config = self._download_json(
|
||||
'http://api.npr.org/query?%s' % compat_urllib_parse_urlencode({
|
||||
'id': playlist_id,
|
||||
'fields': 'audio,multimedia,title',
|
||||
'fields': 'titles,audio,show',
|
||||
'format': 'json',
|
||||
'apiKey': 'MDAzMzQ2MjAyMDEyMzk4MTU1MDg3ZmM3MQ010',
|
||||
})['list']['story'][0]
|
||||
playlist_title = story.get('title', {}).get('$text')
|
||||
}), playlist_id)
|
||||
|
||||
KNOWN_FORMATS = ('threegp', 'm3u8', 'smil', 'mp4', 'mp3')
|
||||
story = config['list']['story'][0]
|
||||
|
||||
KNOWN_FORMATS = ('threegp', 'mp4', 'mp3')
|
||||
quality = qualities(KNOWN_FORMATS)
|
||||
|
||||
entries = []
|
||||
for media in story.get('audio', []) + story.get('multimedia', []):
|
||||
media_id = media['id']
|
||||
|
||||
for audio in story.get('audio', []):
|
||||
title = audio.get('title', {}).get('$text')
|
||||
duration = int_or_none(audio.get('duration', {}).get('$text'))
|
||||
formats = []
|
||||
for format_id, formats_entry in media.get('format', {}).items():
|
||||
for format_id, formats_entry in audio.get('format', {}).items():
|
||||
if not formats_entry:
|
||||
continue
|
||||
if isinstance(formats_entry, list):
|
||||
@@ -79,30 +64,19 @@ class NprIE(InfoExtractor):
|
||||
if not format_url:
|
||||
continue
|
||||
if format_id in KNOWN_FORMATS:
|
||||
if format_id == 'm3u8':
|
||||
formats.extend(self._extract_m3u8_formats(
|
||||
format_url, media_id, 'mp4', 'm3u8_native',
|
||||
m3u8_id='hls', fatal=False))
|
||||
elif format_id == 'smil':
|
||||
smil_formats = self._extract_smil_formats(
|
||||
format_url, media_id, transform_source=lambda s: s.replace(
|
||||
'rtmp://flash.npr.org/ondemand/', 'https://ondemand.npr.org/'))
|
||||
self._check_formats(smil_formats, media_id)
|
||||
formats.extend(smil_formats)
|
||||
else:
|
||||
formats.append({
|
||||
'url': format_url,
|
||||
'format_id': format_id,
|
||||
'quality': quality(format_id),
|
||||
})
|
||||
formats.append({
|
||||
'url': format_url,
|
||||
'format_id': format_id,
|
||||
'ext': formats_entry.get('type'),
|
||||
'quality': quality(format_id),
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
|
||||
entries.append({
|
||||
'id': media_id,
|
||||
'title': media.get('title', {}).get('$text') or playlist_title,
|
||||
'thumbnail': media.get('altImageUrl', {}).get('$text'),
|
||||
'duration': int_or_none(media.get('duration', {}).get('$text')),
|
||||
'id': audio['id'],
|
||||
'title': title,
|
||||
'duration': duration,
|
||||
'formats': formats,
|
||||
})
|
||||
|
||||
playlist_title = story.get('title', {}).get('$text')
|
||||
return self.playlist_result(entries, playlist_id, playlist_title)
|
||||
|
||||
@@ -243,16 +243,18 @@ class PhantomJSwrapper(object):
|
||||
|
||||
|
||||
class OpenloadIE(InfoExtractor):
|
||||
_DOMAINS = r'(?:openload\.(?:co|io|link|pw)|oload\.(?:tv|stream|site|xyz|win|download|cloud|cc|icu|fun|club|info|pw|live|space))'
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://
|
||||
(?P<host>
|
||||
(?:www\.)?
|
||||
%s
|
||||
(?:
|
||||
openload\.(?:co|io|link|pw)|
|
||||
oload\.(?:tv|stream|site|xyz|win|download|cloud|cc|icu|fun|club|info|pw|live|space)
|
||||
)
|
||||
)/
|
||||
(?:f|embed)/
|
||||
(?P<id>[a-zA-Z0-9-_]+)
|
||||
''' % _DOMAINS
|
||||
'''
|
||||
|
||||
_TESTS = [{
|
||||
'url': 'https://openload.co/f/kUEfGclsU9o',
|
||||
@@ -357,8 +359,8 @@ class OpenloadIE(InfoExtractor):
|
||||
@staticmethod
|
||||
def _extract_urls(webpage):
|
||||
return re.findall(
|
||||
r'<iframe[^>]+src=["\']((?:https?://)?%s/embed/[a-zA-Z0-9-_]+)'
|
||||
% OpenloadIE._DOMAINS, webpage)
|
||||
r'<iframe[^>]+src=["\']((?:https?://)?(?:openload\.(?:co|io)|oload\.tv)/embed/[a-zA-Z0-9-_]+)',
|
||||
webpage)
|
||||
|
||||
def _real_extract(self, url):
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
|
||||
@@ -176,8 +176,7 @@ class ORFRadioIE(InfoExtractor):
|
||||
'description': subtitle,
|
||||
'duration': (info['end'] - info['start']) / 1000,
|
||||
'timestamp': info['start'] / 1000,
|
||||
'ext': 'mp3',
|
||||
'series': data.get('programTitle')
|
||||
'ext': 'mp3'
|
||||
}
|
||||
|
||||
entries = [extract_entry_dict(t, data['title'], data['subtitle']) for t in data['streams']]
|
||||
|
||||
@@ -36,7 +36,7 @@ class PandaTVIE(InfoExtractor):
|
||||
'https://www.panda.tv/api_room_v2?roomid=%s' % video_id, video_id)
|
||||
|
||||
error_code = config.get('errno', 0)
|
||||
if error_code != 0:
|
||||
if error_code is not 0:
|
||||
raise ExtractorError(
|
||||
'%s returned error %s: %s'
|
||||
% (self.IE_NAME, error_code, config['errmsg']),
|
||||
|
||||
@@ -14,7 +14,6 @@ from ..compat import (
|
||||
)
|
||||
from .openload import PhantomJSwrapper
|
||||
from ..utils import (
|
||||
determine_ext,
|
||||
ExtractorError,
|
||||
int_or_none,
|
||||
orderedSet,
|
||||
@@ -276,10 +275,6 @@ class PornHubIE(PornHubBaseIE):
|
||||
r'/(\d{6}/\d{2})/', video_url, 'upload data', default=None)
|
||||
if upload_date:
|
||||
upload_date = upload_date.replace('/', '')
|
||||
if determine_ext(video_url) == 'mpd':
|
||||
formats.extend(self._extract_mpd_formats(
|
||||
video_url, video_id, mpd_id='dash', fatal=False))
|
||||
continue
|
||||
tbr = None
|
||||
mobj = re.search(r'(?P<height>\d+)[pP]?_(?P<tbr>\d+)[kK]', video_url)
|
||||
if mobj:
|
||||
|
||||
62
youtube_dl/extractor/primesharetv.py
Normal file
62
youtube_dl/extractor/primesharetv.py
Normal file
@@ -0,0 +1,62 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
ExtractorError,
|
||||
sanitized_Request,
|
||||
urlencode_postdata,
|
||||
)
|
||||
|
||||
|
||||
class PrimeShareTVIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?primeshare\.tv/download/(?P<id>[\da-zA-Z]+)'
|
||||
|
||||
_TEST = {
|
||||
'url': 'http://primeshare.tv/download/238790B611',
|
||||
'md5': 'b92d9bf5461137c36228009f31533fbc',
|
||||
'info_dict': {
|
||||
'id': '238790B611',
|
||||
'ext': 'mp4',
|
||||
'title': 'Public Domain - 1960s Commercial - Crest Toothpaste-YKsuFona',
|
||||
},
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
|
||||
if '>File not exist<' in webpage:
|
||||
raise ExtractorError('Video %s does not exist' % video_id, expected=True)
|
||||
|
||||
fields = self._hidden_inputs(webpage)
|
||||
|
||||
headers = {
|
||||
'Referer': url,
|
||||
'Content-Type': 'application/x-www-form-urlencoded',
|
||||
}
|
||||
|
||||
wait_time = int(self._search_regex(
|
||||
r'var\s+cWaitTime\s*=\s*(\d+)',
|
||||
webpage, 'wait time', default=7)) + 1
|
||||
self._sleep(wait_time, video_id)
|
||||
|
||||
req = sanitized_Request(
|
||||
url, urlencode_postdata(fields), headers)
|
||||
video_page = self._download_webpage(
|
||||
req, video_id, 'Downloading video page')
|
||||
|
||||
video_url = self._search_regex(
|
||||
r"url\s*:\s*'([^']+\.primeshare\.tv(?::443)?/file/[^']+)'",
|
||||
video_page, 'video url')
|
||||
|
||||
title = self._html_search_regex(
|
||||
r'<h1>Watch\s*(?: )?\s*\((.+?)(?:\s*\[\.\.\.\])?\)\s*(?: )?\s*<strong>',
|
||||
video_page, 'title')
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'url': video_url,
|
||||
'title': title,
|
||||
'ext': 'mp4',
|
||||
}
|
||||
@@ -147,7 +147,7 @@ class RadioCanadaIE(InfoExtractor):
|
||||
|
||||
|
||||
class RadioCanadaAudioVideoIE(InfoExtractor):
|
||||
IE_NAME = 'radiocanada:audiovideo'
|
||||
'radiocanada:audiovideo'
|
||||
_VALID_URL = r'https?://ici\.radio-canada\.ca/([^/]+/)*media-(?P<id>[0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://ici.radio-canada.ca/audio-video/media-7527184/barack-obama-au-vietnam',
|
||||
|
||||
44
youtube_dl/extractor/ruleporn.py
Normal file
44
youtube_dl/extractor/ruleporn.py
Normal file
@@ -0,0 +1,44 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .nuevo import NuevoBaseIE
|
||||
|
||||
|
||||
class RulePornIE(NuevoBaseIE):
|
||||
_VALID_URL = r'https?://(?:www\.)?ruleporn\.com/(?:[^/?#&]+/)*(?P<id>[^/?#&]+)'
|
||||
_TEST = {
|
||||
'url': 'http://ruleporn.com/brunette-nympho-chick-takes-her-boyfriend-in-every-angle/',
|
||||
'md5': '86861ebc624a1097c7c10eaf06d7d505',
|
||||
'info_dict': {
|
||||
'id': '48212',
|
||||
'display_id': 'brunette-nympho-chick-takes-her-boyfriend-in-every-angle',
|
||||
'ext': 'mp4',
|
||||
'title': 'Brunette Nympho Chick Takes Her Boyfriend In Every Angle',
|
||||
'description': 'md5:6d28be231b981fff1981deaaa03a04d5',
|
||||
'age_limit': 18,
|
||||
'duration': 635.1,
|
||||
}
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
display_id = self._match_id(url)
|
||||
|
||||
webpage = self._download_webpage(url, display_id)
|
||||
|
||||
video_id = self._search_regex(
|
||||
r'lovehomeporn\.com/embed/(\d+)', webpage, 'video id')
|
||||
|
||||
title = self._search_regex(
|
||||
r'<h2[^>]+title=(["\'])(?P<url>.+?)\1',
|
||||
webpage, 'title', group='url')
|
||||
description = self._html_search_meta('description', webpage)
|
||||
|
||||
info = self._extract_nuevo(
|
||||
'http://lovehomeporn.com/media/nuevo/econfig.php?key=%s&rp=true' % video_id,
|
||||
video_id)
|
||||
info.update({
|
||||
'display_id': display_id,
|
||||
'title': title,
|
||||
'description': description,
|
||||
'age_limit': 18
|
||||
})
|
||||
return info
|
||||
@@ -185,7 +185,7 @@ class SVTPlayIE(SVTPlayBaseIE):
|
||||
|
||||
def _extract_by_video_id(self, video_id, webpage=None):
|
||||
data = self._download_json(
|
||||
'https://api.svt.se/video/%s' % video_id,
|
||||
'https://api.svt.se/videoplayer-api/video/%s' % video_id,
|
||||
video_id, headers=self.geo_verification_headers())
|
||||
info_dict = self._extract_video(data, video_id)
|
||||
if not info_dict.get('title'):
|
||||
|
||||
@@ -1,140 +0,0 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
clean_html,
|
||||
determine_ext,
|
||||
ExtractorError,
|
||||
float_or_none,
|
||||
get_element_by_class,
|
||||
get_element_by_id,
|
||||
parse_duration,
|
||||
remove_end,
|
||||
urlencode_postdata,
|
||||
urljoin,
|
||||
)
|
||||
|
||||
|
||||
class TeamTreeHouseIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?teamtreehouse\.com/library/(?P<id>[^/]+)'
|
||||
_TESTS = [{
|
||||
# Course
|
||||
'url': 'https://teamtreehouse.com/library/introduction-to-user-authentication-in-php',
|
||||
'info_dict': {
|
||||
'id': 'introduction-to-user-authentication-in-php',
|
||||
'title': 'Introduction to User Authentication in PHP',
|
||||
'description': 'md5:405d7b4287a159b27ddf30ca72b5b053',
|
||||
},
|
||||
'playlist_mincount': 24,
|
||||
}, {
|
||||
# WorkShop
|
||||
'url': 'https://teamtreehouse.com/library/deploying-a-react-app',
|
||||
'info_dict': {
|
||||
'id': 'deploying-a-react-app',
|
||||
'title': 'Deploying a React App',
|
||||
'description': 'md5:10a82e3ddff18c14ac13581c9b8e5921',
|
||||
},
|
||||
'playlist_mincount': 4,
|
||||
}, {
|
||||
# Video
|
||||
'url': 'https://teamtreehouse.com/library/application-overview-2',
|
||||
'info_dict': {
|
||||
'id': 'application-overview-2',
|
||||
'ext': 'mp4',
|
||||
'title': 'Application Overview',
|
||||
'description': 'md5:4b0a234385c27140a4378de5f1e15127',
|
||||
},
|
||||
'expected_warnings': ['This is just a preview'],
|
||||
}]
|
||||
_NETRC_MACHINE = 'teamtreehouse'
|
||||
|
||||
def _real_initialize(self):
|
||||
email, password = self._get_login_info()
|
||||
if email is None:
|
||||
return
|
||||
|
||||
signin_page = self._download_webpage(
|
||||
'https://teamtreehouse.com/signin',
|
||||
None, 'Downloading signin page')
|
||||
data = self._form_hidden_inputs('new_user_session', signin_page)
|
||||
data.update({
|
||||
'user_session[email]': email,
|
||||
'user_session[password]': password,
|
||||
})
|
||||
error_message = get_element_by_class('error-message', self._download_webpage(
|
||||
'https://teamtreehouse.com/person_session',
|
||||
None, 'Logging in', data=urlencode_postdata(data)))
|
||||
if error_message:
|
||||
raise ExtractorError(clean_html(error_message), expected=True)
|
||||
|
||||
def _real_extract(self, url):
|
||||
display_id = self._match_id(url)
|
||||
webpage = self._download_webpage(url, display_id)
|
||||
title = self._html_search_meta(['og:title', 'twitter:title'], webpage)
|
||||
description = self._html_search_meta(
|
||||
['description', 'og:description', 'twitter:description'], webpage)
|
||||
entries = self._parse_html5_media_entries(url, webpage, display_id)
|
||||
if entries:
|
||||
info = entries[0]
|
||||
|
||||
for subtitles in info.get('subtitles', {}).values():
|
||||
for subtitle in subtitles:
|
||||
subtitle['ext'] = determine_ext(subtitle['url'], 'srt')
|
||||
|
||||
is_preview = 'data-preview="true"' in webpage
|
||||
if is_preview:
|
||||
self.report_warning(
|
||||
'This is just a preview. You need to be signed in with a Basic account to download the entire video.', display_id)
|
||||
duration = 30
|
||||
else:
|
||||
duration = float_or_none(self._search_regex(
|
||||
r'data-duration="(\d+)"', webpage, 'duration'), 1000)
|
||||
if not duration:
|
||||
duration = parse_duration(get_element_by_id(
|
||||
'video-duration', webpage))
|
||||
|
||||
info.update({
|
||||
'id': display_id,
|
||||
'title': title,
|
||||
'description': description,
|
||||
'duration': duration,
|
||||
})
|
||||
return info
|
||||
else:
|
||||
def extract_urls(html, extract_info=None):
|
||||
for path in re.findall(r'<a[^>]+href="([^"]+)"', html):
|
||||
page_url = urljoin(url, path)
|
||||
entry = {
|
||||
'_type': 'url_transparent',
|
||||
'id': self._match_id(page_url),
|
||||
'url': page_url,
|
||||
'id_key': self.ie_key(),
|
||||
}
|
||||
if extract_info:
|
||||
entry.update(extract_info)
|
||||
entries.append(entry)
|
||||
|
||||
workshop_videos = self._search_regex(
|
||||
r'(?s)<ul[^>]+id="workshop-videos"[^>]*>(.+?)</ul>',
|
||||
webpage, 'workshop videos', default=None)
|
||||
if workshop_videos:
|
||||
extract_urls(workshop_videos)
|
||||
else:
|
||||
stages_path = self._search_regex(
|
||||
r'(?s)<div[^>]+id="syllabus-stages"[^>]+data-url="([^"]+)"',
|
||||
webpage, 'stages path')
|
||||
if stages_path:
|
||||
stages_page = self._download_webpage(
|
||||
urljoin(url, stages_path), display_id, 'Downloading stages page')
|
||||
for chapter_number, (chapter, steps_list) in enumerate(re.findall(r'(?s)<h2[^>]*>\s*(.+?)\s*</h2>.+?<ul[^>]*>(.+?)</ul>', stages_page), 1):
|
||||
extract_urls(steps_list, {
|
||||
'chapter': chapter,
|
||||
'chapter_number': chapter_number,
|
||||
})
|
||||
title = remove_end(title, ' Course')
|
||||
|
||||
return self.playlist_result(
|
||||
entries, display_id, title, description)
|
||||
@@ -1,10 +1,13 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
import json
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
int_or_none,
|
||||
parse_duration,
|
||||
qualities,
|
||||
ExtractorError,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
@@ -13,9 +16,9 @@ class VeohIE(InfoExtractor):
|
||||
|
||||
_TESTS = [{
|
||||
'url': 'http://www.veoh.com/watch/v56314296nk7Zdmz3',
|
||||
'md5': '9e7ecc0fd8bbee7a69fe38953aeebd30',
|
||||
'md5': '620e68e6a3cff80086df3348426c9ca3',
|
||||
'info_dict': {
|
||||
'id': 'v56314296nk7Zdmz3',
|
||||
'id': '56314296',
|
||||
'ext': 'mp4',
|
||||
'title': 'Straight Backs Are Stronger',
|
||||
'uploader': 'LUMOback',
|
||||
@@ -53,6 +56,29 @@ class VeohIE(InfoExtractor):
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _extract_formats(self, source):
|
||||
formats = []
|
||||
link = source.get('aowPermalink')
|
||||
if link:
|
||||
formats.append({
|
||||
'url': link,
|
||||
'ext': 'mp4',
|
||||
'format_id': 'aow',
|
||||
})
|
||||
link = source.get('fullPreviewHashLowPath')
|
||||
if link:
|
||||
formats.append({
|
||||
'url': link,
|
||||
'format_id': 'low',
|
||||
})
|
||||
link = source.get('fullPreviewHashHighPath')
|
||||
if link:
|
||||
formats.append({
|
||||
'url': link,
|
||||
'format_id': 'high',
|
||||
})
|
||||
return formats
|
||||
|
||||
def _extract_video(self, source):
|
||||
return {
|
||||
'id': source.get('videoId'),
|
||||
@@ -67,37 +93,38 @@ class VeohIE(InfoExtractor):
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
video = self._download_json(
|
||||
'https://www.veoh.com/watch/getVideo/' + video_id,
|
||||
video_id)['video']
|
||||
title = video['title']
|
||||
mobj = re.match(self._VALID_URL, url)
|
||||
video_id = mobj.group('id')
|
||||
|
||||
thumbnail_url = None
|
||||
q = qualities(['HQ', 'Regular'])
|
||||
formats = []
|
||||
for f_id, f_url in video.get('src', {}).items():
|
||||
if not f_url:
|
||||
continue
|
||||
if f_id == 'poster':
|
||||
thumbnail_url = f_url
|
||||
else:
|
||||
formats.append({
|
||||
'format_id': f_id,
|
||||
'quality': q(f_id),
|
||||
'url': f_url,
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
if video_id.startswith('v'):
|
||||
rsp = self._download_xml(
|
||||
r'http://www.veoh.com/api/findByPermalink?permalink=%s' % video_id, video_id, 'Downloading video XML')
|
||||
stat = rsp.get('stat')
|
||||
if stat == 'ok':
|
||||
return self._extract_video(rsp.find('./videoList/video'))
|
||||
elif stat == 'fail':
|
||||
raise ExtractorError(
|
||||
'%s said: %s' % (self.IE_NAME, rsp.find('./errorList/error').get('errorMessage')), expected=True)
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'description': video.get('description'),
|
||||
'thumbnail': thumbnail_url,
|
||||
'uploader': video.get('author', {}).get('nickname'),
|
||||
'duration': int_or_none(video.get('lengthBySec')) or parse_duration(video.get('length')),
|
||||
'view_count': int_or_none(video.get('views')),
|
||||
'formats': formats,
|
||||
'average_rating': int_or_none(video.get('rating')),
|
||||
'comment_count': int_or_none(video.get('numOfComments')),
|
||||
}
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
age_limit = 0
|
||||
if 'class="adultwarning-container"' in webpage:
|
||||
self.report_age_confirmation()
|
||||
age_limit = 18
|
||||
request = sanitized_Request(url)
|
||||
request.add_header('Cookie', 'confirmedAdult=true')
|
||||
webpage = self._download_webpage(request, video_id)
|
||||
|
||||
m_youtube = re.search(r'http://www\.youtube\.com/v/(.*?)(\&|"|\?)', webpage)
|
||||
if m_youtube is not None:
|
||||
youtube_id = m_youtube.group(1)
|
||||
self.to_screen('%s: detected Youtube video.' % video_id)
|
||||
return self.url_result(youtube_id, 'Youtube')
|
||||
|
||||
info = json.loads(
|
||||
self._search_regex(r'videoDetailsJSON = \'({.*?})\';', webpage, 'info').replace('\\\'', '\''))
|
||||
|
||||
video = self._extract_video(info)
|
||||
video['age_limit'] = age_limit
|
||||
|
||||
return video
|
||||
|
||||
@@ -1,16 +1,19 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..compat import (
|
||||
compat_urllib_parse_urlencode,
|
||||
compat_urlparse,
|
||||
)
|
||||
from ..utils import (
|
||||
float_or_none,
|
||||
int_or_none,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
class ViddlerIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?viddler\.com/(?:v|embed|player)/(?P<id>[a-z0-9]+)(?:.+?\bsecret=(\d+))?'
|
||||
_VALID_URL = r'https?://(?:www\.)?viddler\.com/(?:v|embed|player)/(?P<id>[a-z0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.viddler.com/v/43903784',
|
||||
'md5': '9eee21161d2c7f5b39690c3e325fab2f',
|
||||
@@ -75,18 +78,23 @@ class ViddlerIE(InfoExtractor):
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id, secret = re.match(self._VALID_URL, url).groups()
|
||||
video_id = self._match_id(url)
|
||||
|
||||
query = {
|
||||
'video_id': video_id,
|
||||
'key': 'v0vhrt7bg2xq1vyxhkct',
|
||||
}
|
||||
|
||||
qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
|
||||
secret = qs.get('secret', [None])[0]
|
||||
if secret:
|
||||
query['secret'] = secret
|
||||
|
||||
data = self._download_json(
|
||||
'http://api.viddler.com/api/v2/viddler.videos.getPlaybackDetails.json',
|
||||
video_id, headers={'Referer': url}, query=query)['video']
|
||||
headers = {'Referer': 'http://static.cdn-ec.viddler.com/js/arpeggio/v2/embed.html'}
|
||||
request = sanitized_Request(
|
||||
'http://api.viddler.com/api/v2/viddler.videos.getPlaybackDetails.json?%s'
|
||||
% compat_urllib_parse_urlencode(query), None, headers)
|
||||
data = self._download_json(request, video_id)['video']
|
||||
|
||||
formats = []
|
||||
for filed in data['files']:
|
||||
|
||||
60
youtube_dl/extractor/videomega.py
Normal file
60
youtube_dl/extractor/videomega.py
Normal file
@@ -0,0 +1,60 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
decode_packed_codes,
|
||||
sanitized_Request,
|
||||
)
|
||||
|
||||
|
||||
class VideoMegaIE(InfoExtractor):
|
||||
_VALID_URL = r'(?:videomega:|https?://(?:www\.)?videomega\.tv/(?:(?:view|iframe|cdn)\.php)?\?ref=)(?P<id>[A-Za-z0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://videomega.tv/cdn.php?ref=AOSQBJYKIDDIKYJBQSOA',
|
||||
'md5': 'cc1920a58add3f05c6a93285b84fb3aa',
|
||||
'info_dict': {
|
||||
'id': 'AOSQBJYKIDDIKYJBQSOA',
|
||||
'ext': 'mp4',
|
||||
'title': '1254207',
|
||||
'thumbnail': r're:^https?://.*\.jpg$',
|
||||
}
|
||||
}, {
|
||||
'url': 'http://videomega.tv/cdn.php?ref=AOSQBJYKIDDIKYJBQSOA&width=1070&height=600',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'http://videomega.tv/view.php?ref=090051111052065112106089103052052103089106112065052111051090',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
iframe_url = 'http://videomega.tv/cdn.php?ref=%s' % video_id
|
||||
req = sanitized_Request(iframe_url)
|
||||
req.add_header('Referer', url)
|
||||
req.add_header('Cookie', 'noadvtday=0')
|
||||
webpage = self._download_webpage(req, video_id)
|
||||
|
||||
title = self._html_search_regex(
|
||||
r'<title>(.+?)</title>', webpage, 'title')
|
||||
title = re.sub(
|
||||
r'(?:^[Vv]ideo[Mm]ega\.tv\s-\s*|\s*-\svideomega\.tv$)', '', title)
|
||||
thumbnail = self._search_regex(
|
||||
r'<video[^>]+?poster="([^"]+)"', webpage, 'thumbnail', fatal=False)
|
||||
|
||||
real_codes = decode_packed_codes(webpage)
|
||||
video_url = self._search_regex(
|
||||
r'"src"\s*,\s*"([^"]+)"', real_codes, 'video URL')
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'url': video_url,
|
||||
'thumbnail': thumbnail,
|
||||
'http_headers': {
|
||||
'Referer': iframe_url,
|
||||
},
|
||||
}
|
||||
@@ -195,32 +195,6 @@ class VimeoBaseInfoExtractor(InfoExtractor):
|
||||
'subtitles': subtitles,
|
||||
}
|
||||
|
||||
def _extract_original_format(self, url, video_id):
|
||||
download_data = self._download_json(
|
||||
url, video_id, fatal=False,
|
||||
query={'action': 'load_download_config'},
|
||||
headers={'X-Requested-With': 'XMLHttpRequest'})
|
||||
if download_data:
|
||||
source_file = download_data.get('source_file')
|
||||
if isinstance(source_file, dict):
|
||||
download_url = source_file.get('download_url')
|
||||
if download_url and not source_file.get('is_cold') and not source_file.get('is_defrosting'):
|
||||
source_name = source_file.get('public_name', 'Original')
|
||||
if self._is_valid_url(download_url, video_id, '%s video' % source_name):
|
||||
ext = (try_get(
|
||||
source_file, lambda x: x['extension'],
|
||||
compat_str) or determine_ext(
|
||||
download_url, None) or 'mp4').lower()
|
||||
return {
|
||||
'url': download_url,
|
||||
'ext': ext,
|
||||
'width': int_or_none(source_file.get('width')),
|
||||
'height': int_or_none(source_file.get('height')),
|
||||
'filesize': parse_filesize(source_file.get('size')),
|
||||
'format_id': source_name,
|
||||
'preference': 1,
|
||||
}
|
||||
|
||||
|
||||
class VimeoIE(VimeoBaseInfoExtractor):
|
||||
"""Information extractor for vimeo.com."""
|
||||
@@ -685,11 +659,29 @@ class VimeoIE(VimeoBaseInfoExtractor):
|
||||
comment_count = None
|
||||
|
||||
formats = []
|
||||
|
||||
source_format = self._extract_original_format(
|
||||
'https://vimeo.com/' + video_id, video_id)
|
||||
if source_format:
|
||||
formats.append(source_format)
|
||||
download_request = sanitized_Request('https://vimeo.com/%s?action=load_download_config' % video_id, headers={
|
||||
'X-Requested-With': 'XMLHttpRequest'})
|
||||
download_data = self._download_json(download_request, video_id, fatal=False)
|
||||
if download_data:
|
||||
source_file = download_data.get('source_file')
|
||||
if isinstance(source_file, dict):
|
||||
download_url = source_file.get('download_url')
|
||||
if download_url and not source_file.get('is_cold') and not source_file.get('is_defrosting'):
|
||||
source_name = source_file.get('public_name', 'Original')
|
||||
if self._is_valid_url(download_url, video_id, '%s video' % source_name):
|
||||
ext = (try_get(
|
||||
source_file, lambda x: x['extension'],
|
||||
compat_str) or determine_ext(
|
||||
download_url, None) or 'mp4').lower()
|
||||
formats.append({
|
||||
'url': download_url,
|
||||
'ext': ext,
|
||||
'width': int_or_none(source_file.get('width')),
|
||||
'height': int_or_none(source_file.get('height')),
|
||||
'filesize': parse_filesize(source_file.get('size')),
|
||||
'format_id': source_name,
|
||||
'preference': 1,
|
||||
})
|
||||
|
||||
info_dict_config = self._parse_config(config, video_id)
|
||||
formats.extend(info_dict_config['formats'])
|
||||
@@ -948,7 +940,7 @@ class VimeoGroupsIE(VimeoAlbumIE):
|
||||
class VimeoReviewIE(VimeoBaseInfoExtractor):
|
||||
IE_NAME = 'vimeo:review'
|
||||
IE_DESC = 'Review pages on vimeo'
|
||||
_VALID_URL = r'(?P<url>https://vimeo\.com/[^/]+/review/(?P<id>[^/]+)/[0-9a-f]{10})'
|
||||
_VALID_URL = r'https://vimeo\.com/[^/]+/review/(?P<id>[^/]+)'
|
||||
_TESTS = [{
|
||||
'url': 'https://vimeo.com/user21297594/review/75524534/3c257a1b5d',
|
||||
'md5': 'c507a72f780cacc12b2248bb4006d253',
|
||||
@@ -1000,8 +992,7 @@ class VimeoReviewIE(VimeoBaseInfoExtractor):
|
||||
data = self._parse_json(self._search_regex(
|
||||
r'window\s*=\s*_extend\(window,\s*({.+?})\);', webpage, 'data',
|
||||
default=NO_DEFAULT if video_password_verified else '{}'), video_id)
|
||||
config = data.get('vimeo_esi', {}).get('config', {})
|
||||
config_url = config.get('configUrl') or try_get(config, lambda x: x['clipData']['configUrl'])
|
||||
config_url = data.get('vimeo_esi', {}).get('config', {}).get('configUrl')
|
||||
if config_url is None:
|
||||
self._verify_video_password(webpage_url, video_id, webpage)
|
||||
config_url = self._get_config_url(
|
||||
@@ -1009,13 +1000,10 @@ class VimeoReviewIE(VimeoBaseInfoExtractor):
|
||||
return config_url
|
||||
|
||||
def _real_extract(self, url):
|
||||
page_url, video_id = re.match(self._VALID_URL, url).groups()
|
||||
video_id = self._match_id(url)
|
||||
config_url = self._get_config_url(url, video_id)
|
||||
config = self._download_json(config_url, video_id)
|
||||
info_dict = self._parse_config(config, video_id)
|
||||
source_format = self._extract_original_format(page_url, video_id)
|
||||
if source_format:
|
||||
info_dict['formats'].append(source_format)
|
||||
self._vimeo_sort_formats(info_dict['formats'])
|
||||
info_dict['id'] = video_id
|
||||
return info_dict
|
||||
|
||||
@@ -19,7 +19,7 @@ from ..utils import (
|
||||
|
||||
|
||||
class WeiboIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?weibo\.com/[0-9]+/(?P<id>[a-zA-Z0-9]+)'
|
||||
_VALID_URL = r'https?://weibo\.com/[0-9]+/(?P<id>[a-zA-Z0-9]+)'
|
||||
_TEST = {
|
||||
'url': 'https://weibo.com/6275294458/Fp6RGfbff?type=comment',
|
||||
'info_dict': {
|
||||
|
||||
@@ -20,7 +20,7 @@ from ..utils import (
|
||||
class XHamsterIE(InfoExtractor):
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://
|
||||
(?:.+?\.)?xhamster\.(?:com|one)/
|
||||
(?:.+?\.)?xhamster\.com/
|
||||
(?:
|
||||
movies/(?P<id>\d+)/(?P<display_id>[^/]*)\.html|
|
||||
videos/(?P<display_id_2>[^/]*)-(?P<id_2>\d+)
|
||||
@@ -91,9 +91,6 @@ class XHamsterIE(InfoExtractor):
|
||||
# new URL schema
|
||||
'url': 'https://pt.xhamster.com/videos/euro-pedal-pumping-7937821',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://xhamster.one/videos/femaleagent-shy-beauty-takes-the-bait-1509445',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
|
||||
@@ -1,90 +0,0 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals
|
||||
|
||||
from .common import InfoExtractor
|
||||
from ..utils import (
|
||||
int_or_none,
|
||||
url_or_none,
|
||||
)
|
||||
|
||||
|
||||
class YandexVideoIE(InfoExtractor):
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://
|
||||
(?:
|
||||
yandex\.ru(?:/portal/(?:video|efir))?/?\?.*?stream_id=|
|
||||
frontend\.vh\.yandex\.ru/player/
|
||||
)
|
||||
(?P<id>[\da-f]+)
|
||||
'''
|
||||
_TESTS = [{
|
||||
'url': 'https://yandex.ru/portal/video?stream_id=4dbb262b4fe5cf15a215de4f34eee34d',
|
||||
'md5': '33955d7ae052f15853dc41f35f17581c',
|
||||
'info_dict': {
|
||||
'id': '4dbb262b4fe5cf15a215de4f34eee34d',
|
||||
'ext': 'mp4',
|
||||
'title': 'В Нью-Йорке баржи и теплоход оторвались от причала и расплылись по Гудзону',
|
||||
'description': '',
|
||||
'thumbnail': r're:^https?://.*\.jpg$',
|
||||
'timestamp': 0,
|
||||
'duration': 30,
|
||||
'age_limit': 18,
|
||||
},
|
||||
}, {
|
||||
'url': 'https://yandex.ru/portal/efir?stream_id=4dbb36ec4e0526d58f9f2dc8f0ecf374&from=morda',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://yandex.ru/?stream_id=4dbb262b4fe5cf15a215de4f34eee34d',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://frontend.vh.yandex.ru/player/4dbb262b4fe5cf15a215de4f34eee34d?from=morda',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
# vod-episode, series episode
|
||||
'url': 'https://yandex.ru/portal/video?stream_id=45b11db6e4b68797919c93751a938cee',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
# episode, sports
|
||||
'url': 'https://yandex.ru/?stream_channel=1538487871&stream_id=4132a07f71fb0396be93d74b3477131d',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
content = self._download_json(
|
||||
'https://frontend.vh.yandex.ru/v22/player/%s.json' % video_id,
|
||||
video_id, query={
|
||||
'stream_options': 'hires',
|
||||
'disable_trackings': 1,
|
||||
})['content']
|
||||
|
||||
m3u8_url = url_or_none(content.get('content_url')) or url_or_none(
|
||||
content['streams'][0]['url'])
|
||||
title = content.get('title') or content.get('computed_title')
|
||||
|
||||
formats = self._extract_m3u8_formats(
|
||||
m3u8_url, video_id, 'mp4', entry_protocol='m3u8_native',
|
||||
m3u8_id='hls')
|
||||
self._sort_formats(formats)
|
||||
|
||||
description = content.get('description')
|
||||
thumbnail = content.get('thumbnail')
|
||||
timestamp = (int_or_none(content.get('release_date')) or
|
||||
int_or_none(content.get('release_date_ut')) or
|
||||
int_or_none(content.get('start_time')))
|
||||
duration = int_or_none(content.get('duration'))
|
||||
series = content.get('program_title')
|
||||
age_limit = int_or_none(content.get('restriction_age'))
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': title,
|
||||
'description': description,
|
||||
'thumbnail': thumbnail,
|
||||
'timestamp': timestamp,
|
||||
'duration': duration,
|
||||
'series': series,
|
||||
'age_limit': age_limit,
|
||||
'formats': formats,
|
||||
}
|
||||
@@ -351,8 +351,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
||||
(?:www\.)?hooktube\.com/|
|
||||
(?:www\.)?yourepeat\.com/|
|
||||
tube\.majestyc\.net/|
|
||||
(?:(?:www|dev)\.)?invidio\.us/|
|
||||
(?:www\.)?invidiou\.sh/|
|
||||
(?:www\.)?invidio\.us/|
|
||||
(?:www\.)?invidious\.snopyta\.org/|
|
||||
(?:www\.)?invidious\.kabi\.tk/|
|
||||
(?:www\.)?vid\.wxzm\.sx/|
|
||||
|
||||
@@ -31,7 +31,7 @@ def rsa_verify(message, signature, key):
|
||||
def update_self(to_screen, verbose, opener):
|
||||
"""Update the program file with the latest version from the repository"""
|
||||
|
||||
UPDATE_URL = 'https://yt-dl.org/update/'
|
||||
UPDATE_URL = 'https://ytdl-org.github.io/youtube-dl/update/'
|
||||
VERSION_URL = UPDATE_URL + 'LATEST_VERSION'
|
||||
JSON_URL = UPDATE_URL + 'versions.json'
|
||||
UPDATES_RSA_KEY = (0x9d60ee4d8f805312fdb15a62f87b95bd66177b91df176765d13514a0f1754bcd2057295c5b6f1d35daa6742c3ffc9a82d3e118861c207995a8031e151d863c9927e304576bc80692bc8e094896fcf11b66f3e29e04e3a71e9a11558558acea1840aec37fc396fb6b65dc81a1c4144e03bd1c011de62e3f1357b327d08426fe93, 65537)
|
||||
|
||||
@@ -1798,14 +1798,6 @@ def parse_resolution(s):
|
||||
return {}
|
||||
|
||||
|
||||
def parse_bitrate(s):
|
||||
if not isinstance(s, compat_str):
|
||||
return
|
||||
mobj = re.search(r'\b(\d+)\s*kbps', s)
|
||||
if mobj:
|
||||
return int(mobj.group(1))
|
||||
|
||||
|
||||
def month_by_name(name, lang='en'):
|
||||
""" Return the number of a month by (locale-independently) English name """
|
||||
|
||||
@@ -1922,7 +1914,7 @@ def int_or_none(v, scale=1, default=None, get_attr=None, invscale=1):
|
||||
return default
|
||||
try:
|
||||
return int(v) * invscale // scale
|
||||
except (ValueError, TypeError):
|
||||
except ValueError:
|
||||
return default
|
||||
|
||||
|
||||
@@ -1943,7 +1935,7 @@ def float_or_none(v, scale=1, invscale=1, default=None):
|
||||
return default
|
||||
try:
|
||||
return float(v) * invscale / scale
|
||||
except (ValueError, TypeError):
|
||||
except ValueError:
|
||||
return default
|
||||
|
||||
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
__version__ = '2019.04.01'
|
||||
__version__ = '2019.03.09'
|
||||
|
||||
Reference in New Issue
Block a user