【开源】某Hub站的视频爬虫脚本开源 # PornHubBot 源代码地址：https://github.com/levphon/pornhubbot PornHubBot项目主要是爬取全球最大成人网站PornHub的视频标题、时长、mp4链接、封面URL和具体的PornHub链接 项目爬的是PornHub.com，结构简单，速度飞快 爬取PornHub视频的速度可以达到500万/天以上。具体视个人网络情况,因为我是家庭网络，所以相对慢一点。 10个线程同时请求，可达到如上速度。若个人网络环境更好，可启动更多线程来请求，具体配置方法见 [启动前配置] # 环境、架构 开发语言: Python2.7 开发环境: MacOS系统、4G内存 数据库: MongoDB 主要使用 scrapy 爬虫框架 从Cookie池和UA池中随机抽取一个加入到Spider start_requests 根据 PorbHub 的分类，启动了5个Request，同时对五个分类进行爬取。 并支持分页爬取数据，并加入到待爬队列。 # 使用说明 启动前配置 安装MongoDB,并启动，不需要配置 安装Scrapy 安装Python的依赖模块：pymongo、json、requests 根据自己需要修改 Scrapy 中关于 间隔时间、启动Requests线程数等得配置 启动 python PornHub/quickstart.py # 数据库说明 数据库中保存数据的表是 PhRes。以下是字段说明: video_title:视频的标题,并作为唯一标识 link_url:视频调转到PornHub的链接 image_url:视频的封面链接 video_duration:视频的时长，以 s 为单位 quality_480p: 视频480p的 mp4 下载地址

网页中显示CloudFlare（CF）节点信息 原理与实现 当一个网站开启了CloudFlare的CDN页面时，会出现一个CF（CloudFlare简称，下同）的CDN测试页面，就在这里/cdn-cgi/trace，我这里访问得到的应答是以下： ```` fl=12f399 h=baidu.com ip=用户IP ts=1592196919.147 visit_scheme=http uag=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3314.0 Safari/537.36 SE 2.X MetaSr 1.0 colo=LAX http=http/1.1 loc=CN tls=off sni=off warp=off ```` 所以我们需要取出colo字段，当然这里只是简写。我们可以在 https://www.cloudflarestatus.com/ 获取全部节点信息。我们刚才拿到的的SJC就是San Jose, CA, United States - (SJC)。这样的话，实现方法就来了，我就直接贴代码了。 ```` 当前CDN节点: unknown getCDNinfo = function() { $.ajax({ url: "/cdn-cgi/trace", success: function(data, status) { let areas = "Antananarivo, Madagascar - (TNR);Cape Town, South Africa - (CPT);Casablanca, Morocco - (CMN);Dar Es Salaam, Tanzania - (DAR);Djibouti City, Djibouti - (JIB);Durban, South Africa - (DUR);Johannesburg, South Africa - (JNB);Kigali, Rwanda - (KGL);Lagos, Nigeria - (LOS);Luanda, Angola - (LAD);Maputo, MZ - (MPM);Mombasa, Kenya - (MBA);Port Louis, Mauritius - (MRU);Réunion, France - (RUN);Bangalore, India - (BLR);Bangkok, Thailand - (BKK);Bandar Seri Begawan, Brunei - (BWN);Cebu, Philippines - (CEB);Chengdu, China - (CTU);Chennai, India - (MAA);Chittagong, Bangladesh - (CGP);Chongqing, China - (CKG);Colombo, Sri Lanka - (CMB);Dhaka, Bangladesh - (DAC);Dongguan, China - (SZX);Foshan, China - (FUO);Fuzhou, China - (FOC);Guangzhou, China - (CAN);Hangzhou, China - (HGH);Hanoi, Vietnam - (HAN);Hengyang, China - (HNY);Ho Chi Minh City, Vietnam - (SGN);Hong Kong - (HKG);Hyderabad, India - (HYD);Islamabad, Pakistan - (ISB);Jakarta, Indonesia - (CGK);Jinan, China - (TNA);Karachi, Pakistan - (KHI);Kathmandu, Nepal - (KTM);Kolkata, India - (CCU);Kuala Lumpur, Malaysia - (KUL);Lahore, Pakistan - (LHE);Langfang, China - (NAY);Luoyang, China - (LYA);Macau - (MFM);Malé, Maldives - (MLE);Manila, Philippines - (MNL);Mumbai, India - (BOM);Nagpur, India - (NAG);Nanning, China - (NNG);New Delhi, India - (DEL);Osaka, Japan - (KIX);Phnom Penh, Cambodia - (PNH);Qingdao, China - (TAO);Seoul, South Korea - (ICN);Shanghai, China - (SHA);Shenyang, China - (SHE);Shijiazhuang, China - (SJW);Singapore, Singapore - (SIN);Suzhou, China - (SZV);Taipei - (TPE);Thimphu, Bhutan - (PBH);Tianjin, China - (TSN);Tokyo, Japan - (NRT);Ulaanbaatar, Mongolia - (ULN);Vientiane, Laos - (VTE);Wuhan, China - (WUH);Wuxi, China - (WUX);Xi'an, China - (XIY);Yerevan, Armenia - (EVN);Zhengzhou, China - (CGO);Zuzhou, China - (CSX);Amsterdam, Netherlands - (AMS);Athens, Greece - (ATH);Barcelona, Spain - (BCN);Belgrade, Serbia - (BEG);Berlin, Germany - (TXL);Brussels, Belgium - (BRU);Bucharest, Romania - (OTP);Budapest, Hungary - (BUD);Chișinău, Moldova - (KIV);Copenhagen, Denmark - (CPH);Cork, Ireland - (ORK);Dublin, Ireland - (DUB);Düsseldorf, Germany - (DUS);Edinburgh, United Kingdom - (EDI);Frankfurt, Germany - (FRA);Geneva, Switzerland - (GVA);Gothenburg, Sweden - (GOT);Hamburg, Germany - (HAM);Helsinki, Finland - (HEL);Istanbul, Turkey - (IST);Kyiv, Ukraine - (KBP);Lisbon, Portugal - (LIS);London, United Kingdom - (LHR);Luxembourg City, Luxembourg - (LUX);Madrid, Spain - (MAD);Manchester, United Kingdom - (MAN);Marseille, France - (MRS);Milan, Italy - (MXP);Moscow, Russia - (DME);Munich, Germany - (MUC);Nicosia, Cyprus - (LCA);Oslo, Norway - (OSL);Paris, France - (CDG);Prague, Czech Republic - (PRG);Reykjavík, Iceland - (KEF);Riga, Latvia - (RIX);Rome, Italy - (FCO);Saint Petersburg, Russia - (LED);Sofia, Bulgaria - (SOF);Stockholm, Sweden - (ARN);Tallinn, Estonia - (TLL);Thessaloniki, Greece - (SKG);Vienna, Austria - (VIE);Vilnius, Lithuania - (VNO);Warsaw, Poland - (WAW);Zagreb, Croatia - (ZAG);Zürich, Switzerland - (ZRH);Arica, Chile - (ARI);Asunción, Paraguay - (ASU);Bogotá, Colombia - (BOG);Buenos Aires, Argentina - (EZE);Curitiba, Brazil - (CWB);Fortaleza, Brazil - (FOR);Guatemala City, Guatemala - (GUA);Lima, Peru - (LIM);Medellín, Colombia - (MDE);Panama City, Panama - (PTY);Porto Alegre, Brazil - (POA);Quito, Ecuador - (UIO);Rio de Janeiro, Brazil - (GIG);São Paulo, Brazil - (GRU);Santiago, Chile - (SCL);Willemstad, Curaçao - (CUR);St. George's, Grenada - (GND);Amman, Jordan - (AMM);Baghdad, Iraq - (BGW);Baku, Azerbaijan - (GYD);Beirut, Lebanon - (BEY);Doha, Qatar - (DOH);Dubai, United Arab Emirates - (DXB);Kuwait City, Kuwait - (KWI);Manama, Bahrain - (BAH);Muscat, Oman - (MCT);Ramallah - (ZDM);Riyadh, Saudi Arabia - (RUH);Tel Aviv, Israel - (TLV);Ashburn, VA, United States - (IAD);Atlanta, GA, United States - (ATL);Boston, MA, United States - (BOS);Buffalo, NY, United States - (BUF);Calgary, AB, Canada - (YYC);Charlotte, NC, United States - (CLT);Chicago, IL, United States - (ORD);Columbus, OH, United States - (CMH);Dallas, TX, United States - (DFW);Denver, CO, United States - (DEN);Detroit, MI, United States - (DTW);Honolulu, HI, United States - (HNL);Houston, TX, United States - (IAH);Indianapolis, IN, United States - (IND);Jacksonville, FL, United States - (JAX);Kansas City, MO, United States - (MCI);Las Vegas, NV, United States - (LAS);Los Angeles, CA, United States - (LAX);McAllen, TX, United States - (MFE);Memphis, TN, United States - (MEM);Mexico City, Mexico - (MEX);Miami, FL, United States - (MIA);Minneapolis, MN, United States - (MSP);Montgomery, AL, United States - (MGM);Montréal, QC, Canada - (YUL);Nashville, TN, United States - (BNA);Newark, NJ, United States - (EWR);Norfolk, VA, United States - (ORF);Omaha, NE, United States - (OMA);Philadelphia, United States - (PHL);Phoenix, AZ, United States - (PHX);Pittsburgh, PA, United States - (PIT);Port-Au-Prince, Haiti - (PAP);Portland, OR, United States - (PDX);Queretaro, MX, Mexico - (QRO);Richmond, Virginia - (RIC);Sacramento, CA, United States - (SMF);Salt Lake City, UT, United States - (SLC);San Diego, CA, United States - (SAN);San Jose, CA, United States - (SJC);Saskatoon, SK, Canada - (YXE);Seattle, WA, United States - (SEA);St. Louis, MO, United States - (STL);Tampa, FL, United States - (TPA);Toronto, ON, Canada - (YYZ);Vancouver, BC, Canada - (YVR);Tallahassee, FL, United States - (TLH);Winnipeg, MB, Canada - (YWG);Adelaide, SA, Australia - (ADL);Auckland, New Zealand - (AKL);Brisbane, QLD, Australia - (BNE);Melbourne, VIC, Australia - (MEL);Noumea, New caledonia - (NOU);Perth, WA, Australia - (PER);Sydney, NSW, Australia - (SYD)".split(";"); let area = data.split("colo=")[1].split("\n")[0]; for (var i = 0; i < areas.length; i++) { if (areas[i].indexOf(area) != -1) { document.getElementById("cdn").innerHTML = areas[i]; break; } } } }) } $(document).ready(function() { getCDNinfo(); //页面加载完毕就获取CDN信息 }); ````

在线解密qmc和ncm等音乐加密格式的网站 可以在线解密QQ音乐、网易云音乐付费音乐加密格式的网站。 由于程序编写使用了ES6语法，Blob，Web Worker特性，过旧的浏览器可能无法使用。 工具地址：http://unlock.musictool.top/ GitHub项目地址：https://github.com/ix64/unlock-music

PHP正则表达式函数preg_replace用法实例分析 本文实例讲述了PHP正则表达式函数preg_replace用法。分享给大家供大家参考，具体如下： preg_replace 执行一个正则表达式的搜索和替换 语法：preg_replace (pattern ,replacement ,subject,limit,count ) 参数 描述 pattern 正则表达式(字符串或字符串数组) replacement 用于替换的字符串或字符串数组 subject 要进行搜索和替换的字符串或字符串数组。 limit 可选。每个模式在每个subject上进行替换的最大次数。默认是 -1(无限)。 count 可选。完成的替换次数 Example 1 $string = 'huang yu xin'; $pattern = '/(\w+) (\w+) (\w+)/i'; $replacement = '${1}a $3'; // $1对应(\w+),${1}a是区别$1a,说明是$1和a不是$1a,$3对应第三个(\w+) echo preg_replace($pattern, $replacement, $string);

PHP时间相关常用函数用法示例 本文实例讲述了PHP时间相关常用函数用法。分享给大家供大家参考，具体如下： <?php /** * 设置时区 */ date_default_timezone_set("Asia/Shanghai"); /** * 获取时区 */ echo date_default_timezone_get(); //结果 UTC echo "<br/>"; /** * 添加时间 */ $date=date_create("2013-03-15"); //创建一个DateTime 对象 date_add($date,date_interval_create_from_date_string("40 month"));//years days //date_interval_create_from_date_string 从字符串的相关部分建立一个DateInterval。 echo date_format($date,"Y-m-d"); //结果2016-07-15 echo "<br/>"; /** * 减去时间 */ $date=date_create("2013-03-15"); date_sub($date,date_interval_create_from_date_string("40 days")); echo date_format($date,"Y-m-d"); //2013-02-03 echo "<br/>"; /** * 获取两个时区的差值 */ $date1=date_create("2013-03-15"); $date2=date_create("2013-12-12"); $diff=date_diff($date1,$date2);//返回的是一个DateInterval对象 echo "<pre>"; var_dump($diff); // object(DateInterval)#4 (15) { // ["y"]=> // int(0) // ["m"]=> // int(8) // ["d"]=> // int(27) // ["h"]=> // int(0) // ["i"]=> // int(0) // ["s"]=> // int(0) // ["weekday"]=> // int(0) // ["weekday_behavior"]=> // int(0) // ["first_last_day_of"]=> // int(0) // ["invert"]=> // int(0) // ["days"]=> // int(272) // ["special_type"]=> // int(0) // ["special_amount"]=> // int(0) // ["have_weekday_relative"]=> // int(0) // ["have_special_relative"]=> // int(0) // } echo "<br/>"; /** * 获取当前时间戳 */ $date=date_create(); echo date_timestamp_get($date) .'<br/>'; $time = time(); echo $time .'<br>'; echo strtotime("now") .'<br/>'; /* *获取今天0点时间戳 */ echo strtotime("today").'<br>'; /** * 获取带微秒的时间 */ echo microtime(true); /* *获取指定时间戳 * mktime(hour,minute,second,month,day,year); */ echo "<br/>"; echo mktime(18,30,15,3,15,2019); /* *获取前一天0点时间戳 */ echo "<br/>"; echo strtotime('yesterday'); /* *获取昨天此时的时间戳 */ echo "<br/>"; echo strtotime('-1 days'); ?>