java多线程    Java入门    vsftp    ftp    linux配置    centos    FRP教程    HBase    Html5缓存    webp    zabbix    分布式    neo4j图数据库    

google chrome内核工具Puppeteer

(一)Puppeteer的简介
(二)nodejs的使用
(三)安装puppeteer
(四)简单得puppeteer开发小例子
(五)Linux系统如何使用puppeteer

(一)Puppeteer的简介

Puppeteer介绍:

  • 1.可以模拟浏览器来下载
  • 2.可以操纵网页内dom
  • 3.可以跨平台执行
  • 4.表面用的js写的,实际可以在后台运行

官方文档说的很谦虚,可以保存浏览器一个网页为png,pdf,这些都是小case

1.其实你可以用这个软件来开发爬虫
2.其实你可以用这个软件来自动化测试,写个脚本自动填充表单,自动提交

月小升今天觉得google的伟大在于技术,技术的作用在于自动化,google从搜索引擎到adwords收费,全部无人值守。用最小的人力完成最多的事情。未来属于机器人时代。

(二)nodejs的使用

简单普及下node.js知识,安装完毕nodejs 你只需要这样执行

a.js

console.log('test');
node a.js

前段工程师不少人对这个很熟悉,作为玩后端的月小升,下载完毕puppeteer直发晕。所以写上面这几句。

(三)安装puppeteer

安装后nodejs系统就有了npm ,如何安装nodejs看上一个博客

npm i puppeteer

(四)简单得puppeteer开发小例子

电脑任意位置写个a.js 内容如下

const puppeteer = require('puppeteer');

(async () => {
	const browser = await puppeteer.launch();
	const page = await browser.newPage();
	page.setDefaultNavigationTimeout(12000);
	await page.goto('https://java-er.com');
	//读取java-er.com首页的宽高
	const bodyHandle = await page.$('body');
	const { width, height } = await bodyHandle.boundingBox();
	var heightx = parseInt(height)+1;
	var widthx = parseInt(width)+1;
	console.log(heightx);
	console.log(widthx);

	await page.setViewport({
		width: 1920,
		height: heightx
	});
	await bodyHandle.dispose();
	await page.screenshot({path: 'example.png',fullpage:true});
	await browser.close();
})();

命令行执行
node a.js
这样就能把我的首页给存成一个png。

(五)Linux系统如何使用puppeteer

在linux服务器上,我把代码删除的只留一句,都无法执行

const puppeteer = require('puppeteer');

(async () => {
	const browser = await puppeteer.launch();
	
})();
TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

    at onClose (/backup/soft/node_modules/puppeteer/lib/Launcher.js:333:14)
    at Interface.helper.addEventListener (/backup/soft/node_modules/puppeteer/lib/Launcher.js:322:50)
    at Interface.emit (events.js:187:15)
    at Interface.close (readline.js:379:8)
    at Socket.onend (readline.js:157:10)
    at Socket.emit (events.js:187:15)
    at endReadableNT (_stream_readable.js:1092:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
(node:24137) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:24137) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md
月小升猜测这玩意依赖chrome内核
安装chrome 到linux服务器
1.cd /etc/yum.repos.d/

2 vi google.repo

[gogle]

name=Google-x86_64

baseurl=http://dl.google.com/linux/rpm/stable/x86_64

enabled=1

gpgcheck=0

gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub

3.安装google chrome
yum install google-chrome-stable

google chrome 在linux下只能无沙盒执行所以代码要改改

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch({args: ['--no-sandbox']});
console.log('OK');
await browser.close();

})(); 

完整的linux 执行代码

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch({args: ['--no-sandbox']});

const page = await browser.newPage();
	page.setDefaultNavigationTimeout(12000);
	await page.goto('https://java-er.com');
	//读取java-er.com首页的宽高
	const bodyHandle = await page.$('body');
	const { width, height } = await bodyHandle.boundingBox();
	var heightx = parseInt(height)+1;
	var widthx = parseInt(width)+1;
	console.log(heightx);
	console.log(widthx);

	await page.setViewport({
		width: 1920,
		height: heightx
	});
	await bodyHandle.dispose();
	await page.screenshot({path: 'example.png',fullpage:true});
	await browser.close();

})(); 

linux上多个example.png

==============================

参考资料:

官方
https://github.com/GoogleChrome/puppeteer

对puppeteer的应用
https://github.com/zhentaoo/puppeteer-deep

中文API手册
https://zhaoqize.github.io/

一个流程测试的案例
https://cnodejs.org/topic/5a041412ad77fa2004549183


This entry was posted in Linux, MAC, WEB前端 and tagged , . Bookmark the permalink.
月小升QQ 2651044202, 技术交流QQ群 178491360
首发地址:月小升博客https://java-er.com/blog/google-chrome-puppeteer/
无特殊说明,文章均为月小升原创,欢迎转载,转载请注明本文地址,谢谢
您的评论是我写作的动力.

Leave a Reply