Python script for crawling ResearchGate.net papers
This code start crawling process by urls in start.txt and give paper details in crawled.json.
First install Python. Then install these libraries:
pip install selenium
pip install webdriver-manager
MAX_FETCH_COUNT: How many pages you want to crawl?
MAX_CACHED_NUM: We renew crawled.json after crawling each MAX_CACHED_NUM papers.
covid-tracker-script A simple python script to fetch the latest covid info How it works First, get the current date in MM-DD-YYYY format. Check if the
🐛 MASS DBLTEK EXPLOIT CHECKER USING SHODAN 🕸 Python script who crawl first shodan page and check DBLTEK vulnerability
quora-plus-bypass A python script to extract answers to any question on Quora (Quora+ included) Requirements Python 3.x
poolbooru_gelscraper a simple python script for scraping images off gelbooru pools. modules required:requests_html, and os by default saves files with
Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins
Aliexpress to telegram post Python script that reads Aliexpress offers urls from a Excel filename (.csv) and post then in a Telegram channel using a b
This script is useful for downloading stock market data for a wide range of companies specified by their respective tickers. The script reads in the d
Auto Invite To The Organization By Star A GitHub Action script to automatically invite everyone to your organization that stars your repository. What
AutoReserve A webdriver-based script for reserving badminton courts. 使用说明 下载 chromedriver 选择当前Chrome对应版本 安装 selenium pip install selenium 更改场次、金额信息dat
Amazon-Web-Scarper Created a web scraper using simple functions to check price of a product on amazon (can be duplicated to check price at other marke
茅台抢购最新优化版本，茅台秒杀，优化了抢购协程队列
Media-WebScraper This program scrapes information and images for movies and TV shows. Summary For more information on the program, read the WebScrape_
Pixiv-spider A Pixiv spider module WARNING It's an unfinished work, browsing the code carefully before using it. Features 0004 - Readme.md updated, co
RoboBrowser: Your friendly neighborhood web scraper Homepage: http://robobrowser.readthedocs.org/ RoboBrowser is a simple, Pythonic library for browsi
Soup Sieve Overview Soup Sieve is a CSS selector library designed to be used with Beautiful Soup 4. It aims to provide selecting, matching, and filter
Photon Incredibly fast crawler designed for OSINT. Photon Wiki • How To Use • Compatibility • Photon Library • Contribution • Roadmap Key Features Dat
Newspaper3k: Article scraping & curation Inspired by requests for its simplicity and powered by lxml for its speed: "Newspaper is an amazing python li
Project A: WebScraper A script that prints out a list of all EXTERNAL references
WebscrapingDatabases a database scraper created with mechanical soup and sqlite author: Mariya Sha Watch on YouTube: This repository was created to su
What is Moon? Moon is a lightweight and fast proxy scraper made by using ProxyScrape's api. What can i do with this? You can use proxies for varietys
中国大学生在线 “四史”学习教育竞答 自动答题 刷分 (现仅支持英雄篇，已更新可用) 若对您有所帮助，记得点个Star 🌟 ！！！ 中国大学生在线 “四史”学习教育竞答 自动答题 刷分 (现仅支持英雄篇，已更新可用) 🥰 🥰 🥰 依赖 本项目依赖的第三方库: requests 在终端执行以下
covid-tracker-script A simple python script to fetch the latest covid info How it works First, get the current date in MM-DD-YYYY format. Check if the
Web-Scraping using Selenium What is the need of Selenium? Some websites don't like to be scrapped and in that case you need to disguise your webscrapi
Gypsylist gypsylist.py is a web scraper for nomadlist.com, made to avoid website restrictions. nomadlist.com is a website with a lot of information fo
cybernews A package that provides you Latest Cyber/Hacker News from website using Web-Scraping. Latest Cyber/Hacker News Using Webscraping Developed b
quora-plus-bypass A python script to extract answers to any question on Quora (Quora+ included) Requirements Python 3.x
Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins
UdemyBot - A Simple Udemy Free Courses Scrapper
Frontera Overview Frontera is a web crawling framework consisting of crawl frontier, and distribution/scaling primitives, allowing to build a large sc
0 Dec 15, 2021
4 Jan 9, 2022
10 Aug 18, 2022
1 Jan 2, 2022
68 Oct 8, 2022
6 Dec 6, 2022
71 Oct 4, 2022
11 Dec 11, 2022
4 Nov 9, 2021
1 Jan 17, 2022
33 Sep 03, 2022
1 Dec 05, 2021
1 Nov 14, 2021
3.7k Dec 27, 2022
151 Dec 23, 2022
9.3k Jan 02, 2023
12.3k Jan 07, 2023
2 Apr 26, 2022
30 Aug 08, 2022
1 Jul 04, 2022
229 Dec 12, 2022
1 Oct 26, 2021
5 Nov 24, 2022
4 Jun 02, 2022
112 Nov 12, 2022
1.2k Jan 02, 2023