news-please is an open source easy-to-use news extractor that just works.
Project Links
Meta
Author: Felix Hamborg
Classifiers
Development Status
- 5 - Production/Stable
Environment
- Console
Intended Audience
- Developers
- Science/Research
License
- OSI Approved :: Apache Software License
Operating System
- MacOS
- Microsoft
- POSIX :: Linux
Programming Language
- Python :: 3.8
- Python :: 3.9
- Python :: 3.10
- Python :: 3.11
- Python :: 3.12
Topic
- Internet
- Scientific/Engineering :: Information Analysis
news-please is an open source, easy-to-use news crawler that extracts structured information from almost any news website. It can follow recursively internal hyperlinks and read RSS feeds to fetch both most recent and also old, archived articles. You only need to provide the root URL of the news website. Furthermore, its API allows developers to access the exctraction functionality within their software. news-please also implements a workflow optimized for the news archive provided by commoncrawl.org, allowing users to efficiently crawl and extract news articles including various filter options.
1.6.16
Sep 21, 2025
1.6.15
May 26, 2025
1.6.14
Mar 25, 2025
1.6.13
Jul 29, 2024
1.6.12
Jul 08, 2024
1.6.11
Jul 02, 2024
1.6.10
Jun 26, 2024
1.5.48
Jun 06, 2024
1.5.47
Jun 04, 2024
1.5.44
Dec 27, 2023
1.5.43
Dec 27, 2023
1.5.41
Dec 27, 2023
1.5.40
Dec 22, 2023
1.5.35
Aug 30, 2023
1.5.34
Aug 29, 2023
1.5.33
Apr 17, 2023
1.5.22
Apr 04, 2022
1.5.21
May 06, 2021
1.5.20
Apr 22, 2021
1.5.18
Mar 16, 2021
1.5.17
Feb 24, 2021
1.5.16
Feb 07, 2021
1.5.15
Feb 05, 2021
1.5.14
Feb 02, 2021
1.5.13
Oct 23, 2020
1.5.3
Apr 30, 2020
1.5.2
Apr 08, 2020
1.5.1
Apr 06, 2020
1.4.26
Mar 20, 2020
1.4.25
Jan 14, 2020
1.4.24
Nov 08, 2019
1.4.23
Aug 06, 2019
1.4.22
Aug 06, 2019
1.4.21
Jul 15, 2019
1.4.20
Jul 15, 2019
1.4.19
Jul 15, 2019
1.4.18
Jul 15, 2019
1.4.17
Jul 15, 2019
1.4.16
Jul 15, 2019
1.4.15
Jul 15, 2019
1.4.14
Jul 12, 2019
1.4.13
Jul 12, 2019
1.4.12
Jul 12, 2019
1.4.11
Jul 11, 2019
1.4.10
Jul 11, 2019
1.3.14
May 13, 2019
1.3.13
Apr 02, 2019
1.3.11
Mar 29, 2019
1.3.10
Mar 28, 2019
1.2.53
Mar 18, 2019
1.2.52
Feb 22, 2019
1.2.51
Feb 21, 2019
1.2.50
Dec 08, 2018
1.2.44
Nov 14, 2018
1.2.43
Nov 14, 2018
1.2.42
Nov 14, 2018
1.2.41
Jun 27, 2018
1.2.40
Jun 05, 2018
1.2.39
May 29, 2018
1.2.36
Feb 01, 2018
1.2.35
Nov 27, 2017
1.2.33
Oct 05, 2017
1.2.32
Oct 05, 2017
1.2.31
Sep 04, 2017
1.2.28
Aug 27, 2017
1.2.27
Aug 26, 2017
1.2.26
Aug 18, 2017
1.2.25
Aug 18, 2017
1.2.24
Aug 17, 2017
1.2.22
Aug 17, 2017
1.2.21
Aug 16, 2017
1.2.20
Aug 16, 2017
1.2.1
Aug 16, 2017
1.1.53
Aug 12, 2017
1.1.52
Aug 07, 2017
1.1.51
Aug 04, 2017
1.1.50
Aug 04, 2017
1.1.49
Jul 17, 2017
1.1.48
Jun 29, 2017
1.1.47
Jun 29, 2017
1.1.45
Jun 29, 2017
1.1.43
Jun 29, 2017
1.1.41
Jun 29, 2017
1.1.40
Jun 28, 2017
1.1.31
Jun 25, 2017
1.1.30
Jun 10, 2017
1.1.29
Jun 08, 2017
1.1.28
Jun 01, 2017
1.1.27
Jun 01, 2017
1.1.26
May 29, 2017
1.1.25
May 29, 2017
1.1.23
May 28, 2017
1.1.22
May 21, 2017
1.1.21
May 18, 2017
1.1.20
May 18, 2017
1.1.19
May 17, 2017
1.1.18
Apr 03, 2017
1.1.17
Mar 29, 2017
1.1.16
Mar 29, 2017
1.1.15
Mar 20, 2017
1.1.14
Feb 25, 2017
1.1.12
Feb 25, 2017
1.1.11
Feb 24, 2017
1.0.27
Feb 24, 2017
1.0.26
Feb 24, 2017
1.0.25
Feb 17, 2017
1.0.24
Dec 18, 2016
1.0.23
Dec 18, 2016
1.0.21
Dec 18, 2016
1.0.20
Dec 18, 2016
1.0.18
Nov 09, 2016
1.0.17
Nov 09, 2016
1.0.16
Nov 09, 2016
1.0.14
Nov 09, 2016
1.0.13
Nov 09, 2016
1.0.12
Nov 09, 2016
1.0.11
Nov 08, 2016
1.0.10
Nov 08, 2016
1.0.9
Nov 08, 2016
1.0.8
Nov 08, 2016
1.0.7
Nov 08, 2016
1.0.6
Nov 08, 2016
1.0.5
Nov 08, 2016
1.0.4
Nov 08, 2016
1.0.3
Nov 08, 2016
1.0.2
Oct 30, 2016
1.0.1
Oct 30, 2016
1.0.0.post6
Oct 30, 2016
1.0.0.post5
Oct 30, 2016
1.0.0.post4
Oct 23, 2016
1.0.0.post3
Oct 23, 2016
1.0.0.post2
Oct 23, 2016
1.0.0.post1
Oct 23, 2016
Wheel compatibility matrix
Files in release
Extras:
None
Dependencies:
Scrapy
(>=1.1.0)
PyMySQL
(>=0.7.9)
psycopg2-binary
(>=2.8.4)
hjson
(>=1.5.8)
elasticsearch
(>=2.4)
beautifulsoup4
(>=4.3.2)
readability-lxml
(>=0.6.2)
langdetect
(>=1.0.7)
python-dateutil
(>=2.4.0)
plac
(>=0.9.6)
dotmap
(>=1.2.17)
PyDispatcher
(>=2.0.5)
warcio
(>=1.3.3)
ago
(>=0.0.9)
six
(>=1.10.0)
lxml
(>=3.3.5)
hurry.filesize
(>=0.9)
bs4
faust-cchardet
(>=2.1.18)
boto3
redis
newspaper4k
(>=0.9.3.1)
lxml-html-clean
(>=0.1.1)
typing-extensions
(>=4.7.0)
pywin32
(>=220)