scraper
componentv1.0.0Web scraper using Python requests and BeautifulSoup
Usage
This is a built-in component — it ships with kdeps and requires no installation. Use it directly in your workflow resource:
run:
scraper:
# see README for options
See the kdeps CLI docs for full component configuration reference.
README
scraper
Web scraper using Python requests and BeautifulSoup. Built-in kdeps component.
Fetches a URL and extracts text content, optionally filtered by CSS selector.
Usage
run:
component:
name: scraper
with:
url: "https://news.ycombinator.com"
selector: ".titleline a"
timeout: 30
Inputs
| Name | Required | Default | Description |
|---|---|---|---|
| url | yes | — | URL to scrape |
| selector | no | — | CSS selector to extract |
| timeout | no | 30 | Request timeout in seconds |
Versions
| Version | Published | Status |
|---|---|---|
| 1.0.0 | 4/12/2026 | active |
Details
- Author
- kdeps
- License
- Apache-2.0
- Latest Version
- 1.0.0
- Published
- 4/12/2026
Tags
scrapingwebbeautifulsouppythonhtml