Workflow Automation
AdsCrawl for Dify
Use the AdsCrawl plugin in Dify workflows and agents to turn public web pages into readable content or structured fields. Install it from Dify Marketplace, connect your AdsCrawl API key, and add Render Page or Extract Fields to your flow.
What you can do
Render Page
Get rendered HTML, readable Markdown or article JSON. Optionally wait for a CSS selector and focus the result on that element.
Extract Fields
Map field names to CSS selectors and return structured page data for the next workflow node.
Connect workflow steps
Pass page text to an LLM for a summary, or use extracted JSON in downstream processing and output nodes.
Before you start
- An AdsCrawl account and API key.
- A Dify workspace that supports installing tool plugins and editing workflows or agents.
SETUP METHOD 01
Dify Marketplace plugin
Install once, authorize your account, then choose either tool in a workflow or agent.
Install AdsCrawl
Open the Marketplace link above and select Install. Confirm the publisher is adscrawl and install the plugin in your Dify workspace.
Configure the provider credential
Enter your key in the provider's AdsCrawl API key secret field. The plugin sends it to the AdsCrawl HTTPS API in x-api-key; you do not need to configure an MCP server.
Read your first page
Add AdsCrawl → Render Page to a workflow. Fill its fields using these reference values, then test the node. The Markdown result should contain Example Domain. These are node parameters, not a workflow import file.
{ "url": "https://example.com", "content_mode": "markdown", "wait_until": "domcontentloaded", "timeout_ms": 60000 }Extract named fields
For structured data, add AdsCrawl → Extract Fields and set Page URL to https://example.com. Paste the object below into Fields JSON (fields_json). It maps the output field title to the page's h1 selector.
{ "title": "h1" }Connect the next node
Build a page-summary flow: URL input → Render Page → LLM → output. Map the tool's text output into the LLM prompt using Dify's variable picker. For Extract Fields or article JSON, use the JSON output instead. Check for an error result before passing content downstream.
Usage & limits
- API requests and browser sessions use your AdsCrawl account credits. Installing a client does not include a separate usage allowance.
- The plugin makes one API request per tool invocation and does not retry automatically. Navigation timeouts range from 1 to 90 seconds; each result is limited to 1 MiB.
- Targets must be complete public HTTP(S) URLs on ports 80 or 443. This plugin does not accept cookies, custom proxy credentials or CDP browser-control commands.