Scrape Examples
Enterprise web scraping patterns and real-world implementations.
Production-Ready Examples
All examples include enterprise-grade error handling, proxy rotation, and optimization strategies for high-volume scraping operations.
Static Content Scraping
Beginner~5 creditsScrape traditional HTML pages with link and metadata extraction
Link extractionMeta dataResidential proxiesFast processing
Static Content Scraping Implementation
JavaScript SPA Scraping
Intermediate~12 creditsScrape Single Page Applications with dynamic content rendering
JavaScript renderingWait conditionsSchema extractionScreenshot capture
JavaScript SPA Scraping Implementation
Bulk Product Scraping
Advanced~35 creditsEfficiently scrape multiple e-commerce sites for price comparison
Bulk processingConcurrent requestsAutomatic retriesPrice extraction
Bulk Product Scraping Implementation
Authenticated Scraping
Expert~20 creditsAccess protected content behind login forms and user sessions
Session managementCookie handlingCustom headersMulti-step flow
Authenticated Scraping Implementation
Industry Use Cases
E-commerce Intelligence
- • Competitor price monitoring
- • Product availability tracking
- • Review sentiment analysis
- • Inventory level monitoring
- • Dynamic pricing optimization
Financial Data
- • Stock price monitoring
- • Financial report extraction
- • Market sentiment analysis
- • Economic indicator tracking
- • Earnings data collection
Real Estate
- • Property listing aggregation
- • Market price analysis
- • Rental rate tracking
- • Property feature extraction
- • Neighborhood data collection
Media & News
- • News article aggregation
- • Social media monitoring
- • Brand mention tracking
- • Content trend analysis
- • Press release collection
Lead Generation
- • Contact information extraction
- • Company directory scraping
- • Social profile discovery
- • Email verification
- • Professional network analysis
Travel & Hospitality
- • Hotel price comparison
- • Flight availability tracking
- • Review aggregation
- • Amenity data extraction
- • Booking trend analysis
Performance Optimization Strategies
Speed Optimization
Selective JavaScript rendering:
Only use render_js when the content requires it
Specific wait conditions:
Use precise CSS selectors for wait_for parameter
Optimal proxy selection:
Choose datacenter proxies for speed, residential for stealth
Concurrent processing:
Use bulk endpoints for multiple URLs
Reliability Best Practices
Implement retry logic:
Handle temporary failures with exponential backoff
Monitor success rates:
Track scraping success and adjust strategies
Use appropriate timeouts:
Set wait_time based on typical page load speeds
Handle edge cases:
Plan for CAPTCHAs, rate limits, and IP blocks
Common Issues & Solutions
Issue: JavaScript Content Not Loading
Symptoms:
- • Empty or incomplete content
- • Missing dynamic elements
- • Placeholder text instead of data
Solutions:
- • Enable render_js: true
- • Use specific wait_for selectors
- • Increase wait_time parameter
- • Add necessary custom_headers
Issue: Getting Blocked or Rate Limited
Symptoms:
- • 403 Forbidden responses
- • CAPTCHA challenges
- • Consistent timeouts
Solutions:
- • Use residential proxies
- • Add realistic user_agent
- • Implement request delays
- • Rotate proxy countries
Issue: High Credit Consumption
Symptoms:
- • Unexpectedly high costs
- • Slow processing times
- • Unnecessary feature usage
Solutions:
- • Disable unused extractions
- • Use datacenter proxies when possible
- • Optimize wait conditions
- • Batch similar requests