This lesson is for subscribers
You've completed the free preview. Subscribe to unlock every lesson in every course.
Lesson 1850 of 1,919·44. Real-World System: Web CrawlerPro lesson
Frontier Scalability: Sharding and Distribution
Distributing the URL frontier across multiple machines by host or URL hash while maintaining politeness guarantees.