All Collections
Web Unblocker
Can rendered HTML size differ from calculated traffic?
Can rendered HTML size differ from calculated traffic?
Josh avatar
Written by Josh
Updated over a week ago

Yes, the size of a rendered HTML file can differ from the traffic calculated for a scraping job due to dynamic content loading.

Simply put, dynamic websites rely on code rendered upon browser loading. As a result, the scraping content doesn't exist before the page load. This mandates including a browser rendering step in the web scraping process.

To summarize, if JavaScript rendering is applied when web scraping, it can result in extra requests to load additional resources, such as CSS or JavaScript files. These resources contribute to the overall calculated traffic because gathering public data from websites without loading them increases the risk of getting blocked significantly.

It's important to note that we're blocking loading unnecessary resources, such as fonts, images, or media files, because loading them would increase traffic noticeably and slow down the web scraping process.

Did this answer your question?