According to an update to Google's help document, Googlebot crawl's the first 15 megabytes of a webpage and anything beyond this limit will not be considered in rankings calculations.
Google has elaborated more details in the help document:
“Any resources referenced in the HTML such as images, videos, CSS and JavaScript are fetched separately.
After the first 15 MB of the file, Googlebot stops crawling and only considers the first 15 MB of the file for indexing.
The file size limit is applied on the uncompressed data.”
Many people were left wondering if this meant that Googlebot would disregard all text below images at the cutoff point in HTML files.
John Mueller, Google Search Advocate, later clarified via Twitter:
“It’s specific to the HTML file itself, like it’s written,”
“Embedded resources/content pulled in with IMG tags is not a part of the HTML file.”
Important content must be near the top of web pages to guarantee it is weighted by Googlebot. Code needs to be structured to put SEO-relevant info inside the first 15 MB within the HTML file.
Whenever possible, both images and videos should be compressed rather than encoded directly into the HTML.
Many sites that have been properly created using SEO best practices are specifically made to be 100 KB or less, these sites will not be affected. You can check page size to verify using a variety of tools, one being Google Page Speed Insights.
While there is a chance that content on a webpage won't get used for indexing, however, 15 MB is actually a pretty large amount of HTML.
Google’s John Mueller has clarified in a tweet: “This is not a change, it’s just not previously been officially documented…”
If a webpage happens to exceed 15 MB of HTML then there are likely issues already and they should be addressed as soon as possible.
