I have a rather large website, with some 1.4 million pages. As a result of this, I've split up my sitemaps into sitemaps with 10,000 pages each and reference these in a sitemap index which is compliant to the sitemap schema. Google search console identifies that this is a sitemap index however it returns the error "Your Sitemap exceeds the maximum file size limit. Please create multiple Sitemaps containing fewer URLs to meet the file size limit." The sitemap index itself is 10KB.
I could submit all the sitemaps individually, however this is not auto-scalable.
Is there a limit for the number of sitemaps that can be provided within a sitemap index before the sitemap index needs to be split into multiple indexes? I'm struggling to find information as it's not a common use-case.
TLDR; Split sitemaps into 10,000 pages each (~1.1MB) and referenced in a sitemap index. This stopped working somewhere around 100 sitemap indexes (not sure when this broke) and while manually referencing the sitemaps in GSC works, it does not allow for automatic detection of new sitemaps. I would rather not have to write a web automation to add these.