How to use search_after in elastic4s

Question

i'm trying to get some data based on query with elastic4s in scala using search_after because i have more then 10 000 items.

I'm trying something like this:

client.execute(
   search(index).query(q).searchAfter(id).from(start).limit(limit)
)

But it's not working. When i delete that searchAfter, it's working fine but only for first 10 000 items. I'm new with elastic so any help will be appreciated. Thanks

Answer 1

elasticsearch will always return a maximum of 10,000 results on a regular search. when you need to page beyond 10,000 results (in a batch process for instance) you should use the Scroll feature and not a regular search. this will let you "scroll" through all the results in your query, page by page, until the query returns no results. note that the Scroll feature does not allow Sorting.

read more here: Scroll API

Answer 2

I have not used elastic4s, but Elasticsearch requires that you'll provide at least one sort value when using search_after.
See the docs for more details.
My guess is that in your case you'll need to sort by id.