DataImport is Processing 867677 records added only 850789 records. Missing only when the record set is >800000

26 views Asked by At

I have configured a SOLR core where the DB config will connect to oracle and get the records.

The total unique records with grouping on all of the columns is 854,076.

But when I execute *:* on the same core there are 100 to 220 records missing every time.

I have tried to increase:

solr.ulog.numVersionBuckets 1000000
solr.autoSoftCommit.maxTime 100
solr.autoCommit.maxTime 500

<processor class="org.apache.solr.update.processor.SignatureUpdateProcessorFactory">
    <bool name="enabled">true</bool>
    <bool name="overwriteDupes">false</bool>
    <str name="signatureField">UUID</str>
    <str name="signatureClass">org.apache.solr.update.processor.TextProfileSignature</str>
</processor>

All the records are unique because we have the Grouping in the query. It can't be duplicate at all.

If there are 854,076 why not same number available in the numOfFound in the core when I query using *:*.

We have the unique field as UUID and its generated based on all of the column value.

0

There are 0 answers