Angular 11 - Data volumetry issues

Question

My application works fine, except when the response from the service brings back a high amount of data.

This would be the response of the service: (the problem comes when the "errors" array has more than 10000 elements)

{
   "output":[
      {
         "id":1,
         "status":"OK",
         "errors":[
            
         ]
      },
      {
         "id":2,
         "status":"OK",
         "errors":[
            
         ]
      },
      {
         "id":3,
         "status":"ERROR",
         "errors":[
            {
               "id":1,
               "name":"Name1"
            },
            {
               "id":2,
               "name":"Name2"
            }
         ]
      }
   ]
}

When this happens the browser crashes and I get a message to close the tab. (I'm working on Chrome, but the App is for all browsers). When the service responds 200, but brings a high volume of data, the loading icon stays for a while and the browser ends up giving an error (The connection to the Chrome debugger is also closed):

I am trying with chunk method (slice function), but I lose information and I am currently investigating "Web Workers" that I have never used

  private chunkArray(arr, size: number) {
    return arr.length > size
    ? [arr.slice(0, size), ...this.chunkArray(arr.slice(size), size)]
    : [arr];
  }

This is the service call from component:

this.processesService.getData(id).subscribe(response => {
    console.log(response)
}, _  => {this.router.navigate(["home/" + id])});

This is the service:

public getData(id: number): Observable<RestResponse<DataOutput[]>> {
      let serviceName = "/processes/data";
      if (id) {
        serviceName = serviceName + '?iddata=' + id;
      }
      return this.service.get<any>(serviceName)
      .pipe(
        tap(_ => this.log('fetched data')),
        catchError(this.handleError<DataOutput[]>('getData', []))
      );
  }

NOTE: Sometimes it randomly loads everything correctly

Any suggestion? Perhaps it would be more convenient to solve the problem from the back-end? or better front end? Thanks,

Answer 1

Yes, you need to use server side rendering on demand with large data sets. On the frontend I can think of two options with Angular to consume your data. You can use

• virtual scroll Or
• data-tables with pagination

There are stable libraries you can use to achieve this, such as AG Grid and Angular Material.

Because you have large data set you will encounter more problems down the line. This is because probably your users will need filtering and sorting as well. These libraries support these features on server side also.