Scalability of Node.js Endpoint for File Operations

Question

Problem Statement

I'm working on a Node.js endpoint that performs the following tasks:

1. Downloading image files from a URL.
2. Zipping these downloaded images.
3. Uploading the zipped file to Azure Blob Storage.

These tasks are being executed directly on the Node.js server using streams.

Scalability Concerns

I'm concerned about the scalability of this approach due to the high I/O operations involved in downloading, zipping, and uploading files. I'm worried about the potential impact on the server's performance as the number of requests increases.

Additional Information

For context, the server is expected to handle files of size up to 40MB for download and compression. The anticipated number of concurrent users is around 1000.

Question

Is this a scalable approach? If not, what are the potential bottlenecks and how can they be addressed? Are there better alternatives to handle such operations that could minimize the load on the server and maximize scalability?

I'm open to any advice or suggestions. Thanks in advance for your help!

Attempts So Far

I've set up the Node.js server and written the code for downloading the images from the URL using the `http` module. I'm using the `archiver` library to zip these files and the `azure-storage` package to upload the zip file to Azure Blob Storage.

The code works fine when handling a small number of requests. However, when I simulate a load of 1000 concurrent users, each downloading and zipping a 40MB file, I notice a significant slowdown and some requests even time out.

Expectations

I was expecting the server to handle the load without significant performance degradation. However, the current setup doesn't seem to scale well with an increasing number of requests. I'm not sure if the issue is with the way I'm using streams for file operations, or if it's a limitation of the server itself.