MongoDB: Reading a large file vs uploading in a collection

38 views Asked by At

I am in a dilemma about the design choice that I need to make for my application.

My application involves the user selecting a domain name for bidding. The valid domain names are stored in a file, and the user should select the domain name from that list only.

A few points about the file containing the valid domain names:

  • Domain name objects are stored in a JSON file (attaching)
  • It will have an average of 560K records.
  • It will be updated every day in an FTP server and should be downloaded from there every day.

Option 1: Read the file every day and upload it to a Mongo collection. When the user types a domain name, it's validated against the records in the collection. Problem: There will be a lot of operations performed on the collection (read, write, update). It will lead to high costs since the collection is hosted on the cloud.

Option 2: Do not upload to a Mongo collection, but read the file to validate the name when the user types a domain name. Problem: It might lead to performance issues if multiple users are using the application.

Guys, can you please help me out here?

Also, I am fairly new to the Nosql database.

Thanks

Edit 1: I can not find how to upload the file. So here is how the object would look like in the file (and there will be 560K such objects in the file):

{
    "domainName": "VETERANFAMILYHOME.COM",
    "link": "https://auctions.godaddy.com/trpItemListing.aspx?miid=547130809&isc=json_expiring",
    "auctionType": "Bid",
    "auctionEndTime": "2024-03-26T18:19:00",
    "price": "$25",
    "numberOfBids": 0,
    "domainAge": 3,
    "pageviews": 2,
    "valuation": "$1,156",
    "monthlyParkingRevenue": "$0",
    "isAdult": false
}
0

There are 0 answers