How to determine the scale factor of image if I want an exact size in bytes

Question

So I would like to store images which are uploaded to an API. However there is a constraint that I don't store images above 1MB, but resize them in order to be lower than 1MB.

I have a very basic formula for this: resizeFactor = MAX_IMAGE_SIZE_IN_BYTES/currentImageSizeinBytes

Then I do this:

 Image image = Image.Load(stream);
 image.Mutate(x => x.Resize((int)(image.Width*resizeFactor), 0));

Now this resizes every image to be under 1MB, but in case of some images they are way more under 1MB, for example an example jpg has 1,785,213 bytes, the resize factor is 0.58 for this, so the result image will be 641,188 bytes.

If I could be able to resize an image to exact 1e^6 bytes +- 50 bytes, would that save more image quality? So with exact example if I could resize the above image from 1.7MB to 1MB instead of 600kB, would it save more image quality?

If yes, how could I resize my images to be very close to 1MB?