Running pool.map when multithreading Mandebrot Set causes Memory Error (Python 3.x)

Question

I'm creating a program that displays the Mandelbrot Set using numpy and plt. I was able to figure out how to do this, but the process took ages to render an image. So, I tried to implement the multiprocessing library. I've seen success when my image is low quality (<= 2^14) but after this point, I run into memory errors. I've included the two errors I got when running at 2^15 and 2^16 resolution. I issue lies with pool.map, and I believe the specific part of the problem is when it tries to re-merge all the chunks into one array.

import numpy as np
import matplotlib.pyplot as plt
import multiprocessing as mp
import warnings
import time

start = time.time() #to time program

warnings.filterwarnings("ignore") #For the overflow and invalid exponent runtime warnings

b = 8192*8 #Length and Width of the array/img

#-----------------------------Calculating converge/diverge------------------------------#

def calculate_mandelbrot_chunk(chunk):
    Y, X, start_row, end_row = chunk
    c = (X[start_row:end_row, :] + Y[start_row:end_row, :] * 1j) * (4/b) #complex numbers array
    c = c.astype(np.complex64)      #save memory
    chunk_result = np.zeros([end_row - start_row, b], dtype=np.uint8) 
    for k in range(64):
        if k in [2,3,4,5]:  #skip the divergent pixels
            try:  #optimize try by making it run for only some iterations
                chunk_result = chunk_result**2 + c
            except RuntimeWarning as e:
                if "in" in str(e):
                    break
        else:
            chunk_result = chunk_result**2 + c    
        
    chunk_result[chunk_result < 100] = 1  #filter divergents out
    chunk_result = np.real(chunk_result)  #type casting 
    return chunk_result


#-----------------------------Multi Threading------------------------------#
if __name__ == "__main__":
    num_cores = mp.cpu_count() 
    Y, X = np.mgrid[-1 * (b // 2):b // 2, -1 * (b // 2):b // 2]  #inclue all four quadrants about (0,0), not just Quadrant I
    
    chunk_size = b // num_cores  
    chunks = []
    
    for i in range(num_cores):   #create the chunks
        start_row = i * chunk_size
        end_row = (i + 1) * chunk_size if i < num_cores - 1 else b
        chunks.append((Y, X, start_row, end_row))

    pool = mp.Pool(processes=num_cores)
    results = pool.map(calculate_mandelbrot_chunk, chunks) #this is the memory error for b > 2^14
    pool.close()
    pool.join()
    a = np.vstack(results)  
#-----------------------------Multi Threading------------------------------#


    end = time.time()
    print(end - start)

    plt.imshow(a, cmap="gray", extent=(-2, 2, -2, 2))
    plt.colorbar() 
    plt.show()

I have a 12th Gen Intel(R) Core(TM) i7-12700 and 32Gb ram. How can I prevent a memory error happening?

I've included some images from task manager:

I've tried optimizing my calculate_mandelbrot_chunk(chunk) method by adding try and except blocks that skip diverging values (hopefully) and I've tried decreasing the amount of cores I use from 20 to 10. I've tried creating a shared memory array, but that was far too complex for me to figure out. I also tried changing the data types np.uint8 and np.complex64, but nothing so far has worked.

If I try using my GPU (RTX 3060TI), I would be able to do everything faster, but I would still run into a memory overflow. (I also don't know how to implement GPU CUDA. If anyone has a tutorial I'd really appreciate it!)

Error at 2^16 x 2^16 image:

Traceback (most recent call last):
  File "d:\Program setups\Visual Studio Code\Projects\MandelbrotSet.py", line 45, in <module>
    results = pool.map(calculate_mandelbrot_chunk, chunks) #this is the memory error for b > 2^14
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\multiprocessing\pool.py", line 367, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\multiprocessing\pool.py", line 774, in get
    raise self._value
  File "C:\Program Files\Python311\Lib\multiprocessing\pool.py", line 540, in _handle_tasks
    put(task)
  File "C:\Program Files\Python311\Lib\multiprocessing\connection.py", line 205, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\multiprocessing\reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
MemoryError

Error at 2^15:

Traceback (most recent call last):
  File "d:\Program setups\Visual Studio Code\Projects\MandelbrotSet.py", line 45, in <module>
    results = pool.map(calculate_mandelbrot_chunk, chunks)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\multiprocessing\pool.py", line 367, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\multiprocessing\pool.py", line 774, in get
    raise self._value
  File "C:\Program Files\Python311\Lib\multiprocessing\pool.py", line 540, in _handle_tasks
    put(task)
  File "C:\Program Files\Python311\Lib\multiprocessing\connection.py", line 205, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "C:\Program Files\Python311\Lib\multiprocessing\connection.py", line 279, in _send_bytes
    ov, err = _winapi.WriteFile(self._handle, buf, overlapped=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 87] The parameter is incorrect

Let me know if there's anything else I can do to provide more information. I looked at the post Memory Error with Multiprocessing in Python, but it wasn't able to answer my question well.

EDIT: Further testing allowed me to narrow the error range down to working at 22500, and crashing at 25000.