When processing large sets of data, out-of-memory errors can occur when the size of an intermediate result exceeds the available memory allocated to the JVM. If you are encountering these errors, you can tune FetchSize, ResultMemorySize, and the JVM heap size to fit your environment.
Reduce FetchSize to reduce demands on memory. By lowering the number of rows as specified by FetchSize, you lower the number of rows the driver is required to process before returning data to the application. Thus, you reduce demands on memory and decrease the likelihood of out-of-memory errors.
Decrease ResultMemorySize until out-of-memory errors are eliminated. Intermediate results larger than the value specified will be written to disk as opposed to held in memory. When configured correctly, this avoids memory limitations by not relying on memory to process larger intermediate results. Be aware that while writing to disk reduces the risk of out-of-memory errors, it also negatively impacts performance. For optimal performance, decrease this value only to a size necessary to avoid errors. By default, ResultMemorySize is set to -1, which sets the maximum size of intermediate results held in memory to a percentage of the max Java heap size. If you received errors using the default configuration, use the max Java heap size divided by 4 as a starting point when tuning this option.
Increase the JVM heap size. By increasing the max Java heap size, you increase the amount of data the driver can accumulate in memory and avoid out-of-memory errors.
Reduce FetchSize to reduce demands on memory. By lowering the number of rows as specified by FetchSize, you lower the number of rows the driver is required to process before returning data to the application. Thus, you reduce demands on memory and decrease the likelihood of out-of-memory errors.