If you're using an advanced lens model and run the (Advanced) Lens Workflow script, you may find that the generated padding is very large, no matter what the settings. That is a result of having source images with “unrelated” horizontal and vertical resolutions. It can addressed with the “Maximum rounding error” value at the bottom of the Lens Workflow panel.
Details: The advanced lens workflow works to maintain the exact aspect ratio of the original image for the highest accuracy results, in order to avoid creating subtle changes that create tiny sliding errors around the edges (or having to specify the exact aspect ratio to 8 digits of precision).
For standard 16:9 footage, you can add any multiple of 16 to width and any multiple of 9 to height and the aspect is identical, as long as the multiples are the same, so the “extra” padding to maintain accuracy is negligible.
But for less regular sizes, the required addition can be much larger. From grade school arithmetic class, you can reduce the Width/Height ratio to a “proper fraction” by removing any factors in common between the two, ie dividing both numbers by the greatest common factor. The proper fraction tells you what must be added to each to keep the aspect the same. For example, one customer asked about 2160H x 882V images; the greatest common factor is 18, so padding can only be added in increments of 120H x 49V pixels. It can be worse for higher-resolution images!
How to reduce the padding: Use the Advance Lens Workflow's “Maximum rounding error” value. The idea is that if the padding is unconscionably large, you can start increasing the error a little bit at a time until it's able to find a size with tighter padding that you can live with, at the expense of a slightly higher error at the edge. You get to make the tradeoff between the small amount of error being introduced and how much padding results. You might try a sequence such as 0.001, 0.002, 0.005, 0.01, 0.05, 0.1. If you reach 0.5, that is the maximum and there will be no additional padding added to maintain the aspect ratio.
This issue is an unfortunate weird and painful consequence of using odd image sizes — but one that's still easily addressable. If you're picking a standardized resolution, you might consider doing so in a way that has a large common factor, ie a proper faction composed of small numerator and denominator.