Site overhead is the time spent starting up (and shutting down) parallel execution. This overhead includes creating threads, scheduling those threads onto cores, and waiting for the threads to begin executing. In some parallel framework implementations, real threads are only created once - rather than destroying them at the end of a parallel site; the implementation suspends the threads. In this case, the full site overhead will be experienced only the first time a site is entered.
Site overhead is proportional to the number of times a site is executed. If you have a site that is executed too frequently or where the average time per instance is too small, you should choose a location for your site that encloses a larger amount of computation.
If the Suitability tool recommends that you reduce site overhead, the parallel site is probably too small.
To reduce Site overhead, have the site do more work during its execution. You might be able to combine multiple site executions into one. For example, consider putting a site outside a loop instead of inside a loop.