While working with a client, we discovered a need to locate which jobs were running directly on their on-controller executors, as the system had become slow and non-responsive when lots of jobs were running. When properly using Jenkins agents, this should not be an issue as the workload should be offloaded to those agent systems.
A simple way to search would be to
grep through the job run logs to see where they’re running directly on the controller.
grep -rn --include='log' "Running on .*Jenkins" .
Using the above snippet, it will dig through all files named
log while recursing subdirectories, and searching specifically for the text “Running on” followed by any series of text, then the word “Jenkins”. In the logs, when running on any node it will say “Running on” followed by a link to the node and then the name of the node, so we’re parsing through to locate the word Jenkins immediately following that URL.
It could definitely be improved, but hopefully this is a good start point to help you slim down the number of jobs eating controller resources.