For Hacktoberfest, I decided my project this year was going to be useful to those who are administrators of a CloudBees CI environment and are always being pestered about Plugins and their Support level (Tier 1, 2, 3 or completely off the radar) to encourage those administrators of Controllers to manage their Plugins more effectively.

With the advent of the Plugin Usage Analyzer Plugin from CloudBees, this job is made much easier, but it’s still all processed through a UI and as an administrator in a past life I realize the value and speed and efficiency gained by using command line tools.

With that, I present my project in two parts – first, a script written in Ruby that parses a Plugins list from a Controller generated via the Jenkins CLI and returns a few lists containing which Plugins fit into which category. Second, a script written in Ruby that parses the analysis.json file output by the CloudBees Plugin Usage Analyzer and returns not only a few lists with which Plugins fit into which Support level, but also how many Pipelines have referenced the plugin and how long ago it was last referenced.

Before We Begin

You can find this code on my GitHub repo. Feel free to add to it if you’d like to contribute.

Running the Scripts

To run these scripts, you’ll need the following:

• Ruby (the latest version will do)
• The HTTParty gem (gem install httparty)
• An output of the Plugins List from a Controller for the first script – java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json
• An output of the Plugin Usage Analyzer from the UI (analysis.json by default)

Plugin List Parse for Support Level

The first script takes a file generated from the Jenkins CLI and returns which Support Level it lives at.

To get the JSON list from a Controller: java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json

# Requiring JSON parsing tools
require 'json'
# httparty is a separate gem - use `gem install httparty` to install
require 'httparty'
# Readline provides autocomplete functionality for file paths
require 'readline'

# Take the input for the filename to parse
puts 'Enter the file name: '
file_path = Readline.readline("> ", true).rstrip

# Validation the file exists
if File.file?(file_path) == false
  puts "The file #{file_path} doesn't exist or is not a file. Please try again."
  exit
end

# This is our JSON file we're parsing
# Generate this from CLI using this CLI command:
# java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json
file = File.read(file_path)

# Parse the file from JSON into a hash
installed_plugin_list = JSON.parse(file)

# Plugin List hash
pl = {
  'proprietary': {'count': 0, 'list': [], 'desc': 'Proprietary/Verified (Tier 1) Plugins'},
  'compatible': {'count': 0, 'list': [], 'desc': 'Compatible (Tier 2) Plugins'},
  'community': {'count': 0, 'list': [], 'desc': 'Community (Tier 3) Plugins'},
  'unsupported': {'count': 0, 'list': [], 'desc': 'Unsupported Plugins'},
  'overall': {'count': 0}
}

# Loop our list of plugins hash
installed_plugin_list['data'].each do |key, _value|
  # Increment overall count number
  pl[:overall][:count] += 1
  # Var builds our URL
  url = "https://docs.cloudbees.com/api/search?provider=ci-plugins&filters%5Btext%5D=#{key['id']}&filters%5Bcategory%5D="
  # HTTParty simplifies retrieving JSON results from a URL
  begin
    response = HTTParty.get(url)
  rescue => error
    puts error
    puts "Failed to get URL Please try again."
    exit
  end
  # If it's not in the CI plugins list, it is missing
  if response['count'] == 0
    # Pushes plugin to the unsupported list
    (pl[:unsupported][:list] ||= []) << key['id']
    # Increment unsupported count
    pl[:unsupported][:count] += 1
  else
    # checking to validate the slug matches the plugin id for accuracy
    # create result hash [empty]
    result = {}

    # check each result to ensure it matches our slug
    response['results'].each do |k,v|
      if k['metadata']['artifactId'] == key['id'] 
        # Slug matches a result
        # assign correct entry to result var
        result = k['metadata']
      end
    end

    if result.empty?
      # Slug doesn't match any results
      # Pushes plugin to the unsupported list
      (pl[:unsupported][:list] ||= []) << key['id']
      # Increment unsupported count
      pl[:unsupported][:count] += 1
      # skip to next iteration of each()
      next
    end
    # verified is Tier 1, so we force it to "proprietary" for the type
    result['tier'] == 'verified' ? type = :proprietary : type = result['tier'].to_sym
    # Pushes the plugin to the list for the type
    (pl[type][:list] ||= []) << key['id']
    # Increments the count of the plugin type
    pl[type][:count] += 1
  end
end

# Header for the output
puts "Plugins List - #{pl[:overall][:count]} Total"

# Loop each grouping and write to output
pl.each do |k, _v|
  # Skip if it's the overall entry
  next if k == :overall
  puts "\n#{pl[k][:desc]} [#{pl[k][:count]}]:"
  puts "====================\n\n"
  puts pl[k][:list]
end

Plugin Usage Analyzer Output Parse

This script will parse the output from the Plugin Usage Analyzer, which can be downloaded via the UI.

# Requiring JSON parsing tools
require 'json'
# httparty is a separate gem - use `gem install httparty` to install
require 'httparty'
# Readline provides autocomplete functionality for file paths
require 'readline'

# Function for parsing number of occurrences
def count_occ(usages_list)
  count = 0
  usages_list.each do |k,v|
    if k['location']['type'] == 'pipeline'
      count += 1
    end
  end
  return count
end

def days_since(usages_list)
  # Returns the last usage of the plugin in a pipeline
  date_stamp = 0
  usages_list.each do |k,v|
    if k['location']['type'] == 'pipeline' && k['instant']['epochSecond'] > date_stamp
      date_stamp = k['instant']['epochSecond'] 
    end
  end
  now = Time.now.to_i
  date_stamp = now if date_stamp == 0 || date_stamp == nil
  since = (now - date_stamp.to_i) / (24 * 60 * 60)
  puts since
  output = since > 0 ? "- #{since} days ago" : ""
  return output
end

puts 'Enter the file name: '
file_path = Readline.readline("> ", true).rstrip

if File.file?(file_path) == false
  puts "The file #{file_path} doesn't exist or is not a file. Please try again."
  exit
end

# This is our JSON file we're parsing
# Generate this from CLI using this CLI command:
# java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json
file = File.read(file_path)

# Parse the file from JSON into a hash
installed_plugin_list = JSON.parse(file)

# Plugin List hash
pl = {
  'proprietary': {'count': 0, 'list': [], 'desc': 'Proprietary/Verified (Tier 1) Plugins'},
  'compatible': {'count': 0, 'list': [], 'desc': 'Compatible (Tier 2) Plugins'},
  'community': {'count': 0, 'list': [], 'desc': 'Community (Tier 3) Plugins'},
  'unsupported': {'count': 0, 'list': [], 'desc': 'Unsupported Plugins'},
  'overall': {'count': 0}
}

# Loop our list of plugins hash
installed_plugin_list['usages'].each do |key, value|
  # Increment overall count number
  pl[:overall][:count] += 1
  # Var builds our URL
  url = "https://docs.cloudbees.com/api/search?provider=ci-plugins&filters%5Btext%5D=#{key}&filters%5Bcategory%5D="
  # HTTParty simplifies retrieving JSON results from a URL
  begin
    response = HTTParty.get(url)
  rescue => error
    puts error
    puts "Failed to get URL Please try again."
    exit
  end
  # If it's not in update center, it is missing

  # TODO: Find the last ['instant']['epochSecond'] entry (most recent) and then find how many days since

  if response['count'] == 0
    # Pushes plugin to the unsupported list
    (pl[:unsupported][:list] ||= []) << "#{key} [#{count_occ(value)}] #{days_since(value)}"
    # Increment unsupported count
    pl[:unsupported][:count] += 1
  else
    # checking to validate the slug matches the plugin id for accuracy
    # create result hash [empty]
    result = {}

    # check each result to ensure it matches our slug
    response['results'].each do |k,v|
      if k['metadata']['artifactId'] == key
        # Slug matches a result
        # assign correct entry to result var
        result = k['metadata']
      end
    end

    if result.empty?
      # Slug doesn't match any results
      # Pushes plugin to the unsupported list
      (pl[:unsupported][:list] ||= []) << "#{key} [#{count_occ(value)}] #{days_since(value)}"
      # Increment unsupported count
      pl[:unsupported][:count] += 1
      # skip to next iteration of each()
      next
    end
    # verified is Tier 1, so we force it to "proprietary" for the type
    result['tier'] == 'verified' ? type = :proprietary : type = result['tier'].to_sym
    # Pushes the plugin to the list for the type
    (pl[type][:list] ||= []) << "#{key} [#{count_occ(value)}] #{days_since(value)}"
    # Increments the count of the plugin type
    pl[type][:count] += 1
  end
end

# Header for the output
puts "Plugins List - #{pl[:overall][:count]} Total"

# Loop each grouping and write to output
pl.each do |k, _v|
  # Skip if it's the overall entry
  next if k == :overall
  puts "\n#{pl[k][:desc]} [#{pl[k][:count]}]:"
  puts "====================\n\n"
  puts pl[k][:list]
end

If you find either of these scripts handy, please let me know via Twitter. Enjoy!

* Updated 10/20/2021 to fix an issue with timestamps in the second script.