SimplyTim

Archives
  • December 2021
  • November 2021
  • October 2021
  • March 2021
Categories
  • Docker
  • GitHub
  • Jenkins
  • Ruby
  • Shell Scripts
Meta
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org
Skip to content

SimplyTim

0
0
315
    • GitHub
    • Jenkins
    • Ruby

Hacktoberfest: Analyzing CloudBees CI Plugins List for Support Level

  • bytim
  • Posted on October 12, 2021October 20, 2021
  • 238 views
  • 6 minute read
0
0
0

For Hacktoberfest, I decided my project this year was going to be useful to those who are administrators of a CloudBees CI environment and are always being pestered about Plugins and their Support level (Tier 1, 2, 3 or completely off the radar) to encourage those administrators of Controllers to manage their Plugins more effectively.

With the advent of the Plugin Usage Analyzer Plugin from CloudBees, this job is made much easier, but it’s still all processed through a UI and as an administrator in a past life I realize the value and speed and efficiency gained by using command line tools.

With that, I present my project in two parts – first, a script written in Ruby that parses a Plugins list from a Controller generated via the Jenkins CLI and returns a few lists containing which Plugins fit into which category. Second, a script written in Ruby that parses the analysis.json file output by the CloudBees Plugin Usage Analyzer and returns not only a few lists with which Plugins fit into which Support level, but also how many Pipelines have referenced the plugin and how long ago it was last referenced.

Before We Begin

You can find this code on my GitHub repo. Feel free to add to it if you’d like to contribute.

Running the Scripts

To run these scripts, you’ll need the following:

  • Ruby (the latest version will do)
  • The HTTParty gem (gem install httparty)
  • An output of the Plugins List from a Controller for the first script – java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json
  • An output of the Plugin Usage Analyzer from the UI (analysis.json by default)

Plugin List Parse for Support Level

The first script takes a file generated from the Jenkins CLI and returns which Support Level it lives at.

To get the JSON list from a Controller: java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json

# Requiring JSON parsing tools
require 'json'
# httparty is a separate gem - use `gem install httparty` to install
require 'httparty'
# Readline provides autocomplete functionality for file paths
require 'readline'

# Take the input for the filename to parse
puts 'Enter the file name: '
file_path = Readline.readline("> ", true).rstrip

# Validation the file exists
if File.file?(file_path) == false
  puts "The file #{file_path} doesn't exist or is not a file. Please try again."
  exit
end

# This is our JSON file we're parsing
# Generate this from CLI using this CLI command:
# java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json
file = File.read(file_path)

# Parse the file from JSON into a hash
installed_plugin_list = JSON.parse(file)

# Plugin List hash
pl = {
  'proprietary': {'count': 0, 'list': [], 'desc': 'Proprietary/Verified (Tier 1) Plugins'},
  'compatible': {'count': 0, 'list': [], 'desc': 'Compatible (Tier 2) Plugins'},
  'community': {'count': 0, 'list': [], 'desc': 'Community (Tier 3) Plugins'},
  'unsupported': {'count': 0, 'list': [], 'desc': 'Unsupported Plugins'},
  'overall': {'count': 0}
}

# Loop our list of plugins hash
installed_plugin_list['data'].each do |key, _value|
  # Increment overall count number
  pl[:overall][:count] += 1
  # Var builds our URL
  url = "https://docs.cloudbees.com/api/search?provider=ci-plugins&filters%5Btext%5D=#{key['id']}&filters%5Bcategory%5D="
  # HTTParty simplifies retrieving JSON results from a URL
  begin
    response = HTTParty.get(url)
  rescue => error
    puts error
    puts "Failed to get URL Please try again."
    exit
  end
  # If it's not in the CI plugins list, it is missing
  if response['count'] == 0
    # Pushes plugin to the unsupported list
    (pl[:unsupported][:list] ||= []) << key['id']
    # Increment unsupported count
    pl[:unsupported][:count] += 1
  else
    # checking to validate the slug matches the plugin id for accuracy
    # create result hash [empty]
    result = {}

    # check each result to ensure it matches our slug
    response['results'].each do |k,v|
      if k['metadata']['artifactId'] == key['id'] 
        # Slug matches a result
        # assign correct entry to result var
        result = k['metadata']
      end
    end

    if result.empty?
      # Slug doesn't match any results
      # Pushes plugin to the unsupported list
      (pl[:unsupported][:list] ||= []) << key['id']
      # Increment unsupported count
      pl[:unsupported][:count] += 1
      # skip to next iteration of each()
      next
    end
    # verified is Tier 1, so we force it to "proprietary" for the type
    result['tier'] == 'verified' ? type = :proprietary : type = result['tier'].to_sym
    # Pushes the plugin to the list for the type
    (pl[type][:list] ||= []) << key['id']
    # Increments the count of the plugin type
    pl[type][:count] += 1
  end
end

# Header for the output
puts "Plugins List - #{pl[:overall][:count]} Total"

# Loop each grouping and write to output
pl.each do |k, _v|
  # Skip if it's the overall entry
  next if k == :overall
  puts "\n#{pl[k][:desc]} [#{pl[k][:count]}]:"
  puts "====================\n\n"
  puts pl[k][:list]
end
Code language: PHP (php)

Plugin Usage Analyzer Output Parse

This script will parse the output from the Plugin Usage Analyzer, which can be downloaded via the UI.

# Requiring JSON parsing tools
require 'json'
# httparty is a separate gem - use `gem install httparty` to install
require 'httparty'
# Readline provides autocomplete functionality for file paths
require 'readline'

# Function for parsing number of occurrences
def count_occ(usages_list)
  count = 0
  usages_list.each do |k,v|
    if k['location']['type'] == 'pipeline'
      count += 1
    end
  end
  return count
end

def days_since(usages_list)
  # Returns the last usage of the plugin in a pipeline
  date_stamp = 0
  usages_list.each do |k,v|
    if k['location']['type'] == 'pipeline' && k['instant']['epochSecond'] > date_stamp
      date_stamp = k['instant']['epochSecond'] 
    end
  end
  now = Time.now.to_i
  date_stamp = now if date_stamp == 0 || date_stamp == nil
  since = (now - date_stamp.to_i) / (24 * 60 * 60)
  puts since
  output = since > 0 ? "- #{since} days ago" : ""
  return output
end

puts 'Enter the file name: '
file_path = Readline.readline("> ", true).rstrip

if File.file?(file_path) == false
  puts "The file #{file_path} doesn't exist or is not a file. Please try again."
  exit
end

# This is our JSON file we're parsing
# Generate this from CLI using this CLI command:
# java -jar jenkins-cli.jar -s http://controller-url -webSocket -auth user:PassOrToken list-plugins-v2 > plugins.json
file = File.read(file_path)

# Parse the file from JSON into a hash
installed_plugin_list = JSON.parse(file)

# Plugin List hash
pl = {
  'proprietary': {'count': 0, 'list': [], 'desc': 'Proprietary/Verified (Tier 1) Plugins'},
  'compatible': {'count': 0, 'list': [], 'desc': 'Compatible (Tier 2) Plugins'},
  'community': {'count': 0, 'list': [], 'desc': 'Community (Tier 3) Plugins'},
  'unsupported': {'count': 0, 'list': [], 'desc': 'Unsupported Plugins'},
  'overall': {'count': 0}
}

# Loop our list of plugins hash
installed_plugin_list['usages'].each do |key, value|
  # Increment overall count number
  pl[:overall][:count] += 1
  # Var builds our URL
  url = "https://docs.cloudbees.com/api/search?provider=ci-plugins&filters%5Btext%5D=#{key}&filters%5Bcategory%5D="
  # HTTParty simplifies retrieving JSON results from a URL
  begin
    response = HTTParty.get(url)
  rescue => error
    puts error
    puts "Failed to get URL Please try again."
    exit
  end
  # If it's not in update center, it is missing

  # TODO: Find the last ['instant']['epochSecond'] entry (most recent) and then find how many days since

  if response['count'] == 0
    # Pushes plugin to the unsupported list
    (pl[:unsupported][:list] ||= []) << "#{key} [#{count_occ(value)}] #{days_since(value)}"
    # Increment unsupported count
    pl[:unsupported][:count] += 1
  else
    # checking to validate the slug matches the plugin id for accuracy
    # create result hash [empty]
    result = {}

    # check each result to ensure it matches our slug
    response['results'].each do |k,v|
      if k['metadata']['artifactId'] == key
        # Slug matches a result
        # assign correct entry to result var
        result = k['metadata']
      end
    end

    if result.empty?
      # Slug doesn't match any results
      # Pushes plugin to the unsupported list
      (pl[:unsupported][:list] ||= []) << "#{key} [#{count_occ(value)}] #{days_since(value)}"
      # Increment unsupported count
      pl[:unsupported][:count] += 1
      # skip to next iteration of each()
      next
    end
    # verified is Tier 1, so we force it to "proprietary" for the type
    result['tier'] == 'verified' ? type = :proprietary : type = result['tier'].to_sym
    # Pushes the plugin to the list for the type
    (pl[type][:list] ||= []) << "#{key} [#{count_occ(value)}] #{days_since(value)}"
    # Increments the count of the plugin type
    pl[type][:count] += 1
  end
end

# Header for the output
puts "Plugins List - #{pl[:overall][:count]} Total"

# Loop each grouping and write to output
pl.each do |k, _v|
  # Skip if it's the overall entry
  next if k == :overall
  puts "\n#{pl[k][:desc]} [#{pl[k][:count]}]:"
  puts "====================\n\n"
  puts pl[k][:list]
end
Code language: PHP (php)

If you find either of these scripts handy, please let me know via Twitter. Enjoy!

* Updated 10/20/2021 to fix an issue with timestamps in the second script.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recent Posts
  • Mounting a SFTP (SSH) Share as a Volume in Docker-Compose
  • Finding Jobs Running On A Controller Executor
  • Hacktoberfest: Analyzing CloudBees CI Plugins List for Support Level
  • Auditing Durability in Jenkins Jobs
Recent Comments
    Copyright 2021 Tim H.
    Cookies Notice
    We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
    Do not sell my personal information.
    Cookie SettingsAccept
    Manage consent

    Privacy Overview

    This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
    Necessary
    Always Enabled
    Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
    CookieDurationDescription
    cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
    cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
    cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
    cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
    cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
    viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
    Functional
    Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
    Performance
    Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
    Analytics
    Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
    Advertisement
    Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
    Others
    Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
    SAVE & ACCEPT