cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2304
Views
0
Helpful
0
Comments
AdvocateRick
Cisco Employee
Cisco Employee

Automation is the one of the keys to a successful DevOps department. There are many workflow engines, like the one in Cisco’s SecureX. This blog discusses one way to automate launching connector runs.

Why automate connector runs? One reason is because you will have vulnerability risk information at the same time each day. This provides consistent data for statistical purposes. Unburdening people from launching connector runs manually gives time to your DevOps personnel for other projects.

Introduction

There are two code examples to draw from:

  • connectors_auto_start.py - scans the list of connectors and determines when to launch a connector run and is discussed in this blog.
  • show_connector_status.py - scans the list of connectors and informs you when a connector would be launched.

After experimenting with the connector APIs, I created connectors_auto_start.py. The code example, show.connector_status.py, is what the experimentation left behind. It is not polished, but has some useful functions.

I decided to use the PrettyTable library for output in connectors_auto_start.py. This library is easy to use and is great for making readable output. Please use the requirements.txt to install the libraries.  I’ll discuss more about PrettyTable later.

The flow of connectors_auto_start.py is:

  1. Obtain a list of connectors.
  2. For each connector, get the latest connector run.
  3. If the criteria are met, launch the connector run.

Obtaining connectors

To obtain a list of connectors, the List Connector API is invoked. The response information contains the connector name, the connector ID, the connector host address, and if the connector is currently running. Later on the host address determines whether this connector is a file based connector or not. The response is returned to the caller.

11 # Returns connector information from the List Connectors API.
12 def get_connectors(base_url, headers):
13.    connectors = []
14.    list_connectors_url = base_url + "connectors"
15     list_connectors_url = f"{base_url}connectors"
16
17.    response = requests.get(list_connectors_url, headers=headers)
18.    if response.status_code != 200:
19.        print(f"List Connector Error: {response.status_code} with {list_connectors_url}")
20.       sys.exit(1)
21
22.       resp_json = response.json()
23.       connectors = resp_json['connectors']
24
25.     return connectors

Obtaining connector runs

With the list of connectors, we eliminate the file-based connectors. Why? Because I didn’t want to code uploading a file at this time. (Another blog perhaps). Non-base file connectors have a host associated with them.

109         # Check if connector is file based.
110         if connector['host'] is None:
111             conn_tbl.add_row([name, "is a file base connector"])
112             

Here the main code obtains a list of connector runs. If a connector doesn’t have any runs, then the automation won’t try to launch it. Why? Because if it hasn’t been run at least once, it hasn’t been tested.

114         # Obtain the connector runs for a connector.
115         connector_runs = get_connector_runs(base_url, headers, id)
116         if len(connector_runs) == 0:
117             conn_tbl.add_row([name, "has no runs"])
118             continue

The get connector runs function invokes the List Connector Runs API to obtain a list of connector runs for each connector. The connector ID is used to specify the connector.

 26 # Gets the connector runs for the specified connector ID.
 27 def get_connector_runs(base_url, headers, connector_id):
 28     get_connector_runs_url = f"{base_url}connectors/{connector_id}/connector_runs"
 29 
 30     response = requests.get(get_connector_runs_url, headers=headers)
 31     if response.status_code != 200:
 32         print(f"List Connector Runs Error: {response.status_code} with {get_connector_runs_url}")
 33         sys.exit(1)
 34 
 35     resp_json = response.json()
 36 
 37     return resp_json

Note that the JSON return is the response since the array does not have a field value.

Launching the connector run

Two more criteria to go. The first criterion checks if the connector is already running by checking the start and end time stamps on the latest run. If the end time stamp is not available, then the connector is running. I think this is a nice feature which I discovered while experimenting, which means that this feature is not in the current API documentation.

120         # Only check the latest run
121         latest_run = connector_runs[0]
122         start_datetime = parse_time_str(latest_run['start_time'])
123         if latest_run['end_time'] is None:
124             conn_tbl.add_row([name, "still running"])
125             continue
126         end_datetime = parse_time_str(latest_run['end_time'])

If the connector is not running, then grab the end time stamp and proceed to the second criterion which checks if enough time has elapsed since the connector ended. The elapsed time can be passed as a command line input parameter. If not specified, it defaults to 24 hours.

128         # Check if the end time was interval hours ago.
129         if (end_datetime + timedelta(hours=interval_hours)) > datetime.now():
130             conn_tbl.add_row([name, f"has to wait {interval_hours} hours past {end_datetime}."])
131             continue

All criteria have been met, so launch the connector with the Run Connector API. The connector run ID is returned.

 39 # Starts a connector run based on the connector ID and returns a connector run ID.
 40 def run_connector(base_url, headers, connector_id):
 41     run_connector_url = f"{base_url}connectors/{connector_id}/run"
 42 
 43     response = requests.get(run_connector_url, headers=headers)
 44     if response.status_code != 200:
 45         print(f"Run Connector Error: {response.status_code} with {run_connector_url}")
 46         sys.exit(1)
 47 
 48     resp_json = response.json()
 49     if not resp_json['success']:
 50         print(f"Running {connector_id} failed.  Check log files.")
 51         sys.exit(1)
 52 
 53     return resp_json['connector_run_id']

The connector run ID is used in the output.

133         # Launch the connector if all tests passed.
134         connector_run_id = run_connector(base_url, headers, id)
135         conn_tbl.add_row([name, f"launched connector run {connector_run_id}."])

PrettyTable

As stated earlier, the PrettyTable library is straight-forward to use. Here is where the table is initialized.

 96     # Set up the output table
 97     conn_tbl = PrettyTable()
 98     conn_tbl.field_names = ["Connector Name", "Status"]
 99     conn_tbl.align["Connector Name"] = "r"
100     conn_tbl.align["Status"] = "l"
101     conn_tbl.border = False
102     conn_tbl.add_row(["~~~~~~~~~~~~~~", "~~~~~~"])

As you can see, there are two columns, “Connector Name” and “Status”. The “Connector Name” column is right aligned, and the “Status” column is left aligned. Borders are turned off, but a row of tildes are added to separate the heading from the rows.

Rows are added with the add_row() method. The table is printed with a print() call.

138     print(conn_tbl)

Conclusion

So now you know how to use the "List Connectors", "List Connector Runs", and "Run Connector" APIs. This blog gives some ideas on how to automate the timing of launching connector runs that should be adaptable to workflow engines.

If you’re interested in playing with these samples, they’re located in the Kenna Security blog_samples repo in the connectors directory. One thought would be to add code to check if a connector is running too long.

Rick Ehrhart

API Evangelist

This blog was originally written for Kenna Security, which has been acquired by Cisco Systems.
Learn more about Cisco Vulnerability Management.

  

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: