As a security professional, you might want a list of vulnerabilities per asset under your purview to assess risk and begin the remediation process. With Kenna Security APIs, you’ll be able to extract the asset and its vulnerability data. This blog will demonstrate different strategies for different numbers of assets.
As you might recall, the Inactive Asset blog demonstrated how to list assets with 500 or less. If you want to list more than 500, you need to fetch pages. With the list assets API, you can list up to 20 pages with 500 assets, each bringing the total to 10,000 assets. If you have more than 10,000 assets, you will need to use the export data APIs. Let’s look at how to fetch pages first.
Fetching Asset Pages
The code snippet above adds the page query parameter to a base URL, in this case: https://api.kennasecurity.com/assets.
The rest of the code just loops through the pages, counting them up to the maximum number of pages obtained from the meta attribute.
This code also checks to see if the number of asset pages is greater than 20, the maximum number of pages, and a message is printed if there are over 20 pages or 10,000 assets. The pages beyond 20 are not accessible; therefore, you will have to export them to obtain all the assets.
Exporting Asset Data
To export all the assets, the request data exports API is used. Since this can take anywhere from 30 seconds to 10 minutes, there is a check data export status API to check when the data can be downloaded via the retrieve data export API.
The code located in export_assets.py requests a data export for assets, checks the status and downloads the gzip file containing all the assets. Let’s look at the code.
The filter_params inform the request data exports API what to request. The model field is set to “asset” because we want assets, while the format field is set to “jsonl,” which indicates JSON lines. I used JSONL so that I didn’t have to suck the whole JSON output in at one time. Only the “active” assets are being requested. The records_updated_since field is commented out for now. You can uncomment it when you want to perform incremental exports. A search ID is returned, which is used to check the export status and retrieve a gzip file.
Here is the code that checks the data export status using the check data export status API:
The key points are that search_id is a query parameter, and the return message we’re looking for is “Export ready for download.”
The code attempts to calculate the amount of time it takes for the export to be ready while checking the status every five or ten seconds, depending on the estimated time.
To retrieve the asset data, use the retrieve data export API:
Again the search_id is a query parameter. One way to download the gzip file is with streaming shown here with stream=True set in the requests.get() call along with the Response.iter_contentwith an open gzip file.
After the gzip is downloaded, it is unzipped, and lines are counted. File names are: assets_
Obtaining Vulnerabilities per Asset
Now that we know how to obtain assets let’s look at how to obtain the vulnerability information for each asset. Each program style, page, and export has an accompanying program with obtaining the vulnerabilities for each asset, page_asset_vulns.py, and export_asset_vulns.py. The additional code is very similar in each style, so I will only look at export_asset_vulns.py.
After the assets file is unzipped, the code looks at each asset. Here’s where the usefulness of JSONL comes in. The code reads each line of JSON and parses it, extracts the vulnerability URL, and calls get_vuln_info()with the vulnerability URL.
The vuln_url is the show asset vulnerability API. In get_vuln_info(), the API URL is invoked with some extra logic for “too many requests” and timeouts.
This function logs the asset_id and the number of vulnerabilities for the asset in a file named asset_vuln_log_
Vulnerability information for each asset is written to asset_vuln_info_
These programs produce a lot of data, so you’ll have to determine what is important, like asset description, CVE ID, or CVE description. The asset and vulnerability data could be sliced differently; for example, the appropriate asset and vulnerability information could be written into a database or separate files.
If you’re interested in playing with these samples, they’re located in a Kenna Security blog_samples repo in the vulns_per_assets directory.