Debugging API Invocations

Created by Ashish Pandit
Oct 05, 2017

Debugging API access could be frustrating. Using splunk will help you reduce this frustration

To get started log onto https://act-search-splunk.ucsd.edu

 

To see everything from this new system use this search: index=soa_api_public_i

(all ITS developers should already have access to this index. Rest, will have to ask for additional permission)

 

The major elements are:

INCOMING_REQUEST

Logged right after the API is identified, before any work like checking authorization.

Contains:

  • RequestInURL – the url that the client invoked on the API Manager.
  • API – The API being invoked.

 

REQUEST_COMPLETE

Logged upon receiving an answer from the back end service.  This can be correlated with a INCOMING_REQUEST but also repeats much of the data from that item.

Contains:

  • RequestOutURL – The URL on the back end service that the API Manager invoked.
  • ResponseTo – The unique message identifier of the matching INCOMING_REQUEST
  • Most of the data from the associated INCOMING_REQUEST

 

ERROR

Logged when an error condition is detected.

Contains:

  • ErrorSource – A identifier of where the error was raised.  This gives an indication of the type of error (e.g. Authorization vs API not found).
  • API – The API that matched the request.  This may not be present if the API couldn’t be identified.
  • ERROR_CODE – The API Manager Error code that will be returned.
  • ERROR_MESSAGE – The error message to be returned.
  • StatusCode – The HTTP status code that will be returned.  Due to implementation limitations, this may be subtly inaccurate (e.g. 401 instead of 403) or missing, depending on the type of error.

 

Error Types

  • _auth_failure_handler_ - Logged when the incoming request did not meet the authorization requirements to invoke the requested API/Resource.  This includes missing credentials, invalid credentials and insufficient permissions.
  • main – Logged when the “main” sequence is invoked.  In practice this only seems to be called if the request couldn’t be matched to an API (i.e. 404 errors)
  • fault – Logged when the “fault” sequence is invoked.  This covers any general error, but most useful / common it is called when the backend service couldn’t be contacted.

 

So now to the fun parts, queries!

 

To see APIs invoked over a period of time: index=soa_api_public_i INCOMING_REQUEST | timechart count by API

 

To see Events regarding certain APIs: index=soa_api_public_i API=*Echo*

 

To exclude the echo API (it’s called A LOT by NAGIOS and the Load Balancer and so spams up the logs): index=soa_api_public_i  NOT API=*Echo*

 

To filter Events based on the incoming request URL (helpful to identify people attempting to access your API):  index=soa_api_public_i RequestInURL="*CCES*"

 

To see the success/failure of calls, by API: index=soa_api_public_i NOT INCOMING_REQUEST | eval result=if(Event=="REQUEST_COMPLETE", "Success", ERROR_MESSAGE) | chart count by API, result

  

NOTE:  Due to implementation issues, APIs will only be added to the log on publish, existing APIs need to be re-published (press “Save & Publish” on the third tab of the publisher interface) to start logging.