Customer Issue Management

The first step after receiving a case is to determine if we can correctly and effectively assist our customers within the bounds of our Tier 1 & Implementation team’s initial triage or if we will require further assistance and investigation from our Tier 2 and/or Engineering teams.

Addressing questions + issues

Banno Content | Banno Apps

Creating an Issue in Jira Service Desk

Banno Content | Banno Apps

Managing Issues through Jira Service Desk

Once an issue has been created within Jira Service Desk, the issue will be managed there until there is a resolution. Tier 2 will be our first line of defense and will perform the initial triage and investigation for all issues submitted. For any issues that require further involvement from one of our Engineering teams, Tier 2 will handle the assignment and work as liaisons between the Operations and Engineering teams until fully resolved.

Communication about the issues will happen within Jira Service Desk on each ticket. The diagram below is a generalization of the workflow of a customer issue through Jira and Jira Service Desk. Note that there could be more than one Jira issue for each customer issue depending on how many teams it touches:

• The Jira Service Desk issue is the single source of truth and is owned by the Operations team. All comments, updates, and interactions should live in this issue.
• An Engineering team’s Jira issue (or possibly multiple team issues) is a team’s representation of the issue and is used solely for tracking a customer issue along an Engineering team(s) workflow.

Automations (in red and blue lines) coordinate the movement of respective Operations and Engineering issues. The workflow is described in more detail in the next sections.

Tier 2 Triage Process

When a customer issue is created, the Tier 2 team will triage and determine the appropriate assignment. Each ticket has a Team(s) field that contains a list of all appropriate teams. This list will initially be empty. Issues will be assigned to the best known Engineering team(s) based on domain ownership by adding that team to the Team(s) field. If at any time, an issue is assigned incorrectly, the Engineering team will provide guidance for Tier 2 to reassign and work to help provide further education. In cases where Tier 2 can resolve the issue without involvement from an Engineering team, the case will remain with the Operations team(s) and will be assigned accordingly within the Tier 2 team.

Once a JSD issue has been opened, Tier 2 will be responsible for managing the ticket to completion. This means that tickets sent to Engineering for assistance are still ultimately owned by Tier 2 and customers should be provided with an update at least weekly even if the status of the issue has not changed. If it is determined that a case will impact or is already impacting multiple FIs, whoever made this determination should document the case by adding the affected FIs (Institution Name, ID, etc…) to the case Description and notify Support and Implementation as appropriate through their respective Slack channels so the teams can monitor for new related cases.

When an issue has been submitted through Jira Service Desk, it should be acknowledged by Tier 2 within 8 business hours of creation. Tier 2 will sweep new cases daily. If you do not receive a response on your ticket in the first 8 business hours of creation, please ping in the #org-banno-tier2-triage Slack channel to notify the team. If you’re having difficulty you can escalate the issue to the Tier 2 manager, Ian Cain. If needed, you can also escalate to the Operations Management team by using @operations-managers in the Slack channel.

Engineering Triage Process

Engineering teams are responsible for monitoring new customer issues with their team assigned in the Team(s) field. Filters were initially created for each team to assist in locating and monitoring assigned issues. They can be found under the Jira Filters page and are named “Customer Support tickets - [Team]” where “[Team]” represents the individual Engineering team name. If a filter does not exist for a team, a new one can be created using a JQL query similar to any of the existing filters. You can also request the Tier 2 manager, Ian Cain, to create the filter.

After reviewing the issue, the Engineering team can

• Accept the issue as their own by running the Claim for [Team] automation.
• Reject the issue by running the Reject Issue automation.
• Request more information and run the Send to Review automation.

Please note that claiming an issue can be performed by multiple teams and ownership will be spread among all teams that have claimed the issue.

If a customer issue is accepted by the Engineering team (using the Claim for [Team] automation) and they do not already have a ticket on their board for this issue that they are working out of, the Operations issue will be cloned to the Engineering team’s project as an Engr Customer Issue in the default To Do status. The Operations issue and the Engineering issue will be linked automatically. If engineers already have a case on their board for this issue, they will need to manually link it to the Operations issue, leave a comment explaining that an existing issue was linked, and then run the Claim for [Team] automation.

The Engineering team may manage the ticket on their side however they like within their team’s workflow. New customer issues are prioritized alongside all other customer issues, using the severity and other issue details to determine the priority.

If an Operations issue is rejected by the Engineering team, the Operations team will be notified of the rejection. The Engineering team is expected to provide comments in the Operations issue why the issue is being rejected, including who they believe the issue should belong to.

If the case does not contain enough information to determine if it should be Claimed or Rejected, an engineer can add a comment and run the Send to Review automation which will send the case to Needs Ops Review while leaving their team still assigned in the Team(s) field. This will allow engineers a way to request information or have a discourse if necessary without Rejecting a case that may end up belonging to them.

Engineering teams are required to have a Needs Ops Review status in their customer issue workflow. Moving an Engineering team’s issue to this status will trigger an automation that will move the corresponding Operations issue to a similar status on the Operations side.

Issues in the Needs Ops Review status will be moved back to the Engineering teams’ default To Do status through an automation that is triggered when the Operations team resolves outstanding questions and feels the issue is ready to be returned to the specific Engineering team(s).

When an Engineering team has resolved a customer issue, the team moves the issue to the Closed status appropriate for their Workflow. This will trigger an automation to alert the Operations team to validate that the customer issue is resolved. The resolution should be completed on the Engineering issue, and comments should be made on the operations issue to indicate how the issue was resolved.

Communication on Issues

General Communication Practice

Where possible, discussions in regards to Jira Service Desk issues should take place in Jira Service Desk. Operations team members can comment on the ticket, when needed, to engage with engineers. The Engineering teams will also comment on the ticket within Jira Service Desk as progress is made. The workflow status of the issue should serve as the source of truth for progress on the issue. (See Operations Workflow Status.)

At times, discussions about an issue may happen outside of Jira Service Desk (e.g. Slack, email). When this happens, relevant details from the discussion should be captured as a comment on the original issue, so the history is complete and can be easily referenced later. This may include screenshotting, linking to Slack archives, or summarizing a discussion/email thread.

Remember that Datadog logs disappear after 14 days. Instead of linking to Datadog logs, relevant logs should be supplied on the issue directly as well as the query used to obtain them.

Receiving & Requesting Updates

Engineering teams should be providing regular updates. If updates and not being made and/or an update is needed from Engineering on a case outside of the regular cadence, requesting it should follow this phased approach: (You should only move to the next phase if the previous phase failed to yield a response)

• Tag the appropriate EM on the case requesting an update. They should be able to either update the case directly or they will leverage the assigned engineer who will provide an update.
• Ping the case to the #org-banno-tier2-triage Slack channel to notify the team.
• Reach out directly to the Tier 2 manager, Ian Cain, or the appropriate Engineering manager on Slack.
• If all of the aforementioned phases fail, please escalate the issue to the Operations Management team by using @operations-managers in the Slack channel.

Non-Responsive Customer Case Closure

If a Jira Service Desk issue needs additional information, confirmation, or testing from the customer, the issue may be closed after 10 business days if no response from the customer has been received on the source case (jSource, Basecamp, Salesforce, etc…). Following the cadence below, Tier 2 should request status updates from the source case owner on the Jira issue after 4 business days with no response:

• Day 1 - Initial update requesting more information or additional testing
• Day 2
• Day 3
• Day 4
• Day 5 - Second update
• Day 6
• Day 7
• Day 8
• Day 9 - Third update and warning that case will be closed in 1 business day if no response is received
• Day 10 - Close Jira issue

It is the responsibility of the source case owner to contact the customer and provide the requested information in the Jira issue. The source case owner should follow our established Best Practices in the Customer Service Associate Guide found on the Case Processes page of Support Central when following up with the customer. If the source case owner updates the Jira issue to advise that the customer has responded on the source case, the non-responsive closure clock should be stopped and reset.

Operations Workflow Status

We use the status field within Jira Service Desk to keep the Engineering teams and Operations teams informed of where we are on each particular issue. As progress is made on an issue, the status should be updated, as well. Operations team members will be able to view the status of each of the issues created within the Customer Support & Content Support boards and can also follow the status of all engineering tickets linked to the original issue on each respective Engineering team’s board to gain further insight and understand prioritization of the work to be done within each team.

While statuses vary on engineering workflows, below is an outline of the various operational statuses and their corresponding descriptions of how and when they should be used. For those without Jira Service Desk licenses, there are automations to transition issues.

Looping in Other Teams

Documentation

If the determination of the firefighters is that the product/feature is working as intended, updates may need to be made to Banno Knowledge and/or Banno Docs to better outline the product or feature functionality. The original reporter will create a Jira Service Desk ticket in the Documentation Requests project for the documentation team to review and manage. Follow our guide to Submitting documentation requests for direction.

Tier 2 should work with Engineering to update runbooks and Ops Docs when there is an obvious gap identified with a given issue. This is outside the scope of the documentation requests project.

• Ops Docs contain useful information about how a feature is configured and how it works. They can be found in the Banno Knowledge base here.
• Domain ownership is tracked here.

Business Analysts

If it is necessary to engage the Technical Business Analyst team, the engineer will capture any relevant logs, details, and questions on the Jira ticket and use the @BusinessAnalysts mention which should notify the team in the analyst Slack channel. If a BA can provide a quick answer on the Ops ticket, they will. If they can’t provide a quick answer, they will open a ticket in their team project to research and respond. The analyst ticket will be linked to the Jira Service Desk ticket, and to the Engineering team’s ticket using the link type that indicates the engineering ticket is blocked. When the analyst effort has been completed, their ticket will be moved to Done and the Service Desk ticket updated accordingly so that the engineer can proceed with their effort (if applicable), and operations has the information they need.

Remember that business analysts are here to help with resolution but they are not expected to be assigned ownership around a ticket.

Product

If it is necessary to engage the Product team in discussion on an issue, the team should be looped in by tagging the primary PM for the domain based on the Product Management Domains documentation and adding a brief summary of what is needed. If a project is required, this should be stated in the note to the Product Manager/s, “Product” should be added to the Team(s) field, the OPS issue should be assigned to the primary PM of the product domain, and the status should be updated to Product Management Ownership. Product Management will create a ticket in the Product Jira project which the team can use to track their research/findings. The engineer will link the Research issue to the Jira Service Desk issue using the “is blocked by” linked issue type.

Many times customer reports will simply be around issues where our systems are working as designed and are not broken. Whenever this is the case, we should explain the behavior to the customer so that they understand while also working to understand the customer’s perspective and the behavior they were expecting to determine if a new project is needed to change the functionality. To create a new project request, the Support and Product teams will work together to determine if a project will be scoped to accomodate the request. The Product team will be responsible for helping to plan for and prioritize work on the roadmap. In this case, the original reporter will create a Jira Service Desk ticket in the New Feature Requests project for the Product team to review and prioritize. Follow our guide to Submitting new feature requests for direction.

Other JHA Teams

Some issues may require assistance from other JHA teams related to products with which Banno services interact. After being provided any necessary information from the engineer(s) working on the issue, the Support team will open or forward a jSource case with the appropriate team. The case number should be added to the issue, a comment regarding what specifically is needed, the status set to Waiting on Info, and an ‘internal JHA member’ label applied.

Closing Tickets

When the engineering work has been deployed to production, the issue’s status will be updated to Needs Ops Review. It is Tier 2’s responsibility to review, test, and verify that the issue has been fully resolved, reach out to the client(s) experiencing the issue to verify it has been resolved, and ensure the ticket gets closed once there is a resolution. Issues should be swept through on a weekly basis to ensure resolved issues are closed in a timely fashion.

Any issues that have been in Waiting on Info and have been awaiting a customer response for more than two weeks should be closed as long as attempts were made to contact them via case notes and phone calls as detailed in the rest of this paragraph. Before closing out an issue where we are waiting on the customer, be sure you have indicated to the customer that you will close the case if you don’t hear back within two weeks, have followed up within the case, and have called to let them know you are awaiting their response. Both the case and issue can be closed after waiting two weeks and a new issue can be recreated that links to the prior closed issue at a future date if the customer reports that the issue remains a problem. Whenever possible, the steps to reproduce the original issue should be included in the case as this will save time in troubleshooting if a new linked case is created.

After a ticket has been closed, Tier 2 should reach out to the Engineering team to discuss if additional documentation/changes needs to be written to prevent this type of issue from being opened in the first place.

To close a ticket, update the status to Issue is Closed. You will be required to select a Resolution in order to indicate why the ticket is being closed and should always leave an internal comment outlining specific details of the resolution.

Below are the resolution options available and a description of when to use each.

Note: only one resolution can be selected and the issue will have to be reopened to change a resolution. If you reopen an issue, the Resolution will be cleared.

Once the ticket is closed, if a customer re-opens their case with any change in the experience of the originally reported issue, the appropriate Operational team should open a new customer issue with the new details and including the prior issue as a reference for the engineering teams.

If a customer is simply delayed in providing verification on the resolution, with no change in the experience of the originally reported issue, the original reporter should re-open the issue and set workflow to Needs Ops Review with a tag to the team or team member associated with the issue to notify of the failed resolution. In this scenario, there is no additional automation to post in the #org-banno-tier2-triage channel so if you need eyes on the issue, you can post there with notification of the failed resolution to gain further visibility as needed.