What is a War Room
Since our culture is deeply remote, the majority of our conversation flows through Slack. While this works well much of the time, we have had issues communicating when certain events will affect large groups of specific people across multiple teams. War Rooms are a mechnaism to keep the right cross-functional teams in place to handle specific events without the need to scramble across multiple Slack channels. We have specific rules around how each of these rooms keeps its information, how often active members are expected to keep up with new messages, and how we triage incoming work. As a baseline, all Slack channels follow our Triage Playbook.
Go Live War Room - #war-room-go-live
Our Go Live War Room is where we keep track of every critical issue for a customer before or near after they go live. We track the progress of each institution and handle incoming issues as we take each customer live. In addition to tracking the status of each institution, this information will be used to help gauge which teams need more firefighter throughput and to help maintain priorities for those deeply committed on this process.
What is a Critical Issue?
Critical issues are issues that need attention right away. While this is a difficult definition to measure, as a baseline the following criteria would all qualify as critical:
- The issue affects a large percentage of users
- The issue is preventing any number of users from logging in
- The issue is preventing any number of users from seeing accounts
- The issue affects business users
- Critical app functionality is not functioning as expected: this includes features like viewing check images, viewing statements, and using remote deposit
As always, exercise your best judgement to ensure the channel is not flooded and that issues are not buried under high volume, when possible.
Which critical issues belong in the Go Live War Room?
We track any critical issues pertaining to launches that match any of the following criteria:
- The customer is going live in a week or less
- The customer has been live for a week or less
- The customer has significant critical issues that may persist into the week before going live
- The customer has open issues that may prevent going live
- The customer has been turned over to Support, but still has open critical issues that are being tracked by the Implementation team
Additionally, very large customers are special cases. They will often have their own channels dedicated to their launch, and communication should be kept in those channels. Links to specific issues that match the above criteria can be linked in #war-room-go-live as needed.
What information can be found here?
Our home base is the Go Live Report, which can always be found in the pinned messages. This gives a full rundown of when each customer is currently scheduled to go live. Each Monday, an Implementation Team member will also post the week’s launches in the channel itself for clarity on that week’s workload.
Relevant information, such as weekly launch lists or important ongoing issues, should be pinned until they are no longer relevant. Those who pin these items are also responsible for unpinning them once they are no longer relevant.
Who needs to be in the War Room?
Anyone who is currently on their firefighter rotation is expected to be present in #war-room-go-live for the duration of their rotation. Issues will be posed with the aforementioned Triage tags, and it is expected that each member is tracking incoming work on a regular basis. Issues that do not receive prompt attention (according to their severity) should be escalated via PagerDuty. It is important to keep in the channel if you are on a firefighter rotations, as issues will not be crossposted to team channels.
Any Implementation Coordinator managing a launch that week will also be present in the channel, as will their immediate escalation points/team leads. The channel will remain public to allow others to monitor as needed, but there are no requirements regarding how often those not listed need to check Slack.
What if our firefighters don’t rotate on Monday?
It is expected that members of this team will rotate throughout the week. When your shift starts, you’ll be expected to check the pinned items, which will contain any relevant posts to keep track of for the week. After your rotation ends and you have handed off any ongoing issues to your replacement, it’s perfectly acceptable to leave the channel until your next rotation. Updates on any issues in the War Room should be a part of Firefighter handoffs.