Detailed Procedures

Incident Identification & Reporting

  • Detection: Multiple monitoring tools, Weekly meetings update

  • Initial Reporting:

    • Reporter: Responsible person in the area of responsibilities

    • Method: Linear as incident tracking tool

    • Required Info: Date/time, description, affected systems, suspected impact.

  • Triage: IRT assesses severity, scope, and priority.

Incident Report Process in Linear:

1) Go to Linear https://linear.app/xcavate/team/XCA/active. Access project view in Linear:

Project view in Linear

2) Access the relevant project to report the incident. If the incident is related to realXmarket, click on realXmarket.

3) Click on All Issues at the top to see all issues.

All Issues tab at the top

4) Scroll down to the relevant milestone if the incident is relevant to any of the milestone. If none of the milestones are relevant, scroll the vertical scroll bar all the way down to No Milestone.

Milestones pointed by the green arrows

5) Incident is something that is happening. Scroll the bottom scroll bar (shown by the blue arrow) to the right to In Progress (circled by yellow highlight) if In Progress is not already in view.

In Progress section

6) If the incident is under No Milestone, scroll down the scroll bar on the right hand side (shown by the blue arrow) click the + sign in the In Progress column as circled by the yellow highlight as shown in the diagram below:

Add an incident issue to the correct column

7) Once the + sign is clicked, an issue will popped up. Click on Template (shown by black arrow) on the popped up issue and select Incident Management (shown by indigo arrow) on the list:

Select Incident Management from the Template list

8) Click on Incident Management, the popped up will change to the Incident Management template. Fill in all the fields pointed to by the green arrows. Scroll down the template using the scroll bar (pointed by blue arrow) to complete the information required.

Incident Management Template

9) Please add all the additional information that are required for the issue if not already covered by the template. There will be continuous improvement for this template going forward. Select the assignee (circled by yellow highlight), possible to change status (circled by green highlight) if the status has changed or not accurate and select priority (circled by blue highlight):

10) There are additional information that can be obtained by clicking the the three dots (circled by yellow highlight below). Set due date and add sub-issue. Sub-issue here could mean all the separate tasks to resolve this incident. Alternatively, sub-issue can also be added after the incident is created, see step (13) below.

11) Once everything is completed, click create issue (circled by yellow highlight) and a confirmation of the incident (called issue in Linear) created appear at the bottom right hand corner of the screen:

Click Create Issue
Confirmation of Incident created at the bottom right hand corner

12) Once the incident is resolved, root cause analysis will be required. I have created an example incident of DPRK backdoor in Xcavate Protocol project

DPRK backdoor issue as an example

13) Open the Incident ticket, you can add sub-issues as shown in the following diagram. Sub-issues can be used during the incident if required.

Sub-issues can also be used for root cause analysis

14) Once add sub-issues is clicked, the issue window will appear. Click on the icon to the right as noted in the following diagram to choose the root cause analysis template.

Apply template icon circled by the yellow highlight

15) The issue templates will appear after the icon is clicked. Select the root cause analysis template.

Select Root Cause Analysis template

16) Once click on Root Cause Analysis, the template will appear.

Root Cause Analysis template

17) Replaced the [Incident title] with the incident title. In this example, it is DPRK Backdoor. Assign the ticket. Click create button in the bottom right hand corner of the template.

Replaced [Incident title] with actual incident title, assign and create the sub-issue for Root Cause Analysis

18) Conduct the Root Cause Analysis and fill in the template. The result from this activity will result in Lessons Learned for continuous improvement. It is possible to add sub-issue(s) to Root Cause Analysis for Lessons Learned. Repeat steps (11) - (15).

Select Add sub-issues in the Root Cause Analysis template
Lessons Learned template
The [Incident title] in this example is DPRK Backdoor
Assign and create

19) The Root Cause Analysis issue will appear connected to the incident. The Lessons Learned will appear connected to the Root Cause Analysis.

Root Cause Analysis issue connected to the incident.
Lessons Learned issue connected to the Root Cause Analysis

20) Project level view for all three issues, Incident, Root Cause Analysis and Lessons Learned:

Project level view of the incident issue and the two sub-issues Root Cause Analysis and Lessons Learned.

Last updated