Preparing for an Incident Response

When an enterprise-level incident transpires, and an incident response is initiated, the success and accuracy of the investigation relies upon having complete and consistent visibility for all systems and network communication paths throughout the enterprise environment. While this sounds like a reasonable expectation, through the years of conducting cyber incident responses, we have observed common challenges that can impact the ability to perform a comprehensive investigation as part the cyber incident response. These challenges represent practical steps that all organizations can take ahead of time to minimize the number of roadblocks and delays that could impact the success of an investigation.

Asset Management

• An accurate asset inventory is key for ensuring optimal visibility of all endpoints. An ideal asset inventory system would categorize systems by function and identify critical systems that provide essential services or provide access to critical data (ex: PII, PHI, intellectual property) – as endpoint visibility should be heightened for these systems.
• If third-party technology may be used to support an investigation, establish a formalized process for rapid software deployment to endpoints, as this is a critical to a timely incident response. An accurate asset inventory can allow an organization to quickly reconcile and identify endpoints where third-party technology coverage may be missing.
• The asset inventory and deployment process should account for the potential need to deploy technology to systems in environments that may not be directly connected to the core of the enterprise infrastructure. Examples include systems or network segments managed by a third-party vendor, operational technology (OT) environments, or cloud-based technologies and platforms.

Network Architecture

• Ensure that accurate and complete network diagrams are available. Accurate diagrams are important when scoping network visibility for network sensor deployment, or when leveraging existing technology coverage for reviewing communication flows both within and at the perimeter of an enterprise.
• Confirm all ingress and egress paths, routes between sites, integration of third-party connectivity, and network locations where encryption is enforced. If not scoped properly, gaps in network visibility can ultimately hamper the accuracy in performing a comprehensive investigation.
• If network sensors or network data collectors will need to be utilized to support enhancing network visibility, ensure that network taps or port mirroring (SPAN) technology is available, and that a process exists to expedite deployment and configuration. 
• Identify surge resources and personnel that can assist with rapid deployment and configuration of network-based technology in preparation for an investigation.

Credential Management

• Maintain an accurate inventory of service and privileged accounts. This inventory can assist with supporting an investigation, in addition to reducing delays in implementing containment and remediation measures. Furthermore, understanding the scope of how these accounts are utilized, and the source endpoints that the accounts should be initiated from, can provide enhanced detection of anomalous events that could be malicious in nature.
• For domain-based service accounts used by applications, document and maintain an inventory that correlates each service account to a specific application. Having this inventory can minimize operational impacts if a coordinated password reset is required as part of an incident remediation event.
• Review, test, and document security controls that are implemented to restrict the exposure and usage of privileged accounts on endpoints. Understanding how an adversary could utilize privileged accounts within an environment can help prioritize scoping of lateral movement during an investigation.

Logging

• Having detailed logging configured and available for review and analysis is a core driver for an effective investigation. Unfortunately, many organizations either do not have logging configured optimally on endpoints, or are not collecting and archiving logs from critical systems and common technologies (ex: DHCP, DNS, Web/Proxy, VPN, net flow).
• Verify that detailed logging is present for core assets and critical systems.
• Ensure that all endpoints, networking devices, and log aggregators are configured for NTP synchronization to an authoritative time source.
• Ensure that logs are collected and archived for internet facing systems and applications (ex: DMZ systems, web logs).
• Verify that network traffic logs support the ability to review communication flows based upon source and destination IP addresses, port, duration, and byte count. In addition, if load balancers are utilized, ensure that the true source and destination IP addresses of a session can be correlated.
• On endpoints, verify log data exists to support a review of (at a minimum):
  • Successful and failed logon events
  • System events
  • Scheduled tasks
  • Process execution events with command line arguments
  • PowerShell activity
  • Security software events (ex: third-party Antivirus alerts and detections)

Playbooks to support incident response activities

• Not all aspects of an incident response can be scripted. Having playbooks and plans that support response and recovery functions – in addition to resource alignment, surge support, and third-party assistance – can greatly reduce delays when responding to a cybersecurity event.
• Adapt, modernize, and test response and recovery playbooks. Ensure that playbooks are relevant to threats and cyber security risks that may impact the organization. In parallel, testing the effectiveness of playbooks can identify potential visibility and protection gaps that may exist, in addition to verifying the efficacy of recovery actions for systems in the environment.
• Document and test playbooks that support containment and remediation activities. A common example aligns to an enterprise-wide password reset. Establish a playbook that supports not only the technical aspects of enforcing an enterprise password reset, but also includes planning, communications for both internal and external users, account dependencies and mappings, processes used to verify users, progress tracking, and surge support to assist with the event. 
• Establishing a documented and process-focused set of response actions can be used to train and educate new personnel joining an incident response team within an organization.

In conclusion

When the need for a coordinated cyber incident response occurs, it can be a stressful and impactful situation for any organization. Organizations that take the time to formulate and plan for an incident response are better able to remain focused, prioritize and allocate resources to support critical milestones and functions, minimize coverage gaps, and ensure that optimal visibility is achieved and maintained throughout the engagement.