Measurement and Metrics in Secure Software Development: CMMI & ISO/IEC 15939

 

Measurement and Metrics in Secure Software Development: CMMI & ISO/IEC 15939

Revised June 25, 2023

Security metrics play a crucial role in enhancing the security characteristics of software during the development process. To effectively measure and analyze software, organizations can refer to industry standards and guidelines such as the ISO/IEC 15939 (Software Measurement Process standard), Capability Maturity Model Integration (CMMI), and Practical Software and Systems Measurement (PSM) project.

Aligning measures outlined in CMMI:

1. Establish measurement objectives: Define the goals and objectives of the measurement process in alignment with organizational and project goals.
2. Specify measures: Determine the specific measures that will be used to assess security-related aspects of the software.
3. Specify data collection and storage: Define the methods and tools for collecting and storing measurement data accurately and securely.
4. Specify analysis procedures: Develop procedures to analyze the collected data and extract meaningful insights.

CMMI Levels:

1. Initial: Organizations at this level have unpredictable processes and their success depends on individual efforts rather than standardized processes.
2. Managed: Organizations at this level have defined processes for project management and can repeat successful practices across projects.
3. Defined: Organizations at this level have defined and documented processes that cover the entire organization. These processes are tailored to specific projects and are consistent across the organization.
4. Quantitatively Managed: Organizations at this level use quantitative data to understand and control their processes. They can measure and analyze process performance to make informed decisions.
5. Optimizing: Organizations at this level focus on continuous process improvement. They continually refine their processes to improve efficiency and effectiveness based on quantitative data and feedback.

 

 

Each level represents a higher level of maturity and capability in managing and improving processes. Organizations progress through these levels by implementing and refining standardized processes, collecting and analyzing data, and continuously improving their practices.

 

ISO/IEC 15939: Software Measurement Process

ISO/IEC 15939 is a standard that provides guidance on software measurement processes. It outlines the activities and practices involved in measuring software characteristics to support decision-making and improvement efforts. The standard emphasizes the importance of collecting reliable and meaningful data to effectively measure and analyze software attributes.

Key aspects of ISO/IEC 15939 include:

1. Measurement Objectives: Establishing clear objectives for the measurement process is crucial. Organizations need to identify what they want to measure, why they want to measure it, and how the measurements will be used to support their goals and objectives.
2. Measures Specification: Once the measurement objectives are defined, specific measures need to be selected and defined. These measures should be tailored to the specific context and goals of the organization. They should provide meaningful and actionable information about the software being measured.
3. Data Collection and Storage: ISO/IEC 15939 emphasizes the importance of collecting accurate and reliable data. Organizations need to establish processes and mechanisms for collecting data, ensuring its quality, and securely storing it for future analysis and reference.
4. Analysis Procedures: Analyzing the collected data is a critical step in deriving insights and making informed decisions. ISO/IEC 15939 encourages organizations to define analysis procedures that enable effective interpretation of the data. Various analytical techniques can be employed to identify trends, patterns, and anomalies in the software metrics.
5. Communication of Results: The standard emphasizes the need to communicate the results of the measurement process effectively. Clear and concise reporting mechanisms should be established to present the findings to relevant stakeholders. The results should be understandable, actionable, and support decision-making processes.

ISO/IEC 15939 provides a framework for organizations to establish a systematic and disciplined approach to software measurement. By following its guidelines, organizations can improve their understanding of software characteristics, identify areas for improvement, and make data-driven decisions to enhance the overall quality and performance of their software products and processes.

Note: It’s important to consult the official ISO/IEC 15939 standard for detailed and specific guidance on implementing the software measurement process.

Practices for performing measurement:

1. Collect measurement data: Gather relevant data related to security vulnerabilities, risks, and compliance with security-related processes and procedures.
2. Analyze measurement data: Use appropriate analysis techniques to interpret the collected data and identify patterns, trends, and areas for improvement.
3. Store data and results: Ensure that measurement data and analysis results are appropriately stored for future reference and analysis.
4. Communicate results: Share the measurement results with stakeholders, including project teams and management, to drive decision-making and process improvement.

By implementing these practices, organizations can achieve the following outcomes:

1. Goal-driven measurement: Aligning measurement activities with organizational goals ensures support from management and a clear focus on meeting objectives.
2. Targeted measurement: By defining specific measures and objectives, the project team can concentrate on measuring the necessary elements to achieve security requirements.
3. Process and product improvement: The measurement process provides a framework for making informed decisions and driving improvements in both the software development process and the final product.

In the context of security, it is essential to identify and address security concerns throughout the measurement and analysis process. Organizations should assess risks, identify potential threats, and translate them into specific security requirements for the software design. This ensures that security requirements are embedded into the development process.

To define measurement objectives related to security, organizations can employ techniques such as threat modeling and analyzing likely sources of attacks. Analytical questions can help guide the measurement objectives, such as:

• What vulnerabilities have been detected, and are current development practices sufficient to prevent their recurrence?
• Which process points are most vulnerable to the introduction of security-related risks?
• What proportion of defects relate to security concerns and requirements?
• Do practitioners comply with security-related processes and procedures?
• Have measures associated with security requirements and implementation been defined and planned?
• What are the critical and vulnerable modules, and have vulnerabilities been identified and addressed?

Measuring security-related aspects does not have to be complex. For example, in the requirements phase, organizations can measure whether security concerns have been included in defining system requirements by a simple yes or no answer. Another example is measuring the percentage of input sources that have validation checks, with the target being 100% for thorough validation. The key principle is to keep measures as simple as possible while meeting the information needs of the organization.

By leveraging measurement and metrics in secure software development, organizations can effectively track and improve the security posture of their software, leading to more robust and resilient systems.

References

https://www.iso.org/obp/ui/#iso:std:iso-iec:15939:ed-2:v2:en

https://ieeexplore.ieee.org/abstract/document/5482589

https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=29529

http://psmsc.com/

https://cmmiinstitute.com/

Additional Articles

Security Policy Example – Remote Access

Cloud Architecture Models

Fast Ethernet Specification – IEEE 802.3u

Mitigating Insider Security Threats

Understanding Business Continuity Planning

Exploring the Implications of Artificial Intelligence

Artificial Intelligence in Texas Higher Education: Ethical Considerations, Privacy, and Security

 

Note: This article has been drafted and improved with the assistance of AI, incorporating ChatGTP suggestions and revisions to enhance clarity and coherence. The original research, decision-making, and final content selection were performed by a human author.

Disclaimer

Terms and Conditions of Use