The quality of a Software Architecture Document is evaluated in two ways:

  • Validation ensures the document satisfies the requirements (right system)
  • Verification ensures the design adheres to good practices (right practices)

Validation

Validation is performed by numerous key people

  • Analysts who defined the system requirements
  • System architect(s)
  • Program designers
  • System tester(s)
  • System maintainer(s)

Verification

Verification ensures that the project adheres to good design principles

  • Is the architecture modular, are the units atomic, is the system easy to understand?
  • Can the structure and understandability of the architecture be improved?
  • Is the architecture portable and platform-agnostic?
  • Are aspects of the architecture reusable?
  • Is the architecture easy to test?
  • Is the architecture performant?
  • Is the architecture fault-tolerant, does it fail if a fault is encountered?
  • Can the architecture accomodate design changes and extensions?

There are active and passive design reviews:

  • Active reviews use the document in the way developers would use the final documents in practice
  • Passive reviews merely read through the documentation in search for problems

Evaluation & Refinement

  • Design is iterative

Measuring Design Quality

  • Metrics can assess key aspects of design quality
  • Chidamber & Kemerer
  • Briand, Morasca & Basili
  • Briand, Devanbu & Melo

Safety Analysis

A safety analysis is used to identify possible faults.
Fault-tree analysis traces backwards through a design

  • Trees are then used to determine which faults to correct/avoid/tolerate
  • Data-flow graph
  • Control-flow graph
flowchart TD
    SB["Security Breach"] --> OR
    OR --> IAG["Illegal access granted"]
    IAG --> AND
    AND --> PE["Password exposed"]
    AND --> PNC["Password not changed"]
    OR --> PLOR["Previous logout not recognised"]
  • The root node states the potential failure that is being analysed
  • This is linked to events
  • Logic gates show the relations between the conditions

Once the fault tree is constructed, weaknesses can be found

Cut-Set tree

A cut-set tree reveals combinations of events that cause failure

  • Assign the top node of the cut-set tree to match the logic gate at the top of the fault tree
  • Working from the top down
    • Expand an or-gate node to have two children, one for each or-gate child
    • Expand an and-gate node to have a child composition node listing both and-gate children
    • Expand a composition node by propagating the node to its children, but expanding one of the gates listed in the node
  • Continue until all leaf nodes are basic events or composition nodes of basic events

[@TODO: Add this diagram example]

Once a fault is found in the design

  • Correct the fault
  • Add components or conditions to prevent it
  • Add components that detect the fault and recover from damage

Security Analysis

  • Software characterization - Documentation must be reviewed for understanding functionality of the system
  • Threat Analysis - Look for threats (wow)
  • Vulnerability Assessment - Includes failure to authenticate user or use of cryptographic algorithm that is easy to break
  • Risk likelihood determination - Considers motivation ability of the threat to exploit, impact of the exploit, and degree to which current controls can prevent
  • Risk impact determination - Business consequences
  • Risk mitigation planning - Planning to reduce likelihood and consequences of most severe risks

Trade-Off Analysis

It should be important to explore alternative designs to the first that comes to mind.
Different design team members may produce different designs
One specification can have many designs, it is important to compare them thoroughly and ensure the right one is chosen.

Cost-Benefit Analysis

A cost-benefit analysis can be used to compare cost, to… benefit… it’s fairly self explanatory
Important to highlight ROI and payback period
It’s useful when it comes to doing things that cost money like program improvements or reworks

Prototyping

Some design decisions are best answered by prototyping

  • Throw-away prototypes are meant to be discarded
  • Rapid prototyping allows you to progressively refine the prototype until it becomes the final system