Semifore’s Experience with Certess Certitude
Add comment November 20th, 2008
The Semifore design verification team currently uses the Certess Certitude tool to qualify the test benches and test cases used within Semifore for quality assurance. The Semifore CSRCompiler product creates synthesizable RTL from a register address map specification, unit level test benches, and testcases to verify that the generated RTL has the specified hardware and software semantics. After considering many techniques, the Semifore design verification team chose the Certess Certitude tool to automate the verification effort and RTL quality. The Certess Certitude tool provides the verification team feedback without a large commitment of team resources. The feedback includes a quality measurement and a detailed list of areas that need additional work.
Initial Porting Experience
The Semifore team has an extensive regression suite that has been in operation for several years and is continually improved as gaps are discovered in the verification coverage. The Certess Certitude tool instruments the regression and reports uncovered faults. The initial porting effort was performed on the Verilog RTL generation and verification outputs of the CSRCompiler. The port involved adjusting the regression execution scripts, test benches and diagnostic Verilog files to conform to the Certess requirements. The changes to the regression scripts were minimal which included replacing the device under test, DUT, Verilog RTL files with the Certess instrumented Verilog RTL files.
A small amount of improvement is also required by the automatically generated test bench and the test case Verilog files. The Certess flow requires that the regression return explicit pass or fail status. If the Semifore regression suite does report explicit failure, it had initially assumed as an implicit pass status with a regression running to completion without failure. The verification Verilog files were updated to return an explicit pass status. The Semifore verification team considers this an improvement and a best practice in the verification environment.
Results
The Semifore verification team was encouraged that the initial fault coverage result was very high; however several uncovered faults were identified. Most of these uncovered faults involved gaps in the verification. Some of the faults required more cross team communication because it was unclear if the design specification, implementation, or verification was incorrect.
One example of a gap in the verification identified by Certess Certitude is an uncovered fault in the read value of unspecified bits in a software accessible register in the address map. The Semifore verification environment has extensive checking to make sure that data for each specified field in register returned by a software read transaction is correct. However, the data returned for any unspecified bits in a register was not checked by the test bench.
Some of the uncovered faults reported were not clear that the verification environment had missed something. These faults were related to grey areas in industry standards. One example is a bus protocol standard which specifies the read data bus must be valid during the read transaction clock cycle when the bus ready signal is asserted. This standard unfortunately does not specify what the read data bus should contain during the other clock cylces. This is a case where the RTL implementation has the choice of many correct behaviors. The Semifore verification environment tests the RTL bus protocol behavior against the industry standard. The Certess Certitude tool identified an uncovered fault in the RTL logic which drives zeros to the bus when valid data is not required. This result is useful to Semifore because it identifies an area in the design that needs more analysis and discussion. Semifore now has the choice to disabling the fault in Certess Certitude, change the verification environment to match the RTL implementation, or change the RTL implementation. There is a value to identify grey areas in industry standards for design and verification teams since many grey areas are overlooked.
Semifore develops software for management of control and status registers. The entire product has been tested and fully functional with Certess Certitude tool. Semifore generates synthesizable RTL code with a test benches and diagnostic Verilog files. The Certess Certitude tool independently certify the quality of the test bench.