Any one can easily understand how Constrained Random Coverage Driven Verification works by referring the good text books or attending the trainings. But you learn certain things only when you do the real verification. Testsuite optimization is one of those gray areas for the beginners.
I do not want to write new articles just for the sake of keeping my blog alive. Sometimes I take my own time and prepare myself to write about some new topics, especially the unusual but important ones. As this topic "Seed Vs Transaction" demands not only my experience, but my learnings from others experience too. I have actually interacted with many of my customers and
peers and collected lot of details about various things like seeds, number of transactions, redundant tests, testcase run time etc.
verification folks who are new to CDV usually ask, which option [1 or 2] is good for achieving 100% coverage:
[1] Minimum number of seeds and maximum transactions per testcase
[2] Minimum number of transactions per testcase and many seeds
Either way you can improve the coverage but neither will get you 100% coverage, as they will definitely reach saturation limit at some point of time.
Considering the productivity, means improvement in the coverage, a testcase with 5 seeds could be same as running the same with even 500 seeds. Similarly a testacse generating 1000 transactions could give the same productivity with even 10000 transactions or sometimes it might improve the coverage hardly 1%.
It's always good to consider various factors like the simulation run time, DUT features, Unusual bugs etc. while defining the testcase, rather than just focusing only on reaching coverage goals.
Let me brief how these factors influence us to choose between seed and transactions.
* Run time - We decide the number of transactions based on the simulation run time of a testcase. Run time is a critical factor which decides how quickly we can reproduce the bug. Once we hit the bug, we want to rerun the testcase and reproduce it to debug the design. Ideal testcases consume 10 -15 minutes of run time.
Longer Tests - Also note that shorter testcases are not the ideal ones. Sometimes we need to pumpin more transactions to the DUV to reach it's deeper states. FIFO overflow, Suspend Data, Loop around etc. features can be verified only by running longer testcases
* Effective Seeds
If you are looking for shorter testcases, running hundreds of seeds would help but still you need to identify the effective test/seed pairs. One should measure the productivity of each testcase and mark the redundant tests as low ranking tests.
* Experience
Learn from others experience. If you are working on new version of the legacy DUT, it's good to interact with the verification folks who verified the legacy. They can tell you exactly what really worked, not worked, what kind of bugs, when they found bugs, how they debugged etc. Why don't you at least scan through the postmortem report of the legacy verification?
