Code Coverage for PERL Scripts

Update[10/9/2012] : Sample Code Coverage reports are available here.
manosoftlive.com/blog/cover_db/coverage.html

Code Coverage : Code Coverage is considered as direct measure of quality of code.

For example, Consider we have some 1000 lines of code for some utility, which is thoroughly tested.
We don't know how many exact lines of code has been tested, here comes code coverage handy.

Uses : 
- Helps in determining the test gaps.
- To know what extent of code is covered.
- Creating additional test cases to increase the coverage.

Code Metrics
A large variety of metrics exists, many organisations have formal requirements for structural coverage.

Code Metrics for PERL
Code metrics for other languages like C C++ Java can be easily calculated as the code is instrumented.
Coming to PERL, here the code is interpreted, here is the catch.
There are very few tools to measure code metrics for PERL.
Here I will explain the code coverage for PERL using the Perl module Devel::Cover from CPAN.


Prerequisities
- Unix machine
- PERL installed
- make and gmake to install Devel::Cover


Get/Install Devel::Cover from CPAN
This module can be directly downloaded from CPAN at http://search.cpan.org/~pjcj/Devel-Cover-0.93/
All dependencies should be satisfied before going ahead.
OR
bash#cpan You will get cpan prompt cpan>get Devel::Cover Downloads Cover module from CPAN cpan>install Devel::Cover Installs Cover module from CPAN, in newer versions of CPAN dependencies are resolved and downloaded accordingly. cpan> quit
Once you have installed all dependencies, you can see "cover" binary in /usr/bin/ or /usr/local/bin
This is used to calculate coverage and generate HTML reports.

If you have problem with resolving dependencies, try manually installing modules as shown below.

bash#cpan cpan[0]> get YAML cpan[1]> install YAML cpan[2]> get Storable Digest::MD5 Tree::DAG_Node Sub::Uplevel Test::Warn Module::CoreList Module::Load::Conditional IPC::Cmd CPAN::Meta::Requirements CPAN::Meta::YAML JSON::PP Parse::CPAN::Meta CPAN::Meta Module::Metadata Module::Build Pod::Man Scalar::Util Test::Harness Test::More ExtUtils::MakeMaker Data::Dumper File::Spec File::Path File::Basename Carp Exporter Test Pod::Simple Pod::Escapes Text::Wrap constant IO::File version File::Temp ExtUtils::Install Perl::OSType ExtUtils::Manifest ExtUtils::CBuilder Locale::Maketext::Simple Params::Check Module::Load Text::ParseWords Text::Abbrev Getopt::Long Pod::Usage ExtUtils::ParseXS DynaLoader XSLoader Digest::base MIME::Base64 cpan[3]> install Storable Digest::MD5 Tree::DAG_Node Sub::Uplevel Test::Warn Module::CoreList Module::Load::Conditional IPC::Cmd CPAN::Meta::Requirements CPAN::Meta::YAML JSON::PP Parse::CPAN::Meta CPAN::Meta Module::Metadata Module::Build Pod::Man Scalar::Util Test::Harness Test::More ExtUtils::MakeMaker Data::Dumper File::Spec File::Path File::Basename Carp Exporter Test Pod::Simple Pod::Escapes Text::Wrap constant IO::File version File::Temp ExtUtils::Install Perl::OSType ExtUtils::Manifest ExtUtils::CBuilder Locale::Maketext::Simple Params::Check Module::Load Text::ParseWords Text::Abbrev Getopt::Long Pod::Usage ExtUtils::ParseXS DynaLoader XSLoader Digest::base MIME::Base64 cpan[4]> get Devel::Cover cpan[5]> install Devel::Cover

Example and getting code Metrics

Save the below as Sample.pl #!/usr/bin/perl # Author : Manoranjan Reddy # Email : mano[at]manosoftlive[dot]com use strict; use Sample; use Devel::Cover; my $i = $ARGV[0]; my $j = $ARGV[1]; if ($i == 1) { Sample::one(); } if ($i == 2) { Sample::one(); Sample::two(); } if ($i == 3) { Sample::one(); Sample::two(); Sample::three(); } if ($i == 4) { Sample::one(); Sample::two(); Sample::three(); Sample::four(); } if ( $i || $j ) { print "i+j"; } Save the below as Sample.pm

Save the below as Sample.pm#!/usr/bin/perl # Author : Manoranjan Reddy # Email : mano[at]manosoftlive[dot]com use strict; package Sample; use Devel::Cover; sub one { print " In one " ; } sub two { print " In two " ; } sub three { print " In three " ; } sub four { print " In four " ; } 1;


If you observer the above code, all i added was one line "use Devel::Cover;" , with which i get code metrics.

Lets see the sample run and results.   bash#perl Sample.pl 3 Devel::Cover 0.93: Collecting coverage data for branch, condition, statement, subroutine and time. Pod coverage is unavailable. Please install Pod::Coverage from CPAN. Selecting packages matching: Ignoring packages matching: /Devel/Cover[./] Ignoring packages in: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . In one In two In three i+jDevel::Cover: Writing coverage database to /Coverage/switch/cover_db/runs/1345125760.14647.30828 ----------------------------------- ------ ------ ------ ------ ------ ------ File stmt bran cond sub time total ----------------------------------- ------ ------ ------ ------ ------ ------ Sample.pl 66.7 50.0 33.3 100.0 100.0 60.0 Sample.pm 75.0 n/a n/a 75.0 0.0 75.0 Total 68.0 50.0 33.3 80.0 100.0 62.8 ----------------------------------- ------ ------ ------ ------ ------ ------

Now a folder is created with name "cover_db", where all the info on runs is saved, now we need to generate reports.

Create HTML reports
"cover" is the utility, with which we can generate HTML reports.
All the HTML reports can be customized.

bash#ls cover_db Sample.pl Sample.pm bash#cover Reading database from /Coverage/switch/cover_db ----------------------------------- ------ ------ ------ ------ ------ ------ File stmt bran cond sub time total ----------------------------------- ------ ------ ------ ------ ------ ------ Sample.pl 66.7 50.0 33.3 100.0 100.0 60.0 Sample.pm 75.0 n/a n/a 75.0 0.0 75.0 Total 68.0 50.0 33.3 80.0 100.0 62.8 ----------------------------------- ------ ------ ------ ------ ------ ------ HTML output written to /Coverage/switch/cover_db/coverage.html done.

Once the HTML reports are generated, the work is damn simple, just open the HTML file in browser.
See the details reports generated, hope self readable and self explainable

Update[10/9/2012] : Sample Code Coverage reports are available here.
manosoftlive.com/blog/cover_db/coverage.html


Branch Coverage

Condition Coverage

File Coverage

Subroutine Coverage

Understand More: 
This is just to have quick understanding of PERL code coverage.
To understand more, save the above sample files and play around.
Try running the above code samples with different inputs like
bash# perl Sample.pl 3

bash# perl Sample.pl 2 1

bash# perl Sample.pl 2 2 

Many things can be customized according to your need, see below link to CPAN module

http://search.cpan.org/~pjcj/Devel-Cover-0.93/lib/Devel/Cover.pm