Message Broker Test Coverage using instrumentation

Unit Test Code Coverage

Unit test coverage in Message Broker works similarly to what developers are used to in other languages such as Java.
It can most easily compared to cobertura, where the initial code is packaged up as a BAR file, that BAR file is then instrumented with additional ESQL, Java and XSL calls.
Currently we support test coverage for:

  • ESQL
  • Msgflows
  • XSL (within WMB/IIB)
  • Java

The instrumented BAR file is deployed to a running environment, and then the test cases are run.

The results are processed into a SonarQube standard file called “Coverage.xml”, which can then be used as part of the Sonar analysis to report code coverage metrics.
From those metrics quality gates can be built as part of a continuous integration/deployment process.

Setup files

The files needed to run the coverage reporting can be found here.
You will also need the latest MB-Precise.jar file as well.

Coverage includes

WHILE .. DO

IF … THEN
ELSEIF … THEN
ELSE
END IF;

X : LOOP

END LOOP X;

X : REPEAT

END REPEAT X;

All message flows executed (view able in the coverage report in Sonar or in the generated flow diagrams)


Coverage example

Coverage xsl example

Coverage example

An example ANT script might look like the following

<?xml version=”1.0″?>
<project name=”project” default=”run”><!–
We have made changes to the way that the instrumentation is done so that you no longer need any external dependencies to make use of the instrumented bar file />
–><!– to access the coverage report when running sonar you need to configure the location in the sonar-project.properties file
# coverage
sonar.genericcoverage.reportPaths=/coveragetemp/coverage.xml
–>

<property name=”IIB.CREATEBAR” value=”C:/Program Files/IBM/IIB/10.0.0.1/server/bin” />
<property name=”IIB.DEPLOYBAR” value=”C:/Program Files/IBM/IIB/10.0.0.1/server/bin” />
<property name=”IIB.CREATEBAR_TOOLS” value=”C:/Program Files/IBM/IIB/10.0.0.1/tools” />

<property name=”IIB.EG” value=”CN_EG1″ />
<property name=”IIB.RUNTIME” value=”CoverageNode” />
<property name=”BUILD.NUMBER” value=”0.01″ />
<property name=”BARDIR” value=”bars” />
<property name=”APP.NAME” value=”Coverage1″ />

<target name=”init” description=”">

<!– the required jars to run the instrumentation and recording process –>
<path id=”project.classpath”>
<!– this could refer to the Sonar extensions directory –>
<pathelement location=”C:\utils\bct\MB-precise-sonarplugin-2.28.jar”/>
<pathelement location=”C:\utils\bct\commons-io-2.0.1.jar”/>
<pathelement location=”C:\utils\bct\slf4j-api-1.7.5.jar”/>
<pathelement location=”C:\utils\bct\slf4j-jdk14-1.7.7.jar”/>
<pathelement location=”C:\utils\bct\commons-lang-2.6.jar”/>
<pathelement location=”C:\utils\bct\xstream-1.4.7.jar”/>
<pathelement location=”C:\utils\bct\plexus-utils-3.0.21.jar”/>
<pathelement location=”C:\utils\bct\xmlpull-1.1.3.1.jar”/>
<pathelement location=”C:\utils\bct\xpp3-1.1.3.3.jar”/>
</path>

<!– setup ant tasks required –>
<typedef name=”instrument”
classname=”au.com.bettercodingtools.sonar.messagebrokersonar.anttasks.BarFileInstrumentTask” >
<classpath refid=”project.classpath” />
</typedef>

<typedef name=”listener”
classname=”au.com.bettercodingtools.sonar.messagebrokersonar.anttasks.TestMessageListenerTask” >
<classpath refid=”project.classpath” />
</typedef>

<typedef name=”stopper”
classname=”au.com.bettercodingtools.sonar.messagebrokersonar.anttasks.StopTestMessageListenerTask” >
<classpath refid=”project.classpath” />
</typedef>

<!– this could be any task that can trigger the test to start such as calling a url, sending an MQ message to SOAP
the urlTest just sends some text to a HttpRequest noode which triggers our example test. –>
<typedef name=”urlTest”
classname=”au.com.bettercodingtools.sonar.messagebrokersonar.anttasks.HttpTestTask” >
<classpath refid=”project.classpath” />
</typedef>

<property environment=”env” />

<!– need to clean bars directry –>
<delete dir=”${BARDIR}”/>

</target>

<target name=”run” description=”" depends=”init”>

<!– build the bar as your normally would –>
<antcall target=”mqsicreatebar.buildbar” />

<!– this task unpacks the bar, updates the esql and repacks the bar –>
<antcall target=”instrument.bar” />

<!– deploy the “instrumented” bar file –>
<antcall target=”mqsideploybar” />

<!– start the results listener and then run the tests –>
<parallel>
<listener port=”9011″ maxWait=”30″ coverageFilePath=”coveragetemp”  />
<antcall target=”runtests” />
</parallel>

<!– this stops the listener if it hasn’t finished already and forces the results to be recorded –>
<stopper />

</target>

<target name=”mqsicreatebar.buildbar”>
<echo message=”(Using mqsicreatebar) building Broker Archive file: ${APP.NAME}-${BUILD.NUMBER}.bar ” />

<echo message=”============= mqsicreatebar =====================” />
<echo message=”${IIB.CREATEBAR_TOOLS}/mqsicreatebar -skipWSErrorCheck -cleanBuild -deployAsSource -data coverage_ws -b ${BARDIR}\${APP.NAME}-${BUILD.NUMBER}.bar -p Coverage1 -o Coverage1/Test1.msgflow -trace -v trace.log ” />
<echo message=”============= mqsicreatebar =====================” />

<exec executable=”${IIB.CREATEBAR_TOOLS}/mqsicreatebar” spawn=”false” failonerror=”true”>
<arg value=”-skipWSErrorCheck” />
<arg value=”-clean” />
<arg value=”-cleanBuild” />
<arg value=”-deployAsSource” />
<arg value=”-data” />
<arg value=”coverage_ws” />
<arg value=”-b” />
<arg value=”${BARDIR}\${APP.NAME}-${BUILD.NUMBER}.bar” />
<arg value=”-p” />
<arg value=”Coverage1″ />
<arg value=”-o” />
<arg value=”Coverage1/Test1.msgflow” />
<arg value=”Coverage1/Backlog2ShouldHaveReply.msgflow” />
<arg value=”-trace” />
<arg value=”-v” />
<arg value=”trace.log” />
</exec>

<echo message=”(Using mqsicreatebar) Completed building Broker Archive file – ${APP.NAME}-${BUILD.NUMBER}.bar ” />
</target>

<target name=”instrument.bar” >
<echo message=”Instrumenting bar file ” />

<echo message=”==================================================” />
<echo message=”instrument ${BARDIR}\${APP.NAME}-${BUILD.NUMBER}.bar” />
<echo message=”==================================================” />

<instrument barFileName=”bars\Coverage1-0.01.bar” coverageFilePath=”coveragetemp” address=”machinenamerunninglistener” port=”9011″ />
</target>

<target name=”mqsideploybar” >

<property environment=”env”/>

<echo message=”==============================================” />
<echo message=”MQSI_WORKPATH = ${env.MQSI_WORKPATH}” />
<echo message=”==============================================” />

<echo message=”Deploying Broker Archive file: ${APP.NAME}-${BUILD.NUMBER}.bar ” />

<java classname=”com.ibm.broker.config.util.Deploy” failonerror=”true” fork=”true”>
<arg value=”${IIB.RUNTIME}” />
<arg value=”-e” />
<arg value=”${IIB.EG}” />
<arg value=”-a” />
<arg value=”${BARDIR}\${APP.NAME}-${BUILD.NUMBER}-instrumented.bar” />
<classpath>
<fileset dir=”C:/Program Files/IBM/IIB/10.0.0.1/tools/plugins/com.ibm.etools.mft.config_10.0.100.v20150515-1900″>
<include name=”**/*.jar”/>
</fileset>
</classpath>
</java>

<echo message=”Completed deploying Broker Archive file – ${BARDIR}\${APP.NAME}-${BUILD.NUMBER}-instrumented.bar” />
</target>

<target name=”runtests” >
<!– HttpTestTask –>
<echo message=”Running test: http://localhost:7800/Test1″ />
<urlTest url=”http://localhost:7800/Test1″ />
<echo message=”Running test: http://localhost:7800/Test1 done” />
</target>

</project>