Jpye: /* Buildbot */

2010-05-19T03:08:09Z

Buildbot

← Older revision		Revision as of 03:08, 19 May 2010
Line 77:		Line 77:
	A buildbot has been set up which automatically compiles ASCEND and runs the CUnit and Python test suites each time changes are committed to the Subversion repository. See [[BuildBot]].		A buildbot has been set up which automatically compiles ASCEND and runs the CUnit and Python test suites each time changes are committed to the Subversion repository. See [[BuildBot]].

			''A missing attachment: Pycunit-0.2-1.src.rpm''.

	* <a href="/images/b/b1/Pycunit-0.2-1.src.rpm" class="internal" title="Pycunit-0.2-1.src.rpm">Media:pycunit-0.2-1.src.rpm</a>		[[Category:Outdated]]
	*  %META:FILEATTACHMENT{name="pycunit-0.2.tar.gz" attr="" comment="Source tarball" date="1167471650" path="pycunit-0.2.tar.gz" size="18916" user="johnpye" version="1.1"}%



	[[Category:Outdated]]
	[[Category:Development]]		[[Category:Development]]

Jpye: /* Strategy for Regression/Unit Testing */

2010-05-19T03:07:39Z

Strategy for Regression/Unit Testing

← Older revision		Revision as of 03:07, 19 May 2010
Line 54:		Line 54:


	* ~~[[StaticCodeAnalysis]]~~		* Static code analysis
	* ~~[[ManualTestingFrameworks]]~~		* Manual testing frameworks
	* Automated Testing Tools: we currently have [[BuildBot]] installed.		* Automated Testing Tools: we currently have [[BuildBot]] installed.

UploadBot: Restored page from Google Cache, uploaded by John Pye

2010-05-13T14:03:44Z

Restored page from Google Cache, uploaded by John Pye

New page

<div class="notice metadata" id="outdated" style="border:solid 2pt orange; width:70ex; background-color:#FFEEDD;padding:4px;margin-bottom:6px">''This article is '''outdated'''. You can help out by <span class="plainlinks">[http://ascendwiki.cheme.cmu.edu/index.php?title=Regression_Testing&action=edit updating it]</span>. ''</div>

Historically, ASCEND was developed by a small group of tightly-coordinated members of the [[Art Westerberg]] research group. Individualized ''ad hoc'' testing of modules and successful running of test models was adequate under this development model.

As the ASCEND project grows, we anticipate that more of our developers will be unfamiliar with parts of the code, and for this reason we want to move towards providing a regression/unit testing suite for ASCEND. This is aim to prevent bugs from re-emerging by testing for expected behaviour following an applied fix (regression tests) and thoroughly testing the behaviour of a component currently under development (unit testing, test-driven development).

== Preliminary test suite ==

Jerry has implemented testing of parts of the ASCEND codebase. To run the tests, read about [[BuildingAscend]] then try the following commands:

<pre>cd ~/src/ascend/trunk;
scons WITH_CUNIT_TESTS=1
./base/generic/test/test
</pre>

The following tests are currently implemented:

<pre>[root@cruncher2 generic]# ls '''/test/test_'''.c
general/test/test_dstring.c utilities/test/test_asc[[DynaLoad]].c
general/test/test_hashpjw.c utilities/test/test_asc[[DynaLoad]]_shlib.c
general/test/test_list.c utilities/test/test_asc[[EnvVar]].c
general/test/test_listio.c utilities/test/test_ascMalloc.c
general/test/test_pool.c utilities/test/test_ascPanic.c
general/test/test_pretty.c utilities/test/test_ascPrint.c
general/test/test_register_general.c utilities/test/test_ascSignal.c
general/test/test_stack.c utilities/test/test_mem.c
general/test/test_table.c utilities/test/test_readln.c
general/test/test_tm_time.c utilities/test/test_register_utilities.c
solver/test/test_register_solver.c utilities/test/test_set.c
solver/test/test_slv_common.c
</pre>

We're looking at how best to integrate tests like this into an automatic testing process. The C-based tests don't provide easy cross-system reporting as Python-based tests suites do.

== Strategy for Regression/Unit Testing ==

Unfortunately, the size and complexity of ASCEND makes manually retrofitting a test suite an enormous task. That's not to say that it isn't worth doing, just that it will consume considerable resources. The availability of an automated testing tool would help a lot, but may not be easily available.

A possible approach to implementing a test suite would be

# identify the best testing protocol(s) to use for ASCEND
# prototype the protocol(s) for a limited subset of ASCEND
# implement test-driven development of new code
# implement tests for legacy code

Here are some general testing approaches that could be used. Please add to the list and edit the topics to discuss ideas, preferences, experiences, etc.

* [[StaticCodeAnalysis]]
* [[ManualTestingFrameworks]]
* Automated Testing Tools: we currently have [[BuildBot]] installed.

== ASCEND Model Self-Testing ==

Some work is also underway to implement a tool which will find models containing their own 'self-testing' methods, and run those tests. This will be a higher-level complement to the low-level tests offered here, and will be used to verify the language features are working correctly.

See [[ModelSelfTesting]].

== Python Test Suite ==

A python test suite is being developed using the <tt>unittest</tt> framework, and is growing quickly to cover parts of the code not currently covered by the CUnit tests. The Python test suite will ultimately sweep in the above [[ModelSelfTesting]] as well.

To run the Python test suite, build the Python bindings for ASCEND (type <tt>scons</tt>) then run <tt>python pygtk/test.py</tt>.

== Buildbot ==

A buildbot has been set up which automatically compiles ASCEND and runs the CUnit and Python test suites each time changes are committed to the Subversion repository. See [[BuildBot]].

* <a href="/images/b/b1/Pycunit-0.2-1.src.rpm" class="internal" title="Pycunit-0.2-1.src.rpm">Media:pycunit-0.2-1.src.rpm</a>
*  %META:FILEATTACHMENT{name="pycunit-0.2.tar.gz" attr="" comment="Source tarball" date="1167471650" path="pycunit-0.2.tar.gz" size="18916" user="johnpye" version="1.1"}%

[[Category:Outdated]]
[[Category:Development]]

Regression Testing - Revision history

Jpye: /* Buildbot */

Jpye: /* Strategy for Regression/Unit Testing */

UploadBot: Restored page from Google Cache, uploaded by John Pye