<title>Installing GCC: Testing</title>
<meta http-equiv="Content-Type" content="text/html">
<meta name="description" content="Installing GCC: Testing">
-<meta name="generator" content="makeinfo 4.5">
-<link href="http://www.gnu.org/software/texinfo/" rel="generator-home">
+<meta name="generator" content="makeinfo 4.13">
+<link title="Top" rel="top" href="#Top">
+<link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage">
+<!--
+Copyright (C) 1988, 1989, 1992, 1993, 1994, 1995, 1996, 1997,
+1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007,
+2008 Free Software Foundation, Inc.
+
+ Permission is granted to copy, distribute and/or modify this document
+under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no
+Invariant Sections, the Front-Cover texts being (a) (see below), and
+with the Back-Cover Texts being (b) (see below). A copy of the
+license is included in the section entitled "GNU Free Documentation License".
+
+(a) The FSF's Front-Cover Text is:
+
+ A GNU Manual
+
+(b) The FSF's Back-Cover Text is:
+
+ You have freedom to copy and modify this GNU Manual, like GNU
+ software. Copies published by the Free Software Foundation raise
+ funds for GNU development.-->
+<meta http-equiv="Content-Style-Type" content="text/css">
+<style type="text/css"><!--
+ pre.display { font-family:inherit }
+ pre.format { font-family:inherit }
+ pre.smalldisplay { font-family:inherit; font-size:smaller }
+ pre.smallformat { font-family:inherit; font-size:smaller }
+ pre.smallexample { font-size:smaller }
+ pre.smalllisp { font-size:smaller }
+ span.sc { font-variant:small-caps }
+ span.roman { font-family:serif; font-weight:normal; }
+ span.sansserif { font-family:sans-serif; font-weight:normal; }
+--></style>
</head>
<body>
<h1 class="settitle">Installing GCC: Testing</h1>
+<a name="index-Testing-1"></a><a name="index-Installing-GCC_003a-Testing-2"></a><a name="index-Testsuite-3"></a>
Before you install GCC, we encourage you to run the testsuites and to
compare your results with results from a similar configuration that have
been submitted to the
<a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>.
+Some of these archived results are linked from the build status lists
+at <a href="http://gcc.gnu.org/buildstat.html">http://gcc.gnu.org/buildstat.html</a>, although not everyone who
+reports a successful build runs the testsuites and submits the results.
This step is optional and may require you to download additional software,
but it can give you confidence in your new GCC installation or point out
problems before you install and start using your new GCC.
<p>First, you must have <a href="download.html">downloaded the testsuites</a>.
These are part of the full distribution, but if you downloaded the
-"core" compiler plus any front ends, you must download the testsuites
+“core” compiler plus any front ends, you must download the testsuites
separately.
<p>Second, you must have the testing tools installed. This includes
-a <a href="http://www.gnu.org/software/dejagnu/">current version of DejaGnu</a>;
-dejagnu 1.3 is not sufficient.
-It also includes Tcl and Expect; the DejaGnu site has links to these.
-
- <p>Now you may need specific preparations:
-
- <ul>
-
- <li>The following environment variables may need to be set appropriately, as in
-the following example (which assumes that DejaGnu has been installed
-under <code>/usr/local</code>):
-
- <pre class="example"> TCL_LIBRARY = /usr/local/share/tcl8.0
- DEJAGNULIBS = /usr/local/share/dejagnu
- </pre>
-
- <p>On systems such as Cygwin, these paths are required to be actual
+<a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect;
+the DejaGnu site has links to these.
+
+ <p>If the directories where <samp><span class="command">runtest</span></samp> and <samp><span class="command">expect</span></samp> were
+installed are not in the <samp><span class="env">PATH</span></samp>, you may need to set the following
+environment variables appropriately, as in the following example (which
+assumes that DejaGnu has been installed under <samp><span class="file">/usr/local</span></samp>):
+
+<pre class="smallexample"> TCL_LIBRARY = /usr/local/share/tcl8.0
+ DEJAGNULIBS = /usr/local/share/dejagnu
+</pre>
+ <p>(On systems such as Cygwin, these paths are required to be actual
paths, not mounts or links; presumably this is due to some lack of
-portability in the DejaGnu code.
-
- <p>If the directories where <code>runtest</code> and <code>expect</code> were
-installed are in the <code>PATH</code>, it should not be necessary to set these
-environment variables.
-
- </ul>
+portability in the DejaGnu code.)
<p>Finally, you can run the testsuite (which may take a long time):
-<pre class="example"> cd <var>objdir</var>; make -k check
- </pre>
-
- <p>The testing process will try to test as many components in the GCC
-distribution as possible, including the C, C++, Objective-C and Fortran
-compilers as well as the C++ and Java runtime libraries.
-
- <p>While running the testsuite, DejaGnu might emit messages resembling
-<code>WARNING: Couldn't find the global config file.</code> or
-<code>WARNING: Couldn't find tool init file</code>.
-These messages are harmless and do not affect the validity of the tests.
-
-<h3 class="section"><a name="TOC0"></a>How can I run the test suite on selected tests?</h3>
-
- <p>As a first possibility to cut down the number of tests that are run it is
-possible to use <code>make check-gcc</code> or <code>make check-g++</code>
-in the <code>gcc</code> subdirectory of the object directory. To further cut down the
-tests the following is possible:
-
-<pre class="example"> make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>"
- </pre>
-
- <p>This will run all <code>gcc</code> execute tests in the testsuite.
-
-<pre class="example"> make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>"
- </pre>
-
- <p>This will run the <code>g++</code> "old-deja" tests in the testsuite where the filename
-matches <code>9805*</code>.
-
- <p>The <code>*.exp</code> files are located in the testsuite directories of the GCC
-source, the most important ones being <code>compile.exp</code>,
-<code>execute.exp</code>, <code>dg.exp</code> and <code>old-deja.exp</code>.
-To get a list of the possible <code>*.exp</code> files, pipe the
-output of <code>make check</code> into a file and look at the
-<code>Running ... .exp</code> lines.
-
- <p>To run only the tests for a library, run <code>make check</code> from the
-the library's testsuite in a subdirectory of the object directory:
-<code>libstdc++-v3/testsuite</code> or <code>libcgj/testsuite</code>.
-
-<h3 class="section"><a name="TOC1"></a>Additional testing for Java Class Libraries</h3>
-
- <p>The <a href="http://sources.redhat.com/mauve/">Mauve Project</a> provides
+<pre class="smallexample"> cd <var>objdir</var>; make -k check
+</pre>
+ <p>This will test various components of GCC, such as compiler
+front ends and runtime libraries. While running the testsuite, DejaGnu
+might emit some harmless messages resembling
+‘<samp><span class="samp">WARNING: Couldn't find the global config file.</span></samp>’ or
+‘<samp><span class="samp">WARNING: Couldn't find tool init file</span></samp>’ that can be ignored.
+
+ <p>If you are testing a cross-compiler, you may want to run the testsuite
+on a simulator as described at <a href="http://gcc.gnu.org/simtest-howto.html">http://gcc.gnu.org/simtest-howto.html</a>.
+
+<h3 class="section"><a name="TOC0"></a>How can you run the testsuite on selected tests?</h3>
+
+<p>In order to run sets of tests selectively, there are targets
+‘<samp><span class="samp">make check-gcc</span></samp>’ and ‘<samp><span class="samp">make check-g++</span></samp>’
+in the <samp><span class="file">gcc</span></samp> subdirectory of the object directory. You can also
+just run ‘<samp><span class="samp">make check</span></samp>’ in a subdirectory of the object directory.
+
+ <p>A more selective way to just run all <samp><span class="command">gcc</span></samp> execute tests in the
+testsuite is to use
+
+<pre class="smallexample"> make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>"
+</pre>
+ <p>Likewise, in order to run only the <samp><span class="command">g++</span></samp> “old-deja” tests in
+the testsuite with filenames matching ‘<samp><span class="samp">9805*</span></samp>’, you would use
+
+<pre class="smallexample"> make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>"
+</pre>
+ <p>The <samp><span class="file">*.exp</span></samp> files are located in the testsuite directories of the GCC
+source, the most important ones being <samp><span class="file">compile.exp</span></samp>,
+<samp><span class="file">execute.exp</span></samp>, <samp><span class="file">dg.exp</span></samp> and <samp><span class="file">old-deja.exp</span></samp>.
+To get a list of the possible <samp><span class="file">*.exp</span></samp> files, pipe the
+output of ‘<samp><span class="samp">make check</span></samp>’ into a file and look at the
+‘<samp><span class="samp">Running ... .exp</span></samp>’ lines.
+
+<h3 class="section"><a name="TOC1"></a>Passing options and running multiple testsuites</h3>
+
+<p>You can pass multiple options to the testsuite using the
+‘<samp><span class="samp">--target_board</span></samp>’ option of DejaGNU, either passed as part of
+‘<samp><span class="samp">RUNTESTFLAGS</span></samp>’, or directly to <samp><span class="command">runtest</span></samp> if you prefer to
+work outside the makefiles. For example,
+
+<pre class="smallexample"> make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants"
+</pre>
+ <p>will run the standard <samp><span class="command">g++</span></samp> testsuites (“unix” is the target name
+for a standard native testsuite situation), passing
+‘<samp><span class="samp">-O3 -fmerge-constants</span></samp>’ to the compiler on every test, i.e.,
+slashes separate options.
+
+ <p>You can run the testsuites multiple times using combinations of options
+with a syntax similar to the brace expansion of popular shells:
+
+<pre class="smallexample"> ..."--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}"
+</pre>
+ <p>(Note the empty option caused by the trailing comma in the final group.)
+The following will run each testsuite eight times using the ‘<samp><span class="samp">arm-sim</span></samp>’
+target, as if you had specified all possible combinations yourself:
+
+<pre class="smallexample"> --target_board=arm-sim/-mhard-float/-O1
+ --target_board=arm-sim/-mhard-float/-O2
+ --target_board=arm-sim/-mhard-float/-O3
+ --target_board=arm-sim/-mhard-float
+ --target_board=arm-sim/-msoft-float/-O1
+ --target_board=arm-sim/-msoft-float/-O2
+ --target_board=arm-sim/-msoft-float/-O3
+ --target_board=arm-sim/-msoft-float
+</pre>
+ <p>They can be combined as many times as you wish, in arbitrary ways. This
+list:
+
+<pre class="smallexample"> ..."--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}"
+</pre>
+ <p>will generate four combinations, all involving ‘<samp><span class="samp">-Wextra</span></samp>’.
+
+ <p>The disadvantage to this method is that the testsuites are run in serial,
+which is a waste on multiprocessor systems. For users with GNU Make and
+a shell which performs brace expansion, you can run the testsuites in
+parallel by having the shell perform the combinations and <samp><span class="command">make</span></samp>
+do the parallel runs. Instead of using ‘<samp><span class="samp">--target_board</span></samp>’, use a
+special makefile target:
+
+<pre class="smallexample"> make -j<var>N</var> check-<var>testsuite</var>//<var>test-target</var>/<var>option1</var>/<var>option2</var>/...
+</pre>
+ <p>For example,
+
+<pre class="smallexample"> make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu}
+</pre>
+ <p>will run three concurrent “make-gcc” testsuites, eventually testing all
+ten combinations as described above. Note that this is currently only
+supported in the <samp><span class="file">gcc</span></samp> subdirectory. (To see how this works, try
+typing <samp><span class="command">echo</span></samp> before the example given here.)
+
+<h3 class="section"><a name="TOC2"></a>Additional testing for Java Class Libraries</h3>
+
+<p>The Java runtime tests can be executed via ‘<samp><span class="samp">make check</span></samp>’
+in the <samp><var>target</var><span class="file">/libjava/testsuite</span></samp> directory in
+the build tree.
+
+ <p>The <a href="http://sourceware.org/mauve/">Mauve Project</a> provides
a suite of tests for the Java Class Libraries. This suite can be run
-as part of libgcj testing by specifying the location of the Mauve tree
-when invoking <code>make</code>, as in <code>make MAUVEDIR=~/mauve check</code>.
+as part of libgcj testing by placing the Mauve tree within the libjava
+testsuite at <samp><span class="file">libjava/testsuite/libjava.mauve/mauve</span></samp>, or by
+specifying the location of that tree when invoking ‘<samp><span class="samp">make</span></samp>’, as in
+‘<samp><span class="samp">make MAUVEDIR=~/mauve check</span></samp>’.
-<h3 class="section"><a name="TOC2"></a>How to interpret test results</h3>
+<h3 class="section"><a name="TOC3"></a>How to interpret test results</h3>
- <p>After the testsuite has run you'll find various <code>*.sum</code> and <code>*.log</code>
-files in the testsuite subdirectories. The <code>*.log</code> files contain a
+<p>The result of running the testsuite are various <samp><span class="file">*.sum</span></samp> and <samp><span class="file">*.log</span></samp>
+files in the testsuite subdirectories. The <samp><span class="file">*.log</span></samp> files contain a
detailed log of the compiler invocations and the corresponding
-results, the <code>*.sum</code> files summarize the results. These summaries list
-all the tests that have been run with a corresponding status code:
+results, the <samp><span class="file">*.sum</span></samp> files summarize the results. These summaries
+contain status codes for all tests:
<ul>
<li>PASS: the test passed as expected
</ul>
<p>It is normal for some tests to report unexpected failures. At the
-current time our testing harness does not allow fine grained control
-over whether or not a test is expected to fail. We expect to fix this
-problem in future releases.
+current time the testing harness does not allow fine grained control
+over whether or not a test is expected to fail. This problem should
+be fixed in future releases.
-<h3 class="section"><a name="TOC3"></a>Submitting test results</h3>
+<h3 class="section"><a name="TOC4"></a>Submitting test results</h3>
-
<p>If you want to report the results to the GCC project, use the
-<code>contrib/test_summary</code> shell script. Start it in the <var>objdir</var> with
+<p>If you want to report the results to the GCC project, use the
+<samp><span class="file">contrib/test_summary</span></samp> shell script. Start it in the <var>objdir</var> with
-<pre class="example"> <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \
+<pre class="smallexample"> <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \
-m gcc-testresults@gcc.gnu.org |sh
- </pre>
-
- <p>This script uses the <code>Mail</code> program to send the results, so
-make sure it is in your <code>PATH</code>. The file <code>your_commentary.txt</code> is
+</pre>
+ <p>This script uses the <samp><span class="command">Mail</span></samp> program to send the results, so
+make sure it is in your <samp><span class="env">PATH</span></samp>. The file <samp><span class="file">your_commentary.txt</span></samp> is
prepended to the testsuite summary and should contain any special
remarks you have on your results or your build environment. Please
do not edit the testsuite result block or the subject line, as these
-messages are automatically parsed and presented at the
-<a href="http://gcc.gnu.org/testresults/">GCC testresults</a> web
-page. Here you can also gather information on how specific tests
-behave on different platforms and compare them with your results. A
-few failing testcases are possible even on released versions and you
-should look here first if you think your results are unreasonable.
-
- <hr />
-<p>
-<a href="./index.html">Return to the GCC Installation page</a>
-
- </body></html>
+messages may be automatically processed.
+
+ <p><hr />
+<p><a href="./index.html">Return to the GCC Installation page</a>
+
+<!-- ***Final install*********************************************************** -->
+<!-- ***Binaries**************************************************************** -->
+<!-- ***Specific**************************************************************** -->
+<!-- ***Old documentation****************************************************** -->
+<!-- ***GFDL******************************************************************** -->
+<!-- *************************************************************************** -->
+<!-- Part 6 The End of the Document -->
+</body></html>
projects / msp430-gcc.git / blobdiff