public class PairedTTester extends java.lang.Object implements OptionHandler, Tester, RevisionHandler
-D <index,index2-index4,...> Specify list of columns that specify a unique dataset. First and last are valid indexes. (default none)
-R <index> Set the index of the column containing the run number
-F <index> Set the index of the column containing the fold number
-G <index1,index2-index4,...> Specify list of columns that specify a unique 'result generator' (eg: classifier name and options). First and last are valid indexes. (default none)
-S <significance level> Set the significance level for comparisons (default 0.05)
-V Show standard deviations
-L Produce table comparisons in Latex table format
-csv Produce table comparisons in CSV table format
-html Produce table comparisons in HTML table format
-significance Produce table comparisons with only the significance values
-gnuplot Produce table comparisons output suitable for GNUPlot
| Constructor and Description |
|---|
PairedTTester() |
| Modifier and Type | Method and Description |
|---|---|
void |
assign(Tester tester)
retrieves all the settings from the given Tester
|
PairedStats |
calculateStatistics(Instance datasetSpecifier,
int resultset1Index,
int resultset2Index,
int comparisonColumn)
Computes a paired t-test comparison for a specified dataset between two
resultsets.
|
boolean |
displayResultset(int index)
Checks whether the resultset with the given index shall be displayed.
|
Range |
getDatasetKeyColumns()
Get the value of DatasetKeyColumns.
|
int[] |
getDisplayedResultsets()
Gets the indices of the the datasets that are displayed (if
null then all are displayed). |
java.lang.String |
getDisplayName()
returns the name of the tester
|
int |
getFoldColumn()
Get the value of FoldColumn.
|
Instances |
getInstances()
Get the value of Instances.
|
int |
getNumDatasets()
Gets the number of datasets in the resultsets
|
int |
getNumResultsets()
Gets the number of resultsets in the data.
|
java.lang.String[] |
getOptions()
Gets current settings of the PairedTTester.
|
ResultMatrix |
getResultMatrix()
Gets the instance that produces the output.
|
Range |
getResultsetKeyColumns()
Get the value of ResultsetKeyColumns.
|
java.lang.String |
getResultsetName(int index)
Gets a string descriptive of the specified resultset.
|
java.lang.String |
getRevision()
Returns the revision string.
|
int |
getRunColumn()
Get the value of RunColumn.
|
boolean |
getShowStdDevs()
Returns true if standard deviations have been requested.
|
double |
getSignificanceLevel()
Get the value of SignificanceLevel.
|
int |
getSortColumn()
Returns the column to sort on, -1 means the default sorting.
|
java.lang.String |
getSortColumnName()
Returns the name of the column to sort on.
|
java.lang.String |
getToolTipText()
returns a string that is displayed as tooltip on the "perform test" button
in the experimenter
|
java.lang.String |
header(int comparisonColumn)
Creates a "header" string describing the current resultsets.
|
java.util.Enumeration<Option> |
listOptions()
Lists options understood by this object.
|
static void |
main(java.lang.String[] args)
Test the class from the command line.
|
java.lang.String |
multiResultsetFull(int baseResultset,
int comparisonColumn)
Creates a comparison table where a base resultset is compared to the other
resultsets.
|
java.lang.String |
multiResultsetRanking(int comparisonColumn)
returns a ranking of the resultsets
|
java.lang.String |
multiResultsetSummary(int comparisonColumn)
Carries out a comparison between all resultsets, counting the number of
datsets where one resultset outperforms the other.
|
int[][] |
multiResultsetWins(int comparisonColumn,
int[][] nonSigWin)
Carries out a comparison between all resultsets, counting the number of
datsets where one resultset outperforms the other.
|
java.lang.String |
resultsetKey()
Creates a key that maps resultset numbers to their descriptions.
|
void |
setDatasetKeyColumns(Range newDatasetKeyColumns)
Set the value of DatasetKeyColumns.
|
void |
setDisplayedResultsets(int[] cols)
Sets the indicies of the datasets to display (
null means all). |
void |
setFoldColumn(int newFoldColumn)
Set the value of FoldColumn.
|
void |
setInstances(Instances newInstances)
Set the value of Instances.
|
void |
setOptions(java.lang.String[] options)
Parses a given list of options.
|
void |
setResultMatrix(ResultMatrix matrix)
Sets the matrix to use to produce the output.
|
void |
setResultsetKeyColumns(Range newResultsetKeyColumns)
Set the value of ResultsetKeyColumns.
|
void |
setRunColumn(int newRunColumn)
Set the value of RunColumn.
|
void |
setShowStdDevs(boolean s)
Set whether standard deviations are displayed or not.
|
void |
setSignificanceLevel(double newSignificanceLevel)
Set the value of SignificanceLevel.
|
void |
setSortColumn(int newSortColumn)
Set the column to sort on, -1 means the default sorting.
|
public void setResultMatrix(ResultMatrix matrix)
setResultMatrix in interface Testermatrix - the instance to use to produce the outputResultMatrixpublic ResultMatrix getResultMatrix()
getResultMatrix in interface Testerpublic void setShowStdDevs(boolean s)
setShowStdDevs in interface Testers - true if standard deviations are to be displayedpublic boolean getShowStdDevs()
getShowStdDevs in interface Testerpublic int getNumDatasets()
getNumDatasets in interface Testerpublic int getNumResultsets()
getNumResultsets in interface Testerpublic java.lang.String getResultsetName(int index)
getResultsetName in interface Testerindex - the index of the resultsetpublic boolean displayResultset(int index)
displayResultset in interface Testerindex - the index of the resultset to check whether it shall be
displayedpublic PairedStats calculateStatistics(Instance datasetSpecifier, int resultset1Index, int resultset2Index, int comparisonColumn) throws java.lang.Exception
calculateStatistics in interface TesterdatasetSpecifier - the dataset specifierresultset1Index - the index of the first resultsetresultset2Index - the index of the second resultsetcomparisonColumn - the column containing values to comparejava.lang.Exception - if an error occurspublic java.lang.String resultsetKey()
resultsetKey in interface Testerpublic java.lang.String header(int comparisonColumn)
public int[][] multiResultsetWins(int comparisonColumn,
int[][] nonSigWin)
throws java.lang.Exception
multiResultsetWins in interface TestercomparisonColumn - the index of the comparison columnnonSigWin - for storing the non-significant winsjava.lang.Exception - if an error occurspublic java.lang.String multiResultsetSummary(int comparisonColumn)
throws java.lang.Exception
multiResultsetSummary in interface TestercomparisonColumn - the index of the comparison columnjava.lang.Exception - if an error occurspublic java.lang.String multiResultsetRanking(int comparisonColumn)
throws java.lang.Exception
multiResultsetRanking in interface TestercomparisonColumn - the column to compare withjava.lang.Exception - if something goes wrongpublic java.lang.String multiResultsetFull(int baseResultset,
int comparisonColumn)
throws java.lang.Exception
multiResultsetFull in interface TesterbaseResultset - the index of the base resultsetcomparisonColumn - the index of the column to compare overjava.lang.Exception - if an error occurspublic java.util.Enumeration<Option> listOptions()
listOptions in interface OptionHandlerpublic void setOptions(java.lang.String[] options)
throws java.lang.Exception
-D <index,index2-index4,...> Specify list of columns that specify a unique dataset. First and last are valid indexes. (default none)
-R <index> Set the index of the column containing the run number
-F <index> Set the index of the column containing the fold number
-G <index1,index2-index4,...> Specify list of columns that specify a unique 'result generator' (eg: classifier name and options). First and last are valid indexes. (default none)
-S <significance level> Set the significance level for comparisons (default 0.05)
-V Show standard deviations
-L Produce table comparisons in Latex table format
-csv Produce table comparisons in CSV table format
-html Produce table comparisons in HTML table format
-significance Produce table comparisons with only the significance values
-gnuplot Produce table comparisons output suitable for GNUPlot
setOptions in interface OptionHandleroptions - an array containing options to set.java.lang.Exception - if invalid options are givenpublic java.lang.String[] getOptions()
getOptions in interface OptionHandlerpublic Range getResultsetKeyColumns()
getResultsetKeyColumns in interface Testerpublic void setResultsetKeyColumns(Range newResultsetKeyColumns)
setResultsetKeyColumns in interface TesternewResultsetKeyColumns - Value to assign to ResultsetKeyColumns.public int[] getDisplayedResultsets()
null then all are displayed). The base is always displayed.getDisplayedResultsets in interface Testerpublic void setDisplayedResultsets(int[] cols)
null means all).
The base is always displayed.setDisplayedResultsets in interface Testercols - the indices of the datasets to displaypublic double getSignificanceLevel()
getSignificanceLevel in interface Testerpublic void setSignificanceLevel(double newSignificanceLevel)
setSignificanceLevel in interface TesternewSignificanceLevel - Value to assign to SignificanceLevel.public Range getDatasetKeyColumns()
getDatasetKeyColumns in interface Testerpublic void setDatasetKeyColumns(Range newDatasetKeyColumns)
setDatasetKeyColumns in interface TesternewDatasetKeyColumns - Value to assign to DatasetKeyColumns.public int getRunColumn()
getRunColumn in interface Testerpublic void setRunColumn(int newRunColumn)
setRunColumn in interface TesternewRunColumn - Value to assign to RunColumn.public int getFoldColumn()
getFoldColumn in interface Testerpublic void setFoldColumn(int newFoldColumn)
setFoldColumn in interface TesternewFoldColumn - Value to assign to FoldColumn.public java.lang.String getSortColumnName()
getSortColumnName in interface Testerpublic int getSortColumn()
getSortColumn in interface Testerpublic void setSortColumn(int newSortColumn)
setSortColumn in interface TesternewSortColumn - the new sort column.public Instances getInstances()
getInstances in interface Testerpublic void setInstances(Instances newInstances)
setInstances in interface TesternewInstances - Value to assign to Instances.public void assign(Tester tester)
public java.lang.String getToolTipText()
getToolTipText in interface Testerpublic java.lang.String getDisplayName()
getDisplayName in interface Testerpublic java.lang.String getRevision()
getRevision in interface RevisionHandlerpublic static void main(java.lang.String[] args)
args - contains options for the instance ttests