DataMining-Components

Overview: This module implements the CLOSET algorithm to generate closed frequent itemsets consisting of items that occur in a sufficient number of examples to satisfy the minimum support criteria.

Detailed Description: This module takes an FPProb object that has been generated by a Large Item Table Generator module and uses the CLOSET algorithm to find the combinations of items that satisfy a minimum support criteria. An item is an [attribute,value] pair that occurs in the set of examples being mined. The user controls the support criteria via the property of LargeItemTableGenerator that specifies the number of all examples that must contain a given combination of items before that combination is included in the generated output. Each combination of items that satisfies the Minimum Support % is called a Frequent Itemset. The CLOSET algorithm further restricts the itemsets returned by returning only the closed sets. (see Han 2000)

References: For more information on the CLOSET frequent pattern mining algorithm, see " CLOSET: An Efficient Algorithm for Mining Frequent Closed Itemsets ", Jian Pei, Jiawei Han, Runying Mao, 2000.

Data Type Restrictions: While this module can operate on attributes of any datatype, in practice it is usually infeasible to use it with continuous-valued attributes. The module considers each [attribute,value] pair that occurs in the examples individually when building the frequent itemsets. Continuous attributes (and categorical attributes with a large number of values) are less likely to meet the Minimum Support requirements and can result in unacceptably long execution time.

This component implements the FPGrowth algorithm to generate frequent itemsets consisting of items that occur in a sufficient number of examples to satisfy the minimum support criteria.

Detailed Description: This component takes an Item Sets object that has been generated by a Table To Item Sets component and uses the FPGrowth algorithm to find the combinations of items that satisfy a minimum support criteria. An item is an [attribute,value] pair that occurs in the set of examples being mined. The user controls the support criteria via the Minimum Support % property that specifies the percentage of all examples that must contain a given combination of items before that combination is included in the generated output. Each combination of items that satisfies the Minimum Support % is called a Frequent Itemset.

The user can restrict the maximum number of items included in any frequent itemset with the Maximum Items Per Rule property. The generation of sets with large number of items can be computationally expensive, so setting this property in conjunction with the Minimum Support % property helps keep the component runtime reasonable.

In a typical flow the Frequent Item Sets output port from this component is connected to a Compute Confidence component which forms association rules that satisfy a minimum confidence value.

References: For more information on the FPGrowth frequent pattern mining algorithm, see "Mining Frequent Patterns without Candidate Generation"Jiawei Han, Jian Pei, and Yiwen Yin, 2000.

Limitations: The FPGrowth and Compute Confidence components currently build rules with a single item in the consequent.

Data Type Restrictions: While this component can operate on attributes of any datatype, in practice it is usually infeasible to use it with continuous-valued attributes. The component considers each [attribute,value] pair that occurs in the examples individually when building the frequent itemsets. Continuous attributes (and categorical attributes with a large number of values) are less likely to meet the Minimum Support requirements and can result in unacceptably long execution time. Typically Choose Attributes and Binning components should appear in the itinerary prior to the Table to Item Sets component, whose output produces the Item Sets object used as input by this component. The Choosing/Binning components can reduce the number of distinct [attribute,value] pairs that must be considered in this component to a reasonable number.

Data Handling: This component does not modify the input Item Sets in any way.

Scalability: This component creates an array of integers to hold the indices of the items in each frequent itemset. The component may be computationally intensive, and scales with the number of Item Sets entries to search. The user can limit the size of the frequent itemsets although this will have little effect on performance for this algorithm. Choosing/Binning components can be included in the itinerary prior to this components to reduce the number of Item Sets entries.

Overview: This module transforms a SparseExampleTable containing term frequency values into a data structure, FPProb, that efficiently represents item occurrences within user supplied support constraints.

NOTE: All non-zero values for term frequency are treated alike -- they are counted as a positive occurrence for that row.

References: N/A.

Data Type Restrictions: The input table must be a SparseExampleTable containing term frequency information.

Data Handling: This module does not modify the input SparseExampleTable

Scalability: This module makes a constant number of passes over the table data. Memory usage is proportional to the size of the input SparseExampleTable

Overview: This module scans the input SparseTable and any term (column) whose support does not fall within the range specified has its column removed from the table. This can greatly reduce the total number features used for learning -- improving accuracy and performance.

Data Type Restrictions: The input Table must be an instance of a SparseTable.

Data Handling: A new SparseTable instance is created and only columns that will be kept are copied into it.

Scalability: Creates a second table on the same order of size as the original. Columns from the first table are inserted into the second table; no copies are made. Algorithm makes one pass over the table columns and one pass over the table data.

Overview:
This module reads URLs residing underneath the URL that is pointed to by the input WebDAV client.

Overview: Outputs a TableFactory suitable for creating D2K Tables from the ncsa.d2k.modules.core.datatype.table.basic package. This is a standard Table suitable for most applications.

Overview:
This Component initializes a JNDI context for datasource objects.These datasource objects can then be used to connect to a database usingThe connect DB component. The web UI allows the userto add and configure datasource objects.

Overview:
Given a resultset object from an SQL query,create a table for data-mining. The table is a D2K Table object. Each column of the table is from a column in the resultset. Each row of the table correspons to a row from the resultset.

Overview:
Read a graph from a local or remote file using GraphML from prefuse.
Note: This module is the same as deprecated module GraphMLToPrefuse, extended to access the data through DataObjectProxy.

Overview: This module will scan the item sets in an FPProb object and determine the minimum and maximum supports. It adds this information to the FPProb object before writing it to output.

References: For more information on the FPGrowth frequent pattern mining algorithm, see "Mining Frequent Patterns without Candidate Generation"Jiawei Han, Jian Pei, and Yiwen Yin, 2000.

Data Handling: This module modifies the support boundary values in the FPProb object.

Detailed Description:
This module creates a parser that can be used to read data from a file that uses a single delimiter character to separate the data into fields. The delimiter can be found automatically, or it can be input in the properties editor. If the delimiter is to be found automatically, the file must contain at least 2 rows. The file can contain a row of labels, and a row of data types. These are also specified via the properties editor.Properties are used to specify the delimiter, the labels row number, and the types row number. The row numbers are indexed from zero.Typically the File Parser output port of this module is connected to the File Parser input port of a module whose name begins with 'Parse File', for example, Parse File To Table or Parse File To Paging Table.Data Type Restrictions: The input to this module must be a delimited file. If the file is large a java OutOfMemory error might occur.

Data Handling: The module does not destroy or modify the input data.

Overview: Converts a JUNG graph (directed or not) to a Prefuse graph.

Acknowledgement: This module uses functionality from the JUNG project. See http://jung.sourceforge.net.

Acknowledgement: This module uses functionality from the Prefuse project. See http://prefuse.org.

Overview:
This Component initializes a JNDI context for datasource object.This datasource object can then be used to connect to a database using the connect DB component.OpenDS Directory Server is recommended to use for JNDI. Before running this component, starting up the server and creating a new entry for saving datasource object. Then providing this compoennt with the username and password of server, and the entry's distinguished name(DN).

This module reads a Table and extracts from it items for use in mining association rules with the Apriori algorithm.

Detailed Description: This module takes as input a Table or an Example Table, and extracts items that are used by the Apriori rule association algorithm. An item is an [attribute,value] pair that occurs in the input table. The module uses information from the original table to determine which attributes should be used to form items being considered as possible rule antecedents and rule consequents. A compact representation is created indicating which items are contained in rows in the original table. The items and other information used by the Apriori algorithm are written to the Item Sets output port.

If a Table or an Example Table with no specified input or output attributes is loaded, all attributes (columns) will be used to form items being considered as possible antecedents and consequents for the association rules. If an Example Table with only input attributes or only output attributes is loaded, the chosen attributes will be used to form items considered as possible rule antecedents and possible rule consequents. If an Example Table with both input and output attributes is loaded, the inputs will be used to form items considered as possible rule antecedents, and the outputs used to form items considered as possible rule consequents.

The computational complexity of the Apriori algorithm depends on the number of possible antecedents and consequents, so narrowing the search prior to this step is highly recommended. Use the module Choose Attributes to specify the subset of table attributes that are of interest. If the table has continuous attributes as possible rule antecedents or targets, a Binning module should be used prior to this module to reduce the number of possible values for those continuous attributes.

In a typical itinerary the Item Sets output port from this module is connected to a Generate Multiple Outputs module and then to an Apriori module which forms frequent itemsets based on a minimum support value, and to a Compute Confidence module which forms association rules that satisfy a minimum confidence value.

Limitations: The Apriori and Compute Confidence modules currently build rules with a single item in the consequent.

Data Handling: This module does not modify the input Table in any way.

Scalability: A representation of each row of the table is stored in memory. The representation is usually smaller than the original data.

Overview:
This component accepts a resultset object generated by a query to a database. The component launches a webUI and displayes the resultset as a table. Each column in the table refers to a column in the resultset and each row to an enrty in the resultset.

Overview:
This component closes a connection object to a database. After all components connecting to the databse have been executed, the connection should be closed.

Overview: This module creates a Jung Graph object from a GraphML file. The input may be a local file or remote.

Note: This module is the same as deprecated module XMLToGraphFromURL, extended to access the data through aDataObjectProxy.

Acknowledgement: This module uses functionality from the JUNG project. See http://jung.sourceforge.net.

Dendrogram visualization of SEASR cluster models.

Properties:
The 'use_local_host' means that you intend to run the server and browser client on the same machine. This is set to false by default. If you do run both on the same machine then it is important to set this property to true. The reason is that when Java queries to OS for the local machines IP it often gets internal LAN IP's that often are not reachable as url's in your browser, and therefore an error will result. By choosing to set this property to true you force the client app to use '127.0.0.1' for your local address.
The sparse_detail_limit property defaults to 10. For sparse tables it is not practical to display the entire table subset that make up a given cluster because the table may have thousands of columns. This value tells the application how many column values to display in descending order by support.

Overview:
This component writes a D2K table into a databse. The user specifies the name of the database table. The user sets createTable = true if the table needs to be created in the databse. If createTable is false, it will append the d2k table to an existing database table.

This module allows the user to choose which columns of a table are inputs and outputs.

Detailed Description: This module outputs an Example Table with the input and output features assigned. Inputs and outputs do not have to be selected, nor do they have to be mutually exclusive.

Data Handling: This module does not modify the data in the table. It only sets the input and output features.

Overview:
This component accepts a connection object to a database and queries the database to get a list of all databse tables.
It then starts a WebUI to display all the tables in the database to the user. It is best used to test connections to databases.

Overview:
This component opens a connection object to a database. This component assumes that a JNDI context with datasources is already configured. It will open a connection to the database specified by the JNDI object at JNDIName.

This module works in conjunction with other modules implementing the Apriori rule association algorithm to generate association rules satisfying a minimum confidence threshold.

Detailed Description: This module takes as input an Item Sets object generated by the Table To Item Sets module, and Frequent Itemsets generated by the Apriori or FPGrowth module. From these inputs, it develops a set of possible association rules, each with a single target item, where an item consists of an [attribute,value] pair. For each possible rule, this module computes the Confidence in the prediction, and accepts those rules that meet a minimum confidence threshold specified via the property editor.

For a rule of the form Antecedent A implies Consequent C, the Confidence is the percentage of examples in the original data that contain A that also contain C. The formula to compute the confidence of the rule A->C is:
Confidence = ( (# of examples with A and C) / (# of examples with A ) ) * 100.00

Limitations: The Apriori, FPGrowthand Compute Confidence modules currently build rules with a single item in the consequent.

Scalability: This module searches all the Items Sets to compute the confidence for each Frequent Itemset. The module allocated memory for the resulting Rule Table.

Overview: Outputs a TableFactory suitable for creating D2K Tables from the org.seasr.datatypes.datamining.table.sparse package.

Overview:
This compnent executes a query and returns the resultset. The user has three options.
First they may specify a path to a query file on the local filesystem using the Query_Path property.
If this property is not blank, the component will attempt to load from the path and execute the query stored in that file.
If the Query_Path property is blank, it will check the Query_Statement Property.
If this is not blank, it will attempt to execute the text in the Query_Statement property as an sql query.
Finally, if both properties are blank, it will present the user with a WebUI to enter a query manually

This module applies a Transformation to a Table.

Detailed Description: This module applies a Transformation to a MutableTable and outputs the transformed table as a MutableTable.

Data Handling: This module modifies the input Table