Additional order info. K educators : This link is for individuals purchasing with credit cards or PayPal only. Introduction to Data Mining presents fundamental concepts and algorithms for those learning data mining for the first time. Each concept is explored thoroughly and supported with numerous examples. The text requires only a modest background in mathematics. Each major topic is organized into two chapters, beginning with basic concepts that provide necessary background for understanding each data mining technique, followed by more advanced concepts and algorithms.
This book provides a comprehensive coverage of important data mining techniques. Numerous examples are provided to lucidly illustrate the key concepts. In my opinion this is currently the best data mining text book on the market. I like the comprehensive coverage which spans all major data mining techniques including classification, clustering, and pattern mining association rules. Pearson offers affordable and accessible purchase options to meet the needs of your students.
This research has resulted in more than papers published in the proceedings of major data mining conferences or computer science or domain journals. Teaching and Learning Experience This program will provide a better teaching and learning experience-for you and your students.
Each major topic kumaf organized into two chapters, beginning with basic concepts that provide necessary background for understanding each data mining technique, followed by more advanced concepts and algorithms.
Previous to his academic career, he held a variety of software engineering, analysis, and design positions in industry at Silicon Biology, Racotek, and NCR. User Review — Flag as inappropriate provide its preview. Product details Format Paperback pages Nung x x The text requires only a modest background in mathematics. Pearson Addison Wesley- Data mining — pages. Check out the top books of the year on our page Best Books of This chapter addresses the increasing concern over the validity and reproducibility of results obtained from data analysis.
Each concept is explored thoroughly and supported with numerous examples. A new appendix provides a brief discussion vi;in scalability in the context of big data. This book provides a comprehensive coverage of important data mining techniques. Ninf to Data Mining presents fundamental concepts and algorithms for those learning data mining for the first time. Each major topic is organized into two chapters, I like the comprehensive coverage which spans all major data mining techniques including classification, clustering, and pattern mining association rules.
Home Contact Us Help Free delivery worldwide. Dispatched from the UK in 2 business days When will my order arrive? Some of the most significant improvements in the text have been in the two chapters on classification. Almost every section of the advanced classification chapter has been significantly updated. It is also suitable for individuals seeking an introduction to data mining.
The reconstruction-based approach is illustrated using autoencoder networks that are vvipin of the deep learning paradigm. Keep track of which object is associated with each distance.
Assume the distance function will only return a distance of 0 for objects that are the same. There are several problems. First, the order of duplicate objects on a nearest neighbor list will depend on details of the algorithm and the order of objects in the data set.
Second, if there are enough duplicates, the nearest neighbor list may consist only of duplicates. Third, an object may not be its own nearest neighbor. There are various approaches depending on the situation. One approach is to to keep only one object for each group of duplicate objects. In this case, each neighbor can represent either a single object or a group of duplicate objects.
The following attributes are measured for members of a herd of Asian ele- phants: weight, height, tusk length, trunk length, and ear area. Based on these measurements, what sort of similarity measure from Section 2.
Justify your answer and ex- plain any special circumstances. These attributes are all numerical, but can have widely varying ranges of values, depending on the scale used to measure them. Furthermore, the attributes are not asymmetric and the magnitude of an attribute matters. These latter two facts eliminate the cosine and correlation measure. Eu- clidean distance, applied after standardizing the attributes to have a mean of 0 and a standard deviation of 1, would be appropriate.
You are given a set of m objects that is divided into K groups, where the ith group is of size mi. Assume sampling with replacement.
Consider a document-term matrix, where tfij is the frequency of the ith word term in the j th document and m is the number of documents. This transformation is known as the inverse document frequency transformation. In every document? Terms that occur in every document have 0 weight, while those that occur in one document have maximum weight, i. This exercise compares and contrasts some similarity and distance measures.
The Jaccard similarity is a measure of the similarity between two binary vectors. Compute the Hamming distance and the Jaccard simi- larity between the following two binary vectors. The Hamming distance is similar to the SMC.
The Jaccard measure is similar to the cosine measure because both ignore matches. Describe which measure, Hamming or Jaccard, you think would be more appropriate for comparing the genetic makeup of two organisms. Assume that each animal is represented as a binary vector, where each attribute is 1 if a particular gene is present in the organism and 0 otherwise. Jaccard is more appropriate for comparing the genetic makeup of two organisms; since we want to see how many genes these two organisms share.
Thus, the Hamming distance is more appropriate in this situation. For the following vectors, x and y, calculate the indicated similarity or dis- tance measures. Here, we further explore the cosine and correlation measures. Many times the data has only positive entries and in that case the range is [0, 1].
Not necessarily. What general observation can you make about the relationship between Euclidean distance and cosine similarity when vectors have an L2 norm of 1? Since all the , points fall on the curve, there is a functional rela- tionship between Euclidean distance and cosine similarity for normal- ized data.
For example, if two data points are identical, their cosine similarity is one and their Euclidean distance is zero, but if two data points have a high Euclidean distance, their cosine value is close to zero. Note that all the sample data points were from the positive quadrant, i. This means that all cosine and correlation values will be positive.
What general observation can you make about the relationship between Euclidean distance and correlation when the vectors have been standardized to have a mean of 0 and a standard deviation of 1? Let x and y be two vectors where each vector has an L2 length of 1. For such vectors, the variance is just n times the sum of its squared attribute values and the correlation between the two vectors is their dot product divided by n.
Let x and y be two vectors where each vector has an a mean of 0 and a standard deviation of 1. For such vectors, the variance standard deviation squared is just n times the sum of its squared attribute values and the correlation between the two vectors is their dot product divided by n.
Figure 2. Figures for exercise Note that the type of transformation that you use might depend on the application that you have in mind. Thus, consider two applications: clustering time series and predicting the behavior of one time series given another. For time series clustering, time series with relatively high positive correlation should be put together. Note that this assumes that you only want to predict magnitude, not direction.
Two examples are the following: i based on pairwise proximity, i. One approach is to compute the distance between the centroids of the two sets of points. One approach is to compute the average pairwise proximity of objects in one group of objects with those objects in the other group.
Other approaches are to take the minimum or maximum proximity. Note that the cohesion of a cluster is related to the notion of the proximity of a group of objects among themselves and that the separation of clusters is related to concept of the proximity of two groups of objects.
See Section 8. Furthermore, the proximity of two clusters is an important concept in agglomerative hierarchical clustering. You are given a set of points S in Euclidean space, as well as the distance of each point in S to a point x. Unfortunately, there is a typo and a lack of clarity in the hint. As x becomes farther away, typically more distance calculations are needed.
If the three vectors lie in a plane then it is obvious that the angle between x and z must be less than or equal to the sum of the angles between x and y and y and z. Explain why computing the proximity between two attributes is often simpler than computing the similarity between two objects. To compute the overall similarity of two objects in this case, we need to decide how to compute the similarity for each attribute and then combine these similarities.
This can be done straightforwardly by using Equations 2. In contrast, the values of an attribute are all of the same type, and thus, if another attribute is of the same type, then the computation of similarity is conceptually and computationally straightforward. The bibliographic notes and book Web site provide pointers to visualization software.
Identify at least two advantages and two disadvantages of using color to visually represent information. Advantages: Color makes it much easier to visually distinguish visual el- ements from one another.
Color can be hard to use properly. For example, a poor color scheme can be garish or can focus attention on unimportant elements. What are the arrangement issues that arise with respect to three-dimensional plots? The key issue for three dimensional plots is how to display information so that as little information is obscured as possible. If the plot is of a two- dimensional surface, then the choice of a viewpoint is critical. For three dimensional solids, the situation is even more challenging.
Typically, portions of the information must be omitted in order to provide the necessary information. For example, a slice or cross-section of a three dimensional object is often shown. In some cases, transparency can be used. Again, the ability to change the arrangement of the visual elements interactively can be helpful. Discuss the advantages and disadvantages of using sampling to reduce the number of data objects that need to be displayed.
Would simple random sampling without replacement be a good approach to sampling? Why or why not? Simple random sampling is not the best approach since it will eliminate most of the points in sparse regions.
It is better to undersample the regions where data objects are too dense while keeping most or all of the data objects from sparse regions. Describe how you would create visualizations to display information that describes the following types of systems. How will you map objects, attributes, and relation- ships to visual elements?
Are there any special considerations that need to be taken into account with respect to how visual elements are displayed? How will you handle a large number of attributes and data objects? The following solutions are intended for illustration. The connectivity of the network would best be represented as a graph, with the nodes being routers, gateways, or other communications de- vices and the links representing the connections.
The bandwidth of the connection could be represented by the width of the links. Color could be used to show the percent usage of the links and nodes. The simplest approach is to display each species on a separate map of the world and to shade the regions of the world where the species occurs.
If several species are to be shown at once, then icons for each species can be placed on a map of the world. The resource usage of each program could be displayed as a bar plot of the three quantities. For example, resource usage could be displayed as a percentage of the total. Alternatively, we could use three bar plots, one for type of resource usage.
On each of these plots there would be a bar whose height represents the usage of the corresponding program. This approach would not require any scaling. For this approach, the resource usage of all programs could be displayed on the same plot.
Assume that you have yearly information about each person that also includes gender and level of education. For each gender, the occupation breakdown could be displayed as an array of pie charts, where each row of pie charts indicates a particu- lar level of education and each column indicates a particular year. For convenience, the time gap between each column could be 5 or ten years.
Alternatively, we could order the occupations and then, for each gen- der, compute the cumulative percent employment for each occupation. If this quantity is plotted for each gender, then the area between two successive lines shows the percentage of employment for this occupa- tion.
If a color is associated with each occupation, then the area between each set of lines can also be colored with the color associated with each occupation. A similar way to show the same information would be to use a sequence of stacked bar graphs. Describe one advantage and one disadvantage of a stem and leaf plot with respect to a standard histogram. A stem and leaf plot shows you the actual distribution of values.
On the other hand, a stem and leaf plot becomes rather unwieldy for a large number of values. How might you address the problem that a histogram depends on the number and location of the bins? Describe how a box plot can give information about whether the value of an attribute is symmetrically distributed.
What can you say about the symme- try of the distributions of the attributes shown in Figure 3. For the remain- ing data, the length of the whiskers and outliers is also an indication, although, since these features do not involve as many points, they may be misleading.
Compare sepal length, sepal width, petal length, and petal width, using Figure 3. Comment on the use of a box plot to explore a data set with four attributes: age, weight, height, and income.
A great deal of information can be obtained by looking at 1 the box plots for each attribute, and 2 the box plots for a particular attribute across various categories of a second attribute. Give a possible explanation as to why most of the values of petal length and width fall in the buckets along the diagonal in Figure 3. We would expect such a distribution if the three species of Iris can be ordered according to their size, and if petal length and width are both correlated to the size of the plant and each other.
Use Figures 3. Simple line plots, such as that displayed in Figure 2. The fact that the attribute values are ordered. Describe the types of situations that produce sparse or dense data cubes.
Illustrate with examples other than those used in the book. Any set of data for which all combinations of values are unlikely to occur would produce sparse data cubes. A dense data cube would tend to arise, when either almost all combinations of the categories of the underlying attributes occur, or the level of aggregation is high enough so that all combinations are likely to have values.
The original data cube would be very sparse, but if it is aggregated to have categories consisting single or multiple car accident, the state of the accident, and the month in which it occurred, then we would obtain a dense data cube. How might you extend the notion of multidimensional data analysis so that the target variable is a qualitative variable? In other words, what sorts of summary statistics or data visualizations would be of interest?
A summary statistics that would be of interest would be the frequencies with which values or combinations of values, target and otherwise, occur. From this we could derive conditional relationships among various values. In turn, these relationships could be displayed using a graph similar to that used to display Bayesian networks.
Construct a data cube from Table 3. Is this a dense or sparse data cube? If it is sparse, identify the cells that are empty. The data cube is shown in Table 3. It is a dense cube; only two cells are empty. Table 3. Fact table for Exercise Data cube for Exercise In aggregation, groups of dimensions are combined. In some cases, as when days are aggregated into months or the sales of a product are aggregated by store location, the aggregation can be viewed as a change of scale.
Is it possible to simplify the tree? Decision tree for parity function of four Boolean attributes. Consider the training examples shown in Table 4.
Table 4. Data set for Exercise 2. Answer: The gini for each Customer ID value is 0. Therefore, the overall gini for Customer ID is 0. The gini for Female is also 0. Therefore, the overall gini for Gender is 0. Data set for Exercise 3. Instance a1 a2 a3 Target Class 1 T T 1. Answer: The gini for Family car is 0. The overall gini is 0. Answer: The gini for Small shirt size is 0.
The overall gini for Shirt Size attribute is 0. Answer: Car Type because it has the lowest gini among the three attributes. Answer: The attribute has no predictive power since new customers are assigned to new Customer IDs.
Answer: According to information gain, a1 produces the best split. Therefore, according to error rate, a1 produces the best split.
Show that the entropy of a node never increases after splitting it into smaller successor nodes. Consider the following data set for a binary class problem.
Which attribute would the decision tree induction algorithm choose? Consider the following set of training examples. What is the overall error rate of the induced tree?
Answer: Splitting Attribute at Level 1. To determine the test condition at the root node, we need to com- pute the error rates for attributes X, Y , and Z. Since Z gives the lowest error rate, it is chosen as the splitting attribute at level 1.
Splitting Attribute at Level 2. After splitting on attribute Z, the subsequent test condition may in- volve either attribute X or Y.
The corresponding two-level decision tree is shown below. What is the error rate of the induced tree? Since attribute Y leads to a smaller error rate, it provides a better split. Comment on the suitability of the greedy heuristic used for splitting attribute selection. This examples shows that a greedy heuris- tic does not always produce an optimal solution.
Build a two-level decision tree. The error rate is Decision tree and data sets for Exercise 8. The overall error rate of the induced tree is 0.
The greedy heuristic does not necessarily lead to the best tree. Consider the decision tree shown in Figure 4. For simplicity, use the strategy of adding a factor of 0. This approach is known as reduced error pruning. Consider the decision trees shown in Figure 4. Assume they are generated from a data set that contains 16 binary attributes and 3 classes, C1 , C2 , and C3.
Compute the total description length of each decision tree according to the minimum description length principle. Decision trees for Exercise 9. If there are m attributes, the cost of encoding each attribute is log2 m bits. If there are k classes, the cost of encoding a class is log2 k bits. To simplify the computation, you can assume that the total cost of the tree is obtained by adding up the costs of encoding each internal node and each leaf node. Each error is encoded by log2 n bits, where n is the total number of training instances.
Which decision tree is better, according to the MDL principle? While the. Consider a two-class problem, where there are equal number of positive and negative examples in the data. Suppose the class labels for the examples are generated randomly.
Substituting this information into the formula for. Answer: The ten-fold cross-validation and holdout method provides a better error estimate than the. Comparing the accuracy of various classification methods. Since X has a binomial distribution, r also has the same distribution. Answer: No b Is the rule set exhaustive?
Answer: Yes c Is ordering needed for this set of rules? Answer: Yes because a test instance may trigger more than one rule. Answer: No because every instance is guaranteed to trigger at least one rule.
Both algorithms apply the reduced-error pruning method to determine whether a rule needs to be pruned. For this question, you will be asked to determine whether R2 is preferred over R1 from the perspectives of rule-growing and rule-pruning.
For R1 , suppose the number of positive examples covered by the rule is , and the number of negative examples covered by the rule is For R2 , suppose the number of positive examples covered by the rule is and the number of negative examples is 5. Compute vIREP for both rules. Which rule does IREP prefer? Answer: The C4.
This is because the rules are derived from decision trees, which are induced with the objective of partitioning the feature space into homogeneous regions, without focusing on any classes. Which method between C4. Answer: The class-ordering scheme used by C4. Consider a training set that contains positive examples and negative examples. Therefore R1 is the best candidate and R3 is the worst candidate according to rule accuracy. Answer: The Laplace measure of the rules are Therefore R2 is the best candidate and R3 is the worst candidate according to the Laplace measure.
Answer: The m-estimate measure of the rules are Therefore R2 is the best candi- date and R3 is the worst candidate according to the m-estimate mea- sure. Figure 5. Determine which is the best and worst rule according to: a The likelihood ratio statistic.
Answer: There are 29 positive examples and 21 negative examples in the data set. R1 covers 12 positive examples and 3 negative examples. R2 covers 7 positive examples and 3 negative examples. Elimination of training records by the sequential covering algorithm. R1, R2, and R3 represent regions covered by three different rules.
R3 covers 8 positive examples and 4 negative examples. R1 is the best rule and R3 is the worst rule according to the likelihood ratio statistic. Answer: The Laplace measure for the rules are Therefore R1 is the best rule and R3 is the worst rule according to the Laplace measure.
Answer: The m-estimate measure for the rules are Therefore R1 is the best rule and R3 is the worst rule according to the m-estimate measure. Therefore R2 is preferred over R3.
In this case, R3 is preferred over R2. We want to compute P G S. According to Bayesian Theorem, 0. If a student smokes and lives in the dorm, is he or she more likely to be a graduate or undergraduate student? You can assume independence between students who live in a dorm and those who smoke.
Answer: First, we need to estimate all the probabilities. Consider the data set shown in Table 5. Data set for Exercise 7. Which method is better and why? Data set for Exercise 8.
State the relationships between A and B. Therefore, A and B are independent. A and B are still independent. Are the variables conditionally independent given the class? Data set for Exercise 9. Answer: NB will not do well on this data set because the conditional probabilities for each distinguishing attribute given the class are the same for both class A and class B. What if there are four classes?
Answer: For the two-class problem, decision tree will not perform well because the entropy will not improve after splitting the data using the distin- guishing attributes. If there are four classes, then decision tree will improve considerably.
Repeat the analysis shown in Example 5. Assume that all the attributes are binary. Bayesian belief network. Table 5. Data set for Exercise Bayesian belief network for Exercise Given the Bayesian network shown in Figure 5. Consider the one-dimensional data set shown in Table 5.
The nearest-neighbor algorithm described in Section 5. Answer: The training set shown in Figure 5. For each of the Boolean functions given below, state whether the problem is linearly separable.
Answer: Let x1 and x2 be a pair of Boolean variables and y be the output. Answer: Multilayer neural networks is useful for modeling nonlinear relation- ships between the input and output attributes.
However, if linear func- tions are used as activation functions instead of sigmoid or hyperbolic tangent function , the output is still a linear combination of its input attributes.
Such a network is just as expressive as a perceptron. The test set you have chosen contains 26 binary attributes, labeled as A through Z. Only the posterior probabilities for the positive class are shown. Assume that we are mostly interested in detecting instances from the positive class. Posterior probabilities for Exercise You should plot them on the same graph. Which model do you think is better? Explain your reasons. ROC curve. Compute the precision, recall, and F-measure for the model at this threshold value.
Compare the F -measure results for both models. Which model is better? Are the results consistent with what you expect from the ROC curve? Based on F-measure, M 1 is still better than M 2. This result is consis- tent with the ROC plot. Since 0. This result is inconsistent with the results using F-measure. Answer: There are 30 positive and negative examples in the data. Hint: only the leaves of the old decision tree need to be changed. Answer: Find the maximum margin linear decision boundary in the transformed space.
Given the data sets shown in Figures 5. Answer: a Both decision tree and NB will do well on this data set because the distinguishing attributes have better discriminating power than noise attributes in terms of entropy gain and conditional probability.
Other schemes will do better than NB. Decision tree will not do as well, due to the relatively large number of distinguishing attributes. Decision trees will also work, but will result in a fairly large decision tree. NB will not perform quite as well due to the attribute dependency. Decision trees will also work, but will result in a large decision tree.
If decision tree uses an oblique split instead of just vertical and horizontal splits, then the resulting decision tree will be more compact and highly accurate.
NB will not perform quite as well due to attribute dependency. NB does not work well for this data set due to attribute dependency. Decision tree will have a large tree in order to capture the circular decision boundaries.
Also, describe whether such rules are subjectively interesting. Such obvious rule tends to be uninteresting. While the sale of tuna and milk may be higher than the support threshold, not all transactions that contain milk also contain tuna. Such rule tends to be interesting. Consider the data set shown in Table 6. Example of market basket transactions. Answer: 0.
Each item should be treated as a binary variable 1 if an item appears in at least one transaction bought by the customer, and 0 otherwise.
Discuss whether there are any relationships between s1 and s2 or c1 and c2. Answer: There are no apparent relationships between s1 , s2 , c1 , and c2. For each of the following measures, determine whether it is monotone, anti- monotone, or non-monotone i. An itemset of size k can produce up to k characteristic rules.
An itemset of size k can produce up to k discriminant rules. Hence, the measure is non-monotone. Therefore, the measure is non-monotone. Prove Equation 6. Hint: First, count the number of ways to create an itemset that forms the left hand side of the rule.
0コメント