This course addresses principles, methods and practical recommendations for extracting interesting and meaningful patterns from structured and unstructured data (numeric and textual data), from a perspective at the interface between Computer Science and Statistics. The course covers fundamental topics and computational methods for the growing field of Data Analytics and Mining.
The course is organized in two modules:
(i) Module I is about data understanding and pre‐processing with exploratory data analysis, and also on data-driven clustering algorithms to induce models from data and on their interpretation.
(ii) Module II is about Relevant Information Extraction, symbolic and statistical analysis of texts, document descriptors, document classification and distribution of words and multi-words in Big Data context.
Knowledge
- Understand the paradigms and challenges of Data Analytics and Text Mining
- Learn the fundamental methods and their applications in the extraction of patterns from data. Understand data features, the selection of models and interpretation of model’s results.
- Understand the advantages and disadvantages of the different methods.
Skills
- Implement and adapt Data Analytics and Text Mining algorithms;
- Model real data experimentally.
- Interpret and evaluate experimental results.
Competences
- Evaluate the suitability of each method to case studies
- Critical evaluation of the results.
Autonomy and self-reliance in the application and furthering studies in Data Analytics and Text Mining.
Introduction
Data Analytics
What is data: Examples of data analytic tasks and various perspectives of them
Visualization as a convenient tool for business analytics
Text Mining
Structured or unstructured data? Why mining texts?
What types of problems can be solved?
Data Understanding
- 1D Summarization and Visualization of a Single Feature
- 2D Analysis: Correlation and Visualization of Two Quantitative Features
- Verification of structure in data
Data Preparation
- Variable cleaning
- Feature creating
- Why normalization matters
Descriptive Modeling I
Principal Component Analysis(PCA): Model and Method
- Summarization versus Correlation
- Matrix spectrum and Singular Value Decomposition (SVD)
- PCA as SVD. Conventional PCA’s.
PCA: Applications
Descriptive Modeling II
-
K‐means, Anomalous clusters, IntelligentK‐Means
- Spectral clustering
- Relational clustering (if time permits)
Interpreting Descriptive Models
- Conventional Cluster Model Interpretation
- Assessing Cluster Tendency
- Least squares principle induced interpretation aids
Data Analytics Case Studies
Relevant Information Extraction
- Relevant Expressions: Multi‐words and single‐words
- Statistical vs symbolic extractors. Algorithms and metrics
- Language‐independence
Symbolic and Statistical Analysis of texts
- Tokenization, Stemming and Part‐Of‐Speech Tagging
- Word and Multi-word distribution in Big Data context. Zipf Law
- Metrics for word association and retrieval
- Document correlation
- Word Sense Disambiguation
Document Descriptors
- Language‐independent Mining of Explicit and Implicit Keywords from documents.
- Semantic Scope of Documents
- Document Summarization
Document Classification
- Relevant Expressions as features for document characterization. Feature selection and reduction.
- Document Similarity
- Supervised vs unsupervised Document Clustering.
- Prediction and evaluation
Text Mining Case Studies(some examples)
- Extraction of Named Entities
- Email filtering
- Language detection
- Efficient Extraction of Multiwords
- Polarity Detection